CN116524118B - Multi-mode rendering method based on three-dimensional tooth CBCT data and oral cavity scanning model - Google Patents
Multi-mode rendering method based on three-dimensional tooth CBCT data and oral cavity scanning model Download PDFInfo
- Publication number
- CN116524118B CN116524118B CN202310430198.6A CN202310430198A CN116524118B CN 116524118 B CN116524118 B CN 116524118B CN 202310430198 A CN202310430198 A CN 202310430198A CN 116524118 B CN116524118 B CN 116524118B
- Authority
- CN
- China
- Prior art keywords
- cbct
- data
- oral cavity
- curvature
- tooth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007408 cone-beam computed tomography Methods 0.000 title claims abstract description 166
- 210000000214 mouth Anatomy 0.000 title claims abstract description 116
- 238000009877 rendering Methods 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 title claims abstract description 41
- 239000011159 matrix material Substances 0.000 claims abstract description 141
- 230000009466 transformation Effects 0.000 claims abstract description 51
- 238000012216 screening Methods 0.000 claims abstract description 22
- 210000004872 soft tissue Anatomy 0.000 claims abstract description 12
- 230000001131 transforming effect Effects 0.000 claims abstract description 12
- 239000013598 vector Substances 0.000 claims description 136
- 230000011218 segmentation Effects 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 13
- 238000004422 calculation algorithm Methods 0.000 claims description 11
- 210000004195 gingiva Anatomy 0.000 claims description 9
- 238000012935 Averaging Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 210000000515 tooth Anatomy 0.000 abstract description 114
- 238000012545 processing Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 239000000284 extract Substances 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 208000012868 Overgrowth Diseases 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 239000006227 byproduct Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000002591 computed tomography Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 208000002925 dental caries Diseases 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 210000004400 mucous membrane Anatomy 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 230000003239 periodontal effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000001356 surgical procedure Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Dental Tools And Instruments Or Auxiliary Dental Instruments (AREA)
Abstract
The application provides a multi-mode rendering method based on three-dimensional tooth CBCT data and an oral cavity scanning model, which comprises the following steps: obtaining dental crown surface data by acquiring CBCT and oral cavity scanning model data and respectively removing gum and soft tissue areas; calculating the curvature of each non-boundary triangular mesh vertex, and screening to obtain a high curvature point set in the CBCT and oral cavity scanning model; and carrying out feature description on the triangular mesh vertexes in each high-curvature point set, calculating to obtain a rigid body transformation matrix through the corresponding point set, and transforming the CBCT and the dental crown surface data of the oral cavity scanning model into the same coordinate system for rendering. According to the scheme, the volume data and the surface data are screened, the advantages of the volume data and the surface data are combined, the volume data and the surface data are transformed to the same coordinate system for rendering, and a dentist can be helped to better judge the situation of teeth.
Description
Technical Field
The application relates to the field of medical multi-mode data processing, in particular to a multi-mode rendering method based on three-dimensional tooth CBCT data and an oral scanning model.
Background
In the field of multi-modal data processing, the basic idea is the process of processing medical images of different modalities (e.g. MRI, CT, PET, etc.), which may have different resolutions, contrasts, sampling rates or other characteristics, and therefore need to be converted into the same data type before reliable analysis and comparison can be performed.
In the field of dental treatment, along with the continuous development and maturity of a digital technology, a dentist can obtain more accurate diagnosis and treatment effects through the digital technology, and meanwhile, the medical efficiency can be improved, the main digital technology comprises technologies such as CBCT (computed tomography) and oral scanning, wherein the Chinese full name of the CBCT is oral and maxillofacial cone beam computed tomography, the CBCT is a low-dose oral X-ray scanning technology, a high-resolution three-dimensional image can be provided, and complete teeth (dental crowns and tooth roots) are contained, so that illness state and planning treatment schemes can be more accurately diagnosed, the CBCT is usually stored in a DICOM (digital imaging and communication) format, and influence information mainly comprises an image pixel matrix, pixel resolution, sampling frequency, patient information and the like; the oral cavity scanning technology can obtain a high-precision three-dimensional dental crown and gum model, the storage format of an oral cavity scanning result is usually standard triangular language stl, the surface of the oral cavity scanning result is composed of a plurality of triangular grids, and stl files have the characteristics of simplicity and universality, and are convenient for subsequent processing, analysis and CAD design.
The CBCT three-dimensional tooth model belongs to volume data, is three-dimensional data composed of voxel grids, can represent the internal structure of an object, has the advantages that the data is more complete, the three-dimensional volume can be calculated through the number of volume elements, has the disadvantages that the details of the surface information of the object are more lost, the oral cavity scanning model belongs to surface data, is three-dimensional data composed of a plurality of triangular grids, can represent the external surface structure of the object, has the advantages that the surface accuracy of the object is higher, and is not suitable for calculating the three-dimensional volume, and the internal structure cannot be represented.
In view of the foregoing, there is a need for a method that allows multi-modal rendering of volume and face data based on a three-dimensional dental model to better aid dentists in further processing of teeth in dental care, in digital dental care and in computer-aided digital high-precision dental modeling.
Disclosure of Invention
The embodiment of the application provides a multi-mode rendering method based on three-dimensional tooth CBCT data and an oral cavity scanning model, which can be used for transforming CBCT data and oral cavity model data into the same coordinate system for rendering according to the screening of curvature points and rigid body transformation matrix, and helps to better observe tooth conditions in digital dental treatment.
In a first aspect, embodiments of the present application provide a multi-modal rendering method based on three-dimensional tooth CBCT data and an oral scan model, the method comprising:
acquiring CBCT data, inputting the CBCT data into a trained tooth segmentation model for segmentation to obtain three-dimensional tooth volume data, extracting the tooth surface of the three-dimensional tooth volume data to obtain tooth surface data, and removing the tooth root part of each tooth in the tooth surface data to obtain CBCT crown surface data, wherein the CBCT crown surface data is stored in a triangular mesh form;
acquiring an oral cavity scanning model corresponding to CBCT data, and removing gingiva and other soft tissue areas in the oral cavity scanning model to obtain dental crown surface data of the oral cavity scanning model, wherein the dental crown surface data of the oral cavity scanning model is stored in a triangular mesh form;
acquiring triangular mesh vertexes of each triangular mesh which is not at a boundary in the CBCT dental crown surface data and the oral cavity scanning model dental crown surface data, calculating a normal vector of each triangular mesh vertex based on a face averaging algorithm, acquiring an area sum of adjacent triangular meshes of each triangular mesh vertex and an angle cosine square sum of the adjacent triangular mesh and the normal vector of the corresponding triangular mesh vertex, calculating the curvature of each triangular mesh vertex according to the angle cosine square sum and the area sum of the adjacent triangular mesh of each triangular mesh vertex, setting a curvature threshold value, and screening the curvature of each triangular mesh vertex to obtain a CBCT high curvature point set and an oral cavity scanning model high curvature point set respectively;
Calculating characteristic column vectors of each high-curvature triangle grid vertex in the CBCT high-curvature point set and the oral cavity scanning model high-curvature point set, splicing the characteristic column vectors of each high-curvature triangle grid vertex to respectively obtain a CBCT characteristic description matrix and an oral cavity scanning model characteristic description matrix, constructing a vector similarity matrix according to the CBCT characteristic description matrix and the oral cavity scanning model characteristic description matrix, and screening similar points in the vector similarity matrix to obtain a corresponding point set;
and calculating by using the corresponding point set to obtain a rigid transformation matrix, transforming the three-dimensional tooth volume data by using the rigid transformation matrix to obtain rigid transformation volume data, and rendering the rigid transformation volume data and the dental crown surface data of the oral scanning model.
In a second aspect, embodiments of the present application provide a multi-modal rendering apparatus based on three-dimensional tooth CBCT data and an oral scan model, comprising:
a first acquisition module: acquiring CBCT data, inputting the CBCT data into a trained tooth segmentation model for segmentation to obtain three-dimensional tooth volume data, extracting the tooth surface of the three-dimensional tooth volume data to obtain tooth surface data, and removing the tooth root part of each tooth in the tooth surface data to obtain CBCT crown surface data, wherein the CBCT crown surface data is stored in a triangular mesh form;
And a second acquisition module: acquiring an oral cavity scanning model corresponding to CBCT data, and removing gingiva and other soft tissue areas in the oral cavity scanning model to obtain dental crown surface data of the oral cavity scanning model, wherein the dental crown surface data of the oral cavity scanning model is stored in a triangular mesh form;
the curvature calculation module: acquiring triangular mesh vertexes of each triangular mesh which is not at a boundary in the CBCT dental crown surface data and the oral cavity scanning model dental crown surface data, calculating a normal vector of each triangular mesh vertex based on a face averaging algorithm, acquiring an area sum of adjacent triangular meshes of each triangular mesh vertex and an angle cosine square sum of the adjacent triangular mesh and the normal vector of the corresponding triangular mesh vertex, calculating the curvature of each triangular mesh vertex according to the angle cosine square sum and the area sum of the adjacent triangular mesh of each triangular mesh vertex, setting a curvature threshold value, and screening the curvature of each triangular mesh vertex to obtain a CBCT high curvature point set and an oral cavity scanning model high curvature point set respectively;
the point set acquisition module: calculating characteristic column vectors of each high-curvature triangle grid vertex in the CBCT high-curvature point set and the oral cavity scanning model high-curvature point set, splicing the characteristic column vectors of each high-curvature triangle grid vertex to respectively obtain a CBCT characteristic description matrix and an oral cavity scanning model characteristic description matrix, constructing a vector similarity matrix according to the CBCT characteristic description matrix and the oral cavity scanning model characteristic description matrix, and screening similar points in the vector similarity matrix to obtain a corresponding point set;
And a rendering module: and calculating by using the corresponding point set to obtain a rigid transformation matrix, transforming the three-dimensional tooth volume data by using the rigid transformation matrix to obtain rigid transformation volume data, and rendering the rigid transformation volume data and the dental crown surface data of the oral scanning model.
In a third aspect, embodiments of the present application provide an electronic device comprising a memory having a computer program stored therein and a processor configured to run the computer program to perform a multi-modal rendering method based on three-dimensional tooth CBCT data and an oral scan model.
In a fourth aspect, embodiments of the present application provide a readable storage medium having a computer program stored therein, the computer program comprising program code for controlling a process to perform a process comprising a multi-modal rendering method based on three-dimensional tooth CBCT data and an oral scan model.
The main contributions and innovation points of the invention are as follows:
according to the embodiment of the application, the CBCT data are calculated to form the normal vector included angle, and subdivision is carried out according to a calculation result by a loop subdivision iteration method, so that the accuracy of the obtained dental crown surface data is higher; according to the scheme, seed points are set for each tooth, and the seed points with high similarity are combined, so that gum and other soft tissue parts of an oral cavity scanning model are removed; according to the scheme, the normal vector included angle of each high-curvature point in the high-curvature point set and each triangular mesh of the neighborhood two-ring is calculated to convert the high-curvature point set into the feature description matrix, so that the accuracy of the feature description matrix is higher; according to the scheme, the coordinates of the three-dimensional tooth body data are preprocessed through homogeneous coordinate transformation, the three-dimensional tooth body data are rendered through the rigid body transformation matrix, and the visual rendering results of the CBCT data and the oral cavity scanning model are rendered into the same coordinate system, so that the digital diagnosis of dental medical treatment is facilitated.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the other features, objects, and advantages of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
FIG. 1 is a flow chart of a multi-modal rendering method based on three-dimensional tooth CBCT data and an oral scan model in accordance with an embodiment of the present application;
FIG. 2 is a schematic diagram of selecting triangular mesh vertices according to an embodiment of the present application;
FIG. 3 is a schematic illustration of computing normal vectors for each triangle mesh vertex based on a face averaging algorithm in accordance with an embodiment of the present application;
FIG. 4 is a schematic illustration of normal vector angles according to an embodiment of the present application;
FIG. 5 is a schematic diagram of normal vector angles between vertices of a triangular mesh and a neighborhood bi-ring according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a structure for eliminating pairs of error points according to an embodiment of the present application;
FIG. 7 is a block diagram of a multi-modal rendering device based on three-dimensional tooth CBCT data and an oral scan model in accordance with an embodiment of the present application;
Fig. 8 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of the present specification. Rather, they are merely examples of apparatus and methods consistent with aspects of one or more embodiments of the present description as detailed in the accompanying claims.
It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than described in this specification. Furthermore, individual steps described in this specification, in other embodiments, may be described as being split into multiple steps; while various steps described in this specification may be combined into a single step in other embodiments.
To facilitate understanding of the present solution, terms appearing in the present solution are explained herein:
CBCT data: CBCT is a short name of Cone beam CT, which is a Cone beam projection computer-based reconstruction tomographic imaging apparatus, and its principle is that an X-ray generator makes a ring DR (digital projection) around a projection object with a relatively low radiation amount (typically, a bulb current is about 10 milliamperes), and then "recombines" data obtained in an "intersection" after digital projection around the projection object multiple times (180-360 times, product-by-product) in a computer, and then obtains a three-dimensional image.
Oral scan model: the oral cavity scanning model is characterized in that a penetrating optical scanning head of a digital scanner is applied in an oral cavity, the oral cavity of a patient is directly scanned, three-dimensional morphology and color texture information of the surfaces of soft and hard tissues such as teeth, gingiva and mucous membrane in the oral cavity are obtained, an acquired image is subjected to computer processing reconstruction through an optical imaging system to obtain three-dimensional data information under the current visual angle, and a complete three-dimensional digital impression is obtained through a matching and splicing technology. Is widely applied to a plurality of fields such as oral restoration, orthodontics, periodontal, maxillofacial surgery and the like.
Example 1
The embodiment of the application provides a multi-modal rendering method based on three-dimensional tooth CBCT data and an oral cavity scanning model, and specifically referring to FIG. 1, the method comprises the following steps:
acquiring CBCT data, inputting the CBCT data into a trained tooth segmentation model for segmentation to obtain three-dimensional tooth volume data, extracting the tooth surface of the three-dimensional tooth volume data to obtain tooth surface data, and removing the tooth root part of each tooth in the tooth surface data to obtain CBCT crown surface data, wherein the CBCT crown surface data is stored in a triangular mesh form;
acquiring an oral cavity scanning model corresponding to CBCT data, and removing gingiva and other soft tissue areas in the oral cavity scanning model to obtain dental crown surface data of the oral cavity scanning model, wherein the dental crown surface data of the oral cavity scanning model is stored in a triangular mesh form;
acquiring triangular mesh vertexes of each triangular mesh which is not at a boundary in the CBCT dental crown surface data and the oral cavity scanning model dental crown surface data, calculating a normal vector of each triangular mesh vertex based on a face averaging algorithm, acquiring an area sum of adjacent triangular meshes of each triangular mesh vertex and an angle cosine square sum of the adjacent triangular mesh and the normal vector of the corresponding triangular mesh vertex, calculating the curvature of each triangular mesh vertex according to the angle cosine square sum and the area sum of the adjacent triangular mesh of each triangular mesh vertex, setting a curvature threshold value, and screening the curvature of each triangular mesh vertex to obtain a CBCT high curvature point set and an oral cavity scanning model high curvature point set respectively;
Calculating characteristic column vectors of each high-curvature triangle grid vertex in the CBCT high-curvature point set and the oral cavity scanning model high-curvature point set, splicing the characteristic column vectors of each high-curvature triangle grid vertex to respectively obtain a CBCT characteristic description matrix and an oral cavity scanning model characteristic description matrix, constructing a vector similarity matrix according to the CBCT characteristic description matrix and the oral cavity scanning model characteristic description matrix, and screening similar points in the vector similarity matrix to obtain a corresponding point set;
and calculating by using the corresponding point set to obtain a rigid transformation matrix, transforming the three-dimensional tooth volume data by using the rigid transformation matrix to obtain rigid transformation volume data, and rendering the rigid transformation volume data and the dental crown surface data of the oral scanning model.
In some embodiments, in the step of inputting the CBCT data into a trained tooth segmentation model to perform segmentation to obtain three-dimensional tooth volume data, the tooth segmentation model processes the CBCT data to obtain a segmentation result, the segmentation result is a tooth probability of each pixel point, a tooth threshold is set, the segmentation result is subjected to threshold segmentation by using the tooth threshold to obtain a tooth binary image, and the tooth binary image is output to obtain the three-dimensional tooth volume data.
Specifically, the tooth segmentation model may be any trained network model, which is not limited in this scheme.
In some embodiments, the tooth segmentation model used in the present solution includes an encoding module and a decoding module, where the encoding module includes a first encoder, a second encoder, and a third encoder that are sequentially connected in series, the decoding module includes a first decoder, a second decoder, and a third decoder that are sequentially connected in series, the first encoder extracts tooth edge information in the CBCT data, the second encoder extracts texture and shape information of a tooth according to the tooth edge information, the third encoder obtains tooth contour features according to the texture and shape information of the tooth, the first decoder decodes the tooth contour information to obtain a first decoding result, the second decoder upsamples the first decoding result and then splices the first decoding result with the texture and shape information of the tooth to obtain a second decoding result, the third decoder upsamples the second decoding result and then splices the second decoding result with the tooth edge information to obtain a third decoding result, and the third decoding result obtains the tooth probability of each pixel through an activation function.
Specifically, the input of the tooth model is a two-dimensional slice sequence of CT data, the encoding module of the tooth segmentation model extracts feature information of the CT data through convolution and pooling operations, the decoding module of the tooth segmentation model restores the feature information to the same size as the input data through operations such as deconvolution and upsampling, the first encoder acquires tooth edge information through extracting low-level features, the second encoder acquires texture and shape features of the teeth, the third encoder extracts high-level tooth contour features, the first decoder upsamples the high-level tooth contour features extracted by the third encoder, then performs stitching with texture and shape features of the teeth extracted by the second encoder, finally performs feature extraction and upsampling through a series of convolution layers to obtain an intermediate result image with the same size as the input image, the intermediate result image is the third decoding result, the second decoder performs upsampling on the third decoding and then performs stitching with texture and shape features of the teeth, finally performs convolutional layer extraction and upsampling on the third convolution layer, the second decoder performs upsampling on the third convolution layer and the second convolution layer decoding result and the third convolution layer decoding result is performed, and the third result is obtained, and the probability point-based on the third result is obtained, and the result is sampled and the third result is obtained after the third point-and the result is sampled.
Specifically, setting a tooth threshold t, setting a pixel point with the tooth probability larger than t as 1, setting the rest pixel points as 0 to obtain a tooth binary image, and outputting the tooth binary image to obtain three-dimensional tooth volume data.
Specifically, the tooth segmentation model continuously updates the weight parameters of the model through back propagation, so that the loss function is minimized.
Specifically, the method evaluates the performance of the model by using the verification set, and judges whether the model has the fitting problem or not by monitoring the change condition of the loss function and the evaluation index.
Specifically, the tooth segmentation model is tested by using the test set to obtain performance indexes such as segmentation accuracy of the model so as to evaluate the generalization capability of the model.
In particular, the tooth segmentation model is trained, validated and tested using the disclosed CT dataset CQ 500.
In some embodiments, in the step of removing the root portion of each tooth in the tooth surface data to obtain CBCT crown surface data, the tooth surface data is composed of a plurality of triangular meshes, a normal vector of each triangular mesh in the tooth surface data is calculated, a normal vector included angle of each triangular mesh and all neighboring triangular meshes is calculated, an included angle threshold is set, and if at least one normal vector included angle is greater than the included angle threshold, the triangular mesh and all neighboring triangular meshes are subdivided until all normal vector included angles are less than the included angle threshold to obtain CBCT crown surface data.
Specifically, considering that in all teeth, the teeth are uneven and have more detail information on the surface of the teeth due to the reasons of tooth grinding, tooth decay and the like, if a traditional Maring cube algorithm (a conventional reconstruction algorithm) is adopted to generate the crown surface, the tooth surface is not smooth or unnecessary triangular grids exist, and the follow-up operation is not facilitated, the scheme determines the triangular grids with smaller precision by calculating the included angle between each triangular grid and the normal vector of the adjacent triangular grids, and subdivides the triangular grids by a Loop subdivision iteration method to obtain the crown surface with higher precision.
Specifically, the inverse vector of each triangle mesh is calculated according to the vector cross multiplication of two sides of the triangle, assuming that three vertices of a certain triangle mesh are respectively denoted as a (x A ,y A ,z A )、B(x B ,y B ,z B ) And C (x) C ,y C ,z C ) The normal vector of the triangular mesh can be expressed as:
where n is the normal vector of the corresponding triangular mesh, AB is the vector representation of the triangular mesh AB edge, and AC is the vector representation of the triangular mesh AC edge.
Specifically, the Loop subdivision iteration method is used for refining the triangular mesh into a triangular mesh with higher resolution.
In the step of calculating the normal vector included angle between each triangular grid and all adjacent triangular grids, if a triangular grid a exists and k triangular grids are adjacent to each triangular grid, calculating the normal vector included angle b between each triangular grid in the k triangular grids and the triangular grid a, setting an included angle threshold t, and if the normal vector included angle b is larger than the included angle threshold t, subdividing the k triangular grids and the triangular grid a by using a Loop subdivision iteration method to obtain triangular grids with higher precision and higher resolution.
The CBCT crown surface data are obtained by carrying out the operation on each triangular mesh, and the tooth root part of each tooth in the CBCT crown surface data can be further refined and removed by a manual point selection mode.
In some embodiments, in the step of removing gums and other soft tissue regions in the oral cavity scan model to obtain dental crown surface data of the oral cavity scan model, a first number of triangular mesh vertices are randomly selected as seed points on the dental crown surface of each tooth in the oral cavity scan model, adjacent triangular meshes of each seed point are obtained, a similarity included angle is obtained by calculating a normal vector of each seed point and normal vector included angles of all adjacent meshes of each seed point, a growth threshold is set, and if the similarity included angle of any two seed points is larger than the growth threshold, the two seed points are combined until no seed points which can be combined exist in the oral cavity scan model or the designated combining times are reached, and dental crown surface data of the oral cavity scan model is obtained according to a seed point combining path.
In some embodiments, as shown in fig. 2, at least two triangular mesh vertices are selected as seed points on the crown surface of each tooth in the oral scan model.
Specifically, since the storage format of the oral cavity scanning model is stl file, which does not contain information such as color, texture, etc., the combination is performed by calculating the similarity included angle of the seed points, and the growth threshold is set for avoiding overgrowth of the region.
Specifically, when no seed points which can be combined exist in the oral cavity scanning model or the specified combination times are reached, the seed points are uniformly distributed in the crown area of each tooth, gums and other soft tissue areas are removed, and finally the seed points are connected along the seed point combination path to obtain the dental crown surface data of the oral cavity scanning model.
In some embodiments, in the "calculate normal vector of each triangle mesh vertex" step, all neighboring triangle meshes of each triangle mesh vertex are obtained as vertex neighboring triangle meshes, the normal vector of each vertex neighboring triangle mesh is calculated, and the normal vectors of all vertex neighboring triangle meshes are summed and averaged to obtain the normal vector of the triangle mesh vertex.
In some embodiments, as shown in FIG. 3, the normal vector for each triangle mesh vertex is calculated based on a face averaging algorithm.
For example, if triangle mesh vertex P has m vertex neighbors of the triangle mesh, this calculates the m vertices respectivelyThe normal vectors of adjacent triangular meshes are summed and averaged, if the normal vector of the ith triangular network in the m vertex adjacent triangular meshes is expressed asThe normal vector of the vertex P can be expressed as:
in some embodiments, the sum of areas of adjacent triangular meshes of each triangular mesh vertex is calculated, the sum of squares of the cosine values of the included angles of the normal vector of each triangular mesh vertex and the normal vector of the adjacent triangular mesh of the corresponding triangular mesh vertex is calculated, and the sum of areas and squares are used to obtain the curvature of the triangular mesh vertex.
Specifically, for the vertex P, the area and area_sum of the adjacent m triangular meshes are calculated, and area is used i Representing the area of the ith triangular mesh in the m adjacent triangular meshes, then for the vertex P, the sum of the areas of the adjacent m triangular meshes is represented as:
specifically, as shown in fig. 4, for the triangle mesh vertex P, the sum of squares cos_sum of angle cosine values of normal vectors of m triangle meshes adjacent to the normal vector of the triangle mesh vertex P is calculated, and the larger the sum of squares is, the more the normal vector is changed, that is, the greater the degree of curvature of the curved surface is, θ is used i Representing the angle between the normal vector of the ith triangular mesh in m adjacent triangular meshes and the normal vector of the triangular mesh vertex P, cos_sum can be expressed as follows:
specifically, the curvature of the triangle mesh vertex P is:
wherein k is p Representing the curvature of the p-point.
In some embodiments, in the step of screening the curvature of each triangular mesh vertex to obtain a CBCT high curvature point set and an oral scan model high curvature point set respectively by setting a curvature threshold, triangular mesh vertices with the triangular mesh vertex curvature greater than the curvature threshold in CBCT crown surface data are selected as the CBCT high curvature point set, and triangular mesh vertices with the triangular mesh vertex curvature greater than the curvature threshold in the oral scan model crown surface data are selected as the oral scan model high curvature point set.
In some embodiments, in the step of calculating the feature column vector of each high curvature triangle mesh vertex in the CBCT high curvature point set and the oral scan model high curvature point set, all triangle mesh normal vectors in each high curvature triangle mesh vertex neighborhood secondary are obtained, the normal vector of each high curvature triangle mesh vertex is obtained, the included angle between the normal vector of each high curvature triangle mesh vertex and all triangle mesh normal vectors in the corresponding high curvature triangle mesh vertex neighborhood secondary is calculated, all triangle meshes in the neighborhood secondary comprise the neighborhood triangle mesh of each high curvature triangle mesh vertex field triangle mesh, and the feature column vector of each high curvature triangle mesh vertex is generated according to the included angle between the normal vector of each high curvature triangle mesh vertex and all triangle mesh normal vectors in the corresponding high curvature triangle mesh vertex neighborhood secondary.
Specifically, as shown in FIG. 5, for CBCT a high curvature point set N 1 Is a high curvature triangle mesh vertex p 1 Calculating to obtain normal vector and p of all triangular grids in the neighborhood second ring 1 And if the included angle of the normal vectors is s triangular grids in the neighborhood second ring, calculating the included angles of the normal vectors of the s triangular grids and the p1 normal vector respectively.
Further, the included angle between the normal vector of each high-curvature triangular grid vertex and the normal vectors of all triangular grids in the neighborhood secondary ring of the corresponding high-curvature triangular grid vertex is rounded, the occurrence times of the included angles with different angles are calculated, and the characteristic column vector corresponding to the high-curvature triangular grid vertex is generated according to the angle and the occurrence times.
The method for calculating the characteristic column vector of each high curvature triangular mesh vertex in the high curvature point set of the oral cavity scanning model is the same as that described above, and the scheme is not repeated here.
Specifically, the feature column vector is a×1 dimension, and the feature column vector is used as a high-dimensional geometric feature corresponding to the high-curvature triangular mesh vertex.
In some embodiments, feature column vectors of each high-curvature triangular mesh vertex in the CBCT high-curvature point set are calculated and spliced to obtain a CBCT feature description matrix.
In some specific embodiments, the characteristic column vector of each high curvature triangle mesh vertex in the high curvature point set of the oral scan model is calculated, and is spliced to obtain the characteristic description matrix of the oral scan model.
Specifically, the CBCT feature description matrix and the oral scan model feature description matrix are geometry description matrices.
In some embodiments, in the step of constructing a vector similarity matrix according to the CBCT feature description matrix and the oral cavity scan model feature description matrix and screening similar points in the vector similarity matrix to obtain a corresponding point set, calculating vector similarity of the CBCT feature description matrix and the oral cavity scan model feature description matrix to obtain a vector similarity matrix, where the vector similarity matrix includes similarity of corresponding points in the CBCT feature description matrix and the oral cavity scan model feature description matrix, setting a similarity threshold, screening the vector similarity matrix by using the similarity threshold to obtain point pair information, and removing erroneous point pairs in the point pair information to obtain the corresponding point set.
Specifically, if CBCT has a high curvature point set N 1 Consensus n 1 A point of the light-emitting diode is located,the CBCT characterization matrix D 1 Is an m x n 1 If the oral cavity scans the matrix of the high curvature point set N 2 Consensus n 2 A plurality of points, the characteristic description matrix D of the oral cavity scanning model 2 Is an m x n 2 The vector similarity matrix S can be expressed as:
S＝D 1 T ·D 2
wherein S is an n 1 X n2 matrix in which the elements s ij Representing a set of points N 1 Points i and point set N in (1) 2 Feature vector similarity of point j in (a).
Screening elements larger than the similarity threshold value from the vector similarity matrix S according to the similarity threshold value to obtain r pairs of corresponding points, wherein r is smaller than or equal to n 1 And r is less than or equal to n 2 The r corresponding points are the corresponding point set fcrr.
Further, any two point pairs are selected from the point pair information to be a first point pair and a second point pair respectively, a first distance absolute value between the first point pair and the second point pair is calculated, a rejection threshold is set, if the first distance absolute value is larger than the rejection threshold, any one point pair except the first point pair and the second point pair is selected from the point pair information to be used as a third point pair, a second distance absolute value between the first point pair and the third point pair is calculated, a third distance absolute value between the second point pair and the third point pair is calculated, if the second distance absolute value is larger than the third distance absolute value, the first point pair is rejected, and if the third distance absolute value is larger than the second distance absolute value, the second point pair is rejected, and each point pair in the point pair information is traversed to obtain a corresponding point set.
Specifically, under normal conditions, the situation that the second distance absolute value is equal to the third distance absolute value does not occur, and if the second distance absolute value is equal to the third distance absolute value, calculation errors are indicated, and parameters of the scheme need to be readjusted.
Specifically, there is a first point pair (p i ，p i ) A second point pair (p j ，p j ) Wherein p is i ，p j Describing the points in the matrix for CBCT features, q i ，q j For points in the characterization matrix of the oral scan model, the first absolute value of the distance of the first and second pairs of points can be expressed as:
d ij ＝|||p i -p j ||-||q i -q j |||
where dij represents the absolute value of the first distance of the first point pair and the second point pair.
Specifically, as shown in fig. 6, a rejection threshold is set based on the distance-keeping property of the rigid transformation, and if the first distance absolute value is greater than the rejection threshold, a third point pair (p k ，q k ) And calculates the second and third absolute distance values dik and djk, if d ik ＞d jk Then reject the first point pair, if d ik ＜d ik The second point pair is culled.
In some embodiments, in the step of calculating the rigid body transformation matrix by using the corresponding point set, the corresponding point set is divided into CBCT point information and oral scan model point information, CBCT data centroid coordinates are calculated according to the CBCT point information, oral scan model centroid coordinates are calculated according to the oral scan model point information, a CBCT coordinate matrix is obtained by obtaining coordinate information of each point bit in the CBCT point information relative to the CBCT data centroid coordinates, an oral scan model coordinate matrix is obtained by obtaining coordinate information of each point bit in the oral scan model point information relative to the oral scan model centroid coordinates, the CBCT coordinate matrix and the oral scan model coordinate matrix are integrated into a singular matrix, singular value decomposition is performed on the singular matrix to obtain a left singular matrix, a singular value matrix and a right singular matrix, a rotation matrix is obtained according to the left singular matrix and the right singular matrix, and a translation vector is calculated according to the CBCT data centroid coordinates, the oral scan model centroid coordinates and the rotation matrix, and the translation vector is calculated to obtain the rigid body transformation matrix.
Specifically, calculating an average value of each point bit coordinate in the CBCT point location information to obtain a CBCT data centroid coordinate, and the specific formula is as follows:
wherein,representing CBCT centroid coordinates, p i Represents one point in the CBCT point information, and n represents the total number of points in the CBCT point information.
Specifically, calculating an average value of each point position coordinate in the point position information of the oral cavity scanning model to obtain a centroid coordinate of the oral cavity scanning model, wherein the specific formula is as follows:
wherein,represents the centroid coordinates of the oral scan model, q i And (3) representing one point in the point information of the oral cavity scanning model, and n represents the total point number in the point information of the oral cavity scanning model.
Specifically, subtracting the CBCT data centroid point location information from each point bit information in the CBCT point location information to obtain coordinate information of each point bit relative to the CBCT data centroid coordinates, and the specific formula is as follows:
wherein,and the coordinate information of any point in the CBCT point information relative to the centroid coordinate of the CBCT data is represented.
Specifically, subtracting the centroid point position information of the oral cavity scanning model from the point position information of each point in the oral cavity scanning model to obtain the coordinate information of each point position relative to the centroid coordinate of the oral cavity scanning model, wherein the specific formula is as follows:
Wherein,coordinate information indicating coordinates of any point in the point information of the oral scan model with respect to centroid coordinates of the oral scan model.
Thereafter, all ofAnd +.>Respectively putting the matrix into the formula:
a is CBCT coordinate matrix, B is oral cavity scanning model coordinate matrix, A, B is 3×s matrix, and S is total point position number.
Specifically, the CBCT coordinate matrix and the transpose matrix of the oral scanning model coordinate matrix are integrated to obtain a singular matrix H, and the specific formula is as follows:
H＝AB T
wherein the singular matrix H is a 3×3 matrix.
Specifically, the singular matrix H is formulated as:
H＝U∑V T
wherein U is a 3×3 matrix, vectors in U are left singular vectors and are all orthogonal, sigma is a 3×3 matrix, diagonal lines in sigma are singular values, and the rest elements are 0, V T Is a 3 x 3 matrix, V T The vectors in (a) are right singular vectors, and are all orthogonal.
Specifically, multiplying the left singular matrix by the right singular matrix to obtain a rotation matrix R, where the formula is expressed as:
R＝UV T
specifically, the product of the rotation matrix and the CBCT data centroid is subtracted from the centroid coordinate of the oral scanning model to obtain a translation vector t, and the formula is as follows:
Specifically, a rigid transformation matrix is obtained according to the rotation matrix R and the translation vector t, and the formula is expressed as follows:
where 0 is a zero vector of 1×3 and 1 is a real number.
In some embodiments, in the step of transforming the three-dimensional tooth volume data using the rigid transformation matrix to obtain rigid transformation volume data, and in the step of rendering the rigid transformation volume data and the oral scan model crown surface data, performing homogeneous coordinate transformation on the three-dimensional tooth volume data to obtain a CBCT homogeneous coordinate matrix, transforming the CBCT homogeneous coordinate matrix into cartesian coordinates to obtain the rigid transformation volume data by using the rigid transformation matrix, and rendering the oral scan model crown surface data and the rigid transformation volume data into a world coordinate system, the CBCT homogeneous coordinate matrix is transformed into cartesian coordinates by using the rigid transformation matrix.
Example two
Based on the same conception, referring to fig. 7, the present application also proposes a multi-modal rendering device based on three-dimensional tooth CBCT data and an oral scan model, comprising:
a first acquisition module: acquiring CBCT data, inputting the CBCT data into a trained tooth segmentation model for segmentation to obtain three-dimensional tooth volume data, extracting the tooth surface of the three-dimensional tooth volume data to obtain tooth surface data, and removing the tooth root part of each tooth in the tooth surface data to obtain CBCT crown surface data, wherein the CBCT crown surface data is stored in a triangular mesh form;
And a second acquisition module: acquiring an oral cavity scanning model corresponding to CBCT data, and removing gingiva and other soft tissue areas in the oral cavity scanning model to obtain dental crown surface data of the oral cavity scanning model, wherein the dental crown surface data of the oral cavity scanning model is stored in a triangular mesh form;
the curvature calculation module: acquiring triangular mesh vertexes of each triangular mesh which is not at a boundary in the CBCT dental crown surface data and the oral cavity scanning model dental crown surface data, calculating a normal vector of each triangular mesh vertex based on a face averaging algorithm, acquiring an area sum of adjacent triangular meshes of each triangular mesh vertex and an angle cosine square sum of the adjacent triangular mesh and the normal vector of the corresponding triangular mesh vertex, calculating the curvature of each triangular mesh vertex according to the angle cosine square sum and the area sum of the adjacent triangular mesh of each triangular mesh vertex, setting a curvature threshold value, and screening the curvature of each triangular mesh vertex to obtain a CBCT high curvature point set and an oral cavity scanning model high curvature point set respectively;
the point set acquisition module: calculating characteristic column vectors of each high-curvature triangle grid vertex in the CBCT high-curvature point set and the oral cavity scanning model high-curvature point set, splicing the characteristic column vectors of each high-curvature triangle grid vertex to respectively obtain a CBCT characteristic description matrix and an oral cavity scanning model characteristic description matrix, constructing a vector similarity matrix according to the CBCT characteristic description matrix and the oral cavity scanning model characteristic description matrix, and screening similar points in the vector similarity matrix to obtain a corresponding point set;
And a rendering module: and calculating by using the corresponding point set to obtain a rigid transformation matrix, transforming the three-dimensional tooth volume data by using the rigid transformation matrix to obtain rigid transformation volume data, and rendering the rigid transformation volume data and the dental crown surface data of the oral scanning model.
Example III
This embodiment also provides an electronic device, referring to fig. 8, comprising a memory 404 and a processor 402, the memory 404 having stored therein a computer program, the processor 402 being arranged to run the computer program to perform the steps of any of the method embodiments described above.
In particular, the processor 402 may include a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or may be configured to implement one or more integrated circuits of embodiments of the present application.
The memory 404 may include, among other things, mass storage 404 for data or instructions. By way of example, and not limitation, memory 404 may comprise a Hard Disk Drive (HDD), floppy disk drive, solid State Drive (SSD), flash memory, optical disk, magneto-optical disk, tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Memory 404 may include removable or non-removable (or fixed) media, where appropriate. Memory 404 may be internal or external to the data processing apparatus, where appropriate. In a particular embodiment, the memory 404 is a Non-Volatile (Non-Volatile) memory. In particular embodiments, memory 404 includes Read-only memory (ROM) and Random Access Memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or FLASH memory (FLASH) or a combination of two or more of these. The RAM may be Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM) where appropriate, and the DRAM may be fast page mode dynamic random access memory 404 (FPMDRAM), extended Data Output Dynamic Random Access Memory (EDODRAM), synchronous Dynamic Random Access Memory (SDRAM), or the like.
Memory 404 may be used to store or cache various data files that need to be processed and/or used for communication, as well as possible computer program instructions for execution by processor 402.
The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any of the multi-modal rendering methods of the above embodiments based on three-dimensional tooth CBCT data and an oral scan model.
Optionally, the electronic apparatus may further include a transmission device 406 and an input/output device 408, where the transmission device 406 is connected to the processor 402 and the input/output device 408 is connected to the processor 402.
The transmission device 406 may be used to receive or transmit data via a network. Specific examples of the network described above may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, simply referred to as NIC) that can connect to other network devices through the base station to communicate with the internet. In one example, the transmission device 406 may be a Radio Frequency (RF) module, which is configured to communicate with the internet wirelessly.
The input-output device 408 is used to input or output information. In this embodiment, the input information may be CBCT data, an oral scan model, and the like, and the output information may be CBCT rendering results, oral scan model rendering results, and the like.
Alternatively, in the present embodiment, the above-mentioned processor 402 may be configured to execute the following steps by a computer program:
s101, acquiring CBCT data, inputting the CBCT data into a trained tooth segmentation model for segmentation to obtain three-dimensional tooth volume data, extracting the tooth surface of the three-dimensional tooth volume data to obtain tooth surface data, and removing the root part of each tooth in the tooth surface data to obtain CBCT crown surface data, wherein the CBCT crown surface data is stored in a triangular mesh form;
s102, acquiring an oral cavity scanning model corresponding to CBCT data, and removing gingiva and other soft tissue areas in the oral cavity scanning model to obtain dental crown surface data of the oral cavity scanning model, wherein the dental crown surface data of the oral cavity scanning model is stored in a triangular mesh form;
s103, triangular mesh vertexes of triangular meshes which are not on the boundary in the CBCT dental crown surface data and the oral cavity scanning model dental crown surface data are obtained, a normal vector of each triangular mesh vertex is calculated based on a face average algorithm, the sum of areas of adjacent triangular meshes of each triangular mesh vertex and the sum of cosine squares of included angles of the adjacent triangular meshes and the normal vector of the corresponding triangular mesh vertex are obtained, the curvature of each triangular mesh vertex is calculated according to the sum of the cosine squares of included angles of the adjacent triangular meshes of each triangular mesh vertex and the sum of the areas, a curvature threshold is set, and the curvature of each triangular mesh vertex is screened to obtain a CBCT high curvature point set and an oral cavity scanning model high curvature point set respectively;
S104, calculating characteristic column vectors of each high-curvature triangular mesh vertex in the CBCT high-curvature point set and the oral cavity scanning model high-curvature point set, splicing the characteristic column vectors of each high-curvature triangular mesh vertex to respectively obtain a CBCT characteristic description matrix and an oral cavity scanning model characteristic description matrix, constructing a vector similarity matrix according to the CBCT characteristic description matrix and the oral cavity scanning model characteristic description matrix, and screening similar points in the vector similarity matrix to obtain a corresponding point set;
and S105, calculating by using the corresponding point set to obtain a rigid transformation matrix, transforming the three-dimensional tooth volume data by using the rigid transformation matrix to obtain rigid transformation volume data, and rendering the rigid transformation volume data and the dental crown surface data of the oral scan model.
It should be noted that, specific examples in this embodiment may refer to examples described in the foregoing embodiments and alternative implementations, and this embodiment is not repeated herein.
In general, the various embodiments may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. While various aspects of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.
Embodiments of the invention may be implemented by computer software executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets, and/or macros can be stored in any apparatus-readable data storage medium and they include program instructions for performing particular tasks. The computer program product may include one or more computer-executable components configured to perform embodiments when the program is run. The one or more computer-executable components may be at least one software code or a portion thereof. In this regard, it should also be noted that any block of the logic flow as in fig. 8 may represent a program step, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on a physical medium such as a memory chip or memory block implemented within a processor, a magnetic medium such as a hard disk or floppy disk, and an optical medium such as, for example, a DVD and its data variants, a CD, etc. The physical medium is a non-transitory medium.
It should be understood by those skilled in the art that the technical features of the above embodiments may be combined in any manner, and for brevity, all of the possible combinations of the technical features of the above embodiments are not described, however, they should be considered as being within the scope of the description provided herein, as long as there is no contradiction between the combinations of the technical features.
The foregoing examples merely represent several embodiments of the present application, the description of which is more specific and detailed and which should not be construed as limiting the scope of the present application in any way. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.
Claims (10)
1. The multi-mode rendering method based on the three-dimensional tooth CBCT data and the oral cavity scanning model is characterized by comprising the following steps of:
acquiring CBCT data, inputting the CBCT data into a trained tooth segmentation model for segmentation to obtain three-dimensional tooth volume data, extracting the tooth surface of the three-dimensional tooth volume data to obtain tooth surface data, and removing the tooth root part of each tooth in the tooth surface data to obtain CBCT crown surface data, wherein the CBCT crown surface data is stored in a triangular mesh form;
Acquiring an oral cavity scanning model corresponding to CBCT data, and removing gingiva and other soft tissue areas in the oral cavity scanning model to obtain dental crown surface data of the oral cavity scanning model, wherein the dental crown surface data of the oral cavity scanning model is stored in a triangular mesh form;
acquiring triangular mesh vertexes of each triangular mesh which is not at a boundary in the CBCT dental crown surface data and the oral cavity scanning model dental crown surface data, calculating a normal vector of each triangular mesh vertex based on a face averaging algorithm, acquiring an area sum of adjacent triangular meshes of each triangular mesh vertex and an angle cosine square sum of the adjacent triangular mesh and the normal vector of the corresponding triangular mesh vertex, calculating the curvature of each triangular mesh vertex according to the angle cosine square sum and the area sum of the adjacent triangular mesh of each triangular mesh vertex, setting a curvature threshold value, and screening the curvature of each triangular mesh vertex to obtain a CBCT high curvature point set and an oral cavity scanning model high curvature point set respectively;
calculating characteristic column vectors of each high-curvature triangle grid vertex in the CBCT high-curvature point set and the oral cavity scanning model high-curvature point set, splicing the characteristic column vectors of each high-curvature triangle grid vertex to respectively obtain a CBCT characteristic description matrix and an oral cavity scanning model characteristic description matrix, constructing a vector similarity matrix according to the CBCT characteristic description matrix and the oral cavity scanning model characteristic description matrix, and screening similar points in the vector similarity matrix to obtain a corresponding point set;
And calculating by using the corresponding point set to obtain a rigid transformation matrix, transforming the three-dimensional tooth volume data by using the rigid transformation matrix to obtain rigid transformation volume data, and rendering the rigid transformation volume data and the dental crown surface data of the oral scanning model.
2. The multi-modal rendering method based on three-dimensional tooth CBCT data and an oral cavity scan model according to claim 1, wherein in the step of removing root portions of each tooth in the tooth surface data to obtain CBCT crown surface data, the tooth surface data is composed of a plurality of triangular meshes, normal vectors of each triangular mesh in the tooth surface data are calculated, normal vector angles of each triangular mesh and all neighboring triangular meshes are calculated, an angle threshold is set, and if at least one normal vector angle is greater than the angle threshold, the triangular mesh and all neighboring triangular meshes are subdivided until all normal vector angles are smaller than the angle threshold to obtain CBCT crown surface data.
3. The multi-mode rendering method based on three-dimensional tooth CBCT data and an oral cavity scanning model according to claim 1, wherein in the step of removing gingiva and other soft tissue areas in the oral cavity scanning model to obtain dental crown surface data of the oral cavity scanning model, a first number of triangular mesh vertexes are randomly selected from the dental crown surface of each tooth in the oral cavity scanning model as seed points, adjacent triangular meshes of each seed point are obtained, a similarity included angle is obtained by calculating a normal vector of each seed point and normal vector included angles of all adjacent meshes, a growth threshold is set, and if the similarity included angle of any two seed points is larger than the growth threshold, the two seed points are combined until no seed points which can be combined exist in the oral cavity scanning model or the specified combination times are reached, and dental crown surface data of the oral cavity scanning model is obtained according to a seed point combination path.
4. The multi-modal rendering method based on three-dimensional tooth CBCT data and an oral scan model according to claim 1, wherein in the step of screening the curvature of each triangular mesh vertex to obtain a CBCT high curvature point set and an oral scan model high curvature point set respectively by setting a curvature threshold, triangular mesh vertices with the triangular mesh vertex curvature greater than the curvature threshold in CBCT crown surface data are selected as the CBCT high curvature point set, and triangular mesh vertices with the triangular mesh vertex curvature greater than the curvature threshold in the oral scan model crown surface data are selected as the oral scan model high curvature point set.
5. The multi-modal rendering method based on three-dimensional tooth CBCT data and an oral scan model according to claim 1, wherein in the step of calculating the feature column vector of each high curvature triangle mesh vertex in the CBCT high curvature point set and the oral scan model high curvature point set, all triangle mesh normal vectors in each high curvature triangle mesh vertex neighborhood loop are obtained, normal vector of each high curvature triangle mesh vertex is obtained, included angles between the normal vector of each high curvature triangle mesh vertex and all triangle mesh normal vectors in the corresponding high curvature triangle mesh vertex neighborhood loop are calculated, all triangle meshes in the neighborhood loop include the neighborhood triangle mesh of each high curvature triangle mesh vertex field triangle mesh, and the feature column vector of each high curvature triangle mesh vertex is generated according to the included angles between the normal vector of each high curvature triangle mesh vertex and all triangle mesh normal vectors in the corresponding high curvature triangle mesh vertex neighborhood loop.
6. The multi-modal rendering method based on three-dimensional tooth CBCT data and an oral cavity scan model according to claim 1, wherein in the step of constructing a vector similarity matrix according to the CBCT feature description matrix and the oral cavity scan model feature description matrix and screening similar points in the vector similarity matrix to obtain a corresponding point set, calculating vector similarity of the CBCT feature description matrix and the oral cavity scan model feature description matrix to obtain a vector similarity matrix, wherein the vector similarity matrix comprises similarity of corresponding points in the CBCT feature description matrix and the oral cavity scan model feature description matrix, setting a similarity threshold, screening the vector similarity matrix by using the similarity threshold to obtain point pair information, and removing incorrect point pairs in the point pair information to obtain a corresponding point set.
7. The multi-modal rendering method based on three-dimensional tooth CBCT data and an oral scan model according to claim 1, wherein in the step of transforming the three-dimensional tooth volume data using the rigid transformation matrix to obtain rigid transformation volume data and rendering the rigid transformation volume data and the oral scan model crown surface data, the three-dimensional tooth volume data is subjected to homogeneous coordinate transformation to obtain a CBCT homogeneous coordinate matrix, the CBCT homogeneous coordinate matrix is transformed into cartesian coordinates by the rigid transformation matrix to obtain the rigid transformation volume data, and the oral scan model crown surface data and the rigid transformation volume data are rendered under a world coordinate system.
8. A multi-modal rendering device based on three-dimensional tooth CBCT data and an oral scan model, comprising:
a first acquisition module: acquiring CBCT data, inputting the CBCT data into a trained tooth segmentation model for segmentation to obtain three-dimensional tooth volume data, extracting the tooth surface of the three-dimensional tooth volume data to obtain tooth surface data, and removing the tooth root part of each tooth in the tooth surface data to obtain CBCT crown surface data, wherein the CBCT crown surface data is stored in a triangular mesh form;
and a second acquisition module: acquiring an oral cavity scanning model corresponding to CBCT data, and removing gingiva and other soft tissue areas in the oral cavity scanning model to obtain dental crown surface data of the oral cavity scanning model, wherein the dental crown surface data of the oral cavity scanning model is stored in a triangular mesh form;
the curvature calculation module: acquiring triangular mesh vertexes of each triangular mesh which is not at a boundary in the CBCT dental crown surface data and the oral cavity scanning model dental crown surface data, calculating a normal vector of each triangular mesh vertex based on a face averaging algorithm, acquiring an area sum of adjacent triangular meshes of each triangular mesh vertex and an angle cosine square sum of the adjacent triangular mesh and the normal vector of the corresponding triangular mesh vertex, calculating the curvature of each triangular mesh vertex according to the angle cosine square sum and the area sum of the adjacent triangular mesh of each triangular mesh vertex, setting a curvature threshold value, and screening the curvature of each triangular mesh vertex to obtain a CBCT high curvature point set and an oral cavity scanning model high curvature point set respectively;
The point set acquisition module: calculating characteristic column vectors of each high-curvature triangle grid vertex in the CBCT high-curvature point set and the oral cavity scanning model high-curvature point set, splicing the characteristic column vectors of each high-curvature triangle grid vertex to respectively obtain a CBCT characteristic description matrix and an oral cavity scanning model characteristic description matrix, constructing a vector similarity matrix according to the CBCT characteristic description matrix and the oral cavity scanning model characteristic description matrix, and screening similar points in the vector similarity matrix to obtain a corresponding point set;
and a rendering module: and calculating by using the corresponding point set to obtain a rigid transformation matrix, transforming the three-dimensional tooth volume data by using the rigid transformation matrix to obtain rigid transformation volume data, and rendering the rigid transformation volume data and the dental crown surface data of the oral scanning model.
9. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, the processor being arranged to run the computer program to perform a multi-modal rendering method based on three-dimensional dental CBCT data and an oral scan model as claimed in any of claims 1 to 7.
10. A readable storage medium, characterized in that the readable storage medium has stored therein a computer program comprising program code for controlling a process to perform a process comprising a multi-modal rendering method based on three-dimensional tooth CBCT data and an oral scan model according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310430198.6A CN116524118B (en) | 2023-04-17 | 2023-04-17 | Multi-mode rendering method based on three-dimensional tooth CBCT data and oral cavity scanning model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310430198.6A CN116524118B (en) | 2023-04-17 | 2023-04-17 | Multi-mode rendering method based on three-dimensional tooth CBCT data and oral cavity scanning model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116524118A CN116524118A (en) | 2023-08-01 |
| CN116524118B true CN116524118B (en) | 2023-12-22 |
Family
ID=87400451
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310430198.6A Active CN116524118B (en) | 2023-04-17 | 2023-04-17 | Multi-mode rendering method based on three-dimensional tooth CBCT data and oral cavity scanning model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116524118B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105447908A (en) * | 2015-12-04 | 2016-03-30 | 山东山大华天软件有限公司 | Dentition model generation method based on oral cavity scanning data and CBCT (Cone Beam Computed Tomography) data |
| WO2020218560A1 (en) * | 2019-04-26 | 2020-10-29 | 株式会社カイ | Tooth position analysis device, tooth region extraction model generation method, tooth position analysis method, program, and recording medium |
| WO2021238673A1 (en) * | 2020-05-29 | 2021-12-02 | 先临三维科技股份有限公司 | Tooth scanning method and apparatus, computer device, and computer-readable storage medium |
-
2023
- 2023-04-17 CN CN202310430198.6A patent/CN116524118B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105447908A (en) * | 2015-12-04 | 2016-03-30 | 山东山大华天软件有限公司 | Dentition model generation method based on oral cavity scanning data and CBCT (Cone Beam Computed Tomography) data |
| WO2020218560A1 (en) * | 2019-04-26 | 2020-10-29 | 株式会社カイ | Tooth position analysis device, tooth region extraction model generation method, tooth position analysis method, program, and recording medium |
| WO2021238673A1 (en) * | 2020-05-29 | 2021-12-02 | 先临三维科技股份有限公司 | Tooth scanning method and apparatus, computer device, and computer-readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116524118A (en) | 2023-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7168644B2 (en) | Select and lock intraoral images | |
| US11017535B2 (en) | Method and system for hybrid mesh segmentation | |
| EP3490489B1 (en) | Method and system for removing orthodontic brackets from the digital mesh of the dentition | |
| EP2011088B1 (en) | Contour triangulation system and method | |
| Kainmueller et al. | Automatic extraction of mandibular nerve and bone from cone-beam CT data | |
| EP3346943A1 (en) | Method and system for hybrid mesh segmentation | |
| US11704819B2 (en) | Apparatus and method for aligning 3-dimensional data | |
| Chougule et al. | Clinical case study: spine modeling for minimum invasive spine surgeries (MISS) using rapid prototyping | |
| CN115619773B (en) | Three-dimensional tooth multi-mode data registration method and system | |
| CN116205956A (en) | Point cloud registration method and device and medical imaging equipment | |
| CN109598703B (en) | Method, system, computer-readable storage medium and device for processing dental image | |
| Yau et al. | An adaptive region growing method to segment inferior alveolar nerve canal from 3D medical images for dental implant surgery | |
| Nassef et al. | Extraction of human mandible bones from multi-slice computed tomographic data | |
| CN116524118B (en) | Multi-mode rendering method based on three-dimensional tooth CBCT data and oral cavity scanning model | |
| Qian et al. | An automatic tooth reconstruction method based on multimodal data | |
| CN115409811A (en) | Tooth model reconstruction method, device, equipment and medium based on curvature enhancement | |
| CN114299096A (en) | Outline delineation method, device, equipment and storage medium | |
| EP4365855A1 (en) | Method, system, and computer program for generating 3d models of dentures in occlusion conditions | |
| CN113140016B (en) | Metal artifact correction method and system for CBCT equipment | |
| WO2024097286A1 (en) | Method, system, and computer program for generating 3d models of dentures in occlusion conditions | |
| Barone et al. | Three-dimensional tooth segmentation by integrating multiple ray-sum images from CBCT data | |
| Soltaninejad et al. | Automatic crown surface reconstruction using tooth statistical model for dental prosthesis planning | |
| Harris et al. | Automatic Pulp and Teeth Three-Dimensional Modeling of Single and Multi-Rooted Teeth Based on Cone-Beam Computed Tomography Imaging: A Promising Approach With Clinical and Therapeutic Outcomes | |
| CN116563455A (en) | Tooth three-dimensional reconstruction method based on panoramic X-ray film and mouth scanning data | |
| Li et al. | A Cloud Collaborative Healthcare Platform Based on Deep Learning in The Segmentation of Maxillary Sinus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240412 Address after: 310000 room 1911, 19 / F, Huachuang building, No. 511, Jianye Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province Patentee after: Hangzhou Fengya Medical Technology Co.,Ltd. Country or region after: China Address before: 310000 room 1910, 19th floor, Huachuang building, 511 Jianye Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province Patentee before: HANGZHOU YAZHI MEDICAL TECHNOLOGY Co.,Ltd. Country or region before: China |
|
| TR01 | Transfer of patent right |