CN114224528B - Oral cavity digital twin model system based on virtual reality interaction and establishment method - Google Patents
Oral cavity digital twin model system based on virtual reality interaction and establishment method Download PDFInfo
- Publication number
- CN114224528B CN114224528B CN202111244755.2A CN202111244755A CN114224528B CN 114224528 B CN114224528 B CN 114224528B CN 202111244755 A CN202111244755 A CN 202111244755A CN 114224528 B CN114224528 B CN 114224528B
- Authority
- CN
- China
- Prior art keywords
- data
- virtual reality
- module
- information
- digital twin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 53
- 238000000034 method Methods 0.000 title claims abstract description 49
- 210000000214 mouth Anatomy 0.000 title claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 122
- 230000001815 facial effect Effects 0.000 claims abstract description 33
- 108090000623 proteins and genes Proteins 0.000 claims abstract description 30
- 210000003296 saliva Anatomy 0.000 claims abstract description 30
- 238000004458 analytical method Methods 0.000 claims abstract description 26
- 239000008280 blood Substances 0.000 claims abstract description 23
- 210000004369 blood Anatomy 0.000 claims abstract description 23
- 210000001519 tissue Anatomy 0.000 claims abstract description 18
- 238000004088 simulation Methods 0.000 claims abstract description 17
- 230000003190 augmentative effect Effects 0.000 claims abstract description 10
- 230000029058 respiratory gaseous exchange Effects 0.000 claims abstract description 10
- 210000000515 tooth Anatomy 0.000 claims description 58
- 230000008859 change Effects 0.000 claims description 40
- 238000011282 treatment Methods 0.000 claims description 32
- 230000008569 process Effects 0.000 claims description 30
- 238000012937 correction Methods 0.000 claims description 19
- 201000010099 disease Diseases 0.000 claims description 17
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 17
- 230000033001 locomotion Effects 0.000 claims description 15
- 102000004169 proteins and genes Human genes 0.000 claims description 14
- 238000011161 development Methods 0.000 claims description 13
- 230000018109 developmental process Effects 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 13
- 230000000694 effects Effects 0.000 claims description 12
- 210000000988 bone and bone Anatomy 0.000 claims description 11
- 230000000241 respiratory effect Effects 0.000 claims description 11
- 238000010521 absorption reaction Methods 0.000 claims description 8
- 238000012216 screening Methods 0.000 claims description 8
- 238000005516 engineering process Methods 0.000 claims description 7
- 238000009434 installation Methods 0.000 claims description 7
- 230000009885 systemic effect Effects 0.000 claims description 7
- 238000013461 design Methods 0.000 claims description 6
- 230000008449 language Effects 0.000 claims description 6
- 230000003068 static effect Effects 0.000 claims description 6
- 210000003781 tooth socket Anatomy 0.000 claims description 6
- 241001672694 Citrus reticulata Species 0.000 claims description 5
- 210000002379 periodontal ligament Anatomy 0.000 claims description 5
- 230000028327 secretion Effects 0.000 claims description 5
- 210000004513 dentition Anatomy 0.000 claims description 4
- 238000002474 experimental method Methods 0.000 claims description 4
- 230000008921 facial expression Effects 0.000 claims description 4
- 238000003709 image segmentation Methods 0.000 claims description 4
- 230000006872 improvement Effects 0.000 claims description 4
- 230000036346 tooth eruption Effects 0.000 claims description 4
- 208000031220 Hemophilia Diseases 0.000 claims description 3
- 208000009292 Hemophilia A Diseases 0.000 claims description 3
- 210000000845 cartilage Anatomy 0.000 claims description 3
- 238000001514 detection method Methods 0.000 claims description 3
- 206010012601 diabetes mellitus Diseases 0.000 claims description 3
- 206010015037 epilepsy Diseases 0.000 claims description 3
- 208000019622 heart disease Diseases 0.000 claims description 3
- 208000006454 hepatitis Diseases 0.000 claims description 3
- 231100000283 hepatitis Toxicity 0.000 claims description 3
- 239000007943 implant Substances 0.000 claims description 3
- 238000002513 implantation Methods 0.000 claims description 3
- 208000007442 rickets Diseases 0.000 claims description 3
- 210000004872 soft tissue Anatomy 0.000 claims description 3
- 210000001738 temporomandibular joint Anatomy 0.000 claims description 3
- 201000008827 tuberculosis Diseases 0.000 claims description 3
- 201000008383 nephritis Diseases 0.000 claims description 2
- 238000012549 training Methods 0.000 abstract description 8
- 238000002560 therapeutic procedure Methods 0.000 abstract 1
- 238000011160 research Methods 0.000 description 11
- 238000013473 artificial intelligence Methods 0.000 description 7
- 230000036541 health Effects 0.000 description 4
- 210000001847 jaw Anatomy 0.000 description 4
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000010438 heat treatment Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000000338 in vitro Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 239000012188 paraffin wax Substances 0.000 description 2
- 201000002859 sleep apnea Diseases 0.000 description 2
- 239000001993 wax Substances 0.000 description 2
- 206010009260 Cleft lip and palate Diseases 0.000 description 1
- 206010049119 Emotional distress Diseases 0.000 description 1
- 206010061274 Malocclusion Diseases 0.000 description 1
- 206010029164 Nephrotic syndrome Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007321 biological mechanism Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000001055 chewing effect Effects 0.000 description 1
- 208000016653 cleft lip/palate Diseases 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000002591 computed tomography Methods 0.000 description 1
- 210000002455 dental arch Anatomy 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000009429 distress Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000005802 health problem Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 208000009928 nephrosis Diseases 0.000 description 1
- 231100001027 nephrosis Toxicity 0.000 description 1
- 238000011369 optimal treatment Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 210000003254 palate Anatomy 0.000 description 1
- 230000003239 periodontal effect Effects 0.000 description 1
- 239000011505 plaster Substances 0.000 description 1
- 238000004393 prognosis Methods 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 238000000611 regression analysis Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000035882 stress Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61C—DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE
- A61C7/00—Orthodontics, i.e. obtaining or maintaining the desired position of teeth, e.g. by straightening, evening, regulating, separating, or by correcting malocclusions
- A61C7/002—Orthodontic computer assisted systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/50—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
- A61B6/51—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications for dentistry
- A61B6/512—Intraoral means
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
- A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Animal Behavior & Ethology (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- High Energy & Nuclear Physics (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Optics & Photonics (AREA)
- Physics & Mathematics (AREA)
- Dentistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Epidemiology (AREA)
- Dental Tools And Instruments Or Auxiliary Dental Instruments (AREA)
Abstract
The invention belongs to the field of oral cavity digital models based on virtual reality interaction, and particularly discloses an oral cavity digital twin model system based on virtual reality interaction and an establishment method thereof, wherein the oral cavity digital twin model system comprises a data acquisition module, a data processing module, a mechanical analysis module, a virtual reality interaction module, an augmented reality interaction module, a force feedback module, a digital twin module and an output module, wherein the data acquisition module acquires image data of soft and hard tissues of a jaw face of a patient, tooth scanning, facial images, acquisition of breathing, acquisition of pronunciation, acquisition of saliva, acquisition of blood and acquisition of genes; the method comprises the steps of collecting the soft and hard tissue image data of the jaw face of a patient, including collecting the data of the jaw bone, the face, the neck of the patient by CT, and the like. The invention can realize the simulation of the orthodontic virtual reality and the non-orthodontic virtual reality through virtual reality interaction, and can simulate mechanical analysis, thereby enabling the system to achieve the purpose of improving rehabilitation therapy and operation clinical skill training.
Description
Technical Field
The invention relates to the field of oral cavity digital models based on virtual reality interaction, in particular to an oral cavity digital twin model system based on virtual reality interaction and an establishment method.
Background
With the economic development and the continuous improvement of the living standard of people, the medical and health problems become the focus of social attention. In recent years, digital twinning, internet of things, big data and other information technologies are gradually applied to the medical industry, and traditional medical treatment is being transformed into digital medical treatment and intelligent medical treatment. The intelligent medical treatment is to combine the traditional medical treatment through the Internet, and informationize the medical data by applying related technologies such as big data and Internet of things, so as to realize information sharing among patients, medical staff and medical equipment and construct an informationized medical service system.
As the living standard of people increases, more patients are selecting orthodontic treatment. The accurate measurement of orthodontic force and moment of an appliance on teeth in an orthodontic process is a key for analyzing biological mechanism in an orthodontic treatment process, visual observation and prediction of different orthodontic force loading modes are of great importance to the final face shape and tooth arrangement effect, and the orthodontic treatment device has the advantages of improving orthodontic treatment effect, shortening treatment period and training operation skills of clinicians.
Currently, prior to orthodontic treatment, doctors also show and observe the movement of teeth in paraffin by using a Typodont model (wax-shaped dental model). On a Typodont simulated jaw frame, relevant analysis of crowding degree, bolton index, spee curvature and the like can be carried out on a model, so that a proper orthodontic treatment scheme is formulated, judgment of tooth extraction correction or non-tooth extraction correction is made, and technology used in the whole correction process and archwires required in each stage are formulated. The model can also be used for training and learning of bracket, band pasting, placing and fixing ligature archwires and the like. The Typodant orthodontic model is immersed in the water bath box, so that the whole clinical correction process can be concentrated, the movement of teeth along with the deformation of the archwire can be clearly observed in a short time, and the movement plays a very important role in the correction process of a simulated patient, so that some problems of clinical correction are found. The Typodont model is a method capable of truly simulating orthodontic diagnosis, design and treatment in vitro, and can truly simulate various vertical, sagittal or transverse malocclusions in vitro, and the archwire is bent and placed by bonding and fixing the appliance, so that the effect of different appliance forces on teeth is visually observed, and the movement process of the teeth for 2-3 years is completed in a short time. Typodont is an important auxiliary tool for orthodontic study, and is an internationally recognized optimal means for orthodontic teaching training. More students use different types of Typodont to simulate and study orthodontic tooth movement, such as the simulation of overall receding of anterior teeth, the study of the friction force between a bracket groove and an arch wire, the study of lingual correction and the like, and satisfactory research results are obtained. Compared with human body experiments, the initial conditions of scientific research and research performed by adopting the Typodont model to simulate the correction process are strictly controlled, so that the comparability among research groups is greatly improved, the interference caused by individual differences such as age, sex, bad habit, oral hygiene, organism reaction and the like in clinical research is made up, and the distribution of a force system on a simulated dental arch and the trend of tooth movement are better shown.
Conventional Typodont devices utilize water bath heating to soften the wax to simulate orthodontic tooth movement. 1. The temperature of the water bath is about 45-50 ℃, and the Typodont device is heated from the outside by utilizing the water bath, so that the actual movement of teeth in the oral cavity can not be well simulated. Previously, the improved Typodont has been reported by the scholars, and the method is characterized in that an instrument is connected with a metal tooth root through a connecting wire so as to control the heating of the tooth root, so that the heating mode is changed from outside to inside, but the actual use of the type of Typodont is relatively inconvenient. However, the wax dike in the Typodont model has no real periodontal tissue, is greatly different from the real biological environment in the oral cavity, and lacks simulation of chewing and biting pressure in the Typodont model experiment compared with the real conditions in the oral cavity. Therefore, the development of a model capable of replacing Typodont has important clinical significance, wherein the model of digital twin orthodontic Typodont has important research potential.
Digital twinning (Digital Twin) is the creation of a dynamic virtual model of physical entities in a Digital manner, with multidimensional, multi-space-time scale, multidisciplinary, multi-physical quantity, to simulate and characterize physical entities' properties, behaviors, rules, etc. in a real environment. The concept of digital twinning was originally proposed in 2003 by grives teachings on the american michigan university product lifecycle management program, and was primarily applied in the military and aerospace fields early on. Such as the united states air force research laboratory, the united states national aerospace agency (NASA) developed aircraft health management applications based on digital twinning. Because digital twinning has the characteristics of virtual-real fusion and real-time interaction, iterative operation and optimization, full factor/full flow/full service data driving and the like, the digital twinning is applied to various stages of a product life cycle, including product design, manufacturing, service, operation and maintenance and the like. Digital twinning is widely applied in manufacturing industry, is applied to aerospace at the earliest, is widely applied in different research directions along with deep research, and has reports and practical cases in the fields of electric power, communication, automobile manufacturing and the like.
In recent years, the digital twin research field is gradually expanded to the medical industry, tao Fei et al propose the concept of digital twin medical treatment, apply the digital twin five-dimensional model to a medical health system, realize the functions of prognosis, remote treatment and the like of human diseases, perform virtual human training and training of medical staff, and effectively change the current state of medical health; methods such as artificial intelligence, machine learning, big data and the like have been widely used in the medical field and even in the orthodontic field. For example, the artificial intelligence is applied to a head measurement analysis system, and the artificial intelligence is used for automatic identification and segmentation of teeth in CT scan data and research of predicting orthodontic effect of the artificial intelligence. However, such studies can only digitally predict and analyze the orthodontic effect of teeth and cannot be combined with actual plaster or paraffin models.
In recent years, virtual reality technology and augmented reality technology are rapidly developed, and gradually blend into the field of medical health. Virtual reality is to present the virtual world to you before you can make you appear to be real. Augmented reality is to superimpose a virtual world onto the real world.
In the traditional three-dimensional finite element model analysis, only the stress of a certain part of the tooth after orthodontic force loading can be analyzed, and the movement effect of the tooth can not be well simulated. The method of finite element analysis is one of the important methods of creating digital twin models. Therefore, the digital twin model created based on finite element analysis, big data, virtual reality, augmented reality and artificial intelligence analysis has the potential to replace the traditional Typodont device.
Disclosure of Invention
The invention aims to provide an oral cavity digital twin model system based on virtual reality interaction and a cube building method, so as to solve the problems in the background technology.
In order to achieve the above purpose, the present invention provides the following technical solutions: the oral cavity digital twin model system based on virtual reality interaction comprises a data acquisition module, a data processing module and a processing module, wherein the data acquisition module is used for acquiring dental jaw face neck soft and hard tissue image data, tooth scanning, facial image data, breathing data, pronunciation data, saliva data, blood data, gene data and whole body disease data of a patient, and the facial image comprises a two-dimensional facial photograph, three-dimensional, static and dynamic images; the data processing module is used for processing the acquired images and data, wherein the processing process comprises image segmentation, face extraction, respiratory data processing, pronunciation analysis processing, saliva processing and systemic disease data processing, and the data processing module comprises a dento-maxillofacial neck data processing unit, a face image processing unit, a respiratory data processing unit, a pronunciation data processing unit, a saliva data processing unit, a blood data processing unit and a gene data processing unit; the virtual reality interaction module is used for simulating tooth extraction orthodontic and non-tooth extraction orthodontic processes, acquires segmented dentognathic surface neck information and generates an interaction model, and the tooth interaction model is provided with a bracket, an installation arch wire, an invisible tooth socket design installation, accessory bonding and implant anchorage implantation; the mechanical analysis module is used for carrying out three-dimensional finite element analysis processing on the segmented tooth structure and generating a three-dimensional finite element model, wherein the tooth three-dimensional model comprises teeth, periodontal ligament and alveolar bone parts; the force feedback model module is used for simulating the pressure sense and the touch sense in the real operation, and the hand feeling and the touch sense comprise the holding sense and the touch sense of a needle holder, forceps, ligature wires, rubber rings, push springs, tension springs, supporting nails and orthodontic pliers; the digital twin module is used for integrating the information of the bracket, the arch wire, the invisible tooth socket, the accessory information, the mechanical analysis and the like installed in the data acquisition information, the processing information and the virtual reality interaction module uniformly, and constructing a digital twin model and carrying out operation processing; the output module is used for outputting the soft and hard tissue change condition after the tooth correction (including the oral orthodontic fixing appliance, the invisible appliance and the accessory loading), predicting the development change, prompting the treatment risk and needing to improve the treatment point.
Preferably, the collecting process of the data collecting module comprises the following steps: acquiring data of jawbone, face and neck of a patient based on CT, and scanning intraoral tooth data by an intraoral scanner and facial soft tissue contours by a facial scanner; based on the camera, two-dimensional and three-dimensional face photos and videos, including static and dynamic face contours and surface condition changes; collecting pronunciation characteristics, frequency and depth of breath based on a microphone sensor; collecting pronunciation characteristics of a section of sentences of a patient, including a seesaw sound, a flat tongue sound, a vowel sound and a nasal sound, based on a microphone, wherein the sentences include but are not limited to mandarin, dialect and foreign language; detecting the secretion amount, viscosity and secreted protein molecular components and content of saliva based on a saliva detector; collecting protein molecular components and contents in blood based on a blood collection analyzer; analyzing and screening root absorption and alveolar bone wall genes based on an oral cavity detection process; screening whether the patients have epilepsy, rheumatism, hepatitis, nephritis, tuberculosis, diabetes, heart disease, hemophilia and rickets based on the whole body disease collecting process.
Preferably, the image segmentation in the processing process of the data processing module comprises tooth separation, periodontal ligament, jawbone, tooth position and crowding degree, upper and lower jaw dentition relation, tooth midline position and airway width; the processing process of the facial image processing unit comprises extraction of facial contours and expression features, nose-lip distance and facial midline position; the processing process of the breathing data processing unit comprises the steps of analyzing the pronunciation characteristics, frequency and depth of breathing, and judging whether an apnea syndrome exists or not; the processing procedure of the pronunciation data processing unit comprises extracting and analyzing the characteristic points of the seesaw sound, the flat tongue sound, the vowel and the nasal sound, the characteristic points of mandarin, dialect and foreign language voice, and the characteristic points of cleft lip and palate voice, and carrying out pronunciation data analysis and processing; the saliva data processing unit is used for processing saliva data, wherein the saliva data processing unit comprises a saliva secretion amount, viscosity, and secreted protein molecule composition and content; the blood data processing unit comprises a processing process for analyzing the protein molecular components and the protein molecular content in blood; the processing procedure based on the data processing unit comprises screening and analyzing genes which are easy to cause root absorption and alveolar bone wall; the processing process of the whole body disease data comprises analyzing the disease of the whole body acquired image orthodontic treatment effect and evaluating whether orthodontic treatment can be carried out.
Preferably, the digital twin module is used for integrating the acquired information uniformly based on artificial intelligence and/or virtual reality and/or augmented reality and/or big data, outputting the integrated information to Matlab, constructing a digital twin model by the Simulink module, and carrying out operation processing.
Preferably, the soft and hard tissue change condition after the tooth correction output by the output module comprises arrangement change, facial expression change, breathing voice change, prediction development change, early warning of tooth root, alveolar bone wall and temporomandibular joint cartilage absorption risk, protein molecule change in saliva and blood, and the output module outputs treatment points needing improvement, including bonding positions of brackets, thickness and shape change of archwires, and mechanical change of invisible dental shell design, invisible dental shell accessory installation and invisible dental shell.
Preferably, the virtual reality interaction module comprises a virtual reality simulation unit and a data processing unit, wherein the data processing unit acquires the information of the dento-maxillofacial neck and preprocesses the data information, and the virtual reality simulation unit performs orthodontic virtual reality and non-orthodontic virtual reality simulation based on the information.
Preferably, the force feedback model module comprises a force feedback unit and a processing unit, wherein the force feedback unit is used for calculating force data pre-applied by the hand of the user according to the operation action information of the hand of the user, simulating real feedback to the hand of the user according to the force data, feeding back the operation action information to the processing unit, and analyzing and processing the action data by the processing unit.
The invention also provides a method for establishing the digital twin model of the oral cavity, the dental jaw face and the neck, which comprises the following steps:
s1: collecting dental maxillofacial neck soft and hard tissue image data, tooth scanning, facial image data, respiratory data, pronunciation data, saliva data, blood data, gene data and systemic disease data of a patient;
s2: processing the acquired image and data;
s3: generating a finite element model and establishing a virtual reality interaction model;
s4: uniformly integrating data acquisition information, processing information, virtual reality interaction information and mechanical analysis information, and constructing a digital twin model and operation processing;
s5: outputting the change condition of the soft and hard tissues after the tooth correction, predicting the development change, treating the risk presentation and the treatment point needing to be improved, and establishing a personalized virtual reality interaction model based on the facial change, the correction effect, the development change and the like.
Compared with the prior art, the invention has the beneficial effects that:
1. the system model fully considers the variability of three-dimensional model data, biochemical data, language, respiration and facial expression changes of different individuals, integrates the advantages of various modes, and can improve the operation level of clinicians, screen the optimal treatment scheme and visualize the effect of the distress treatment.
2. The system model is also integrated with a virtual reality interaction module and a force feedback model module, the simulation of the orthodontic virtual reality and the non-orthodontic virtual reality can be realized through the virtual reality interaction module, and the force feedback model module can simulate and carry out mechanical analysis, so that the system can achieve the aim of improving rehabilitation and clinical skill training of operations.
Drawings
FIG. 1 is a block diagram of a system module of the present invention;
fig. 2 is a flow chart of a system setup implementation of the present invention.
In the figure: 1. a data acquisition module; 2. a data processing module; 3. a virtual reality interaction module; 4. a mechanical analysis module; 5. a force feedback model module; 6. a digital twinning module; 7. and an output module.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without creative efforts, are within the protection scope of the invention.
Specific examples:
referring to fig. 1-2, the present invention provides a technical solution: an oral digital twin model system based on virtual reality interactions, comprising:
the data acquisition module 1 is used for acquiring dental maxillofacial neck soft and hard tissue image data, dental scan, facial image data, respiratory data, pronunciation data, saliva data, blood data, gene data and systemic disease data of a patient, and the facial image comprises a two-dimensional facial photo, a three-dimensional, static and dynamic image;
the data processing module 2 is used for processing the acquired images and data, wherein the processing process comprises image segmentation, face extraction, respiratory data processing, pronunciation analysis processing, saliva processing and whole body disease data processing, and the data processing module 2 comprises a dento-maxillofacial neck data processing unit, a face image processing unit, a respiratory data processing unit, a pronunciation data processing unit, a saliva data processing unit, a blood data processing unit and a gene data processing unit;
the virtual reality interaction module 3 is used for simulating tooth extraction orthodontic and non-tooth extraction orthodontic processes, the virtual reality interaction module 3 obtains segmented dentognathic face neck information and generates an interaction model, and the tooth interaction model is provided with a bracket, an installation arch wire, an invisible tooth socket design installation, accessory bonding and implant anchorage implantation;
the mechanical analysis module 4 is used for carrying out three-dimensional finite element analysis processing on the segmented tooth structure and generating a three-dimensional finite element model, wherein the tooth three-dimensional model comprises teeth, periodontal ligament and alveolar bone parts;
the force feedback model module 5 is used for simulating the pressure feeling and the touch feeling in the real operation, and the touch feeling comprise the holding feeling and the touch feeling of a needle holder, forceps, ligature wires, rubber rings, push springs, tension springs, supporting nails and orthodontic pliers;
the digital twin module 6 is used for integrating the data acquisition information, the processing information, the bracket, the arch wire, the invisible tooth socket, the accessory information, the mechanical analysis and other information installed in the virtual reality interaction module 3 uniformly, and constructing a digital twin model and carrying out operation processing;
and the output module 7 is used for outputting the change condition of the soft and hard tissues after the tooth correction, predicting the development change, prompting the treatment risk and requiring an improved treatment point.
In this embodiment, the acquisition process of the data acquisition module 1 includes: acquiring data of jawbone, face and neck of a patient based on CT, and scanning intraoral tooth data by an intraoral scanner and facial soft tissue contours by a facial scanner; taking two-dimensional and three-dimensional facial photos and videos based on cameras, wherein the two-dimensional and three-dimensional facial photos comprise static and dynamic facial contours and expression changes; collecting pronunciation characteristics, frequency and depth of breath based on a microphone sensor; collecting pronunciation characteristics of a section of sentences of a patient, including a seesaw sound, a flat tongue sound, a vowel sound and a nasal sound, based on a microphone, wherein the section of sentences include but are not limited to mandarin, dialect and foreign language; detecting the secretion amount, viscosity and secreted protein molecular components and content of saliva based on a saliva detector; collecting protein molecular components and contents in blood based on a blood collection analyzer; analyzing and screening root absorption and alveolar bone wall genes based on an oral cavity detection process; screening whether the patients have epilepsy, rheumatism, hepatitis, nephrosis, tuberculosis, diabetes, heart disease, hemophilia and rickets based on the whole body disease collecting process.
In the present embodiment, the data processing module 2 is used in the processing processThe DGCNN separates the oral scan data into clinical dental crown data, and then uses the manually marked CT sample to supervise and train the UNet neural network feature function, usingAs a loss function, dividing CT image data, establishing a model file comprising separated teeth, periodontal ligament and jaw bone, and carrying out tooth position, crowding degree, upper and lower jaw dentition relation and tooth center line position on the basis of the model file; performing feature point positioning and regression analysis on medical images such as X-ray side position plates by using a deep convolutional neural network CNN, calculating quantitative analysis indexes of the air passage, and determining the width of the air passage; the processing procedure of the facial image processing unit comprises the steps of extracting facial contours and expression features by using Sift feature seeds, and calculating nose lip distance and facial midline position on the basis of feature seed determination;
the processing process of the breathing data processing unit comprises the steps of analyzing the pronunciation characteristics, frequency and depth of breathing, and judging whether an apnea syndrome exists or not; the processing process of the pronunciation data processing unit comprises extracting and analyzing the characteristic points of the seesaw sound, the flat tongue sound, the vowel sound and the nasal sound, the characteristic points of mandarin, dialect and foreign language voice, and the characteristic points of lip palate crack voice, extracting the characteristics of voice signals by using an LSTM (least squares) circulating neural network, and analyzing and processing pronunciation data; the saliva data processing unit is used for processing saliva data, wherein the saliva data processing unit comprises a saliva secretion amount, viscosity, and secreted protein molecule composition and content;
the processing procedure of the blood data processing unit comprises analyzing the protein molecular components and the content in blood; the processing procedure of the gene data processing unit comprises screening and analyzing genes which are easy to cause root absorption and alveolar bone wall; the processing of systemic disease data includes analyzing the disease of the systemic acquired image orthodontic treatment effect and evaluating whether orthodontic treatment can be performed.
In this embodiment, the digital twin module 6 is configured to integrate the collected information uniformly based on artificial intelligence and/or virtual reality and/or big data, output the integrated information to Matlab, and build a digital twin model by using a Simulink module, and perform operation processing.
In this embodiment, the soft and hard tissue change condition after the tooth correction output by the output module 7 includes the arrangement change of dentition, facial expression change, breathing voice change, and prediction development change, and early warning of the absorption risk of tooth root, alveolar bone wall, temporomandibular joint cartilage, protein molecule change in saliva and blood, and the output module 7 outputs the treatment points needing improvement, including the bonding position of bracket, thickness of archwire and shape change.
In this embodiment, the virtual reality interaction module 3 includes a virtual reality simulation unit and a data processing unit, where the data processing unit obtains information of the dento-maxillofacial neck and pre-processes the data information, and the virtual reality simulation unit performs orthodontic virtual reality and non-orthodontic virtual reality simulation based on the information.
In this embodiment, the force feedback model module 5 includes a force feedback unit and a processing unit, where the force feedback unit is configured to calculate force data pre-applied by a user's hand according to operation motion information of the user's hand, simulate real feedback to the user's hand according to the force data, feedback the operation motion information to the processing unit, and perform motion data analysis processing by the processing unit. The virtual reality interaction module 3 and the force feedback model module 5 can realize the simulation of the orthodontic virtual reality and the non-orthodontic virtual reality, and the force feedback model module 5 can simulate and carry out mechanical analysis, so that the system can achieve the purposes of improving rehabilitation and training of clinical skills of operations.
The method for establishing the oral cavity digital twin model based on virtual reality interaction comprises the following steps:
s1: collecting dental maxillofacial neck soft and hard tissue image data, tooth scanning, facial image data, respiratory data, pronunciation data, saliva data, blood data, gene data and systemic disease data of a patient;
s2: processing the acquired image and data;
s3: generating a finite element model and establishing a virtual reality interaction model;
s4: uniformly integrating data acquisition information, processing information, virtual reality interaction information and mechanical analysis information, and constructing a digital twin model and operation processing;
s5: outputting the change condition of the soft and hard tissues after the tooth correction, predicting the development change, treating the risk presentation and the treatment point needing to be improved, and establishing a personalized virtual reality interaction model based on the facial change, the correction effect, the development change and the like.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (7)
1. An oral digital twin model system based on virtual reality interactions, comprising:
the data acquisition module (1) is used for acquiring dental maxillofacial neck soft and hard tissue image data, tooth scanning, facial image data, breathing data, pronunciation data, saliva data, blood data, gene data and whole body disease data of a patient, and the facial image comprises a two-dimensional facial photo, a three-dimensional, static and dynamic image;
the data processing module (2) is used for processing the acquired images and data, wherein the processing process comprises image segmentation, face extraction, respiratory data processing, pronunciation analysis processing, saliva data processing and whole body disease data processing, and the data processing module (2) comprises a dentofacial neck data processing unit, a face image processing unit, a respiratory data processing unit, a pronunciation data processing unit, a saliva data processing unit, a blood data processing unit and a gene data processing unit;
the virtual reality interaction module (3) is used for simulating tooth extraction orthodontic and non-tooth extraction orthodontic processes, the virtual reality interaction module (3) acquires segmented dentognathic face neck information and generates an interaction model, and the tooth interaction model is provided with a bracket, an installation arch wire, an invisible tooth socket design installation, accessory bonding and implant anchorage implantation;
the mechanical analysis module (4) is used for carrying out three-dimensional finite element analysis processing on the segmented tooth structure and generating a three-dimensional finite element model, wherein the tooth three-dimensional model comprises teeth, periodontal ligament and alveolar bone parts;
the force feedback model module (5) is used for simulating the pressure feeling and the touch feeling in the real operation, and the touch feeling comprise the holding feeling and the touch feeling of a needle holder, forceps, ligature wires, rubber rings, push springs, tension springs, support nails and orthodontic forceps;
the digital twin module (6) is used for integrating the data acquisition information, the processing information, the bracket, the arch wire, the invisible tooth socket, the accessory information and the mechanical analysis information which are installed in the virtual reality interaction module (3) uniformly, and constructing a digital twin model and carrying out operation treatment;
and the output module (7) is used for outputting the change condition of the soft and hard tissues after the tooth correction, predicting the development change, prompting the treatment risk and needing an improved treatment point.
2. The oral digital twin model system based on virtual reality interaction according to claim 1, characterized in that the acquisition process of the data acquisition module (1) comprises: acquiring data of jawbone, face and neck of a patient based on CT, and scanning intraoral tooth data by an intraoral scanner and facial soft tissue contours by a facial scanner; taking two-dimensional and three-dimensional facial photos and videos based on cameras, wherein the two-dimensional and three-dimensional facial photos comprise static and dynamic facial contours and expression changes; collecting pronunciation characteristics, frequency and depth of breath based on a microphone sensor; collecting pronunciation characteristics of a section of sentences of a patient, including a seesaw sound, a flat tongue sound, a vowel and a nasal sound, based on a microphone, wherein the section of sentences include mandarin, dialect and foreign language; detecting the secretion amount, viscosity and secreted protein molecular components and content of saliva based on a saliva detector; collecting protein molecular components and contents in blood based on a blood collection analyzer; analyzing and screening root absorption and alveolar bone wall genes based on an oral detection process; screening whether the patients have epilepsy, rheumatism, hepatitis, nephritis, tuberculosis, diabetes, heart disease, hemophilia and rickets based on the whole body disease collecting process.
3. The oral digital twin model system based on virtual reality interaction according to claim 1, wherein the soft and hard tissue change conditions after the teeth correction output by the output module (7) comprise arrangement change of dentition, facial expression change, respiratory voice change, and prediction development change, and early warning of tooth root, alveolar bone wall and temporomandibular joint cartilage absorption risk, protein molecule change in saliva and blood, and the output module (7) outputs treatment points needing improvement, including bonding position of brackets, thickness of archwire and change of shape.
4. The oral digital twin model system based on virtual reality interaction according to claim 1, wherein the virtual reality interaction module (3) comprises a virtual reality simulation unit and a data processing unit, the data processing unit obtains the information of the dento-maxillofacial neck and preprocesses the data information, and the virtual reality simulation unit performs orthodontic virtual reality of tooth extraction and non-orthodontic virtual reality simulation based on the information.
5. The oral cavity digital twin model system based on virtual reality interaction according to claim 1, wherein the virtual reality interaction module (3) can also experiment an augmented reality technology experiment, and comprises an augmented reality simulation unit and a data processing unit, wherein the data processing unit obtains the information of the dento-maxillofacial neck and preprocesses the data information, and the augmented reality simulation unit performs orthodontic augmented reality based on the information and non-orthodontic virtual reality simulation.
6. The oral cavity digital twin model system based on virtual reality interaction according to claim 1, wherein the force feedback model module (5) comprises a force feedback unit and a processing unit, the force feedback unit is used for calculating pre-applied force data of a user hand according to operation motion information of the user hand, simulating real feedback to the user hand according to the force data, feeding back the operation motion information to the processing unit, and analyzing and processing the motion data by the processing unit.
7. The method for establishing the oral cavity digital twin model in the oral cavity digital twin model system based on virtual reality interaction according to any one of claims 1 to 6, which is characterized by comprising the following steps:
s1: collecting dental maxillofacial neck soft and hard tissue image data, tooth scanning, facial image data, respiratory data, pronunciation data, saliva data, blood data, gene data and systemic disease data of a patient;
s2: processing the acquired image and data;
s3: generating a finite element model, establishing a virtual reality interaction model and an augmented reality interaction model;
s4: uniformly integrating data acquisition information, processing information, virtual reality interaction information and mechanical analysis information, and constructing a digital twin model and operation processing;
s5: outputting the change condition of the soft and hard tissues after the teeth are corrected, and establishing a personalized virtual reality interaction model based on the face change, the correction effect and the development change.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111244755.2A CN114224528B (en) | 2021-10-26 | 2021-10-26 | Oral cavity digital twin model system based on virtual reality interaction and establishment method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111244755.2A CN114224528B (en) | 2021-10-26 | 2021-10-26 | Oral cavity digital twin model system based on virtual reality interaction and establishment method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114224528A CN114224528A (en) | 2022-03-25 |
| CN114224528B true CN114224528B (en) | 2023-07-18 |
Family
ID=80743207
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111244755.2A Active CN114224528B (en) | 2021-10-26 | 2021-10-26 | Oral cavity digital twin model system based on virtual reality interaction and establishment method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114224528B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115862821B (en) * | 2023-02-16 | 2023-05-05 | 深圳市汇健智慧医疗有限公司 | Digital twinning-based intelligent operating room construction method and related device |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102215771A (en) * | 2008-11-20 | 2011-10-12 | 矫正技术公司 | Orthodontic systems and methods including parametric attachments |
| CN102663818A (en) * | 2012-04-16 | 2012-09-12 | 上海交通大学医学院附属第九人民医院 | Method and device for establishing three-dimensional craniomaxillofacial morphology model |
| CN103054658A (en) * | 2012-12-18 | 2013-04-24 | 浙江工业大学 | Modeling method of oral cavity model used for individual three-dimensional orthodontic force test |
| CN108268673A (en) * | 2016-12-30 | 2018-07-10 | 无锡时代天使医疗器械科技有限公司 | The method of emulation is digitized for the rectifying effect to dental appliance |
| CN110623760A (en) * | 2019-09-20 | 2019-12-31 | 上海正雅齿科科技股份有限公司 | Tooth correction scheme generation method based on pre-experience and electronic commerce system |
| CN112022384A (en) * | 2020-09-04 | 2020-12-04 | 上海交通大学医学院附属第九人民医院 | Digital resin plate based on CAD/CAM, preparation method and application |
| CN112087985A (en) * | 2018-05-10 | 2020-12-15 | 3M创新有限公司 | Simulated orthodontic treatment via real-time enhanced visualization |
| KR20210069172A (en) * | 2019-12-02 | 2021-06-11 | 오스템임플란트 주식회사 | Method for orthodontic treatment and apparatus thereof |
| CN112932703A (en) * | 2021-02-05 | 2021-06-11 | 四川大学 | Orthodontic bracket bonding method utilizing mixed reality technology |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7717708B2 (en) * | 2001-04-13 | 2010-05-18 | Orametrix, Inc. | Method and system for integrated orthodontic treatment planning using unified workstation |
| US9421074B2 (en) * | 2001-04-13 | 2016-08-23 | Orametrix, Inc. | Unified three dimensional virtual craniofacial and dentition model and uses thereof |
-
2021
- 2021-10-26 CN CN202111244755.2A patent/CN114224528B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102215771A (en) * | 2008-11-20 | 2011-10-12 | 矫正技术公司 | Orthodontic systems and methods including parametric attachments |
| CN102663818A (en) * | 2012-04-16 | 2012-09-12 | 上海交通大学医学院附属第九人民医院 | Method and device for establishing three-dimensional craniomaxillofacial morphology model |
| CN103054658A (en) * | 2012-12-18 | 2013-04-24 | 浙江工业大学 | Modeling method of oral cavity model used for individual three-dimensional orthodontic force test |
| CN108268673A (en) * | 2016-12-30 | 2018-07-10 | 无锡时代天使医疗器械科技有限公司 | The method of emulation is digitized for the rectifying effect to dental appliance |
| CN112087985A (en) * | 2018-05-10 | 2020-12-15 | 3M创新有限公司 | Simulated orthodontic treatment via real-time enhanced visualization |
| CN110623760A (en) * | 2019-09-20 | 2019-12-31 | 上海正雅齿科科技股份有限公司 | Tooth correction scheme generation method based on pre-experience and electronic commerce system |
| KR20210069172A (en) * | 2019-12-02 | 2021-06-11 | 오스템임플란트 주식회사 | Method for orthodontic treatment and apparatus thereof |
| CN112022384A (en) * | 2020-09-04 | 2020-12-04 | 上海交通大学医学院附属第九人民医院 | Digital resin plate based on CAD/CAM, preparation method and application |
| CN112932703A (en) * | 2021-02-05 | 2021-06-11 | 四川大学 | Orthodontic bracket bonding method utilizing mixed reality technology |
Non-Patent Citations (1)
| Title |
|---|
| 虚拟现实技术在口腔颌面部解剖及三维数字模型构建中的应用;高璐;张晓红;金海威;丛蔚;王福;;中国组织工程研究(第46期);第7492-7497页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114224528A (en) | 2022-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Rekow | Digital dentistry: The new state of the art—Is it disruptive or destructive? | |
| Sarrafpour et al. | Tooth eruption results from bone remodelling driven by bite forces sensed by soft tissue dental follicles: a finite element analysis | |
| Hiiemae et al. | Tongue movements in feeding and speech | |
| Salagare et al. | An overview of internet of dental things: new frontier in advanced dentistry | |
| CN113571151A (en) | Device and method for monitoring orofacial muscle training | |
| CN111265316B (en) | Tooth model stress analysis method and device for simulating wearing of orthodontic appliance | |
| CN114224528B (en) | Oral cavity digital twin model system based on virtual reality interaction and establishment method | |
| CN114099016B (en) | Digital twin model-based fixed correction and invisible correction hybrid treatment system | |
| Wai Kan Yeung et al. | Citation Network Analysis of Dental Implant Literature from 2007 to 2016. | |
| US20220265392A1 (en) | Method for providing orthodontic treatment | |
| Shaweesh et al. | Studying the relationships between the outlines of the face, maxillary central incisor, and maxillary arch in Jordanian adults by using Fourier analysis | |
| CN114224529B (en) | Digital twin model mapping system for oral cavity, dental jaw face and neck and establishing method | |
| Brown et al. | A survey of tissue-depth landmarks for facial approximation | |
| US20220211469A1 (en) | Systems and methods for determining an updated target orthodontic force | |
| Shaikh et al. | Artificial Intelligence in Dentistry | |
| Paskay | Instrumentation and measurement procedures in orofacial myology | |
| Khan et al. | Artificial intelligence and 3D printing technology in orthodontics: future and scope. | |
| CN111275808A (en) | Method and device for establishing orthodontic model | |
| Fawaz et al. | What is the current state of artificial intelligence applications in dentistry and orthodontics? | |
| Vermeulen | Manual forensic facial reconstruction | |
| Abdurazzokovich | Diagnosis of children with dentoalveolural anomalies and deformations | |
| Slavicek et al. | Importance of skeletal location of maxilla to determine the vertical dimension of occlusion | |
| BURLUI et al. | INTERDISCIPLINARY APPROACH IN DIGITAL DENTISTRY PLANNING. | |
| Gokdeniz et al. | Artificial Intelligence in Dentistry | |
| Daniels | Defining a method to predict mouth morphology in edentulous skulls |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |