WO2023198101A1

WO2023198101A1 - Artificial intelligence-based oral cavity examination method and apparatus, electronic device, and medium

Info

Publication number: WO2023198101A1
Application number: PCT/CN2023/087798
Authority: WO
Inventors: 王嘉磊; 皮成祥; 张健; 江腾飞
Original assignee: 先临三维科技股份有限公司
Priority date: 2022-04-12
Filing date: 2023-04-12
Publication date: 2023-10-19
Also published as: CN114782345A

Abstract

The present disclosure relates to an artificial intelligence-based oral cavity examination method and apparatus, an electronic device, and a medium. The method comprises: obtaining a two-dimensional texture map corresponding to a three-dimensional tooth model; processing the two-dimensional texture map on the basis of a pre-trained oral cavity examination model to obtain a lesion area; and back-projecting the lesion area to the three-dimensional tooth model to obtain a lesion position of the three-dimensional tooth model. By means of the described technical solution, detection is performed on a two-dimensional texture map to avoid the problem that a lesion area is missed; a lesion area is recognized on the basis of a pre-trained oral cavity examination model, thereby improving the precision of recognition of the lesion area; and a lesion position is presented in the form of a three-dimensional model, thereby implementing a more intuitive presentation effect, and further improving the examination efficiency and effect in an oral cavity examination scene.

Description

Oral examination methods, devices, electronic equipment and media based on artificial intelligence

This disclosure requires the priority of the Chinese patent application submitted to the China Patent Office on April 12, 2022, with the application number 2022103836043 and the invention name "Oral Detection Method, Device, Electronic Equipment and Media Based on Artificial Intelligence", and its entire content is approved by This reference is incorporated into this disclosure.

Technical field

Embodiments of the present disclosure relate to the technical field of intelligent oral medicine, and in particular to an oral detection method, device, electronic device and medium based on artificial intelligence.

Background technique

With the continuous development and progress of society and the continuous improvement of people's living standards, people are paying more and more attention to the condition of their oral teeth.

In related technologies, dentists identify and judge oral diseases based on their own experience, which requires the dentist to have certain clinical experience and consumes a lot of energy. In addition, there are cases where the diseased parts are misdetected or missed.

Contents of the invention

(1) Technical problems to be solved

In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an oral cavity detection method, device, electronic device and medium based on artificial intelligence.

(2) Technical solutions

Embodiments of the present disclosure provide an oral cavity detection method based on artificial intelligence, which method includes:

Obtain the two-dimensional texture map corresponding to the three-dimensional tooth model;

The two-dimensional texture map is processed based on the pre-trained oral cavity detection model to obtain the lesion area;

Back-project the diseased area to the three-dimensional tooth model to obtain the three-dimensional tooth The location of the lesion in the model.

An embodiment of the present disclosure also provides an oral cavity detection device based on artificial intelligence, which device includes:

The image acquisition module is used to obtain the two-dimensional texture map corresponding to the three-dimensional tooth model;

A processing module, used to process the two-dimensional texture map based on a pre-trained oral cavity detection model to obtain the lesion area;

A back-projection module is used to back-project the lesion area to the three-dimensional tooth model to obtain the lesion position of the three-dimensional tooth model.

An embodiment of the present disclosure also provides an electronic device. The electronic device includes: a processor; a memory used to store instructions executable by the processor; and the processor is used to read the instruction from the memory. The instructions can be executed and executed to implement the artificial intelligence-based oral cavity detection method provided by the embodiments of the present disclosure.

Embodiments of the present disclosure also provide a computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to execute the oral cavity detection method based on artificial intelligence as provided by the embodiments of the present disclosure.

(3) Beneficial effects

Compared with the existing technology, the above technical solutions provided by the embodiments of the present disclosure have the following advantages:

The artificial intelligence-based oral detection solution provided by the embodiment of the present disclosure obtains the two-dimensional texture map corresponding to the three-dimensional tooth model, processes the two-dimensional texture map based on the pre-trained oral cavity detection model, obtains the diseased area, and back-projects the diseased area to The three-dimensional tooth model is used to obtain the lesion location of the three-dimensional tooth model. Using the above technical solution, during the oral inspection process, the two-dimensional texture map corresponding to the three-dimensional tooth model can be detected to completely observe the information of the entire set of teeth, avoiding the problem of the diseased area being missed due to improper selection of the observation angle of the three-dimensional tooth model. And identifying the lesion area based on the pre-trained oral detection model can greatly improve the recognition accuracy of the lesion area. In addition, the location of the lesion can be presented in the form of a three-dimensional model, making the presentation effect more intuitive and further improving the detection efficiency and effect in the oral detection scenario.

It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and do not limit the present disclosure.

Description of the drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those of ordinary skill in the art, It is said that other drawings can be obtained based on these drawings without exerting creative labor.

Figure 1 is a schematic flowchart of an artificial intelligence-based oral cavity detection method in one or more embodiments of the present disclosure;

Figure 2 is a schematic flow chart of another oral cavity detection method based on artificial intelligence in one or more embodiments of the present disclosure;

Figure 3a is a schematic diagram of a three-dimensional tooth model in one or more embodiments of the present disclosure;

Figure 3b is a schematic diagram of a two-dimensional texture map in one or more embodiments of the present disclosure;

Figure 3c is a schematic diagram of another two-dimensional texture map in one or more embodiments of the present disclosure;

Figure 3d is a schematic diagram of yet another two-dimensional texture map in one or more embodiments of the present disclosure;

Figure 4 is a schematic diagram of another three-dimensional tooth model in one or more embodiments of the present disclosure;

Figure 5 is a schematic diagram of yet another three-dimensional tooth model in one or more embodiments of the present disclosure;

Figure 6 is a schematic structural diagram of an artificial intelligence-based oral cavity detection device in one or more embodiments of the present disclosure;

Figure 7 is a schematic structural diagram of an electronic device in one or more embodiments of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than All examples. Based on the embodiments in this disclosure, all other embodiments obtained by those of ordinary skill in the art without any creative efforts fall within the scope of protection of this disclosure.

In practical applications, the process of detecting and identifying common oral diseases such as dental caries, dental calculus, and pigmentation requires a dentist to identify the lesion, which requires the dentist to have certain clinical experience and consumes a lot of energy from the doctor; In addition, during the ongoing disease The identification of variable parts and the improper selection of observation angles by the dentist may lead to misdetection or missed detection of the diseased parts.

In response to the above problems, the present disclosure proposes an oral cavity detection method based on artificial intelligence. By obtaining the two-dimensional texture map corresponding to the three-dimensional tooth model, the two-dimensional texture map is processed based on the pre-trained oral cavity detection model to obtain the lesion area, and the lesion is The area is back-projected to the three-dimensional tooth model to obtain the lesion location of the three-dimensional tooth model, so that the location of the lesion can be quickly and accurately located and identified in an automated manner.

Specifically, FIG. 1 is a schematic flowchart of an artificial intelligence-based oral cavity detection method provided by an embodiment of the present disclosure. The method can be executed by an artificial intelligence-based oral cavity detection device, where the device can be implemented using software and/or hardware. Generally can be integrated in electronic equipment. As shown in Figure 1, the method includes:

Step 101: Obtain the two-dimensional texture map corresponding to the three-dimensional tooth model.

The three-dimensional tooth model can be any three-dimensional tooth model. The embodiment of the present disclosure does not limit the source of the three-dimensional tooth model. For example, the three-dimensional tooth model can be obtained by scanning the upper row of teeth or the lower row of teeth in the human mouth in real time with a scanning device. It can also be a three-dimensional tooth model corresponding to the upper row of teeth or the lower row of teeth sent based on the download address or other devices. The two-dimensional texture map refers to a two-dimensional plane texture image after the three-dimensional tooth model is expanded by network parameters, so that the information of the entire set of teeth can be completely observed, and the problem of the diseased area being missed due to improper selection of the observation perspective can be avoided.

In the embodiments of the present disclosure, there are many ways to obtain the two-dimensional texture map corresponding to the three-dimensional tooth model. In some embodiments, the three-dimensional tooth model is subjected to network parameterization processing to obtain the two-dimensional texture map, such as obtaining the two-dimensional texture map corresponding to the three-dimensional tooth model. Multiple three-dimensional vertices, based on multiple three-dimensional vertices, obtain multiple triangles, establish a linear equation system based on the angle between two adjacent sides of each triangle and the ratio of the two sides, and solve the linear equation system based on the preset initial conditions , two-dimensional parameters, construct a two-dimensional texture map based on two-dimensional parameters.

In other embodiments, multiple three-dimensional coordinate points corresponding to the three-dimensional tooth model are obtained, the multiple three-dimensional coordinate points are converted based on the dimension conversion model to obtain multiple two-dimensional coordinate points, and a two-dimensional texture map is constructed based on the two-dimensional coordinate points . The above two methods are only examples of obtaining the two-dimensional texture map corresponding to the three-dimensional tooth model. The embodiments of the present disclosure do not limit the specific methods of obtaining the two-dimensional texture map corresponding to the three-dimensional tooth model.

Specifically, after obtaining the three-dimensional tooth model, which is composed of multiple three-dimensional vertices, the mapping relationship between the three-dimensional vertex coordinates on the three-dimensional tooth model and the two-dimensional coordinates of the parameter domain plane is obtained, thereby converting a three-dimensional tooth model into a three-dimensional tooth model. The tooth model is flattened into a two-dimensional plane texture image.

Step 102: Process the two-dimensional texture map based on the pre-trained oral cavity detection model to obtain the lesion area.

Among them, the oral cavity detection model is an oral cavity detection model pre-trained for lesion area recognition. The specific training process will be described in detail in subsequent embodiments and will not be described in detail here. The lesion area refers to the area corresponding to dental lesions such as dental caries, dental calculus, and pigmentation on the two-dimensional texture map.

In the embodiments of the present disclosure, the two-dimensional texture map is processed based on the pre-trained oral cavity detection model to obtain the lesion area in many ways. In some embodiments, the two-dimensional texture map is processed based on the target detection network in the oral cavity detection model. The image is detected and multiple tooth regions are obtained. Each tooth region is identified based on the semantic segmentation network in the oral cavity detection model, and the recognition result of each tooth region is obtained. The lesion area is determined based on the recognition result of each tooth region.

In other embodiments, feature extraction is performed on the two-dimensional texture map based on the oral cavity detection model to obtain multiple feature values, each feature value is compared with a preset feature threshold, and the lesion area is determined based on the comparison results. The above two methods are only examples of processing the two-dimensional texture map based on the pre-trained oral cavity detection model to obtain the lesion area. The embodiments of the present disclosure do not process the two-dimensional texture map based on the pre-trained oral cavity detection model to obtain the lesion area. be limited in specific ways.

In the embodiment of the present disclosure, after obtaining the two-dimensional texture map, the two-dimensional texture map can be processed based on the pre-trained oral cavity detection model to obtain the lesion area.

Step 103: Back-project the lesion area to the three-dimensional tooth model to obtain the lesion position of the three-dimensional tooth model.

Among them, the lesion area refers to the area corresponding to dental lesions such as dental caries, dental calculus, and pigmentation on the two-dimensional texture map. Therefore, the lesion area needs to be back-projected to the three-dimensional tooth model to obtain the lesion location of the three-dimensional tooth model. Among them, the location of the lesion refers to the location corresponding to dental lesions such as dental caries, dental calculus, and pigmentation on the three-dimensional tooth model.

In the embodiments of the present disclosure, there are many ways to back-project the lesion area to the three-dimensional tooth model to obtain the lesion position of the three-dimensional tooth model. In some embodiments, the two-dimensional coordinate points corresponding to the lesion area are obtained based on the pre-stored The dimensional coordinate mapping relationship table obtains the three-dimensional coordinate points corresponding to the two-dimensional coordinate points, and determines the lesion location of the three-dimensional tooth model based on the three-dimensional coordinate points.

In other embodiments, each coordinate point of the three-dimensional tooth model projected onto the two-dimensional plane is obtained, a target coordinate point matching the lesion area is obtained, and the position of the target coordinate point reflected onto the three-dimensional tooth model is the lesion position. The above two methods are only examples of back-projecting the lesion area to the three-dimensional tooth model to obtain the lesion position of the three-dimensional tooth model. The embodiments of the present disclosure do not back-project the lesion area to the three-dimensional tooth model to obtain the specific lesion position of the three-dimensional tooth model. way to limit.

Specifically, after obtaining the lesion area, the lesion area can be back-projected to the three-dimensional tooth model to obtain the lesion position of the three-dimensional tooth model, and the lesion position can be presented in the form of the three-dimensional model, making the presentation effect more intuitive.

Based on the description of the above embodiments, through parametric expansion of the grid, detection in the form of a two-dimensional texture map can completely observe the information of the entire set of teeth, avoiding the problem of missing the diseased area in the three-dimensional model due to improper selection of observation angles. , and can efficiently and accurately identify lesions through deep learning technology, and by back-projecting the recognition results of the two-dimensional texture map after the grid parameters are expanded back to the three-dimensional grid, the lesion location can be presented in the form of a three-dimensional model, making the presentation The effect is more intuitive, and finally non-occlusion is calculated based on the location of the lesion The observation angle allows dental doctors to quickly locate and count the location of lesions, effectively avoiding omissions.

It can be understood that based on the deep learning network using two stages of detection and segmentation, the tooth area is first detected in the two-dimensional texture map, and then the tooth area is segmented and identified to identify the lesion, which can further improve the detection of small lesion areas such as dental caries. The recognition accuracy, how to train the oral detection model, and associate the observation angle of the screenshot to the observation angle of the three-dimensional tooth model further facilitates the oral surgeon to quickly locate and count the location of the lesion. This is described in detail below in conjunction with Figure 2.

Specifically, FIG. 2 is a schematic flow chart of another oral cavity detection method based on artificial intelligence provided by an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes the above oral cavity detection method based on artificial intelligence. As shown in Figure 2, the method includes:

Step 201: Perform network parameterization processing on the three-dimensional tooth model to obtain a two-dimensional texture map.

Specifically, multiple three-dimensional vertices corresponding to the three-dimensional tooth model are obtained. Based on the multiple three-dimensional vertices, multiple triangles are obtained. A linear equation system is established based on the angle between two adjacent sides of each triangle and the ratio of the two sides. Based on the preset The initial conditions are to solve a system of linear equations, two-dimensional parameters, and a two-dimensional texture map is constructed based on the two-dimensional parameters.

Specifically, the three-dimensional tooth model is composed of multiple three-dimensional vertices. The mapping relationship between the three-dimensional vertex coordinates on the three-dimensional tooth model and the two-dimensional plane coordinates of the parameter domain plane is obtained, thereby flattening a three-dimensional tooth model into a two-dimensional texture. picture.

In order to quickly solve and reduce the deformation and distortion of the triangle on the three-dimensional tooth model caused by parameterization, a linear system of equations is established through the angle between the two adjacent sides of the triangle and the proportion of the two sides. The linear equation can be quickly solved after the initial conditions are given. The system of equations is used to obtain the parameterized two-dimensional texture map.

Among them, the preset initial conditions are conditions that restrict the solution. For example, it is necessary to ensure that the three included angles of the triangle remain unchanged as much as possible. Another example is to ensure that the expanded two-dimensional texture map is as compact as possible. Another example is to ensure that the expanded two-dimensional texture map is as compact as possible. The time required for the two-dimensional texture map should be as short as possible, which should be set according to the needs of the application scenario. In the embodiment of the present disclosure, try to ensure that the angles of the triangular patches are consistent, so that the teeth on the expanded two-dimensional texture map will not be distorted, thereby affecting subsequent recognition and further improving the accuracy of oral cavity detection.

Illustratively, Figure 3a is a schematic diagram of a three-dimensional tooth model provided by an embodiment of the present disclosure. Figure 3a shows the original three-dimensional tooth model. The obtained two-dimensional texture map corresponding to the three-dimensional tooth model is shown in Figure 3b. Figure 3b shows a schematic diagram of the obtained two-dimensional texture map.

Step 202: Obtain the two-dimensional texture map sample corresponding to each three-dimensional tooth model sample; wherein, the two-dimensional texture map sample includes marked tooth areas and marked lesion areas. The two-dimensional texture map samples are input into the target detection network for detection to obtain the training teeth. area, input the training tooth area into the semantic segmentation network for recognition, and obtain the training lesion area.

Step 203: Adjust the network parameters of the target detection network based on the first comparison result between the training tooth area and the marked tooth area, and adjust the network parameters of the semantic segmentation network based on the second comparison result between the training lesion area and the marked lesion area, to obtain an oral cavity detection model.

In the embodiment of the present disclosure, the annotated two-dimensional texture map constitutes a training sample. The target detection network identifies the position of the teeth in the two-dimensional texture map to obtain each tooth region. The semantic segmentation network detects each tooth region in the corresponding image. Classify a pixel to determine whether the category of each pixel belongs to the lesion area.

In the embodiment of the present disclosure, the network parameters of the target detection network are adjusted based on the first comparison result of the training tooth area and the marked tooth area, and the network parameters of the semantic segmentation network are adjusted based on the second comparison result of the training lesion area and the marked lesion area, continuously. Optimize the target detection network and semantic segmentation network until the loss value is less than the preset threshold to obtain the oral cavity detection model.

Step 204: Detect the two-dimensional texture map based on the target detection network in the oral cavity detection model to obtain multiple tooth regions. Identify each tooth region based on the semantic segmentation network in the oral cavity detection model to obtain the identification of each tooth region. As a result, the lesion area is determined based on the recognition result of each tooth area.

Specifically, the target detection network detects a single tooth region, the semantic segmentation network identifies the lesion area of a single tooth region, and re-overlays the lesion area within a single tooth region to the original two-dimensional texture map.

Illustratively, taking the two-dimensional texture map in Figure 3b as an example, Figure 3c is a schematic diagram of another two-dimensional texture map provided by an embodiment of the present disclosure. Figure 3c shows the tooth area obtained by target detection, and based on The semantic segmentation network in the oral cavity detection model identifies each tooth area, such as the lesion area 11 shown in Figure 3d.

Specifically, the small box in Figure 3c is obtained through the target detection technology of deep learning. Using multiple boxes is used to select the tooth area, which can focus attention on a single area (that is, the subsequent segmentation network only processes the image data in a single box), which can be more accurate than segmenting the entire expanded image. Identify small diseased areas such as dental caries.

In addition, based on the detection of a single tooth area based on the deep learning network, only the tooth area is identified. Compared with the identification of the entire picture, smaller diseased areas can be identified. Similar to the human eye observation, the observation window is fixed. When the local area is When magnified, it is easier to observe small objects in the area, further improving the accuracy of oral examination.

Step 205: Obtain the two-dimensional coordinate points corresponding to the lesion area, obtain the three-dimensional coordinate points corresponding to the two-dimensional coordinate points based on the pre-stored dimensional coordinate mapping relationship table, and determine the lesion location of the three-dimensional tooth model based on the three-dimensional coordinate points.

Specifically, when the three-dimensional tooth model is expanded into a two-dimensional texture map, the one-to-one correspondence between the vertices of the three-dimensional tooth model and the coordinates of the two-dimensional points is saved. When the two-dimensional texture map identifies that a certain point is a lesion, the corresponding point can be queried. The coordinates of a three-dimensional point.

For example, taking the two-dimensional texture map in Figure 3d as an example, the lesion area 11 is projected onto the three-dimensional tooth model, as shown in Figure 4, showing the lesion location 21 of the three-dimensional tooth model.

Step 206: Calculate the observation angle based on the lesion location, and obtain and display the target image based on the observation angle.

Among them, the observation angle refers to the visual angle from which the lesion location is observed without obstruction.

In the embodiments of the present disclosure, there are many ways to calculate the observation angle based on the lesion position. In some specific implementations, the average normal direction of the lesion position is obtained, the positions of two adjacent teeth are determined based on the lesion position, and the position of the two adjacent teeth is determined based on the two phases. The position of the adjacent teeth determines the target straight line direction, the target angle is determined based on the average normal direction and the target straight line direction, and the observation angle is determined based on the target included angle and the preset angle threshold.

Specifically, the non-occluded observation angle is calculated based on the location of the lesion, and the target image is obtained and displayed based on the observation angle. The target image of the specific lesion location is given, which can facilitate the oral surgeon to quickly locate and count the location of the lesion, and effectively avoid omissions.

Step 207: Correlate the observation perspective with the perspective of the three-dimensional tooth model.

Exemplarily, Figure 5 is a schematic diagram of yet another three-dimensional tooth model provided by an embodiment of the present disclosure. The tooth number of each tooth can be obtained through automated or manual marking, such as marking numbers 1-16 in Figure 5. Assuming that the location of the lesion is position A in Figure 5, the observation of position A The average normal direction of the triangular patch in the measurement area is the direction of arrow 1 (it can be seen that if position A is observed from this direction, it will be blocked by tooth No. 5).

Therefore, first, according to the angle between the average normal arrow 1 of the triangular patch of the observation area and the z-axis direction, choose to observe from the top (z-axis direction in Figure 5) or the side (x, y plane direction in Figure 5), and There will be no occlusion at the top. Obtain the two tooth positions adjacent to position A (tooth No. 3 and tooth No. 4 in Figure 5), and calculate the average normal direction of the observation area (arrow 1) and the straight line direction connecting the two tooth position numbers (arrow 2). The formed angle α, when the angle α is less than the preset threshold, it indicates that there is occlusion, and the observation angle is forcibly changed to the vertical direction of the two tooth numbers (the direction of arrow 3 in Figure 5), thereby avoiding observation View angle occlusion problem.

From this, the non-occluded observation angle is calculated based on the location of the lesion, and is displayed by automatically obtaining the target image. The observation angle of the target image is associated with the observation angle of the three-dimensional tooth model, and the observation area of the specific lesion location is given to facilitate oral cavity diagnosis. Doctors can quickly locate and count the location of lesions to effectively avoid omissions.

The oral cavity detection solution based on artificial intelligence provided by the embodiment of the present disclosure performs network parameterization processing on the three-dimensional tooth model to obtain the two-dimensional texture, and obtains the two-dimensional texture map sample corresponding to each three-dimensional tooth model sample; wherein, the two-dimensional texture map sample It includes marking the tooth area and marking the lesion area. Input the two-dimensional texture map sample into the target detection network for detection to obtain the training tooth area. Enter the training tooth area into the semantic segmentation network for recognition to obtain the training lesion area. Based on the training tooth area and marking The first comparison result of the tooth area adjusts the network parameters of the target detection network, and the second comparison result of the training lesion area and the marked lesion area adjusts the network parameters of the semantic segmentation network to obtain the oral cavity detection model, which is based on the target detection network in the oral cavity detection model. The two-dimensional texture map is detected to obtain multiple tooth regions. Each tooth region is identified based on the semantic segmentation network in the oral cavity detection model to obtain the recognition result of each tooth region. The lesion is determined based on the recognition result of each tooth region. area, obtain the two-dimensional coordinate points corresponding to the lesion area, obtain the three-dimensional coordinate points corresponding to the two-dimensional coordinate points based on the pre-stored dimensional coordinate mapping relationship table, determine the lesion position of the three-dimensional tooth model based on the three-dimensional coordinate points, and obtain observations based on the lesion position area, and obtain the average normal direction of the observation area, determine the position of two adjacent teeth based on the location of the lesion, and determine the direction of the target straight line based on the positions of the two adjacent teeth, based on the average normal direction and the target straight line The direction determines the target angle, determines the observation angle based on the target angle and the preset angle threshold, and associates the observation angle with the angle of the three-dimensional tooth model. Using the above technical solution, during the oral inspection process, the two-dimensional texture map corresponding to the three-dimensional tooth model can be detected to completely observe the information of the entire set of teeth, avoiding the problem of the diseased area being missed due to improper selection of the observation angle of the three-dimensional tooth model. And detecting the tooth area in the two-dimensional texture map, and then segmenting and identifying the lesion problem in the tooth area, can further improve the recognition accuracy of small lesion areas such as dental caries, and how to train the oral cavity detection model, and associate the observation perspective of the screenshot to The observation perspective of the three-dimensional tooth model further facilitates the dentist to quickly locate and count the location of the lesion. In addition, the location of the lesion can be presented in the form of a three-dimensional model, making the presentation more intuitive. Finally, a target picture corresponding to the specific location of the lesion is given, which is convenient Dental doctors can quickly locate and count the location of lesions, effectively avoiding omissions and further improving detection efficiency and effectiveness in oral examination scenarios.

Figure 6 is a schematic structural diagram of an oral cavity detection device based on artificial intelligence provided by an embodiment of the present disclosure. The device can be implemented by software and/or hardware, and can generally be integrated in electronic equipment. As shown in Figure 6, the device includes:

The picture acquisition module 301 is used to obtain the two-dimensional texture map corresponding to the three-dimensional tooth model;

The processing module 302 is used to process the two-dimensional texture map based on the pre-trained oral cavity detection model to obtain the lesion area;

The back-projection module 303 is used to back-project the lesion area to the three-dimensional tooth model to obtain the lesion position of the three-dimensional tooth model.

Optionally, the picture acquisition module 301 is specifically used to:

Perform network parameterization processing on the three-dimensional tooth model to obtain the two-dimensional texture map.

Optionally, the processing module 302 is specifically used to:

Detect the two-dimensional texture map based on the target detection network in the oral cavity detection model to obtain multiple tooth regions;

Identify each tooth area based on the semantic segmentation network in the oral cavity detection model, and obtain the identification result of each tooth area;

The diseased area is determined based on the identification result of each of the tooth areas.

Optionally, the back-projection module 303 is specifically used for:

Obtain the two-dimensional coordinate points corresponding to the lesion area;

Obtain the three-dimensional coordinate point corresponding to the two-dimensional coordinate point based on the pre-stored dimensional coordinate mapping relationship table;

The lesion location of the three-dimensional tooth model is determined based on the three-dimensional coordinate points.

Optionally, the device also includes:

Calculation module 304, used to calculate the observation angle based on the lesion location;

The acquisition and display module 305 is used to acquire and display a target picture based on the observation perspective.

Optionally, the calculation module 304 is specifically used to:

Obtain an observation area based on the lesion location, and obtain the average normal direction of the observation area;

Determine the positions of two adjacent teeth based on the location of the lesion, and determine the target straight line direction based on the positions of the two adjacent teeth;

Determine the target angle based on the average normal direction and the target straight line direction;

The observation angle is determined based on the target angle and a preset angle threshold.

Optionally, the device also includes:

A sample acquisition module is used to obtain a two-dimensional texture map sample corresponding to each three-dimensional tooth model sample; wherein the two-dimensional texture map sample includes a marked tooth area and a marked lesion area;

A detection module, used to input the two-dimensional texture map sample into the target detection network for detection to obtain the training tooth area;

A recognition module, used to input the training tooth area into the semantic segmentation network for recognition, and obtain the training lesion area;

A training module configured to adjust the network parameters of the target detection network based on the first comparison result of the training tooth area and the marked tooth area, and adjust the second comparison result of the training lesion area and the marked lesion area. The network parameters of the semantic segmentation network are used to obtain the oral cavity detection model.

Optionally, the device also includes an association module for:

The observation angle is associated with the angle of view of the three-dimensional tooth model.

The artificial intelligence-based oral cavity detection device provided by the embodiments of the present disclosure can execute the artificial intelligence-based oral cavity detection method provided by any embodiment of the present disclosure, and has an execution method Corresponding functional modules and beneficial effects.

Embodiments of the present disclosure also provide a computer program product, which includes a computer program/instruction. When executed by a processor, the computer program/instruction implements the artificial intelligence-based oral cavity detection method provided by any embodiment of the present disclosure.

FIG. 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. Referring specifically to FIG. 7 below, a schematic structural diagram of an electronic device 400 suitable for implementing an embodiment of the present disclosure is shown. The electronic device 400 in the embodiment of the present disclosure may include, but is not limited to, mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals ( Mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG. 7 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 7 , the electronic device 400 may include a processing device (eg, central processing unit, graphics processor, etc.) 401 , which may be loaded into a random access device according to a program stored in a read-only memory (ROM) 402 or from a storage device 408 . The program in the memory (RAM) 403 executes various appropriate actions and processes. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, ROM 402 and RAM 403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.

Generally, the following devices may be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 407 such as a computer; a storage device 408 including a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although FIG. 7 illustrates electronic device 400 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a non-transitory computer-readable medium, the computer program product The computer program contains program code for performing the methods illustrated in the flowcharts. In such embodiments, the computer program may be downloaded and installed from the network via communication device 409, or from storage device 408, or from ROM 402. When the computer program is executed by the processing device 401, the above functions defined in the artificial intelligence-based oral cavity detection method of the embodiment of the present disclosure are performed.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmed read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device . Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and server can communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can communicate with digital data in any form or medium. Communications (e.g., communications network) interconnections. Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or developed in the future network of.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.

The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: receives the user's information display triggering operation during the playback of the video; obtains the At least two target information associated with the video; display the first target information among the at least two target information in the information display area of the play page of the video, wherein the size of the information display area is smaller than the size of the play page. Size: Receive the user's first switching triggering operation, and switch the first target information displayed in the information display area to the second target information among the at least two target information.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages—such as "C" or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In situations involving remote computers, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider through Internet connection).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , Or it can be implemented using a combination of specialized hardware and computer instructions.

The units involved in the embodiments of the present disclosure can be implemented in software or hardware. Among them, the name of a unit does not constitute a limitation on the unit itself under certain circumstances.

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logical device (CPLD) and so on.

In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

processor;

memory for storing instructions executable by the processor;

The processor is configured to read the executable instructions from the memory and execute the instructions to implement any of the artificial intelligence-based oral cavity detection methods provided by this disclosure.

According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium, the storage medium stores a computer program, the computer program is used to execute any one of the methods provided by the present disclosure based on Artificial intelligence-based oral detection method.

The above description is only a description of the preferred embodiments of the present disclosure and the technical principles applied. Those skilled in the art should understand that the disclosure scope involved in the present disclosure is not limited to technical solutions composed of specific combinations of the above technical features, but should also cover solutions composed of the above technical features or without departing from the above disclosed concept. Other technical solutions formed by any combination of equivalent features. For example, a technical solution is formed by replacing the above features with technical features with similar functions disclosed in this disclosure (but not limited to).

Furthermore, although operations are depicted in a specific order, this should not be understood as requiring that these operations be performed in the specific order shown or performed in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Industrial applicability

The oral detection solution based on artificial intelligence provided by the present disclosure avoids the problem of missing the lesion area due to improper selection of the observation angle of the three-dimensional tooth model, and identifies the lesion area based on the pre-trained oral detection model, which can greatly improve the identification accuracy of the lesion area. In addition, the location of the lesion can be presented in the form of a three-dimensional model, making the presentation effect more intuitive and further improving the detection efficiency and effect in oral detection scenarios.

Claims

An oral cavity detection method based on artificial intelligence, which is characterized by including:

Obtain the two-dimensional texture map corresponding to the three-dimensional tooth model;

The two-dimensional texture map is processed based on the pre-trained oral cavity detection model to obtain the lesion area;

The lesion area is back-projected to the three-dimensional tooth model to obtain the lesion position of the three-dimensional tooth model.
The oral cavity detection method based on artificial intelligence according to claim 1, characterized in that said obtaining the two-dimensional texture map corresponding to the three-dimensional tooth model includes:

Perform network parameterization processing on the three-dimensional tooth model to obtain the two-dimensional texture map.
The oral cavity detection method based on artificial intelligence according to claim 1, characterized in that the pre-trained oral cavity detection model detects the two-dimensional texture map to obtain the lesion area, including:

Detect the two-dimensional texture map based on the target detection network in the oral cavity detection model to obtain multiple tooth regions;

Identify each tooth area based on the semantic segmentation network in the oral cavity detection model, and obtain the identification result of each tooth area;

The diseased area is determined based on the identification result of each of the tooth areas.
The oral cavity detection method based on artificial intelligence according to claim 1, characterized in that said back-projecting the lesion area to the three-dimensional tooth model to obtain the lesion position of the three-dimensional tooth model includes:

Obtain the two-dimensional coordinate points corresponding to the lesion area;

Obtain the three-dimensional coordinate point corresponding to the two-dimensional coordinate point based on the pre-stored dimensional coordinate mapping relationship table;

The lesion location of the three-dimensional tooth model is determined based on the three-dimensional coordinate points.
The oral cavity detection method based on artificial intelligence according to claim 1, further comprising:

An observation angle is calculated based on the lesion location, and a target image is obtained and displayed based on the observation angle.
The oral cavity detection method based on artificial intelligence according to claim 5, characterized in that calculating the observation angle based on the lesion position includes:

Obtain the average normal direction of the lesion location;

Determine the positions of two adjacent teeth based on the location of the lesion, and determine the target straight line direction based on the positions of the two adjacent teeth;

Determine the target angle based on the average normal direction and the target straight line direction;

The observation angle is determined based on the target angle and a preset angle threshold.
The oral cavity detection method based on artificial intelligence according to claim 1, characterized in that, before the oral cavity detection model based on pre-training detects the two-dimensional texture map to obtain the lesion area, it also includes:

Obtain the two-dimensional texture map sample corresponding to each three-dimensional tooth model sample; wherein the two-dimensional texture map sample includes a marked tooth area and a marked lesion area;

Input the two-dimensional texture map sample into the target detection network for detection to obtain the training tooth area;

Input the training tooth area into the semantic segmentation network for recognition, and obtain the training lesion area;

Adjust the network parameters of the target detection network based on the first comparison result between the training tooth area and the marked tooth area, and adjust the semantic segmentation network based on the second comparison result between the training lesion area and the marked lesion area. network parameters to obtain the oral cavity detection model.
The oral cavity detection method based on artificial intelligence according to claim 1, further comprising:

The observation angle is associated with the angle of view of the three-dimensional tooth model.
An oral cavity detection device based on artificial intelligence, which is characterized by including:

The image acquisition module is used to obtain the two-dimensional texture map corresponding to the three-dimensional tooth model;

A processing module, used to process the two-dimensional texture map based on a pre-trained oral cavity detection model to obtain the lesion area;

A back-projection module is used to back-project the lesion area to the three-dimensional tooth model to obtain the lesion position of the three-dimensional tooth model.
An electronic device, characterized in that the electronic device includes:

processor;

memory for storing instructions executable by the processor;

The processor is configured to read the executable instructions from the memory and execute the instructions to implement the artificial intelligence-based oral cavity detection method described in any one of claims 1-8.
A computer-readable storage medium, characterized in that the storage medium stores a computer program, and the computer program is used to execute the oral cavity detection method based on artificial intelligence according to any one of the above claims 1-8.