EP4165595A1

EP4165595A1 - Method for automatic segmentation of a dental arch

Info

Publication number: EP4165595A1
Application number: EP21731156.2A
Authority: EP
Inventors: Olivier QUERBES; Véronique QUERBES-DURET
Original assignee: Pearl 3d
Current assignee: Pearl 3d
Priority date: 2020-06-10
Filing date: 2021-06-09
Publication date: 2023-04-19
Also published as: WO2021250091A1; FR3111268B1; FR3111268A1; US20230206451A1

Abstract

The invention relates to a method for automatic segmentation of a dental arch that comprises acquiring a three-dimensional surface of the dental arch, in order to obtain a three-dimensional representation comprising a set of vertices, generating virtual views from the three-dimensional representation, projecting the three-dimensional representation onto each two-dimensional virtual view, in order to obtain an image representing each vertex on the virtual view, processing each image by means of a deep learning network, carrying out inverse projection of each image in order to assign, to each vertex of the three-dimensional representation, one or more pixels of the images in which the vertex appears and to which it corresponds, and assigning one or more probability vectors to each vertex, determining the class of dental tissue to which each vertex most probably belongs based on the probability vector or vectors.

Description

AUTOMATIC ARCADE SEGMENTATION PROCESS

DENTAL

Technical field of the invention

The invention relates to a method of automatically segmenting a dental arch. In particular, the invention relates to a method for accurately and automatically segmenting a three-dimensional reconstruction of a dental arch, total or partial.

Technological background

The segmentation of a dental arch allows, from a representation of the dental arch, to identify each of the teeth and the tissues of the dental arch. This segmented dental arch is used as a basis for various dental treatments (prosthetic, aesthetic, etc.).

The dental arch refers to a set of teeth aligned in the shape of an arc, which is carried by the bones forming the jaw: in humans, the mandible (or mandibular bone) forms the lower jaw and carries the lower dental arch. and the maxilla (or maxillary bone) forms the upper jaw and carries the upper dental arch.

As part of the dental treatments mentioned above, the impression of the dental arch was previously done by taking a physical impression based on alginate, but this technique is gradually being replaced by digital three-dimensional reconstructions obtained. thanks to intraoral cameras. These digital three-dimensional reconstructions of dental arches allow the use of Computer Aided Design (CAD) tools, which offer the advantages of expanding and simplifying the possibilities of treatment and / or prosthesis design.

CAD sometimes requires segmenting the dental arch, ie identifying for each part of its three-dimensional representation to which tissue it belongs. The segmentation of these digital three-dimensional reconstructions is usually done manually, which is a tedious, expensive operation, and can slow down the entire digital process. creation of a prosthesis for example.

Automatic or semi-automatic digital processes have been proposed to overcome this problem. In these methods, an algorithm processes this digital three-dimensional representation so as to output a segmented version of this representation.

The existing automatic or semi-automatic processes have some major drawbacks.

On the one hand, most of these processes are based on mainly geometric considerations. Thus, if they present satisfactory results for the segmentation of a classic dentition, defects appear quickly as soon as the dental arch has a significant offset with the "average" dentition. However, it is mainly these dental arches that deviate from the average that are concerned by most prosthetic, aesthetic or orthodontic treatments.

On the other hand, these methods still too often rely on manual intervention by a human operator, either upstream of the method, or during the implementation of the method, or following a first execution of the method to manually recover local errors. These manual interventions to aid the process can sometimes present greater constraints than manual segmentation, which is counterproductive.

Objectives of the invention

The invention aims to provide a method of automatic segmentation of a dental arch.

The invention aims in particular to provide, in at least one embodiment, a method of automatic segmentation of a dental arch that does not require manual intervention of a human operator for the segmentation.

The invention also aims to provide, in at least one embodiment of the invention, a method of automatic segmentation of a robust dental arch, which can be used for dental arches exhibiting defects that are far from conventional dentition. The invention also aims to provide, in at least one embodiment of the invention, a rapid and inexpensive method of automatic segmentation of a dental arch.

Disclosure of the invention To do this, the invention relates to a method of automatic segmentation of a dental arch, characterized in that it comprises:

- a step of acquiring a three-dimensional surface of the dental arch, to obtain a three-dimensional representation of the dental arch in a three-dimensional space, said three-dimensional representation comprising a set of Np three-dimensional points, called vertices, forming vertices of Nf polygonal, preferably triangular faces;

a step of generating M two-dimensional virtual views from the three-dimensional representation, comprising a step of determining the characteristics of the virtual views comprising a sub-step of determining a skeleton representing the general shape of the dental arch and a sub-step of step of determining the characteristics of the virtual views by selecting virtual views distributed along the skeleton and directed towards the skeleton;

a step of projecting the three-dimensional representation on each two-dimensional virtual view, configured to obtain, for each virtual view, an image representing each vertex and each polygonal face visible on this virtual view; a step of processing each image by a previously trained deep learning network, associating with each pixel of each image a probability vector of size N, each index of the vector representing the probability of belonging of said pixel to a class of tissues dental, among N classes of dental tissue; a step of reverse projection of each image so as to attribute to each vertex of the three-dimensional representation, a pixel for each image on which the vertex appears and to which it corresponds, and by assigning to each vertex the probability vector (s) associated with said or to said pixels;

a step of determining, for each vertex, the class of dental tissue to which said vertex most probably belongs from the probability vector (s) attributed to said vertex.

An automatic segmentation method according to the invention therefore allows automatic segmentation of the dental arch that is efficient, robust and does not require manual intervention during its execution.

The dental tissue classes are of different types depending on the practical application of automatic segmentation. At a minimum, one can for example distinguish the tissues corresponding to a gum and the tissues corresponding to a tooth, in general one distinguishes in particular the gum and each single tooth independently. Teeth are thus categorized independently by their unique dental numbering, for example the rating of the Fédération Dentaire Internationale (FDI World Dental Federation rating in English), the universal rating system (Universal Numbering System in English), the Palmer rating system, etc. Other types of tissue can be distinguished: for example, on one, several or each tooth, the tooth faces can be distinguished (e.g. lingual / platinum face, occlusal face, buccal face, etc. ). The segmentation can also include tissues corresponding to prosthetic material (, scan post, scan body, etc.). When the tissue is prosthetic material, the brand and / or model thereof can optionally be recognized, for example by its shape. Recognizing a prosthetic material model can allow a pre-recorded 3D representation of that model to be used in the three-dimensional representation of the dental arch.

The process can be used for a single dental arch (upper or lower), or for all dental arches.

The previously trained deep learning network is for example a network of the convolutional neural network (CNN) type, this type of network being particularly advantageous for processing two-dimensional images. The method using CNNs more particularly for image segmentation tasks, it uses a subclass of fully convolutional CNNs, fully convolutional FCN networks (FCNs for Fully Convolutional Network). This type of network is described in particular in the article: “Fully Convolutional Networks for Semantic Segmentation”, Long et al., 2014. More generally, deep learning networks are called Deep Learning Network in English or Deep Learning Neural Network for deep learning neural network.

The three-dimensional representation of the dental arch is easily obtained by acquisition of the three-dimensional surface, for example by an intraoral camera with structured light, or with passive light (stereoscopy). Instead of directly processing the three-dimensional representation, which requires a lot of resources, the method according to the invention uses two-dimensional virtual views of this three-dimensional representation. The use of two-dimensional virtual views greatly simplifies the modeling of the dental arch for its treatment, which in fact simplifies the treatment. Using a two-dimensional projective space instead of considering a three-dimensional space as a whole also makes it easier to use CNN-type deep learning networks. The determination of the characteristics of the virtual views via the sub-step of determining a skeleton representing the general shape of the dental arch makes it possible to ensure that all the vertices are visible in the virtual views, that enough vertices are visible in several virtual views and that the images obtained from the virtual views can be used by the deep learning network. The automatic segmentation process according to the invention is therefore optimized for the automatic segmentation of a dental arch by taking into account the skeleton of the dental arch for the generation of virtual views.

This determination of the characteristics can be carried out independently of the knowledge of the real spatial positions mentioned above. In addition, two-dimensional data management is less expensive in storage space, and requires less resource to process.

Using a multitude of two-dimensional images makes it possible to assign a plurality of probability vectors to a vertex. By combining the probabilities of several of these probability vectors, we obtain a more reliable determination of the dental tissue class to which the vertex belongs, which reduces the risk of errors in a classification of a vertex and therefore the need to resort to subsequent manual corrections.

The invention is therefore distinguished from automatic processing carried out directly on three-dimensional representations of the prior art, thanks to the use of two-dimensional data which allows a more robust, faster and inexpensive processing in terms of resources for a less favorable result. to local errors.

The steps of generating two-dimensional virtual views, of projecting the three-dimensional representation, of processing each image by a deep learning network, of reverse projection of each image, and of determining the class of dental tissue, are preferably implemented. implemented by a computing device, such as a computer or a plurality of computers, in particular by one or more computer programs executed on the computing device. More particularly, the method is preferably implemented in a module specialized in the processing of three-dimensional and two-dimensional graphics data, such as a graphics processor or GPU (for Graphics Processing Unit in English), executing a computer program specialized in the processing. three-dimensional and two-dimensional data, for example OpenGL.

The step of acquiring the surface of the arch is preferably implemented by the combination of an acquisition device, for example an intraoral camera, and an acquisition computer program processing the data acquired by the device. acquisition to obtain the three-dimensional representation of the dental arch. Examples of oral cameras that can be used are, for example, the “Wow ™” camera from Biotech Dental. Advantageously and according to the invention, the method comprises, prior to the step of processing each image by the learning network, a step of attributing to each pixel of each image at least one discriminatory value, said discriminatory value being a digital value representative of a characteristic of the vertex when said pixel corresponds to a vertex in the virtual view, and to an interpolation of the characteristics of the vertex vertices of the polygonal face when said pixel corresponds to a polygonal face in the virtual view.

According to this aspect of the invention, the discriminatory value is a numerical value attributed to each pixel which allows the composition of an image in which each pixel is representative of the vertex corresponding to this pixel or of the vertex vertices of a polygonal face corresponding to this pixel. The resulting image thus resembles a 2D matrix in which each pixel has a discriminatory value, and can be processed by the deep learning network.

Advantageously and according to the invention, the discriminatory value can be of a type of value selected from the following list of types of values:

- RGB value of the vertex, obtained during the acquisition of a three-dimensional surface;

- three-dimensional curvature value at the vertex;

- distance value between the vertex and an optical center of the virtual view onto which it is projected;

- an angle between a normal of the vertex and a sighting direction of the virtual view.

According to this aspect of the invention, the discriminatory value can be of different types, which makes it possible to obtain images of different types, for example an RGB image if the acquisition was carried out by an intraoral camera allowing reconstruction in color. , an image related to a depth map (case of the optical vertex-center distance), an image where the curvatures are visible, etc. This value can thus be inherent in the vertex independently of its relation to the virtual views (for example its three-dimensional curvature), or represent a relation between the vertex and the virtual view (for example the distance between the vertex and the optical center of the virtual view, or the angle between its normal and the direction of sight of the virtual view).

Other types of discriminatory values can be used depending on the device used for image acquisition, the desired dental treatment, the dental tissue classes to be identified during segmentation, etc.

Advantageously and according to the invention, several discriminatory values can be combined with one another, in order to produce two-dimensional virtual views comprising several channels according to a third dimension, each of these channels carrying information of a particular type of data (for example first channel for curvature, second channel for vertex - optical center distance, etc.).

Advantageously and according to the invention, each virtual view is defined by an optical center included in three-dimensional space, and by a shooting direction along a shooting axis.

According to this aspect of the invention, the virtual view is akin to a virtual camera view which may, in certain cases, correspond to a real camera view obtained in the step of acquiring the three-dimensional surface. More particularly, during the acquisition phase of the three-dimensional surface, the intraoral camera records, in addition to the three-dimensional reconstruction, the successive spatial positions of the camera relative to the three-dimensional surface. These positions can be taken advantage of during the segmentation calculation, by recreating the virtual views according to these same spatial positions. In this way, all of these virtual views cover the entire three-dimensional surface, since they are the ones that were used to generate it.

The optical center corresponds to the point of view, which is a point in three-dimensional space and corresponds to the location of the virtual camera associated with the virtual view.

Other characteristics can be associated with each virtual view, such as the field of view, height and width in pixels of the virtual view. generated, the intrinsic calibration matrix (focal length and optical center). These other characteristics will be chosen as fixed upstream of the design of the segmentation method described in this invention.

As an indication, we can choose the width and height of the virtual view (for example 1280 pixels * 960 pixels), decide for example that the virtual view must cover a planar surface of 3 * 2.25 centimeters located at 1 centimeter from its center optical, and place the optical center in the center of the image. These constraints make it easy to find the focal length of the calibration matrix.

Advantageously and according to the invention, the number of two-dimensional virtual views generated is between 30 and 90 views, preferably between 50 and 70 views.

According to this aspect of the invention, this number of virtual views makes it possible to sufficiently cover a conventional adult human dentition to obtain a robust result, while limiting the number of views to be processed by the deep learning network.

Advantageously and according to the invention, the step of determining, for each vertex, the class of dental tissue to which it most probably belongs comprises the execution of an algorithm of the graph cut type taking as a parameter for each vertex the or said probability vectors attributed to said vertex.

The graph cut algorithm is better known as graph-cut in English.

According to this aspect of the invention, this algorithm makes it possible not to consider only the probability vector (s) associated with a vertex to determine the class of dental tissue, but also to consider parameters originating from neighboring vertices in order to guarantee the absence. artefacts and point errors in determining dental tissue classes.

The invention also relates to an automatic segmentation device. of a dental arch, characterized in that it comprises:

a module for acquiring a three-dimensional surface of the dental arch, configured to obtain a three-dimensional representation of the dental arch in a three-dimensional space, said three-dimensional representation comprising a set of Np three-dimensional points, called vertices, forming vertices Nf polygonal, preferably triangular faces;

a module for generating M two-dimensional virtual views from the three-dimensional representation configured to determine characteristics of the virtual views by determining a skeleton representing the general shape of the dental arch and by determining the characteristics of the virtual views by the selection virtual views distributed along the skeleton and directed towards the skeleton;

a module for projecting the three-dimensional representation on each two-dimensional virtual view, configured to obtain, for each virtual view, an image representing each vertex and each polygonal face visible on the virtual view;

a module for processing each image by a previously trained deep learning network, associating with each pixel of each image a probability vector of size N, each index of the vector representing the probability of belonging of said pixel to a class of tissues dental, among N classes of dental tissue;

- a module for the inverse projection of each image so as to attribute to each vertex of the three-dimensional representation, one or more pixels of the images in which the vertex appears and to which it corresponds, and by assigning to each vertex the associated probability vector (s) said at least one pixel;

- a module for determining, for each vertex, the class of dental tissue to which said vertex most probably belongs to from the probability vector (s) attributed to said vertex.

Advantageously, the automatic segmentation method according to the invention is implemented by the automatic segmentation device according to the invention.

Advantageously, the automatic segmentation device according to the invention implements the automatic segmentation method according to the invention.

A module may consist, for example, of a computing device such as a computer, of a set of computing devices, of an electronic component or of a set of electronic components, or for example of a computer program, d 'a set of computer programs, a library of a computer program or a function of a computer program executed by a computing device such as a computer, a set of computing devices, an electronic component or a set of electronic components.

The invention also relates to a method of supervised learning of a deep learning network, characterized in that it comprises:

- a step of acquiring a plurality of three-dimensional surfaces of several dental arches, to obtain a plurality of three-dimensional learning representations of the dental arches in a three-dimensional space, said three-dimensional learning representations each comprising a set of Np three-dimensional points , called vertex, forming vertices of Nf polygonal faces, preferably triangular;

- a step of manual segmentation, by a human operator, of each three-dimensional learning representation of the dental arch, in which each vertex of the three-dimensional representation is assigned a class of dental tissue, so as to obtain a segmented three-dimensional representation for each three-dimensional representation learning;

a generation step, for each three-dimensional representation of M two-dimensional virtual views from the three-dimensional learning representation comprising a sub-step of determining a skeleton representing the general shape of the dental arch and a sub-step of determining the characteristics of the virtual views by selecting virtual views distributed along the skeleton and directed towards the skeleton;

- a projection step, for each three-dimensional training representation, of the discriminatory value or values chosen for the three-dimensional training representation on each two-dimensional virtual view, configured to obtain, for each virtual view, an input two-dimensional image representing in each pixel the discriminatory value of the vertex or of the polygonal face projected onto the virtual view;

a projection step, for each segmented three-dimensional representation, of the segmented three-dimensional representation on each two-dimensional virtual view, configured to obtain, for each virtual view, an output two-dimensional image representing in each pixel the class of dental tissue of the vertex or of the polygonal face projecting onto the output two-dimensional image;

a step of learning the deep learning network via processing of each pair of images comprising an input image and an output image respectively resulting from the projection with the same virtual view of the discriminatory value (s) for each three-dimensional learning representation and its associated segmented three-dimensional representation.

The deep learning process constitutes a preliminary step to the automatic segmentation carried out in the automatic segmentation process. according to the invention, and can thus be advantageously integrated into the automatic segmentation method as a preliminary step to the step of processing the images obtained via the virtual views. The supervised learning method makes it possible in particular to train the deep learning network used in the automatic segmentation method, in the step of processing the images obtained via the virtual views.

The learning network is thus trained beforehand by providing already segmented images from virtual views. This supervised learning using images from virtual views thus processes only two-dimensional data, which allows rapid and robust learning. The training network thus trained requires fewer resources, is faster and more efficient to perform processing on new non-segmented images, compared to a training network which would be trained only on three-dimensional data without the use of virtual views. and two-dimensional images. Although supervised learning involves a human operator, no manual intervention is required to intervene in the automatic segmentation process once the deep learning network is sufficiently trained.

The invention also relates to an automatic segmentation method, an automatic segmentation device and a method for supervised learning of a deep learning network, characterized in combination by all or part of the characteristics mentioned above or below.

List of Figures

Other aims, characteristics and advantages of the invention will become apparent on reading the following description, given only without limitation and which refers to the appended figures in which:

[Fig. 1] is a dental diagram showing the numbering of human teeth according to the notation system of the Fédération Dentaire Internationale

[Fig. 2a] is a schematic view of a three-dimensional representation of a dental arch obtained following a step of acquiring the surface. three-dimensional view of the dental arch by a segmentation method according to one embodiment of the invention;

[Fig. 2b] is a schematic view of a three-dimensional representation of a dental arch segmented by an automatic segmentation process according to the invention;

[Fig. 3] is a schematic view of a plurality of vertices and a face formed by said vertices of a three-dimensional representation of a dental arch;

[Fig. 4] is a schematic view of the steps of an automatic segmentation process according to the invention;

[Fig. 5a] is a schematic view of a step for determining the characteristics of the virtual views of an automatic segmentation method according to one embodiment of the invention;

[Fig. 5b] is a schematic view of a sub-step for determining the characteristics of the virtual views by the selection of virtual views distributed along the skeleton and directed towards the skeleton of an automatic segmentation method according to an embodiment of the invention;

[Fig. 6] is a schematic view of a method of supervised learning of a deep learning network according to one embodiment of the invention.

Detailed description of an embodiment of the invention

In the figures, the scales and proportions are not strictly observed, for purposes of illustration and clarity.

In addition, identical, similar or analogous elements are designated by the same references in all the figures.

FIG. 1 is a diagram showing an adult human dentition, in which each tooth is associated with its rating according to the rating of the Fédération Dentaire Internationale (FDI World Dental Federation rating in English). In this notation, each tooth is identified by two notation digits; The dentition is separated into four quadrants, and the quadrant in which the tooth is located corresponds to the first digit of the notation: quadrant 1 above left, quadrant 2 above right, quadrant 3 below right, quadrant 4 below left ("right" and “left” being understood from the point of view of a dentist observing the dentition of a patient). The second number of the notation indicates the corresponding tooth, in the quadrant, from 1 the central incisor to 8 the wisdom tooth.

There is also a rating from the International Dental Federation for temporary teeth, not detailed here.

The automatic segmentation process according to the invention makes it possible to distinguish each of these teeth from the dental arch and can assign each of these teeth the numbering shown.

FIG. 2a schematically represents a three-dimensional digital representation 100 obtained following a step of acquiring the three-dimensional surface of the dental arch by a segmentation process according to one embodiment of the invention.

The three-dimensional digital representation 100 is performed for example by an intraoral camera, using several different technologies impacting the three-dimensional digital representation and the characteristics of this three-dimensional digital representation: for example, the camera can obtain RGB data making it possible to identify colors, curvature data to identify the general shape of the dental arch, depth data by passive light stereoscopy or structured light, etc.

The three-dimensional digital representation 100 comprises a set of Np three-dimensional points, called vertices, forming the vertices of Nf polygonal faces, preferably triangular. This polygonal representation is common in three-dimensional surface management methods.

For illustration, FIG. 3 schematically represents a plurality of vertices 300a, 300b, 300c and a triangular face 302 formed by said vertices of a three-dimensional representation of a dental arch. The use of triangular faces is classic and makes it easy to identify each face from three vertices without additional parameters, but polygonal faces with more than three vertices can be used. Each vertex 300a, 300b, 300c is linked to the other two vertices to form face 302, and each vertex can be linked to others vertex via for example edges 304 (shown partially in dotted lines) to form other faces, not shown.

In connection with Figure 2a, Figure 2b illustrates the three-dimensional representation 200 of the dental arch after segmentation, in which a gray level has been associated with each triangular face whose vertex-vertices belong to the same class of dental tissue. Each identified dental tissue is represented in a shade of gray different from at least its neighbors, in order to show the distinction between two adjacent and different dental tissues. For example, are referenced in Figure 2b a premolar 202, two molars 204 and 210, an incisor 206 and a gum 208.

This representation has a mainly illustrative objective: in practice, the segmentation consists at least in assigning to each vertex a class of dental tissue, without requiring to provide a graphical representation in gray level or in color.

FIG. 4 schematically represents the steps of an automatic segmentation method according to one embodiment of the invention.

The method comprises in particular the steps described below.

A step 402 of acquiring a three-dimensional surface of the dental arch makes it possible to obtain a three-dimensional representation of the dental arch in a three-dimensional space, said three-dimensional representation comprising a set of Np three-dimensional points, called vertices, forming vertices of Nf polygonal, preferably triangular faces: this step makes it possible in particular to obtain a three-dimensional representation of the type of that shown with reference to FIG. 2a. Acquisition of the three-dimensional surface of the dental arch is usually done on a patient by a dentist using an intraoral camera, without surgery. The three-dimensional representation of the dental arch is then transmitted to the prosthetist who, from the three-dimensional representation, plans a particular treatment (eg making prostheses). In this context, the three-dimensional representation in the form of vertices and faces is common.

The method then comprises a step 404 for generating M views two-dimensional virtual images from the three-dimensional representation, the objective of which is to reproduce quantitative information from this three-dimensional representation in a two-dimensional projective space; information is easily represented in this space via two-dimensional images. The M virtual views correspond for example to a virtual camera directed towards different locations of the three-dimensional representation of the dental arch. In some cases, none or some or all of the virtual views may correspond to actual views acquired by the intraoral camera during the acquisition of the three-dimensional surface.

Virtual views can be defined by different characteristics, in particular a centroid defining an optical center, i.e. the point where the virtual camera is placed, and a shooting direction, i.e. the direction in which the virtual camera is directed to obtain the virtual view. The number M of virtual views and the characteristics of the virtual views are defined so as to make it possible to see all of the vertices of the three-dimensional representation, preferably several times for each vertex, that is to say that the set of virtual views covers the entire three-dimensional reconstruction.

The step 404 for generating M virtual views can include sub-steps (not shown) making it possible to obtain this number M of virtual views and the characteristics of each virtual view:

A first sub-step is a sub-step for determining a skeleton representing the general shape of the dental arch. A second substep is a substep for determining the characteristics of the virtual views by selecting virtual views distributed along the skeleton and directed towards the skeleton. The step 404 for generating M virtual views can thus be composed, in one of the embodiments, of the following substeps, described with reference to FIG. 5 a: a substep 502 of calculating a three-dimensional curvature of the three-dimensional representation 100; a sub-step 504 for detecting cusps of the dental arch in the three-dimensional representation, by thresholding the negative curvatures of the three-dimensional curvature; The dental arch cusps define the hollow portions of each tooth and are easily detected by detecting curvatures. a substep 506 for estimating a normal Z axis, representing the average of the normals at each cusp detected; The normal Z axis thus represents a Z axis normal to the plane formed by the dental arch, a substep 508 of defining a plane P orthogonal to the normal Z axis and comprising the barycenter G of the three-dimensional reconstruction; The plane P thus substantially represents the plane formed by the dental arch, a sub-step 510 of projecting the three-dimensional surface on the plane.

P; a sub-step 512 of projecting the vertices and faces of the three-dimensional representation on this plane, so as to generate a binary two-dimensional mask from this projection, representing in two-dimensional space a binary value corresponding to the projection of at minus one face of the dental arch, i.e. representing the areas belonging to the dental arch, or the absence of face projection of the dental arch, i.e. representing the zones not belonging to the dental arch a sub-step 514 for determining a skeleton S of the dental arch, from said two-dimensional mask, for example using a topological skeletonization algorithm of morphological type , the skeleton corresponding to an average curve of the zone where the binary value corresponds to the presence of the dental arch. This skeleton S is related to the three-dimensional representation, so as to form a three-dimensional skeleton that is to say characterized in the three-dimensional space of the three-dimensional representation. a sub-step 516 of determining the characteristics of the virtual views by selecting virtual views distributed along the skeleton and directed towards the skeleton. This sub-step can be carried out differently according to the embodiments, for example according to the morphology of the arch. studied and / or the number of views desired, but may for example consist of the following sub-steps, described with reference to FIG. 5b: a sub-step 520 of uniform distribution of three-dimensional anchoring points Vk along the skeleton S, for example starting from one of its ends and inserting a point every centimeter, o a sub-step 522 of creation of three-dimensional semicircles (for illustration, only two semicircles are represented) for each anchor point Vk, whose center is the anchor point Vk, whose radius is a few centimeters (typically 4 centimeters), and whose axis of rotation is defined as the three-dimensional orientation of the skeleton in the vicinity of the point anchoring Vk, o a sub-step 524 of uniform distribution of virtual views along each of these semicircles, for example by inserting the optical center Co of a new virtual view every millimeter and by defining the direction of taking d e view of this virtual view starting from this new point of view and looking towards the anchor point Vk (for illustration, only five optical centers are represented for each of the two semi-circles represented). These characteristics therefore make it possible to obtain M shots.

Again with reference to FIG. 4, the automatic segmentation method then comprises a step 406 of projecting the three-dimensional representation on each two-dimensional virtual view, configured to obtain, for each virtual view, an image representing each vertex and each visible polygonal face on the virtual view: The projection makes it possible to make the vertices visible in the virtual view corresponding to the image correspond to the image. The projection uses for example a ray casting process to make each pixel correspond to a vertex or a face of the three-dimensional representation. The image obtained is composed of pixels having an assigned discriminatory value representative of the vertex or of the polygonal face to which the pixel corresponds. This discriminatory value is a numerical value which can be representative of the vertex, and depend on the characteristics attributed to the vertex during the acquisition of the three-dimensional representation or during subsequent calculations. For example, if the camera is an RGB camera, the discriminatory value includes a triplet of RGB value (the triplet can be represented, in a known manner, by a single discriminatory value, for example FFFFFF for white by its hexadecimal RGB representation), or each RGB value is included in a different channel. The value may in other cases represent a relation between the vertex and the virtual view (for example the distance between the vertex and the optical center of the virtual view, or the angle between its normal and the direction of sight of the view. Virtual). The discriminatory value can also be representative of the depth with respect to the virtual camera forming the virtual view; in other words, it can represent the distance between the vertex considered and the optical center of the virtual view. Other data may form the discriminatory value, for example the three-dimensional curvature, obtained from the three-dimensional representation.

In addition, several discriminatory values can be used for image formation, each of the discriminatory values being stored in a channel

Following the projection, when the pixel does not correspond to a vertex but to a triangular face, then the discriminatory value of the pixel is based on an interpolation of the value of the three vertices forming the vertices of the face (for example a linear interpolation) .

Each image, in which each pixel has an assigned discriminatory value, is processed in a step 408 of processing each image by a previously trained deep learning network, associating with each pixel of each image a probability vector of size N, each index of the vector representing the probability of belonging of said pixel to a class of dental tissues, among N classes of dental tissues. The deep learning network, or deep learning neural network, is previously trained according to a supervised learning method described below with reference to FIG. 6. The deep learning network could assign to each pixel of the image a class of dental tissue, which would be the class with the highest probability of belonging. However, better results and a reduction in errors are allowed in the following steps by using a probability vector of the pixel belonging to a dental tissue class.

The probability vector is of size N, corresponding to the N predetermined dental tissue classes which can be attributed to each pixel. These N classes can correspond to the gum, to a particular tooth identified by its dental notation, to a prosthetic material, etc.

The method then comprises a step 410 of reverse projection of each image so as to attribute to each vertex of the three-dimensional representation, one or more pixels of the images in which the vertex appears and to which it corresponds, and by assigning to each vertex the one or more probability vectors associated with said pixel or pixels.

The reverse projection makes it possible to return to the three-dimensional representation after the passage through the two-dimensional images. The link between a pixel and a vertex, which has already been established in the projection step, is preferably recalculated to avoid having to store the link between each vertex and its projection (s) in each image which may require storage space. important and which does not necessarily allow faster processing than a recalculation.

Each vertex is assigned a probability vector if it is visible only in a virtual image, and otherwise to as many probability vectors as the images in which it was projected. Since virtual views have been set up so that each vertex is projected onto an image, no vertex should have any probability vector assigned to it. Preferably, each vertex has several probability vectors assigned to it in order to maximize the chances of identifying the correct class of dental tissue.

The method finally comprises a step 412 of determination, for each vertex, of the class of dental tissue to which said vertex most probably belongs from the probability vector (s) attributed to said vertex.

This allocation to each vertex of the associated class corresponds to automatic segmentation. As already described with reference to FIG. 2b, the segmentation makes it possible to obtain three-dimensional representations differentiating, for example by gray levels or colors, each element belonging to the same dental tissue, but in practice the segmentation consists only in attributing to each vertex has a class of dental tissue and this data can be used as is.

The dental tissue class used could simply be the one with the highest probability by averaging the set of probability vectors assigned to the vertex. However, to avoid artefacts and local errors, it is preferable to use a graph-cut or graph-cut type combination algorithm in English, which also allows the probability vectors of neighboring vertices to be taken into account.

The aim of the graph-cut algorithm is to assign for each vertex the class having a high probability, while promoting local class homogeneity. Indeed, on the dental arch, there is a high probability that the neighboring vertices have the same class, except if they are separated by a zone of high curvature. For example, two neighboring vertices on the same molar have the same class, and the local curvature between them is small (a tooth is relatively smooth). Conversely, a vertex on a molar and a vertex on the gum (therefore belonging to two different classes) are separated by a zone of high spatial curvature (the insertion of the tooth into the gum creating a spatial “break”). To account for this phenomenon, the graph used can take as a parameter a unit term, for each vertex, the mean probability vector Vp: in this way, it will try to maximize the class probability. The graph can take as binary term (ie connecting two neighboring vertices) the spatial curvature separating these two vertices, or the scalar product between their respective normal. In this way, the graph will try to respect the spatial homogeneity of class as well as possible except when crossing areas of high curvature, while trying to each vertex to maximize the probability according to the vector Vp.

Figure 6 schematically shows a method 600 of supervised learning of a deep learning network according to one embodiment of the invention. The supervised learning method is applied to a CNN convolutional neuron network or more particularly to a fully convolutional FCN network. The deep learning network is trained by providing input two-dimensional images such as will be provided to it by the automatic segmentation process described above. The learning is supervised, that is to say that the deep learning network is also supplied, during its training, with the output two-dimensional images associated with the input two-dimensional images in which the tissue classes are assigned. pixels, that is, the output images are segmented.

To do this, the process comprises the following steps:

a step 602 of acquiring a plurality of three-dimensional surfaces of several dental arches, to obtain a plurality of three-dimensional learning representations of the dental arches in a three-dimensional space, said three-dimensional learning representations each comprising a set of Np points three-dimensional, called vertex, forming vertices of Nf polygonal faces, preferably triangular;

a step 604 of manual segmentation, by a human operator, of each three-dimensional learning representation of the dental arch, in which each vertex of the three-dimensional representation is assigned a class of dental tissue, so as to obtain a three-dimensional representation segmented for each three-dimensional learning representation;

a step 606 of generation, for each three-dimensional representation of M two-dimensional virtual views from the three-dimensional training representation; For each three-dimensional training representation, M two-dimensional virtual views are generated in the same way as in the automatic segmentation method, as described previously with reference to FIGS. 4, 5a and 5b.

a step 608 of projection, for each three-dimensional training representation, of the discriminatory value or values chosen for the three-dimensional training representation on each two-dimensional virtual view, configured to obtain, for each virtual view, an input two-dimensional image representing in each pixel the discriminatory value of the vertex or of the polygonal face projected onto the virtual view; this step generates the data that will be supplied to the input of the learning network, which are input images 620;

a step 610 of projecting, for each segmented three-dimensional representation, the segmented three-dimensional representation on each two-dimensional virtual view, configured to obtain, for each virtual view, an output two-dimensional image representing in each pixel the class of dental tissue of the vertex, or of the polygonal face projecting onto the output two-dimensional image; this step generates the data expected at the output of the learning network which are output images 622;

a step 612 of learning the deep learning network 630 via processing of each pair 624a, 624b, 624c of images comprising an input image and an output image respectively resulting from the projection with the same virtual view of each three-dimensional learning representation and its associated segmented three-dimensional representation. The deep learning network 630 is thus trained by the pairs 624a; 624b, 624c of images, knowing the input and output expected from the segmentation process.

Once the learning network has been sufficiently trained, the process of Auto segmentation can perform auto segmentation without manual intervention.

Claims

1. A method of automatic segmentation of a dental arch, characterized in that it comprises:

- a step (402) of acquiring a three-dimensional surface of the dental arch, to obtain a three-dimensional representation of the dental arch in a three-dimensional space, said three-dimensional representation comprising a set of Np three-dimensional points, called vertex, forming vertices of Nf polygonal faces, preferably triangular;

- a step (404) for generating M two-dimensional virtual views from the three-dimensional representation, comprising a step of determining the characteristics of the virtual views comprising a sub-step of determining a skeleton representing the general shape of the dental arch and a sub-step of determining the characteristics of the virtual views by selecting virtual views distributed along the skeleton and directed towards the skeleton;

a step (406) of projecting the three-dimensional representation on each two-dimensional virtual view, configured to obtain, for each virtual view, an image representing each vertex and each polygonal face visible on this virtual view;

a step (408) of processing each image by a previously trained deep learning network, associating with each pixel of each image a probability vector of size N, each index of the vector representing the probability of belonging of said pixel to a dental tissue class, among N dental tissue classes;

- a step (410) of reverse projection of each image so as to attribute to each vertex of the three-dimensional representation, a pixel for each image on which the vertex appears and to which it corresponds, and by assigning to each vertex the vector or vectors of probability associated with said pixel (s);

a step (412) of determining, for each vertex, the class of dental tissue to which said vertex most probably belongs from the probability vector (s) attributed to said vertex.

2. A method of automatic segmentation of a dental arch according to claim 1, characterized in that it comprises, prior to the step of processing each image by the learning network, a step of attributing to each pixel of each image of at least one discriminatory value, said discriminatory value being representative of a characteristic of the vertex when said pixel corresponds to a vertex on the virtual view, and to an interpolation of the characteristics of the vertices and vertices of the polygonal face when said pixel corresponds to a polygonal face on the virtual view.

3. Automatic segmentation method according to claim 2, characterized in that the discriminatory value can be of a type of value selected from the following list of types of values:

- three-dimensional curvature value at the vertex;

4. A method of automatic segmentation of a dental arch according to one of claims 1 to 3, characterized in that each virtual view is defined by an optical center (Co) included in the three-dimensional space, and by a direction of engagement. view along a shooting axis.

5. A method of automatic segmentation of a dental arch according to one of claims 1 to 4, characterized in that the number of two-dimensional virtual views generated is between 30 and 90 views, preferably between 50 and 70 views.

6. A method of automatic segmentation of a dental arch according to one of claims 1 to 5, characterized in that the step of determining, for each vertex, the class of dental tissue to which it most probably belongs comprises l '' execution of a graph cut-type algorithm taking as a parameter for each vertex the one or more probability vectors attributed to said vertex.

7. Device for automatic segmentation of a dental arch, characterized in that it comprises:

a module for determining, for each vertex, the class of dental tissue to which said vertex most probably belongs from the one or more probability vectors attributed to said vertex.

8. A method of supervised learning of a deep learning network (630), characterized in that it comprises:

- a step of manual segmentation, by a human operator, of each three-dimensional learning representation of the dental arch, in which each vertex of the three-dimensional representation is assigned a class of dental tissue, so as to obtain a segmented three-dimensional representation for each three-dimensional learning representation;

- a projection step, for each segmented three-dimensional representation, of the segmented three-dimensional representation on each two-dimensional virtual view, configured to obtain, for each virtual view, an output two-dimensional image representing in each pixel the dental tissue class of the vertex or the polygonal face projecting on the output two-dimensional image;