WO2022262201A1

WO2022262201A1 - Facial three-dimensional model visualization method and apparatus, electronic device, and storage medium

Info

Publication number: WO2022262201A1
Application number: PCT/CN2021/132548
Authority: WO
Inventors: 罗家祯; 陈福兴; 刘兴云; 李志阳; 齐子铭
Original assignee: 厦门美图宜肤科技有限公司
Priority date: 2021-06-18
Filing date: 2021-11-23
Publication date: 2022-12-22
Also published as: JP7433444B2; KR102622215B1; JP2023534327A; CN113345079A; KR20220169471A; CN113345079B

Abstract

The present application relates to the technical field of three-dimensional models, and provides a facial three-dimensional model visualization method and apparatus, an electronic device, and a storage medium. The method comprises: obtaining a first facial three-dimensional model and a second facial three-dimensional model of a tested person at two different moments, the first facial three-dimensional model and the second facial three-dimensional model being triangular patch formats; calculating the distance between coordinate points of the first facial three-dimensional model and the second facial three-dimensional model; rendering the first facial three-dimensional model according to the distance to obtain a thermodynamic diagram model; and respectively calculating three-dimensional contour lines according to the first facial three-dimensional model and the second facial three-dimensional model to obtain a first contour line model and a second contour line model. According to the present application, the change in the face can be visually displayed.

Description

Facial three-dimensional model visualization method, device, electronic equipment and storage medium

Cross References to Related Applications

This application claims the priority of the Chinese patent application with the application number 2021106824522 and titled "Facial 3D Model Visualization Method, Device, Electronic Equipment, and Storage Medium" filed with the China Patent Office on June 18, 2021, the entire contents of which are incorporated by reference incorporated in this application.

technical field

The present application relates to the technical field of three-dimensional models, and in particular, relates to a visualization method, device, electronic equipment and storage medium of a three-dimensional facial model.

Background technique

In the medical aesthetics industry, accurate measurement, comparison and visualization of facial 3D models play a very important role in evaluating treatment effects and allowing patients to intuitively observe their own changes before and after treatment.

In the existing 3D facial modeling and measurement schemes, it is mainly to measure a single 3D facial model, so as to analyze the changes of the face through data comparison, such as eye width, nose height, nose bridge angle, etc.

However, the comparative analysis of data cannot intuitively allow patients to observe the changes before and after treatment, and some medical cosmetic items only have some small deformations, and the data may not reflect the difference at all.

Contents of the invention

The present application provides a method, device, electronic device and storage medium for visualizing a three-dimensional facial model, so as to intuitively display changes in the face, thereby solving the above-mentioned deficiencies in the prior art.

Some embodiments of the present application provide a method for visualizing a three-dimensional facial model, which may include:

Obtaining the first three-dimensional facial model and the second three-dimensional facial model of the subject at two different moments; the first three-dimensional facial model and the second three-dimensional facial model are in triangular patch format;

calculating the distance between the coordinate points in the first three-dimensional facial model and the second three-dimensional facial model;

Rendering the first facial three-dimensional model according to the distance to obtain a heat map model;

The three-dimensional contour lines are respectively calculated according to the first three-dimensional facial model and the second three-dimensional facial model to obtain a first contour line model and a second contour line model.

Optionally, the acquisition of the first three-dimensional facial model and the second three-dimensional facial model of the subject at two different moments includes:

Obtaining the first initial facial three-dimensional model and the second facial three-dimensional model of the subject at the two different moments;

calculating a pose transformation matrix of the first initial facial three-dimensional model and the second facial three-dimensional model;

Aligning the first initial facial three-dimensional model to the second facial three-dimensional model according to the pose transformation matrix to obtain the first facial three-dimensional model.

Optionally, the calculating the pose transformation matrix of the first initial facial three-dimensional model and the second facial three-dimensional model may include:

Taking the position of the tip of the nose of the first initial facial three-dimensional model as the center of the sphere, intercepting the first initial three-dimensional facial model with a preset radius to obtain a first sphere interception model corresponding to the first initial three-dimensional facial model;

Taking the position of the tip of the nose of the second facial three-dimensional model as the center of the sphere, intercepting the second facial three-dimensional model with the preset radius, to obtain a second sphere interception model corresponding to the second facial three-dimensional model;

The attitude transformation matrix is calculated and acquired by using a preset transformation matrix calculation method, and the first sphere interception model and the second sphere interception model.

Optionally, the calculating the distance between the coordinate points in the first three-dimensional facial model and the second three-dimensional facial model may include:

According to the coordinates of each vertex in the first 3D facial model, calculate and acquire the distance from each vertex in the 1st 3D facial model to the closest triangular patch of the second 3D facial model.

Optionally, according to the coordinates of each vertex in the first three-dimensional facial model, calculate and acquire the distance between each vertex in the first three-dimensional facial model and the nearest triangle surface of the second three-dimensional facial model distance, which can include:

Calculating the minimum common cube bounding box of the first facial three-dimensional model and the second facial three-dimensional model;

Establishing an octree structure for the minimum common cube bounding box of the first facial three-dimensional model;

Divide the minimum common cube bounding box of the second facial three-dimensional model equally with the depth of the octree structure to obtain a preset number of small cubes;

Associating the triangular patch of the second facial three-dimensional model with the small cube;

According to the index relationship between the vertex coordinates of the first facial three-dimensional model and the small cube, associate the vertices of the first facial three-dimensional model with the triangular patches in the small cube;

According to the distance between the vertex coordinates and the associated triangular patch, the distance from each vertex in the first facial 3D model to the closest triangular patch of the second facial 3D model is obtained.

Optionally, according to the index relationship between the vertex coordinates of the first three-dimensional facial model and the small cube, associating the vertices of the first three-dimensional facial model with the triangular patches in the small cube, Can include:

calculating the vertex codes of the first facial three-dimensional model according to the octree structure;

According to the vertex code and the index relationship of the small cube, determine if the triangle patch in the small cube associated with the vertex is not an empty set, then associate the vertex with the triangle patch in the small cube .

Optionally, the method may also include:

If the triangular faces in the small cube associated with the vertex are an empty set, take the small cube as the center, and judge whether the triangular faces in other small cubes in the neighborhood of the preset radius of the small cube are empty set;

If the triangle faces in the other small cubes are not an empty set, associate the vertex with the triangle faces in the other small cubes.

Optionally, rendering the first facial three-dimensional model according to the distance to obtain a heat map model may include:

Perform color rendering on any region of the first facial three-dimensional model according to the mapping relationship between the distance and the color table to obtain the heat map model.

Optionally, the calculating the three-dimensional contours respectively according to the first facial three-dimensional model and the second facial three-dimensional model to obtain the first contour model and the second contour model may include:

calculating a first height map of the first facial 3D model and a second height map of the second facial 3D model;

calculating a first two-dimensional contour line of the first facial three-dimensional model according to the first height map;

calculating a second two-dimensional contour line of the second facial three-dimensional model according to the second height map;

converting the first two-dimensional contour to a first three-dimensional contour, and converting the second two-dimensional contour to a second three-dimensional contour;

Rendering the first facial three-dimensional model according to the first three-dimensional contour line to obtain the first contour line model;

The second facial three-dimensional model is rendered according to the second three-dimensional contour line to obtain the second contour line model.

Other embodiments of the present application also provide a three-dimensional facial model visualization device, and the three-dimensional facial model visualization device may include:

The three-dimensional model acquisition module can be configured to acquire the first three-dimensional facial model and the second three-dimensional facial model of the subject at two different moments; the first three-dimensional facial model and the second three-dimensional facial model are Triangular patch format;

A distance calculation module, which may be configured to calculate the distance between coordinate points in the first three-dimensional facial model and the second three-dimensional facial model;

The heat map visualization module may be configured to render the first facial three-dimensional model according to the distance to obtain a heat map model;

The contour line visualization module may be configured to calculate three-dimensional contour lines respectively according to the first three-dimensional facial model and the second three-dimensional facial model to obtain a first contour line model and a second contour line model.

Optionally, the 3D model acquisition module may include:

A three-dimensional model acquisition unit may be configured to acquire the first initial facial three-dimensional model and the second facial three-dimensional model of the subject at the two different moments;

A pose transformation matrix calculation unit may be configured to calculate a pose transformation matrix between the first initial three-dimensional facial model and the second three-dimensional facial model;

The alignment unit may be configured to align the first initial facial three-dimensional model to the second facial three-dimensional model according to the pose transformation matrix to obtain the first facial three-dimensional model.

Optionally, the attitude transformation matrix calculation unit may include:

The first sphere interception subunit may be configured to intercept the first initial facial three-dimensional model with a preset radius with the nose tip position of the first initial facial three-dimensional model as the center of the sphere, to obtain the first initial A first sphere interception model corresponding to the facial three-dimensional model;

The second sphere intercepting subunit can be configured to use the position of the tip of the nose of the second facial three-dimensional model as the center of the sphere, and intercept the second facial three-dimensional model with the preset radius to obtain the second facial A second sphere interception model corresponding to the three-dimensional model;

The attitude transformation matrix calculation subunit may be configured to calculate and obtain the attitude transformation matrix by using a preset transformation matrix calculation method, and the first sphere interception model and the second sphere interception model.

Optionally, the distance calculation module may be configured to calculate and obtain the distance from each vertex in the first 3D facial model to the second vertex according to the coordinates of each vertex in the first 3D facial model. The distance of the nearest triangle of the facial 3D model.

Optionally, the distance calculation module may include:

A bounding box calculation unit may be configured to calculate the smallest common cube bounding box of the first facial three-dimensional model and the second facial three-dimensional model;

An octree building unit may be configured to build an octree structure for the minimum common cube bounding box of the first facial three-dimensional model;

The equalizing unit may be configured to equally divide the minimum common cube bounding box of the second facial three-dimensional model with the depth of the octree structure to obtain a preset number of small cubes;

The first associating unit may be configured to associate the triangular patch of the second facial three-dimensional model with the small cube;

The second associating unit may be configured to associate the vertices of the first three-dimensional facial model with the triangles in the small cube according to the index relationship between the coordinates of the vertices of the first three-dimensional facial model and the small cube. patch association;

The distance calculation unit may be configured to obtain the distance between each vertex in the first facial three-dimensional model and the nearest triangle surface of the second facial three-dimensional model according to the distance between the vertex coordinates and the associated triangle surface. distance.

Optionally, the second association unit may include:

The encoding calculation subunit may be configured to calculate the vertex encoding of the first facial three-dimensional model according to the octree structure;

The second associating subunit can be configured to, according to the vertex code and the index relationship of the small cube, determine that if the triangle patch in the small cube associated with the vertex is not an empty set, then associate the vertex with the Describe the association of triangular patches in the small cube.

Optionally, the device may also include:

The neighborhood judging unit may be configured to: if the triangular patch in the small cube associated with the vertex is an empty set, then use the small cube as the center to judge other small cubes in the neighborhood of the preset radius of the small cube Whether the triangles in the cube are an empty set;

The third associating subunit may be configured to: if the triangle faces in the other small cubes are not an empty set, associate the vertex with the triangle faces in the other small cubes.

Optionally, the heat map visualization module may be configured to perform color rendering on any region of the first facial three-dimensional model according to the mapping relationship between the distance and the color table, to obtain the heat map model.

Optionally, the contour visualization module may include:

a height map calculation unit configured to calculate a first height map of the first facial three-dimensional model and a second height map of the second facial three-dimensional model;

A first two-dimensional contour calculation unit may be configured to calculate a first two-dimensional contour of the first facial three-dimensional model according to the first height map;

A second two-dimensional contour calculation unit may be configured to calculate a second two-dimensional contour of the second facial three-dimensional model according to the second height map;

The conversion unit may be configured to convert the first two-dimensional contour line into a first three-dimensional contour line, and convert the second two-dimensional contour line into a second three-dimensional contour line;

The first contour line visualization unit may be configured to render the first facial three-dimensional model according to the first three-dimensional contour line to obtain the first contour line model;

The second contour line visualization unit may be configured to render the second facial three-dimensional model according to the second three-dimensional contour line to obtain the second contour line model.

Some other embodiments of the present application also provide an electronic device, which may include: a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the electronic device During operation, the processor communicates with the storage medium through a bus, and the processor executes the program instructions to execute the steps of the facial three-dimensional model visualization method described in any one of the above embodiments.

Further embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the facial three-dimensional model as described in any one of the above-mentioned embodiments is executed. Steps of the visualization method.

The beneficial effects of the application are at least:

The present application provides a method, device, electronic device and storage medium for visualizing a three-dimensional facial model, by obtaining the first three-dimensional facial model and the second three-dimensional facial model of the subject at two different moments; the first three-dimensional facial model and the second facial three-dimensional model in triangular patch format; calculating the distance between the coordinate points in the first facial three-dimensional model and the second facial three-dimensional model; rendering the first facial three-dimensional model according to the distance to obtain a heat map model; The three-dimensional contour lines are respectively calculated according to the first three-dimensional facial model and the second three-dimensional facial model to obtain a first contour line model and a second contour line model. Through the scheme provided by this application, the facial depression or protrusion of the first three-dimensional facial model relative to the second three-dimensional facial model can be characterized according to the distance between the coordinate points of the first three-dimensional facial model and the second three-dimensional facial model, and Through the heat map model, the difference between the two models' facial depressions or protrusions can be intuitively displayed, and the deformation and movement of the subject's face can be intuitively understood through the first contour model and the second contour model, so as to realize facial changes visualization.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, so It should be regarded as a limitation on the scope, and those skilled in the art can also obtain other related drawings based on these drawings without creative work.

Fig. 1 is a schematic flow chart of the first facial three-dimensional model visualization method provided by the embodiment of the present application;

FIG. 2 is a schematic flow diagram of a second facial three-dimensional model visualization method provided by an embodiment of the present application;

FIG. 3 is a schematic flow diagram of a third facial three-dimensional model visualization method provided in an embodiment of the present application;

FIG. 4 is a schematic flow diagram of a fourth facial three-dimensional model visualization method provided in an embodiment of the present application;

FIG. 5 is a schematic flow diagram of a fifth facial three-dimensional model visualization method provided in an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a neighborhood small cube provided by an embodiment of the present application;

FIG. 7 is a schematic flow diagram of a sixth facial three-dimensional model visualization method provided by the embodiment of the present application;

FIG. 8 is a schematic structural diagram of a facial three-dimensional model visualization device provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of an electronic device provided by an embodiment of the present application.

detailed description

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of this application, not all of them.

Accordingly, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the application. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.

In the description of this application, it should be noted that if the orientation or positional relationship indicated by the terms "upper", "lower", etc. appear, it is based on the orientation or positional relationship shown in the drawings, or is the usual practice when using the product of this application. The orientation or positional relationship of placement is only for the convenience of describing the application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be interpreted as a reference to this application. limits.

In addition, the terms "first", "second" and the like in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

It should be noted that, in the case of no conflict, the features in the embodiments of the present application may be combined with each other.

The facial three-dimensional model visualization method provided by the embodiment of the present application is executed by an electronic device with a facial three-dimensional model visualization function. The electronic device connected by device communication receives the facial three-dimensional model sent by the modeling device. The electronic device may be a beauty instrument, or may be a smart phone, a tablet computer, a server, a desktop computer, a notebook computer, etc., which is not limited here.

Fig. 1 is a schematic flow chart of the first facial three-dimensional model visualization method provided by the embodiment of the present application; as shown in Fig. 1, the method may include:

S10: Acquiring the first three-dimensional facial model and the second three-dimensional facial model of the subject at two different moments.

Specifically, the first three-dimensional facial model and the second three-dimensional facial model are facial three-dimensional models of the same subject at two different moments, and both the first three-dimensional facial model and the second three-dimensional facial model can be acquired by a three-dimensional scanning device. Furthermore, the first three-dimensional facial model is obtained through a three-dimensional scanning model, and the second three-dimensional facial model is a professional, such as a plastic surgeon, who simulates plastic surgery on the first three-dimensional facial model. By adjusting the first three-dimensional facial model derived from the facial structure. To facilitate subsequent calculations, it is necessary to ensure that the first facial 3D model and the second facial 3D model are in the triangular patch format, which means that the facial 3D model is composed of multiple triangular surfaces.

It should be noted that the first three-dimensional facial model and the second three-dimensional facial model are high-precision, high-resolution three-dimensional models with texture maps.

S20: Calculate the distance between the coordinate points in the first three-dimensional facial model and the second three-dimensional facial model.

Specifically, calculating the distance between the coordinate points in the first three-dimensional facial model and the second three-dimensional facial model may include: calculating the distance from the vertices of the first three-dimensional facial model to the second three-dimensional facial model and calculating the second three-dimensional facial model The distance from the vertices of to the first facial 3D model.

Take the calculation of the distance from the vertices of the first facial 3D model to the second facial 3D model as an example:

The vertices of the first facial three-dimensional model are the key points for establishing the first facial three-dimensional model, or the triangle vertices of the triangular patch in the first facial three-dimensional model, and the vertices of the first facial three-dimensional model are calculated to the second facial three-dimensional model The process of the distance is: calculate the shortest distance from the vertex of the first facial 3D model to the second facial 3D model. In an optional calculation method, you can calculate the distance from each vertex of the first facial 3D model to the second The distance of all vertices of the face 3D model to determine the closest distance.

In an optional implementation manner, according to the coordinates of each vertex in the first three-dimensional facial model, the distance from each vertex in the first three-dimensional facial model to the nearest triangle patch of the second three-dimensional facial model is calculated and acquired.

Specifically, since the first facial 3D model and the second facial 3D model are high-precision models with a very large number of vertices, it is possible to choose to calculate the nearest triangle surface from the vertices in the first facial 3D model to the second facial 3D model distance. Wherein, by calculating the distance from each vertex to all triangular patches, the distance to the nearest triangular patch to the vertex can be determined as the first distance, and the calculation method of the point-to-surface distance will not be described in detail here.

S30: Render the first facial three-dimensional model according to the distance to obtain a heat map model.

Specifically, the heat map model is used to display the muscle condition of any region of the subject's face. According to the directed distance between the coordinate points of the first three-dimensional facial model and the second three-dimensional facial model, the directed distance is positive, indicating that The first three-dimensional facial model is convex relative to the second three-dimensional facial model at this coordinate point, and the directional distance is negative, indicating that the first three-dimensional facial model is concave relative to the second three-dimensional facial model at this coordinate point, and the calculation The color value of each coordinate point in the area where the first facial three-dimensional model is located, and the first facial three-dimensional model is rendered according to the color value to obtain a heat map model.

S40: Calculating three-dimensional contour lines respectively according to the first three-dimensional facial model and the second three-dimensional facial model, to obtain a first contour line model and a second contour line model.

Specifically, the contour line model is used to show the trend of any area of the subject's face, and the three-dimensional contour line of the first three-dimensional face model is determined according to the height value of each vertex of the first three-dimensional face model in the preset height direction , rendering the 3D contour line of the first facial 3D model on the first facial 3D model to obtain the first contour line model.

The calculation process of the second contour model is the same as that of the first contour model, and will not be repeated here.

The facial three-dimensional model visualization method provided in the embodiment of the present application obtains the first three-dimensional facial model and the second three-dimensional facial model of the subject at two different moments; the first three-dimensional facial model and the second three-dimensional facial model are triangular surfaces slice format; calculate the distance between the coordinate points in the first facial three-dimensional model and the second facial three-dimensional model; render the first facial three-dimensional model according to the distance, and obtain a heat map model; according to the first facial three-dimensional model and the second facial three-dimensional model The three-dimensional contour lines of the two facial 3D models are calculated respectively to obtain the first contour line model and the second contour line model. Through the method provided by the embodiment of the present application, the facial depression or protrusion of the first 3D facial model relative to the second 3D facial model can be characterized according to the distance between the coordinate points of the first 3D facial model and the second 3D facial model , and intuitively display the difference between the two models' facial depression or protrusion through the heat map model, and intuitively understand the deformation and movement of the subject's face through the first contour model and the second contour model, so as to realize the Visualization of facial changes.

On the basis of the above-mentioned embodiments, the embodiment of the present application also provides a method for visualizing a three-dimensional facial model. FIG. 2 is a schematic flow chart of the second method for visualizing a three-dimensional facial model provided in the embodiment of the present application. As shown in FIG. 2 , The above S10 may include:

S11: Acquire the first initial three-dimensional facial model and the second three-dimensional facial model of the subject at the two different moments.

Specifically, the acquisition method of the first initial facial three-dimensional model and the second facial three-dimensional model is as described above in S10, and will not be described in detail here. Since the angles and facial movements of the facial three-dimensional models acquired at different times are different, it is necessary to One of the three-dimensional facial models is corrected, and the solution of the embodiment of the present application is to correct the first initial three-dimensional facial model.

S12: Calculate a pose transformation matrix of the first initial three-dimensional facial model and the second three-dimensional facial model.

Specifically, perform rotation and translation transformation on the first initial facial 3D model, align the first initial facial 3D model to the second facial 3D model, and use a preset transformation method to calculate the attitude transformation matrix for rotation and translation transformation

S13: Align the first initial facial three-dimensional model to the second facial three-dimensional model according to the pose transformation matrix, to obtain the first facial three-dimensional model.

Specifically, the attitude transformation matrix

Correction may be performed on the first initial three-dimensional facial model, so as to align the first initial three-dimensional facial model in the coordinate system where the second three-dimensional facial model is located, to obtain a corrected first three-dimensional facial model. For example, the alignment formula is:

The facial three-dimensional model visualization method provided in the embodiment of the present application obtains the first initial facial three-dimensional model and the second facial three-dimensional model of the subject at the two different moments, and calculates the first initial facial three-dimensional model and the second facial three-dimensional model The attitude transformation matrix of the first facial three-dimensional model is aligned to the second facial three-dimensional model according to the attitude transformation matrix to obtain the first facial three-dimensional model. Through the method provided in the embodiment of the present application, the first initial facial 3D model can be aligned with the second facial 3D model, avoiding the inclination deviation between the facial 3D models at different times, resulting in the thermal map model and the contour model When the visual comparison is performed, there is a deviation in the result, which improves the accuracy of the comparison result.

On the basis of the above-mentioned embodiments, the embodiment of the present application also provides a method for visualizing a three-dimensional facial model. FIG. 3 is a schematic flowchart of the third method for visualizing a three-dimensional facial model provided in the embodiment of the present application. As shown in FIG. 3 , The above S12 may include:

S121: Taking the position of the tip of the nose of the first initial facial three-dimensional model as the center of the sphere, intercepting the first initial three-dimensional facial model with a preset radius to obtain a first sphere interception model corresponding to the first initial three-dimensional facial model.

Specifically, since the cheek regions on both sides of the facial 3D model contribute little to the calculation of the pose transformation matrix, in order to reduce the amount of calculation and improve the calculation efficiency, the position of the tip of the nose can be used as the center of the sphere, and the first initial facial 3D model can be calculated with a preset radius Perform interception to obtain the first sphere interception model.

S122: Take the position of the tip of the nose of the second facial three-dimensional model as the center of the sphere, intercept the second facial three-dimensional model with a preset radius, and obtain a second spherical intercepting model corresponding to the second facial three-dimensional model.

Specifically, the process of intercepting the second 3D facial model to obtain the second sphere interception model is the same as the process of the first initial facial 3D model, and will not be repeated here.

S123: Using a preset transformation matrix calculation method, and the first sphere interception model and the second sphere interception model, to calculate and obtain an attitude transformation matrix.

Specifically, the preset transformation matrix calculation method is used to calculate the attitude transformation matrix that aligns the first sphere interception model to the second sphere interception model

Exemplarily, the preset transformation matrix calculation method may be an iterative closest point (Iterative Closest Point, ICP) algorithm.

The facial 3D model visualization method provided in the embodiment of the present application takes the position of the tip of the nose of the first initial facial 3D model as the center of the sphere, intercepts the first initial facial 3D model with a preset radius, and obtains the first initial facial 3D model corresponding to the first A sphere interception model, taking the position of the tip of the nose of the second facial 3D model as the center of the sphere, and intercepting the second facial 3D model with a preset radius to obtain a second sphere interception model corresponding to the second facial 3D model, using a preset transformation matrix The calculation method, as well as the first sphere interception model and the second sphere interception model, calculate and obtain the attitude transformation matrix. Through the method provided in the embodiment of the present application, the first initial facial 3D model and the second facial 3D model are respectively sphere intercepted to omit the cheek regions on both sides that contribute less to the calculation of the pose transformation matrix, and the first The sphere interception model and the second sphere interception model reduce the calculation amount and improve the calculation efficiency when the attitude transformation matrix is calculated using the preset transformation matrix calculation method.

On the basis of the above-mentioned embodiments, the embodiment of the present application also provides a method for visualizing a three-dimensional facial model. FIG. 4 is a schematic flowchart of the fourth method for visualizing a three-dimensional facial model provided in the embodiment of the present application. As shown in FIG. 4 , The above S20 may include:

S21: Calculate the smallest common cube bounding box of the first facial three-dimensional model and the second facial three-dimensional model.

Specifically, according to the length, width and height of the first 3D facial model and the length, width and height of the second 3D facial model, the maximum length is obtained from the length, width and height of the first 3D facial model and the length, width and height of the second 3D facial model. , width, and height, and according to the maximum length, width, and height, establish the minimum common cube bounding boxes of the first facial three-dimensional model and the second facial three-dimensional model, and the minimum common cube bounding boxes can completely surround the first facial three-dimensional respectively model and a second facial 3D model.

S22: Establish an octree structure for the minimum common cube bounding box of the first facial three-dimensional model.

Specifically, the octree structure (Octree) is a tree-like data structure used to describe a three-dimensional space. By segmenting the bounding box of the smallest common cube, 2*2*2=8 small cubes are obtained, and each small cube The cube stores multiple vertices of the 3D model of the first face, and then sequentially splits the small cubes stored with multiple vertices. The depth N of the split is used to indicate the number of times the small cube is split. The depth can be determined according to the requirements. Before reaching the depth of segmentation, if there is no vertex in the small cube, the segmentation will be stopped in advance. The depth of the octree structure is the number of times the minimum common cube bounding box of the first facial three-dimensional model is segmented plus one, that is, once divided, the depth of the octree structure is 2.

S23: Divide the minimum common cube bounding box of the second facial three-dimensional model equally with the depth of the octree structure to obtain a preset number of small cubes.

Specifically, according to the depth of the octree structure in the above-mentioned S22, the minimum common cube bounding box of the second facial three-dimensional model is equally divided into N3 small cubes.

S24: Associating the triangular patch of the second facial three-dimensional model with the small cube.

Specifically, do an intersection test on each small cube and all the triangular faces in the 3D model of the second face, if the intersection of the small cube and all the triangular faces is not an empty set, then add the intersecting triangular faces to the corresponding small cube The associated patch sequence of .

S25: According to the index relationship between the vertex coordinates of the first three-dimensional facial model and the small cube, associate the vertices of the first three-dimensional facial model with the triangular patches in the small cube.

Specifically, the small cube is coded according to preset rules, and the vertex coordinates of the first three-dimensional facial model and the small cube of the second three-dimensional facial model are determined according to the position of the vertex coordinates of the first three-dimensional facial model in the octree structure. According to the index relationship, the vertices of the first facial three-dimensional model are associated with the corresponding triangular faces in the small cube.

S26: Obtain the distance from each vertex in the first 3D facial model to the closest triangular patch of the second 3D facial model according to the distance between the vertex coordinates and the associated triangular patch.

Specifically, since each small cube contains multiple triangular faces, the distance between the vertex and each triangular face is calculated according to the vertex coordinates of the vertex, so as to determine the distance between the vertex and the nearest triangular face.

The facial three-dimensional model visualization method provided in the embodiment of the present application calculates the smallest common cube bounding box of the first facial three-dimensional model and the second facial three-dimensional model, and establishes an octree structure for the smallest common cube bounding box of the first facial three-dimensional model , with the depth of the octree structure, the minimum common cube bounding box of the second facial 3D model is equally divided to obtain a preset number of small cubes. According to the index relationship between the vertex coordinates of the first facial 3D model and the small cubes, the The vertices of the first facial 3D model are associated with the triangular patches in the small cube, and according to the distance between the vertex coordinates and the associated triangular patches, the nearest triangle from each vertex in the first facial 3D model to the second facial 3D model is obtained The patch distance. The method provided in the embodiment of the present application calculates the distance between the vertices and the associated triangles by establishing the association relationship between the vertices of the first three-dimensional facial model and the triangular patches of the second facial three-dimensional model, so as to determine the shortest distance. Avoid calculating the distance between vertices and all triangular faces, reduce the amount of calculation, and improve calculation efficiency.

On the basis of the above-mentioned embodiments, the embodiment of the present application also provides a method for visualizing a three-dimensional facial model. FIG. 5 is a schematic flow chart of the fifth method for visualizing a three-dimensional facial model provided in the embodiment of the present application. As shown in FIG. 5 , The above S25 may include:

S251: Calculate the vertex codes of the first facial three-dimensional model according to the octree structure.

Specifically, the vertex encoding is binary encoding, and the specific encoding process is illustrated with the following example:

Assuming the three-dimensional coordinates (x, y, z) of the vertex P = (13,5,4), the vertex coordinates (x _min , y _min , z _min , x _max , y _max , z _max )=(0,0,0,16,16,16).

The vertex encoding adopts Morton encoding, and the index value (i, j, k) of the vertex P is calculated. The specific calculation formula is:

in,

For the symbol of rounding down, (i,j,k)=(6,2,2), convert the index value of vertex P into binary code: (i ₁ i ₂ i ₃ ,j ₁ j ₂ j ₃ ,k ₁ k ₂ k ₃ ) = (110,010,010).

S252: According to the vertex code and the index relationship of the small cube, determine if the triangle patch in the small cube associated with the vertex is not an empty set, associate the vertex with the triangle patch in the small cube.

Specifically, according to the vertex code and the index relationship of the small cube, determine the small cube associated with the vertex, and determine whether the associated patch sequence of the small cube is an empty set.

If the triangular patches in the associated patch sequence of the small cube associated with the vertex are not an empty set, then associate the vertex with all the triangular patches in the associated patch sequence of the small cube.

In an optional implementation, the index relationship between the vertex code and the small cube can be expressed by the following formula:

Q＝q ₁ q ₂ …q _l

Wherein, Q represents the index code of the small cube, q _l =4i _l +2j _l +k _l .

For example, the index code of the small cube corresponding to the above binary code (i ₁ i ₂ i ₃ , j ₁ j ₂ j ₃ , k ₁ k ₂ k ₃ )=(110,010,010) is: Q=470.

The facial three-dimensional model visualization method provided by the embodiment of the present application calculates the vertex code of the first facial three-dimensional model according to the octree structure, and determines the triangular surface in the small cube associated with the vertex according to the index relationship between the vertex code and the small cube Piece is not the empty set, associate the vertices with the triangular patches in the cuboid. Through the method provided by the embodiment of the present application, the vertices can be associated with the triangular patches in the small cube through calculation, so as to reduce the number of calculations for the distance between the vertices and the triangular patches, and improve the calculation efficiency.

On the basis of the above-mentioned embodiments, the embodiment of the present application also provides a method for visualizing a three-dimensional facial model, as shown in FIG. 5, the above-mentioned method may also include:

S253: If the triangle patch in the small cube associated with the vertex is an empty set, then judge whether the triangle patches in other small cubes in the neighborhood of the preset radius of the small cube are empty sets with the small cube as the center.

Specifically, if the triangular facets in the triangular facet sequence of the small cube associated with the vertex are an empty set, that is, the small cube does not contain a triangular facet, then use the small cube as the center to determine the corresponding For other small cubes in the neighborhood, initially, the preset radius is set to 2, and the neighborhood corresponding to the radius of the small cube 2 contains 8 small cubes. Figure 6 is a small cube in the neighborhood provided by the embodiment of the present application As shown in Figure 6, the radius of the small cube on the left is 1, centered on the small cube on the left, and the preset radius is 2. The resulting small cube in the domain is shown on the right. It is judged whether the triangular patches in the triangular patch sequences of other small cubes in the neighborhood of the preset radius are empty sets.

S254: If the triangular faces in other small cubes are not an empty set, associate the vertices with the triangular faces in other small cubes.

Specifically, if the triangular patches in the triangular patch sequence in other small cubes are not empty sets, associate the vertices with all the triangular patches in the triangular patch sequence of the corresponding small cube, if there are multiple other small cubes If the triangles in the sequence of triangles in the middle triangle are not an empty set, associate the vertices with all the triangles in the triangles of other small cubes that are not empty sets.

In an optional implementation manner, if the triangular faces in other small cubes are an empty set, the preset radius is increased by 1, and S253-S254 is continued.

In the facial three-dimensional model visualization method provided by the embodiment of the present application, if the triangular faces in the small cube associated with the vertices are an empty set, then the small cube is used as the center to judge the triangular faces in other small cubes in the neighborhood of the preset radius of the small cube. Whether the triangle face is an empty set, if the triangle face in other small cubes is not an empty set, associate the vertex with the triangle face in other small cubes. The method provided in the embodiment of the present application can ensure that each vertex has an associated triangular surface, so as to ensure the accuracy of the results when calculating the heat map and contour lines.

On the basis of the above embodiments, the embodiment of the present application also provides a method for visualizing a three-dimensional facial model, and the above S30 may include:

According to the mapping relationship between the distance and the color table, color rendering is performed on any area of the first facial 3D model to obtain a heat map model.

Specifically, the color table is a strip that changes gradually according to "blue-cyan-green-yellow-orange-red", which contains a total of N _C colors, and the kth (0≤k≤N _C ) color in the color table Expressed by C(k).

Establish the mapping relationship between the distance and the color table, the mapping relationship between the distance and the color in the interval [a, b] is:

If x<a, C _x =C(0), if x>b, C _x =C(255).

Among them, a and b respectively represent the boundary of the mapping interval, and a<b.

According to the distance from each vertex to the nearest triangular surface, determine the color value corresponding to the preset range of the vertex, and perform color rendering on the preset range of the vertex, by calculating the multiplicity of any region of the first facial three-dimensional model A color value can be used to color render any region to obtain a heat map model.

Count the number of occurrences of multiple colors in the arbitrary area, draw the color bar chart of the first facial 3D model, and display the heat map model, color bar chart, color table, and distance at the same time when displaying on the user interface The mapping relationship with the color table.

The method for visualizing a 3D facial model provided in the embodiment of the present application performs color rendering on any region of the first 3D facial model according to the mapping relationship between distance and color table, to obtain a heat map model. Through the method provided in the embodiment of this application, the difference in facial muscle changes can be visualized through the thermal map model. If the second facial three-dimensional model is a simulation method made by professionals, the required medicine in the plastic surgery process can be determined according to the facial differences. Dosage is convenient for preoperative program design.

It should be noted that the heat map model provided in the embodiment of the present application is obtained by rendering on the first 3D facial model, or by calculating the nearest triangular surface from the vertices of the second 3D facial model to the first 3D facial model The distance of the slice, and according to the mapping relationship between the distance and the color table, render any area of the 3D model of the second face to obtain a heat map model, which is not limited in this application.

On the basis of the above-mentioned embodiments, the embodiment of the present application also provides a method for visualizing a three-dimensional facial model. FIG. 7 is a schematic flowchart of the sixth method for visualizing a three-dimensional facial model provided in the embodiment of the present application. As shown in FIG. 7 , The above S40 may include:

S41: Calculate a first height map of the first three-dimensional facial model and a second height map of the second three-dimensional facial model.

Specifically, taking the first facial three-dimensional model as an example for illustration, the method for calculating the first height map may be:

Straighten the 3D model of the first face according to the 3D Cartesian coordinate system. The specific straightening method is: make the center point of the eyes of the 3D model of the first face parallel to the X-axis of the 3D Cartesian coordinate system, and make the 3D model of the first face The central axis of the head of the 3D model is parallel to the Y axis of the 3D Cartesian coordinate system, so that the Z axis of the 3D Cartesian coordinate system passes through the nose tip of the first facial 3D model, and the nose tip is located at a preset coordinate point of the Z axis. For example, the distance between the tip of the nose and the origin of the Z axis is 150mm.

Expanding the first facial three-dimensional model into a two-dimensional plane by using a preset unfolding method to obtain the corresponding relationship between the three-dimensional coordinates (x, y, z) and the two-dimensional coordinates (x, y) of the vertices of the first facial three-dimensional model. A coordinate map of the first three-dimensional facial model is established. In the coordinate map, the two-dimensional coordinates (x, y) of the vertices store the three-dimensional coordinates (x, y, z) of the vertices.

Define the height direction hn in the two-dimensional coordinate system, calculate the height value h of each vertex in the height direction hn according to the defined height direction hn, and establish the first height map of the first facial three-dimensional model, in the first height map In , the two-dimensional coordinate (x, y) position of the vertex stores the height value h of the vertex.

For example, the calculation method of the height value h is: according to the pixel P(x,y) of the two-dimensional coordinates (x,y) in the two-dimensional plane, the inner product of the pixel P(x,y) and the height direction hn Calculation, the specific calculation formula is: D(x,y)=innerProduct(P(x,y),hn).

The calculation method of the second height map is the same as that of the above first height map, and will not be repeated here.

S42: Calculate the first two-dimensional contour line of the first facial three-dimensional model according to the first height map.

Specifically, a preset contour line calculation method is used to calculate the first height map of the first three-dimensional facial model to obtain the first two-dimensional contour line of the first three-dimensional facial model. For example, the preset contour calculation method is the MarchingSquares algorithm.

S43: Calculate a second two-dimensional contour line of the second facial three-dimensional model according to the second height map.

Specifically, the calculation method of the second two-dimensional contour line is the same as the calculation method of the first two-dimensional contour line in S42 above, which will not be repeated here.

S44: Convert the first two-dimensional contour line into a first three-dimensional contour line, and convert the second two-dimensional contour line into a second three-dimensional contour line.

Specifically, according to the coordinate map of the first 3D facial model and the first 2D contour line, the vertices of the first 2D contour line are converted into 3D coordinates to obtain the first 3D contour of the first 3D facial model line; according to the coordinate map of the second facial three-dimensional model and the second two-dimensional contour line, the vertices of the second two-dimensional contour line are converted into three-dimensional coordinates, and the second three-dimensional contour line of the second facial three-dimensional model is obtained.

S45: Render the first facial three-dimensional model according to the first three-dimensional contour line to obtain a first contour line model.

Specifically, according to the three-dimensional coordinates of the vertices of the first three-dimensional contour line, the first three-dimensional contour line is rendered in the first facial three-dimensional model according to the three-dimensional coordinates of the vertices to obtain the first contour line model.

In an optional implementation manner, the first three-dimensional contour line is colored according to a mapping relationship between a height difference of the first three-dimensional contour line and a color table.

Specifically, according to the mapping relationship between the first three-dimensional contour line and the height difference, and the mapping relationship between the height difference and the color table, the mapping relationship between the first three-dimensional contour line and the color table is determined, and according to the mapping relationship, the first three-dimensional contour line Line coloring to obtain the first contour line model after coloring. The mapping relationship between the height difference and the color table can be defined according to the requirements, and there is no limitation here.

For example, the mapping relationship between the first three-dimensional contour line and the height difference is: height difference=height value of the first three-dimensional contour line-maximum height value, and the height value of the first three-dimensional contour line can be based on the first three-dimensional contour line The three-dimensional coordinates of the line, determine the corresponding two-dimensional coordinates from the coordinate map, and then determine the height value corresponding to the two-dimensional coordinates from the height map.

S46: Render the second facial three-dimensional model according to the second three-dimensional contour line to obtain a second contour line model.

Specifically, the specific rendering method is the same as the above-mentioned S45, and will not be repeated here.

In an optional implementation manner, the second three-dimensional contour line is colored according to the mapping relationship between the height difference of the second three-dimensional contour line and the color table, and the calculation method of the height difference of the second three-dimensional contour line is the same as that of the first three-dimensional contour line. The calculation method of the height difference of the three-dimensional contour line is the same, and will not be repeated here.

In the first contour model and the second contour model after coloring, the points with the same height difference as their respective highest points have the same color. By comparing the colors of the contour lines, we can understand the facial changes in the contour area .

The facial three-dimensional model visualization method provided in the embodiment of the present application calculates the first height map of the first facial three-dimensional model and the second height map of the second facial three-dimensional model, and calculates the second height map of the first facial three-dimensional model according to the first height map A two-dimensional contour line, calculate the second two-dimensional contour line of the second facial three-dimensional model according to the second height map, convert the first two-dimensional contour line into the first three-dimensional contour line, the second two-dimensional contour line Convert the line to the second 3D contour line, render the first 3D facial model according to the first 3D contour line, obtain the first contour line model, and render the second 3D facial model according to the second 3D contour line , to get the second contour model. Through the method provided in the embodiment of the present application, through the first contour model and the second contour model, the deformation and movement of the subject's face can be intuitively understood, so as to realize the visualization of facial changes.

On the basis of the above embodiments, the embodiment of the present application also provides a three-dimensional facial model visualization device. FIG. 8 is a schematic structural diagram of a three-dimensional facial model visualization device provided by the embodiment of the present application. As shown in FIG. 8, the The facial three-dimensional model visualization device may include:

The three-dimensional model acquisition module 10 can be configured to obtain the first three-dimensional facial model and the second three-dimensional facial model of the subject at two different moments; the first three-dimensional facial model and the second three-dimensional facial model are triangle faces Format;

The distance calculation module 20 may be configured to calculate the distance between the coordinate points in the first three-dimensional facial model and the second three-dimensional facial model;

The thermal map visualization module 30 may be configured to render the first facial three-dimensional model according to the distance to obtain a thermal map model;

The contour line visualization module 40 may be configured to calculate the three-dimensional contour line according to the first three-dimensional facial model and the second three-dimensional facial model, respectively, to obtain the first contour line model and the second contour line model.

Optionally, the three-dimensional model acquisition module 10 may include:

The three-dimensional model acquisition unit may be configured to acquire the first initial facial three-dimensional model and the second facial three-dimensional model of the subject at two different moments;

A posture transformation matrix calculation unit may be configured to calculate the posture transformation matrix of the first initial facial three-dimensional model and the second facial three-dimensional model;

Optionally, the attitude transformation matrix calculation unit may include:

The first sphere intercepting subunit can be configured to use the nose tip position of the first initial facial three-dimensional model as the center of the sphere, and intercept the first initial three-dimensional facial model with a preset radius to obtain the first initial three-dimensional facial model corresponding to the first three-dimensional facial model. A sphere intercept model;

The second sphere intercepting subunit can be configured to use the nose tip position of the second facial three-dimensional model as the center of the sphere, and intercept the second facial three-dimensional model with a preset radius to obtain a second spherical intercept corresponding to the second facial three-dimensional model Model;

The attitude transformation matrix calculation subunit can be configured to calculate and obtain the attitude transformation matrix by adopting a preset transformation matrix calculation method, and the first sphere interception model and the second sphere interception model.

Optionally, the distance calculation module 20 may be configured to calculate and obtain the closest triangular surface from each vertex in the first three-dimensional facial model to the second three-dimensional facial model according to the coordinates of each vertex in the first three-dimensional facial model slice distance.

Optionally, the distance calculation module 20 may include:

A bounding box calculation unit may be configured to calculate the minimum common cube bounding box of the first facial three-dimensional model and the second facial three-dimensional model;

An octree building unit, which can be configured to build an octree structure for the minimum common cube bounding box of the first facial three-dimensional model;

The equalizing unit can be configured to equally divide the minimum common cube bounding box of the second facial three-dimensional model with the depth of the octree structure to obtain a preset number of small cubes;

The second associating unit may be configured to associate the vertices of the first three-dimensional facial model with the triangular patches in the small cube according to the index relationship between the coordinates of the vertices of the first three-dimensional facial model and the small cube;

The distance calculation unit may be configured to obtain the distance from each vertex in the first three-dimensional facial model to the nearest triangle surface of the second three-dimensional facial model according to the distance between the vertex coordinates and the associated triangle surface.

Optionally, the second association unit may include:

The encoding calculation subunit can be configured to calculate the vertex encoding of the first facial three-dimensional model according to the octree structure;

The second association subunit can be configured to: according to the vertex code and the index relationship of the small cube, determine if the triangle patch in the small cube associated with the vertex is not an empty set, then associate the vertex with the triangle patch in the small cube .

Optionally, the device may also include:

The neighborhood judging unit can be configured as follows: if the triangular patch in the small cube associated with the vertex is an empty set, then use the small cube as the center to judge the triangular patches in other small cubes in the neighborhood of the small cube’s preset radius Is it an empty set;

The third associating subunit may be configured as: if the triangular faces in other small cubes are not an empty set, associate the vertices with the triangular faces in other small cubes.

Optionally, the heat map visualization module 30 may also be configured to perform color rendering on any region of the first three-dimensional facial model according to the mapping relationship between the distance and the color table to obtain a heat map model.

Optionally, the contour visualization module 40 may include:

a height map calculation unit, which may be configured to calculate a first height map of the first facial three-dimensional model and a second height map of the second facial three-dimensional model;

The first two-dimensional contour calculation unit may be configured to calculate the first two-dimensional contour of the first facial three-dimensional model according to the first height map;

The second two-dimensional contour calculation unit may be configured to calculate the second two-dimensional contour of the second facial three-dimensional model according to the second height map;

A conversion unit may be configured to convert the first two-dimensional contour line into a first three-dimensional contour line, and convert the second two-dimensional contour line into a second three-dimensional contour line;

The above-mentioned apparatus is used to execute the methods provided in the foregoing embodiments, and its implementation principles and technical effects are similar, and details are not repeated here.

The above modules may be one or more integrated circuits configured to implement the above method, for example: one or more specific integrated circuits (Application Specific Integrated Circuit, referred to as ASIC), or, one or more microprocessors, or, One or more Field Programmable Gate Arrays (Field Programmable Gate Array, FPGA for short), etc. For another example, when one of the above modules is implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, referred to as CPU) or other processors that can call program codes. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC for short).

FIG. 9 is a schematic diagram of an electronic device provided by an embodiment of the present application. The electronic device 100 may include: a processor 101 , a storage medium 102 and a bus.

The storage medium 102 stores program instructions executable by the processor 101. When the electronic device 100 is running, the processor 101 communicates with the storage medium 102 through a bus, and the processor 101 executes the program instructions to implement the above method embodiments. The specific implementation manner and technical effect are similar, and will not be repeated here.

Optionally, the present application further provides a program product, such as a computer-readable storage medium, which may include a program, and the program is used to execute the above-mentioned method embodiments when executed by a processor.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.

The above-mentioned integrated units implemented in the form of software functional units may be stored in a computer-readable storage medium. The above-mentioned software functional units are stored in a storage medium, and include several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) or a processor (English: processor) to execute the functions described in various embodiments of the present application. part of the method. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), magnetic disk or optical disc, etc. Various media that can store program code.

The above is only a specific embodiment of the application, but the scope of protection of the application is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the application, and should be covered in Within the protection scope of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Industrial Applicability

The present application provides a method, device, electronic equipment and storage medium for visualizing a three-dimensional facial model. The method includes: obtaining the first three-dimensional facial model and the second three-dimensional facial model of the subject at two different moments; the first three-dimensional facial model and the second three-dimensional facial model are in the form of triangular patches; The distance between the coordinate points in the first 3D model and the second facial 3D model; render the first facial 3D model according to the distance to obtain a heat map model; calculate the 3D Contour, get the first contour model and the second contour model. Through this application, the changes of the face can be displayed intuitively.

In addition, it can be understood that the facial three-dimensional model visualization method, device, electronic equipment and storage medium of the present application are reproducible and can be used in various industrial applications. For example, the facial three-dimensional model visualization method, device, electronic equipment and storage medium of the present application can be used in the technical field of three-dimensional models.

Claims

A method for visualizing a three-dimensional facial model, comprising:

Obtaining the first three-dimensional facial model and the second three-dimensional facial model of the subject at two different moments; the first three-dimensional facial model and the second three-dimensional facial model are in triangular patch format;

calculating the distance between the coordinate points in the first three-dimensional facial model and the second three-dimensional facial model;

Rendering the first facial three-dimensional model according to the distance to obtain a heat map model;

The three-dimensional contour lines are respectively calculated according to the first three-dimensional facial model and the second three-dimensional facial model to obtain a first contour line model and a second contour line model.
The method according to claim 1, wherein said obtaining the first three-dimensional facial model and the second three-dimensional facial model of the subject at two different moments comprises:

Obtaining the first initial facial three-dimensional model and the second facial three-dimensional model of the subject at the two different moments;

calculating a pose transformation matrix of the first initial facial three-dimensional model and the second facial three-dimensional model;

Aligning the first initial facial three-dimensional model to the second facial three-dimensional model according to the pose transformation matrix to obtain the first facial three-dimensional model.
The method according to claim 2, wherein the calculating the pose transformation matrix of the first initial facial three-dimensional model and the second facial three-dimensional model comprises:

Taking the position of the tip of the nose of the first initial facial three-dimensional model as the center of the sphere, intercepting the first initial three-dimensional facial model with a preset radius to obtain a first sphere interception model corresponding to the first initial three-dimensional facial model;

Taking the position of the tip of the nose of the second facial three-dimensional model as the center of the sphere, intercepting the second facial three-dimensional model with the preset radius, to obtain a second spherical intercepting model corresponding to the second facial three-dimensional model;

The attitude transformation matrix is calculated and obtained by using a preset transformation matrix calculation method, and the first sphere interception model and the second sphere interception model.
The method according to any one of claims 1 to 3, wherein the calculating the distance between the coordinate points in the first three-dimensional facial model and the second three-dimensional facial model comprises:

According to the coordinates of each vertex in the first 3D facial model, calculate and acquire the distance from each vertex in the 1st 3D facial model to the closest triangular patch of the second 3D facial model.
The method according to claim 4, characterized in that, according to the coordinates of each vertex in the first facial three-dimensional model, calculating and obtaining the coordinates of each vertex in the first facial three-dimensional model to the second The distance of the nearest triangular patch of the facial 3D model, including:

Calculating the minimum common cube bounding box of the first facial three-dimensional model and the second facial three-dimensional model;

Establishing an octree structure for the minimum common cube bounding box of the first facial three-dimensional model;

Divide the minimum common cube bounding box of the second facial three-dimensional model equally with the depth of the octree structure to obtain a preset number of small cubes;

Associating the triangular patch of the second facial three-dimensional model with the small cube;

According to the index relationship between the vertex coordinates of the first facial three-dimensional model and the small cube, associate the vertices of the first facial three-dimensional model with the triangular patches in the small cube;

According to the distance between the vertex coordinates and the associated triangular patch, the distance from each vertex in the first facial 3D model to the closest triangular patch of the second facial 3D model is acquired.
The method according to claim 5, wherein, according to the index relationship between the vertex coordinates of the first three-dimensional facial model and the small cube, the vertices of the first three-dimensional facial model and the Triangular patch association in small cubes, including:

calculating the vertex codes of the first facial three-dimensional model according to the octree structure;

According to the vertex code and the index relationship of the small cube, determine if the triangle patch in the small cube associated with the vertex is not an empty set, then associate the vertex with the triangle patch in the small cube .
The method according to claim 6, further comprising:

If the triangular faces in the small cube associated with the vertex are an empty set, take the small cube as the center, and judge whether the triangular faces in other small cubes in the neighborhood of the preset radius of the small cube are empty set;

If the triangle faces in the other small cubes are not an empty set, associate the vertex with the triangle faces in the other small cubes.
The method according to any one of claims 1 to 7, wherein the rendering of the first facial three-dimensional model according to the distance to obtain a heat map model includes:

Perform color rendering on any region of the first facial three-dimensional model according to the mapping relationship between the distance and the color table to obtain the heat map model.
The method according to any one of claims 1 to 8, wherein the three-dimensional contour lines are respectively calculated according to the first three-dimensional facial model and the second three-dimensional facial model to obtain the first contour line Highline models and second contour models, including:

calculating a first height map of the first facial 3D model and a second height map of the second facial 3D model;

calculating a first two-dimensional contour line of the first facial three-dimensional model according to the first height map;

calculating a second two-dimensional contour line of the second facial three-dimensional model according to the second height map;

converting the first two-dimensional contour to a first three-dimensional contour, and converting the second two-dimensional contour to a second three-dimensional contour;

Rendering the first facial three-dimensional model according to the first three-dimensional contour line to obtain the first contour line model;

Rendering the second facial three-dimensional model according to the second three-dimensional contour line to obtain the second contour line model.
A facial three-dimensional model visualization device is characterized in that it comprises:

A three-dimensional model acquisition module configured to acquire the first three-dimensional facial model and the second three-dimensional facial model of the subject at two different moments; the first three-dimensional facial model and the second three-dimensional facial model are triangular patch format;

a distance calculation module configured to calculate the distance between coordinate points in the first three-dimensional facial model and the second three-dimensional facial model;

A heat map visualization module configured to render the first facial three-dimensional model according to the distance to obtain a heat map model;

The contour visualization module is used to calculate the three-dimensional contours respectively according to the first three-dimensional facial model and the second three-dimensional facial model, so as to obtain the first contour model and the second contour model.
The facial three-dimensional model visualization device according to claim 10, wherein the three-dimensional model acquisition module includes a three-dimensional model acquisition unit, a pose transformation matrix calculation unit and an alignment unit, wherein,

The 3D model acquiring unit is configured to acquire the first initial facial 3D model and the second facial 3D model of the subject at the two different moments;

The posture transformation matrix calculation unit is configured to calculate the posture transformation matrix of the first initial three-dimensional facial model and the second three-dimensional facial model;

The aligning unit is configured to align the first initial facial three-dimensional model to the second facial three-dimensional model according to the pose transformation matrix to obtain a first facial three-dimensional model.
The facial three-dimensional model visualization device according to claim 11, wherein the posture transformation matrix calculation unit includes a first sphere interception subunit, a second sphere interception subunit and a posture transformation matrix calculation subunit, wherein,

The first sphere interception subunit is configured to intercept the first initial facial three-dimensional model with a preset radius, taking the position of the tip of the nose of the first initial facial three-dimensional model as the center of the sphere, to obtain the first initial facial three-dimensional model. A first sphere interception model corresponding to the facial three-dimensional model;

The second sphere interception subunit is configured to intercept the second facial three-dimensional model with the preset radius using the position of the tip of the nose of the second facial three-dimensional model as the center of the sphere to obtain the second facial A second sphere interception model corresponding to the three-dimensional model;

The attitude transformation matrix calculation subunit is configured to use a preset transformation matrix calculation method, and the first sphere interception model and the second sphere interception model to calculate and obtain the attitude transformation matrix.
The facial three-dimensional model visualization device according to any one of claims 10 to 12, characterized in that,

The distance calculation module is also configured to calculate and obtain the closest distance from each vertex in the first 3D facial model to the second 3D facial model according to the coordinates of each vertex in the first 3D facial model. The distance of the triangles.
The facial three-dimensional model visualization device according to any one of claims 10 to 12, wherein the distance calculation module includes a bounding box calculation unit, an octree establishment unit, an equal division unit, a first association unit, The second association unit and the distance calculation unit, wherein,

The bounding box calculation unit is configured to calculate the minimum common cube bounding box of the first facial three-dimensional model and the second facial three-dimensional model;

The octree building unit is configured to build an octree structure for the minimum common cube bounding box of the first facial three-dimensional model;

The equalizing unit is configured to equally divide the minimum common cube bounding box of the second facial three-dimensional model with the depth of the octree structure to obtain a preset number of small cubes;

The first association unit is configured to associate the triangular patch of the second facial three-dimensional model with the small cube;

The second associating unit is configured to associate the vertices of the first three-dimensional facial model with the triangle faces in the small cube according to the index relationship between the vertex coordinates of the first three-dimensional facial model and the small cube association;

The distance calculation unit is configured to obtain the distance from each vertex in the first three-dimensional facial model to the nearest triangle surface of the second three-dimensional facial model according to the distance between the vertex coordinates and the associated triangle surface.
The facial three-dimensional model visualization device according to claim 14, wherein the second association unit includes an encoding calculation subunit and a second association subunit, wherein,

The encoding calculation subunit is configured to calculate the vertex encoding of the first facial three-dimensional model according to the octree structure;

The second associating subunit is configured to, according to the vertex code and the index relationship of the small cube, determine that if the triangle facet in the small cube associated with the vertex is not an empty set, associate the vertex with the small cube Triangular patch association in a cube.
The three-dimensional facial model visualization device according to claim 14 or 15, wherein the three-dimensional facial model visualization device further comprises a neighborhood judging unit and a third association subunit, wherein,

The neighborhood judging unit is configured to: if the triangular patch in the small cube associated with the vertex is an empty set, take the small cube as the center and judge other small cubes in the neighborhood of the preset radius of the small cube. Whether the triangles in the cube are an empty set;

The third associating subunit is configured to associate the vertex with the triangle faces in the other small cubes if the triangle faces in the other small cubes are not an empty set.
The facial three-dimensional model visualization device according to any one of claims 10 to 16, wherein the thermal map visualization module is further configured to: according to the mapping relationship between the distance and the color table, the first Color rendering is performed on any region of the three-dimensional face model to obtain the heat map model.
The facial three-dimensional model visualization device according to any one of claims 10 to 17, wherein:

The contour visualization module includes a height map calculation unit, a first two-dimensional contour calculation unit, a second two-dimensional contour calculation unit, a conversion unit, a first contour visualization unit and a second contour visualization unit, where

The height map calculation unit is configured to calculate a first height map of the first facial three-dimensional model and a second height map of the second facial three-dimensional model;

The first two-dimensional contour calculation unit is configured to calculate a first two-dimensional contour of the first facial three-dimensional model according to the first height map;

The second two-dimensional contour calculation unit is configured to calculate a second two-dimensional contour of the second facial three-dimensional model according to the second height map;

The conversion unit is configured to convert the first two-dimensional contour line into a first three-dimensional contour line, and convert the second two-dimensional contour line into a second three-dimensional contour line;

The first contour line visualization unit is configured to render the first facial three-dimensional model according to the first three-dimensional contour line to obtain the first contour line model;

The second contour line visualization unit is configured to render the second facial three-dimensional model according to the second three-dimensional contour line to obtain the second contour line model.
An electronic device, characterized in that it includes: a processor, a storage medium and a bus, the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor and the The storage media communicate through the bus, and the processor executes the program instructions to execute the steps of the facial three-dimensional model visualization method according to any one of claims 1 to 9.
A computer-readable storage medium, characterized in that, a computer program is stored on the storage medium, and when the computer program is run by a processor, the steps of the facial three-dimensional model visualization method according to any one of claims 1 to 9 are executed .