CN109729169B - Tooth based on C/S framework beautifies AR intelligence householder method - Google Patents
Tooth based on C/S framework beautifies AR intelligence householder method Download PDFInfo
- Publication number
- CN109729169B CN109729169B CN201910015140.9A CN201910015140A CN109729169B CN 109729169 B CN109729169 B CN 109729169B CN 201910015140 A CN201910015140 A CN 201910015140A CN 109729169 B CN109729169 B CN 109729169B
- Authority
- CN
- China
- Prior art keywords
- tooth
- frame
- key
- beautification
- patient
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention belongs to dentistry to beautify technical field, in particular to a kind of tooth based on C/S framework beautifies AR intelligence householder method.Specifically include: client shooting receives patient's oral video；Representative picture is selected from patient's oral video；Choose the position of tooth to be beautified in picture；The tooth after beautification is generated on corresponding position in picture；Server calculates position and the colouring information of tooth to be beautified in patient's oral video, and is sent to client；Client will replace with the tooth after beautifying wait beautify tooth in patient's oral video；Client is to patient's bandwagon effect picture and effect video.The present invention obtains patient's oral video, carries out tooth selection, shows landscaping effect in which can be convenient, and different therapeutic schemes can also be provided and choose for patient, effectively increase the enthusiasm of patient's beautification.
Description
Technical field
The present invention relates to dentistry to beautify technical field, in particular to a kind of tooth beautification AR based on C/S framework is intelligently auxiliary
Aid method.
Background technique
Tooth beautification includes reparation, correction, whitening, tooth movement and the other dental prosthetics/shaping projects of tooth.At present
Tooth beautification it is universal time-consuming, price is high, patient is after uncertain beautification in the case where tooth visual effect, it is likely that more inclines
To in not receiving beautification.Meanwhile because can not see the effect after beautification in advance, beautification conceptual design mistake also can not be directly participated in
Journey, so patient can only carry out selection scheme by the verbal communication with doctor substantially, this is also likely to cause patient to final
It is dissatisfied to beautify result.If during patient can participate in oneself tooth beautification scheme of design, and doctor can give in advance
Patient shows the kinds of schemes effect after beautification, and patient is allowed to select in several beautification schemes that dentist provides, will be very big
Improve the enthusiasm that patient receives beautification.Currently, facing the above problem, solution mainly schemes patient teeth in PC machine
Piece video carries out processing of growing permanent teeth, however PC machine the problems such as there are inconvenient for operation, portable degree and inadequate flexibilities；If directly will be auxiliary
The disadvantages of auxiliary system is deployed on intelligent terminal, and there are computing capability deficiencies, insufficient memory.
Summary of the invention
The AR intelligence householder method for the tooth beautification based on C/S framework that the purpose of the present invention is to provide a kind of, facilitates exhibition
Show calibration result, improves doctor patient communication efficiency.
In order to achieve the above object, the technical solution adopted by the present invention are as follows: a kind of tooth beautification householder method utilizes client
It is communicated with server,
The client obtains patient oral cavity information and analyzes, handles, and/or, it receives user instructions, generates the first beautification
Reference picture；
Patient oral cavity information is reached server by the client, and the server analysis, the processing oral cavity information are simultaneously
Generate the second beautification reference picture；
The first beautification reference picture and the second beautification reference picture are showed into patient respectively using the client, for it
Selection.
The client carries out tooth beautification auxiliary as follows:
(A) client shooting or reception patient's oral video；
(B) representative picture is selected from patient's oral video；
(C) corresponding position of tooth to be beautified is chosen from the picture；
(D) tooth after beautification is generated on the corresponding position, obtains the first beautification reference picture.
The server carries out tooth beautification auxiliary as follows:
(E) patient's oral video is transmitted to server by client, and server calculates tooth to be beautified in oral video
Relevant information, and it is sent to client；
(F) information obtains the tooth in oral video after beautifying tooth and replacing with beautification to client based on the received
Second beautification reference picture.
Compared with prior art, beneficial effects of the present invention are as follows:
Client and server of the invention can provide different beautification reference pictures respectively.Client can be based on trouble
The tooth of person itself, or the data based on database quickly generate a beautification reference picture, and patient can participate in the beauty
In the generating process for changing reference picture.At the same time, oral video information is also reached server by client, and server solves
The disadvantages of client computing capability is insufficient, insufficient memory, can calculate another more with powerful computing capability
Objectively beautification reference picture.
Multiple beautification reference pictures that client can be generated to present customers client and server respectively simultaneously, effectively
The in-mind anticipation that the objective beautification reference picture that pure algorithm obtains does not meet patient is avoided, also avoids artificially participating in generation
Beautify reference picture and situations such as subjective fault occurs.Using the present invention, the waiting time of patient is short, sense of participation is strong, and can be from
It selects most to be satisfied in multiple beautification reference pictures, most suitable scheme；Effectively increase the enthusiasm of patient's beautification.
Detailed description of the invention
Fig. 1 is flow diagram of the invention；
Fig. 2 is Tooth number schematic diagram of the present invention；
Fig. 3 is the schematic diagram of client-side editing tooth of the present invention；
Fig. 4 is the 83 key point position views to be detected in the present invention；
Fig. 5 is the overall architecture schematic diagram of CNN cascade network in the present invention；
Fig. 6 is the CNN network structure of CNN cascade network level1；
Fig. 7 is the CNN network structure of CNN cascade network level2.
Specific embodiment
As described in the background art, tooth beautification described herein specifically includes reparation, correction, whitening, the tooth shifting of tooth
Dynamic and other dental prosthetics/shaping projects.Below with reference to Fig. 1 to Fig. 7, the present invention is described in further detail.
As shown in Figure 1, a kind of AR intelligence householder method of the tooth beautification based on C/S framework, including client and service
Device.C/S, i.e. client and server structure, the advantage of both ends hardware environment can be made full use of by it, and task is rationally divided
It is fitted on the end Client and the end Server to realize respectively, reduces the communication-cost of system.AR is the letter of Augmented Reality
Claim, i.e. augmented reality, by computer technology, by virtual Information application to real world.The present invention creatively by C/S,
AR technical application beautifies field to tooth, the visual effect of tooth after helping client to experience beautification in advance.
Specifically: a kind of tooth beautification householder method is communicated using client and server,
The client obtains patient oral cavity information and analyzes, handles, and/or, it receives user instructions, generates the first beautification
Reference picture；
Patient oral cavity information is reached server by the client, and the server analysis, the processing oral cavity information are simultaneously
Generate the second beautification reference picture；
The beautification reference picture that client and server generates respectively is showed into patient using the client, for its choosing
It selects.
The client carries out tooth beautification auxiliary as follows:
(A) client shooting or reception patient's oral video；
(B) representative picture is selected from patient's oral video；
(C) corresponding position of tooth to be beautified is chosen from the picture；
(D) tooth after beautification is generated on the corresponding position, obtains the first beautification reference picture.
The server carries out tooth beautification auxiliary as follows:
(E) patient's oral video is transmitted to server by client, and server calculates tooth to be beautified in oral video
Relevant information, and it is sent to client；
(F) information obtains the tooth in oral video after beautifying tooth and replacing with beautification to client based on the received
Second beautification reference picture.
Each step is described in detail below.
(A) client shooting or reception patient's oral video.
Here client can generally use intelligent terminal, such as smart phone, tablet computer, and present smart phone is very
Universal, portability and functionality are more and more stronger, are highly suitable as client.Meanwhile patient's oral video, Ke Yitong
It is transmitted in client after crossing the shooting of the equipment such as the camera of profession, or directly carries out shooting acquisition with client.No matter which kind of
Situation should be assisted using tools such as brackets when shooting video, keep video record more preferably steady, reduced shake and obscured equal shadows
It rings, so that the quality of recorded video is higher, such subsequent processing is more accurate.
(B) representative picture is chosen in selection from patient's oral video.
Simultaneous selection front, left side, three picture of right side.Picture can be selected automatically by system, can also be by
Doctor manually selects.It should be noted that the picture quality chosen is better, subsequent tooth position, color identification are more quasi-
Really.The picture is the effective information display diagram of patient, and effective information includes visual tooth, lip, eyes in daily life
And nose, do not include cheek, does not also include non-visual tooth under normal circumstances.In order to facilitate showing and operating, the present invention is such as
Shown in Fig. 4, using front elevation as example.
When being chosen manually by doctor, doctor's workload on the one hand can be increased, is on the other hand influenced by subjective factor, is not allowed
The easily selection optimal picture of quality, therefore the step of automatically selecting there is also provided system, reduce doctor's workload, improve choosing
Select objectivity.When required picture is automatically selected by system, following steps can be used:
(B1) face of every frame picture and standard faces model are compared, calculate face deviation angle, obtains just
The candidate figure in face, left side, right side；(B2) mouth area in candidate figure is taken out, grayscale image is calculated, gray level image is each
Individual element is cumulative again after the two gray scale differences multiplications of a pixel neighborhoods, calculates score, formula is as follows:
D (f)=∑ y ∑ x | f (x, y)-f (x+1, y) | * | f (x, y)-f (x, y+1) |；
Wherein, x, y respectively represent the abscissa and ordinate of pixel；
(B3) each one of the front, left side, right hand view picture of highest scoring is selected from candidate figure.
Optimal mode are as follows: carry out primary dcreening operation by the step of above-mentioned (B1)~(B3) by system, then secondary screening is carried out by doctor, most
Representative picture is determined eventually.
(C) position of tooth to be beautified in the picture is chosen.
This step preferably determines after doctors and patients sufficiently link up again.The tooth for needing to beautify first is marked, then is chosen wait beautify
The corresponding position of tooth.Choose the specific method of corresponding position can be with are as follows: to treat beautification tooth using rectangle frame and carry out frame choosing, make
Rectangle frame just frames tooth, i.e. the frame contains the length and width of tooth to be beautified, so as to be subsequent to tooth effect picture
Foundation is provided when piece zooms in and out.It, can be by when frame selects tooth, determines symmetrical tooth etc. and confirms the position wait beautify tooth
Computerized algorithm is selected, can also be with artificial selection.
As shown in Fig. 2, in order to more easily confirm tooth to be beautified position and other information needed, can also in advance it is right
Tooth position is numbered.The number method particularly includes: right maxillary teeth is successively compiled outward along face center line be
11,12,13······1m；Left maxillary teeth is successively compiled as 21,22,23,2m；By lower-left jaw tooth
Tooth is successively compiled as 31,32,33,3m；Right mandibular teeth is successively compiled for 41,42,43,
4m.Wherein { 1,2,3,4,5,6,7,8 } m=.Tooth 1m and 2m is mutually symmetrical, and tooth 3m and 4m is mutually symmetrical.The number data
In the step of when continuing to be in use to hereinafter server process tooth, in specially step E.
(D) tooth after beautification is generated on corresponding position in picture, obtains the first beautification reference picture.
There are a large amount of all kinds of tooth databases on the market now, by use the company/dental hospital of this householder method according to
It needs voluntarily to select/provide.The database can be set in client background, can also store on the server, can be with
It is stored in other device/equipment being connect with client communication.It is determining after the tooth beautified, it can be directly from tooth data
Matching tooth model is selected in library, and replaces tooth to be beautified in picture.But if to the tooth for needing to be beautified
It is all as above operated, will increase data processing amount；And it can also be deposited between perfect tooth model and the tooth of patient itself again
In certain difference.
Therefore, more preferably mode is, in this step, is handled as follows each tooth to be beautified: (D1) judgement
Whether need to beautify with the symmetrical tooth of tooth to be beautified, if symmetrical tooth does not need to beautify, thens follow the steps D2~D4, or,
Execute step D2~D8.If symmetrical tooth needs to beautify, directly execution step D5~D8.(D2) width of tooth to be beautified is calculated
And height, it will cut out to obtain replacement dental imaging according to profile with the symmetrical tooth of tooth to be beautified.(D3) dental imaging will be replaced
Scaling processing is carried out after horizontal mirror image switch, the replacement dental imaging after scaling is wide, high consistent with width, the height of tooth to be beautified.
(D4) the replacement dental imaging after scaling is attached in step C on the corresponding position of picture, obtains the first beautification reference picture I.
(D5) according to integrated informations such as patient age, gender, tooth positions, candidate tooth model is selected from tooth model library.(D6)
According to adjacent not selected color of teeth, the immediate replacement tooth model of color is selected from candidate tooth model.(D7)
The dental imaging for replacing tooth model front angle is zoomed in and out processing, after scaling by the width and height for calculating tooth to be beautified
Dental imaging is wide, high consistent with width, the height of tooth to be beautified.(D8) dental imaging after scaling is attached to three figures in step C
It is corresponding wait beautify on tooth position in piece, obtain the first beautification reference picture II.
It should be understood that the physical meaning of above-mentioned steps are as follows: if the symmetrical tooth of tooth to be beautified does not need to beautify,
The first beautification reference picture I then can be only obtained, the first beautification reference picture I and the first beautification can also be obtained simultaneously with reference to figure
As II；If the symmetrical tooth of tooth to be beautified is also required to beautify, then the first beautification reference picture II is obtained.In practical operation,
According to reasonable step selection is carried out the case where patient, patient is allowed also to save operating procedure while obtaining more options.
Fig. 3 is illustrated in one embodiment, edits the interface of tooth to be beautified on the client.Shown in Fig. 3
For front elevation, other pictures can also be switched to by clicking corresponding button.The interface increases editting function simultaneously, can be with
Pass through the color of the adjustment tooth such as modification hue, saturation, intensity, transparency；Can also by selection " Quan Xuan, neck, in, cut "
Wait the full tooth of adjustment or some portion of color of tooth；It can also be by taking color device to transfer adjacent on the picture and face without beautifying tooth
Color of the color as tooth to be beautified, or to preset several color of teeth selective；The button of " selecting tooth entirely " can also be passed through
It is unified to edit all teeth, such as unified adjustment institute color dentulous, position etc.；It can also be selected to the tooth of different numbers
Different tooth models is taken, or chooses different tooth models to the tooth of identical number in different schemes；It can also pass through
It clicks form button and adjusts teeth patterning；Interface further includes the functions such as rotation, mobile, scaling, revocation, reset.It should be understood that
It is that the interface that Fig. 3 is shown is one of embodiment, in practical operation, can according to need and editting function is taken
House, increase and decrease.
In the case where patient and doctor sufficiently link up, participate in jointly, generated on the client using step A~D first
First beautification reference picture, for patient's reference.Make in this way, the waiting time of patient is short, and sense of participation is strong, can be enhanced
User experience.While carrying out step A~D, the oral cavity information (oral video) that step A is obtained is transmitted to server, In
The upper common progress step E~F of server and client side, generates more objective second beautification reference picture.Because server into
Row Data Analysis Services need the regular hour, so carrying out step A~D simultaneously using this period, can also reduce client
The idle waiting time, further enhance user experience.
Step E~F is specific as follows:
(E) patient's oral video is transmitted to server by client, and server calculates tooth to be beautified in patient's oral video
The location information and colouring information of tooth, and it is sent to client.
As also shown in e.g. figs. 4-7, in the step E, server calculates the location information of tooth to be beautified as follows:
(E1) patient's oral video is decoded, obtains key frame images and non-key frame image.(E2) it for key frame images, adopts
Tooth position is calculated with cascade convolutional neural networks CNN positioning key point coordinate.(E3) it for non-key frame image, adopts
Tooth position is calculated according to key point coordinate with Lucas-Kanade sparse optical flow algorithm.Here by critical point detection and
The algorithm that combines is tracked to calculate the tooth position in video, critical point detection uses cascade convolutional neural networks CNN, closes
The tracking of key point uses Lucas-Kanade sparse optical flow algorithm.Since critical point detection is than relatively time-consuming, it is advantageous to scheme are as follows:
Critical point detection only is carried out in the key frame of video, non-key frame carries out a tracking using sparse optical flow algorithm, to improve processing
Speed.
The step E2 includes the following steps: that (E21) carries out Face datection to key frame images, using OpenCV vision
Library tape some is detected based on the human-face detector of haar feature, is intercepted the facial image detected, is normalized to 128*128
Size.(E22) the CNN network of Level1, width 128, a height of 128, image channel 3 are set.Each convolution in the CNN network
There is an active coating after layer, for active coating using Sigmoid function as activation primitive, last two layers is full articulamentum.(E23) will
CNN network inputs of the normalized facial image as Level1 in step E21.The output of Level1 is 83 of global prediction
Key point, the attribute of each key point are x, y-coordinate value and the attribute that whether is blocked.As shown in figure 4,83 key points include tooth
50, tooth, 30, lip, 1, nose, eyes 2.Wherein, the coordinate of 50 key points of tooth is numbered in step C is
11~14,21~24,31~34,41~44 16 teeth.16 teeth are daily visual tooth.Due to the pass of tooth
Key point position and eyes, nose, lip key point be equipped with relative coordinate relationship, so final choice CNN network output
For 83 key point coordinates.(E24) for 16 teeth, the image that 18*18 pixel size is intercepted centered on every tooth is made
For the CNN network inputs of Level2, the CNN network output of Level2 measures 4 key points around every tooth in advance again
The coordinate value of 50 key points around 16 teeth after to fine tuning, wherein adjacent teeth or so shares the seat of a key point
Mark shares the average value that the coordinate value of key point takes two teeth to predict.(E25) its week is determined according to the number of tooth to be beautified
Enclose the number of 4 key points.(E26) it is obtained according to the 50 key point coordinate values exported in step E25 wait beautify around tooth
Four key point coordinates.(E27) tooth is obtained in the key frame according to wait beautify four key point coordinates around tooth
Location information.Level1 and Level2 selects L2Loss as loss function, sums to squared differences；Its formula isWherein f represents the operation result of CNN, and N represents the number for the coordinate that the network to be predicted,
XiIt is the input of network, YiIt is the correct coordinate value of training sample mark, i indicates frame number.
In the step E3, if two adjacent key frames are the i-th frame and the i-th+k frame, then to described two adjacent key frames
Between non-key frame tracked as follows: (E31) is for non-key frame: i+1~i-th+k/2 frame tracking
For, choose the i-th frame as first frame initialization, with the position of the position prediction i+1 frame of the i-th frame, then it is pre- with the position of i+1 frame
The position of the i-th+2 frame is surveyed, and so on, track to the i-th+k/2 frame；(E32) for non-key frame :+the k-1 of the i-th+k/2+1~i-th
The tracking of frame are as follows: choose the i-th+k frame as first frame initialization, with the position of position prediction the i-th+k-1 frame of the i-th+k frame,
Again with the position of position prediction the i-th+k-2 frame of the i-th+k-1 frame, and so on, track to the i-th+k/2+1 frame.It should be understood that
It is, for the tracking of non-key frame are as follows: select two neighboring key frame respectively as first frame, from both ends to centre to non-pass
The position of key frame is tracked.For non-key frame image, only tracking is needed wait beautify periodontal 4 key points, remaining
The key point of tooth and lip nose is without tracking.
It is more preferably square in order to avoid such case since non-key frame may have jitter conditions appearance using optical flow tracking
Case are as follows: the non-key frame position after tracking is smoothed using curve quadratic fit.In the step E3, by following
Step is smoothed each key point in non-key frame image:
(E33) for any key point, quadratic equation f (z)=b is selected0+b1z+b2z2。
(E34) all non-passes between two adjacent key frames and two adjacent key frames are being brought in the quadratic equation into
The value of key frame: (h, dh), h ∈ [0, H], wherein H is number of two key frames plus non-key frame between key frame, with two
The smaller key frame of index value is renumberd as the 0th frame in key frame, and h indicates the h frame after renumbeing, dhIndicate h
The coordinate value of the key point in frame.
(E35) loss function Q=∑ is definedh(f(z)-b0-b1z-b2z2), h ∈ [0, H], respectively by abscissa z value and vertical
Coordinate f (z) value brings the loss function Q into, and find out makes Q value reach the smallest b respectively0、b1、b2, obtain two different curves
Equation；
(E36) h is brought into described two curvilinear equations respectively, is calculated up to the smoothed out seat of the key point in non-key frame
Scale value.
The color of replacement tooth is calculated in server-side, may be implemented there are many kinds of scheme.The present invention provides a kind of embodiment party
Formula is mainly fitted the color change of replacement tooth, i.e. institute by calculating the color change trend of whole tooth regions entirety
In the step E stated, comprising: (E4) calculates the colouring information of tooth to be beautified；Wherein, step E4 includes: (E41) according to step
50 key point coordinate values of E26 output intercept the full tooth regions in every frame；(E42) space LAB is chosen, respectively in L, A, B
The average value that full tooth regions all pixels are calculated under channel obtains (La_i, Aa_i, Ba_i), and wherein a represents full tooth regions,
I represents the i-th frame；(E43) on the basis of with the LAB value (La_front, Aa_front, Ba_front) of front elevation, as follows
Calculate the changing value in each channel in every frame backward forward respectively:
(E44) channel the LAB color value of j-th of pixel of tooth to be beautified in the i-th frame image is calculated as follows:
By above step, it can handle to obtain more accurately color of teeth information.
(F) information will replace with the tooth after beautifying wait beautify tooth in patient's oral video to client based on the received,
Obtain the second beautification reference picture.
By the above-mentioned step A~D carried out in client the and step E~F carried out jointly in server and client side, divide
The first relatively subjective beautification reference picture and the second relatively objective beautification reference picture are not obtained.Followed by step
G。
(G) each beautification reference picture is showed patient by client, for its selection.
Beautification preferably, can also be showed patient with reference to effect video, for its reference by the present invention.I.e. in above-mentioned steps
On the basis of, the primitive oral cavity video of patient is handled, the tooth for needing to correct in video is replaced with into the tooth after beautification
Afterwards, then Dynamic Display comes out, to further increase the accuracy of displaying.
Claims (4)
1. a kind of tooth beautifies householder method, it is characterised in that:
It is communicated using client and server,
The client obtains patient oral cavity information and analyzes, handles, and/or, it receives user instructions, generates the first beautification reference
Image；
Patient oral cavity information is reached server by the client, and the server analysis, the processing oral cavity information simultaneously generate
Second beautification reference picture；
The first beautification reference picture and the second beautification reference picture are showed into patient respectively using the client, for its choosing
It selects；
The client carries out tooth beautification auxiliary as follows:
(A) client shooting or reception patient's oral video；
(B) representative picture is selected from patient's oral video；
(C) corresponding position of tooth to be beautified is chosen from the picture；
(D) tooth after beautification is generated on the corresponding position, obtains the first beautification reference picture；
In the step B, representative picture is three, is respectively as follows: front elevation, left side figure and right hand view, described
Three pictures are manually selected by doctor or are automatically selected as follows:
(B1) face of every frame picture and standard faces model are compared, calculate face deviation angle, obtain front, a left side
The candidate figure of side, right side；
(B2) mouth area in candidate figure is taken out, grayscale image is calculated, by gray level image two gray scale differences of each pixel neighborhoods
Individual element point is cumulative again after multiplication, calculates score, formula is as follows:
D (f)=∑ y ∑ x | f (x, y)-f (x+1, y) | * | f (x, y)-f (x, y+1) |；
Wherein, x, y respectively represent the abscissa and ordinate of pixel；
(B3) each one of the front, left side, right hand view picture of highest scoring, that is, having needed for obtaining are selected from candidate figure
Representative picture；In the step C, the information of corresponding position includes numbering teeth, specific method for numbering serial are as follows: along face center
It is 11,12,13 to 1m that line, which outward successively compiles right maxillary teeth, and it is 21,22,23 to 2m that left maxillary teeth, which is successively compiled, will
Left mandibular teeth is successively compiled as 31,32,33 to 3m, and it is 41,42,43 to 4m that right mandibular teeth, which is successively compiled, wherein m=1,2,
3,4,5,6,7,8 }, tooth 1m and 2m are mutually symmetrical, and tooth 3m and 4m is mutually symmetrical；
The server carries out tooth beautification auxiliary as follows:
(E) patient's oral video is transmitted to server by client, and server calculates the correlation of tooth to be beautified in oral video
Information, and it is sent to client；
(F) tooth in oral video after beautifying tooth and replacing with beautification is obtained second to client by information based on the received
Beautify reference picture；
In the step E, the relevant information includes location information, specifically calculates the position letter of tooth to be beautified as follows
Breath:
(E1) patient's oral video is decoded to obtain key frame images and non-key frame image；
(E2) for key frame images, tooth position is calculated using cascade convolutional neural networks CNN positioning key point coordinate
It sets；
(E3) for non-key frame image, tooth is calculated according to key point coordinate using Lucas-Kanade sparse optical flow algorithm
Tooth position is set；
The step E2 includes the following steps:
(E21) Face datection is carried out to key frame images, is examined using OpenCV vision library tape some based on the face of haar feature
It surveys device to be detected, intercepts the facial image detected, be normalized to 128*128 size；
(E22) be arranged the CNN network width of Level1 be 128, a height of 128, image channel 3, each convolutional layer in the CNN network
After have an active coating, active coating is used as activation primitive using Sigmoid function, and last two layers is full articulamentum；
(E23) using facial image normalized in step E21 as the CNN network inputs of Level1, the output of Level1 is complete
The coordinate value of 83 key points of office's prediction, 83 key points include tooth 50, and 30, lip, 1, nose, eyes 2,
It is 11~14,21~24,31~34,41~44 this 16 teeth that the coordinate of 50 key points of Tooth is numbered in step C
Tooth；
(E24) for 16 teeth, the image of 18*18 pixel size is intercepted using centered on every tooth as Level2's
4 key points around every tooth are predicted in the CNN network output of CNN network inputs, Level2 again, after obtaining prediction
The coordinate of 50 key points around 16 teeth, wherein adjacent teeth or so shares the coordinate of a key point, shared key
The average value that the coordinate value of point takes two teeth to predict；
(E25) number of 4 key points around it is determined according to the number of tooth to be beautified；
(E26) it is obtained according to the 50 key point coordinate values exported in step E25 wait beautify four key point coordinates around tooth；
(E27) location information of the tooth in the key frame is obtained according to wait beautify four key point coordinates around tooth.
2. tooth as described in claim 1 beautifies householder method, it is characterised in that: adjacent to two in the step E3
Key frame: the non-key frame between the i-th frame and the i-th+k frame is tracked as follows:
(E31) for non-key frame: i+1~i-th+k/2 frame tracking are as follows: it chooses the i-th frame and is initialized as first frame, with
The position of the position prediction i+1 frame of i-th frame, then with the position of the i-th+2 frame of position prediction of i+1 frame, and so on, tracking
To the i-th+k/2 frame；
(E32) for non-key frame: the tracking of+k-1 frame of the i-th+k/2+1~i-th are as follows: choose the i-th+k frame as at the beginning of first frame
Beginningization, with the position of position prediction the i-th+k-1 frame of the i-th+k frame, then with the position of position prediction the i-th+k-2 frame of the i-th+k-1 frame
It sets, and so on, track to the i-th+k/2+1 frame；
And/or；
Each key point in tracking treated the non-key frame image is smoothed according to the following steps:
(E33) for any key point, quadratic equation f (z)=b is selected0+b1z+b2z2；
(E34) value (h, the d of all non-key frames between two key frames and two key frames are brought intoh), h ∈ [0, H], wherein H
It is two key frames plus the number summation of non-key frame between key frame, with the smaller key frame of index value in two key frames
It is renumberd as the 0th frame, h indicates the h frame after renumbeing, dhIndicate the coordinate value of the key point in h frame；
(E35) a loss function Q=∑ is definedh(f(z)-b0-b1z-b2z2), h ∈ [0, H], respectively by abscissa z value and vertical
Coordinate f (z) value brings the loss function Q into, and find out makes Q value reach the smallest b respectively0、b1、b2, obtain two different curves
Equation；
(E36) h is brought into described two curvilinear equations respectively, is calculated up to the smoothed out coordinate of the key point in non-key frame
Value.
3. tooth as described in claim 1 beautifies householder method, it is characterised in that: the relevant information in the step E is also wrapped
Include colouring information；The step E further include: (E4) calculates the colouring information of tooth to be beautified；The specific steps packet of the E4
It includes:
(E41) the full tooth regions in every frame are intercepted according to 50 key point coordinate values that step E26 is exported；
(E42) space LAB is chosen, the average value for calculating full tooth regions all pixels under L, A, channel B respectively obtains (La_
I, Aa_i, Ba_i), wherein a represents full tooth regions, and i represents the i-th frame；
(E43) on the basis of with the LAB value (La_front, Aa_front, Ba_front) of front elevation, as follows respectively forward
The changing value in each channel in every frame is calculated backward:
(E44) channel the LAB color value of j-th of pixel of tooth to be beautified in the i-th frame image is calculated as follows:
4. tooth as described in claim 1 beautifies householder method, it is characterised in that: in the step D, to each wait beautify
Tooth is handled as follows:
(D1) whether judgement needs to beautify with the symmetrical tooth of tooth to be beautified, if symmetrical tooth does not need to beautify, executes step
Rapid D2~D4, or, executing step D2~D8；If symmetrical tooth needs to beautify, D5~D8 is thened follow the steps；
(D2) width and height of tooth to be beautified are calculated, will cut out to obtain replacement tooth according to profile with the symmetrical tooth of tooth to be beautified
Tooth image；
(D3) carry out scaling processing after the horizontal mirror image switch of dental imaging will be replaced, the replacement dental imaging after scaling is wide, it is high with
Width, the height of tooth to be beautified are consistent；
(D4) the replacement dental imaging after scaling is attached in step C on the corresponding position of picture, obtains the first beautification with reference to figure
As I；
(D5) according to patient age, gender, tooth corresponding position, candidate tooth model is selected from tooth model library；
(D6) according to the color of adjacent and not selected Tooth, the immediate replacement tooth of color is selected from candidate tooth model
Model；
(D7) dental imaging for replacing tooth model front angle is zoomed in and out processing by the width and height for calculating tooth to be beautified,
Dental imaging after scaling is wide, high consistent with width, the height of tooth to be beautified；
(D8) dental imaging after scaling is attached in step C on the corresponding position of picture, obtains the first beautification reference picture II.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910015140.9A CN109729169B (en) | 2019-01-08 | 2019-01-08 | Tooth based on C/S framework beautifies AR intelligence householder method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910015140.9A CN109729169B (en) | 2019-01-08 | 2019-01-08 | Tooth based on C/S framework beautifies AR intelligence householder method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109729169A CN109729169A (en) | 2019-05-07 |
| CN109729169B true CN109729169B (en) | 2019-10-29 |
Family
ID=66298918
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910015140.9A Active CN109729169B (en) | 2019-01-08 | 2019-01-08 | Tooth based on C/S framework beautifies AR intelligence householder method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109729169B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110349224B (en) * | 2019-06-14 | 2022-01-25 | 众安信息技术服务有限公司 | Tooth color value judgment method and system based on deep learning |
| CN111128321A (en) * | 2019-11-20 | 2020-05-08 | 泰康保险集团股份有限公司 | Information display method and system, device, electronic equipment and readable medium |
| CN113223140A (en) * | 2020-01-20 | 2021-08-06 | 杭州朝厚信息科技有限公司 | Method for generating image of orthodontic treatment effect by using artificial neural network |
| CN112101258A (en) * | 2020-09-21 | 2020-12-18 | 北京字节跳动网络技术有限公司 | Image processing method, image processing device, electronic equipment and computer readable medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101862175A (en) * | 2010-06-01 | 2010-10-20 | 苏州生物医学工程技术研究所 | Digitalized oral cavity intelligent auxiliary diagnosis and treatment system and diagnosis and treatment method thereof |
| CN106504321A (en) * | 2016-11-07 | 2017-03-15 | 达理 | Method using the method for photo or video reconstruction three-dimensional tooth mould and using RGBD image reconstructions three-dimensional tooth mould |
| CN106618760A (en) * | 2016-12-07 | 2017-05-10 | 上海牙典医疗器械有限公司 | Method of designing orthodontic correction scheme |
| CN108210095A (en) * | 2017-11-24 | 2018-06-29 | 上海牙典医疗器械有限公司 | A kind of correction arranges tooth method |
| CN108305689A (en) * | 2018-01-18 | 2018-07-20 | 成都贝施美医疗科技股份有限公司 | Dental Erosion effect generation method and device |
-
2019
- 2019-01-08 CN CN201910015140.9A patent/CN109729169B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101862175A (en) * | 2010-06-01 | 2010-10-20 | 苏州生物医学工程技术研究所 | Digitalized oral cavity intelligent auxiliary diagnosis and treatment system and diagnosis and treatment method thereof |
| CN106504321A (en) * | 2016-11-07 | 2017-03-15 | 达理 | Method using the method for photo or video reconstruction three-dimensional tooth mould and using RGBD image reconstructions three-dimensional tooth mould |
| CN106618760A (en) * | 2016-12-07 | 2017-05-10 | 上海牙典医疗器械有限公司 | Method of designing orthodontic correction scheme |
| CN108210095A (en) * | 2017-11-24 | 2018-06-29 | 上海牙典医疗器械有限公司 | A kind of correction arranges tooth method |
| CN108305689A (en) * | 2018-01-18 | 2018-07-20 | 成都贝施美医疗科技股份有限公司 | Dental Erosion effect generation method and device |
Non-Patent Citations (1)
| Title |
|---|
| 一种快速高灵敏度聚焦评价函数;李郁峰等;《计算机应用研究》;20100430;基于灰度梯度聚焦评价函数设计 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109729169A (en) | 2019-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109729169B (en) | Tooth based on C/S framework beautifies AR intelligence householder method | |
| US10997792B2 (en) | Kiosk for viewing of dental treatment outcomes | |
| US11030746B2 (en) | Assisted dental beautification method and apparatus for implementing the same | |
| US20240000545A1 (en) | Smile prediction | |
| US20210121271A1 (en) | Augmented reality enhancements for dental practitioners | |
| US11642195B2 (en) | Visual presentation of gingival line generated based on 3D tooth model | |
| US11727607B2 (en) | Generation of images with tooth color determined using depth information | |
| US10503962B2 (en) | Navigating among images of an object in 3D space | |
| US11517272B2 (en) | Simulated orthodontic treatment via augmented visualization in real-time | |
| CN103347436B (en) | Display system is shot in oral cavity | |
| CN111784754B (en) | Tooth orthodontic method, device, equipment and storage medium based on computer vision | |
| KR20200058316A (en) | Automatic tracking method of cephalometric point of dental head using dental artificial intelligence technology and service system | |
| US20230222750A1 (en) | Method and device for three-dimensional reconstruction of a face with toothed portion from a single image | |
| CN111437057B (en) | Three-dimensional tooth shape restoration method and system based on two-dimensional tooth beautifying characteristic line | |
| CN108597004A (en) | Occlusal surfaces of teeth Panorama Mosaic method based on Local Optimization Algorithm | |
| Amirkhanov et al. | Visual analytics in dental aesthetics | |
| US20240024076A1 (en) | Combined face scanning and intraoral scanning | |
| CN109597488B (en) | active adaptation method for angular distance of space display platform | |
| WO2023203385A1 (en) | Systems, methods, and devices for facial and oral static and dynamic analysis | |
| CN114582498A (en) | Correction scheme generation method, display method, device and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |