CN109858410A - Service evaluation method, apparatus, equipment and storage medium based on Expression analysis - Google Patents

Service evaluation method, apparatus, equipment and storage medium based on Expression analysis Download PDF

Info

Publication number
CN109858410A
CN109858410A CN201910048303.3A CN201910048303A CN109858410A CN 109858410 A CN109858410 A CN 109858410A CN 201910048303 A CN201910048303 A CN 201910048303A CN 109858410 A CN109858410 A CN 109858410A
Authority
CN
China
Prior art keywords
facial
picture
value
expression
deep learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910048303.3A
Other languages
Chinese (zh)
Inventor
刘慧众
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
OneConnect Smart Technology Co Ltd
Original Assignee
OneConnect Smart Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by OneConnect Smart Technology Co Ltd filed Critical OneConnect Smart Technology Co Ltd
Priority to CN201910048303.3A priority Critical patent/CN109858410A/en
Publication of CN109858410A publication Critical patent/CN109858410A/en
Pending legal-status Critical Current

Links

Landscapes

  • Image Analysis (AREA)

Abstract

The invention discloses a kind of, and service evaluation method, apparatus, equipment and storage medium based on Expression analysis are difficult to get customer evaluation from objective angle applied to depth learning technology field for solving the problems, such as.The method include that obtaining target video;Each facial picture is extracted from target video;Each facial picture is put into respectively to deep learning model, facial characteristics value set is obtained;For each facial picture, the set registration between facial characteristics value set and standard feature value set is calculated separately, the corresponding each set registration of facial picture is obtained;For each facial picture, each score value of facial picture is determined respectively according to the corresponding each set registration of facial picture;For each facial picture, the expression score value of facial picture is calculated according to each score value and the corresponding preset weights of each default expression classification;Scoring mean value is calculated according to each expression score value, as final scoring.

Description

Service evaluation method, apparatus, equipment and storage medium based on Expression analysis
Technical field
The present invention relates to depth learning technology fields, more particularly to the service evaluation method, apparatus based on Expression analysis, set Standby and storage medium.
Background technique
Most enterprise is equipped with the contact staff or business personnel of oneself, and in promoting service, business personnel needs and client Face-to-face exchange, and follow up the stage in business, then contact staff needs to client's return visit or the complaint suggestion for receiving client etc., It may also need and client's face-to-face exchange.Client's bring satisfaction pair is given during these staff and customer communication It is most important for enterprise maintenance client, therefore more and more enterprises pay attention to evaluation of the client to business service process.
However, the existing mode for obtaining customer evaluation mainly independently evaluates service process by client, for example allow Client chooses one from several evaluation options such as " satisfaction ", " general ", " dissatisfied ", since client individual has strong master See wish, evaluation when often relate to the factors such as worldly wisdom, the good custom of wind sequence, this result in the customer evaluation got be difficult to from Objective angle reflects the truth of business service process.
Therefore, finding one kind can need from the method that objective angle gets customer evaluation as those skilled in the art It solves the problems, such as.
Summary of the invention
The embodiment of the present invention provides a kind of service evaluation method, apparatus, computer equipment and storage based on Expression analysis Medium, to solve the problems, such as to be difficult to get customer evaluation from objective angle.
A kind of service evaluation method based on Expression analysis, comprising:
Obtain the target video recorded in target service service process for the facial expression of target customer;
Each facial picture is extracted from the target video, the face picture includes the face of the target customer Portion's expression;
It is put into respectively using each Zhang Suoshu face picture as input to preparatory trained deep learning model, is obtained described The output of deep learning model, each corresponding facial characteristics value set of Zhang Suoshu face picture;
For face picture described in every, calculate separately the corresponding facial characteristics value set of the facial picture with it is each pre- If the set registration between the corresponding standard feature value set of expression classification, the corresponding each set of the facial picture is obtained Registration;
For face picture described in every, according to the corresponding each set registration of the facial picture it is determining respectively described in Each score value of facial picture;
For face picture described in every, according to each score value and the corresponding default power of each default expression classification The expression score value of the facial picture is calculated in value;
After the corresponding expression score value of each Zhang Suoshu face picture being calculated, according to each expression Scoring mean value is calculated in score value, the scoring as the target customer to the target service service process.
A kind of service evaluation device based on Expression analysis, comprising:
Target video obtains module, records in target service service process for the facial expression of target customer for obtaining The target video of system;
Facial picture extraction module, for extracting each facial picture, the face picture from the target video It include the facial expression of the target customer;
Model identification module, for being put into respectively using each Zhang Suoshu face picture as input to preparatory trained depth Learning model obtains corresponding facial characteristics value set of Zhang Suoshu face picture that the deep learning model exports, each;
Registration computing module, for calculating separately the corresponding face of the facial picture for every facial picture Set registration between portion's characteristic value collection standard feature value set corresponding with each default expression classification, obtains the face The corresponding each set registration of portion's picture;
Score value determining module, for being directed to every facial picture, according to the corresponding each collection of the face picture Close each score value that registration determines the facial picture respectively;
Score value computing module, for for every facial picture, according to each score value and each default The expression score value of the facial picture is calculated in the corresponding preset weights of expression classification;
Score mean value computation module, in the corresponding expression score value of each Zhang Suoshu face picture being calculated Later, scoring mean value is calculated according to each expression score value, the target service is taken as the target customer The scoring of business process.
A kind of computer equipment, including memory, processor and storage are in the memory and can be in the processing The computer program run on device, the processor realize the above-mentioned service based on Expression analysis when executing the computer program The step of evaluation method.
A kind of computer readable storage medium, the computer-readable recording medium storage have computer program, the meter Calculation machine program realizes the step of above-mentioned service evaluation method based on Expression analysis when being executed by processor.
Above-mentioned service evaluation method, apparatus, computer equipment and storage medium based on Expression analysis, firstly, obtaining The target video recorded in target service service process for the facial expression of target customer;Then, from the target video Each facial picture is extracted, the face picture includes the facial expression of the target customer;Then, by each face Zhang Suoshu Portion's picture is put into respectively as input to preparatory trained deep learning model, obtain the deep learning model output, Each corresponding facial characteristics value set of Zhang Suoshu face picture;For face picture described in every, the face is calculated separately Set weight between the corresponding facial characteristics value set of portion's picture standard feature value set corresponding with each default expression classification It is right, obtain the corresponding each set registration of the facial picture;For face picture described in every, schemed according to the face The corresponding each set registration of piece determines each score value of the facial picture respectively;For face picture described in every, The expression of the facial picture is calculated according to each score value and the corresponding preset weights of each default expression classification Score value;After the corresponding expression score value of each Zhang Suoshu face picture being calculated, according to each expression Scoring mean value is calculated in score value, the scoring as the target customer to the target service service process.As it can be seen that this hair Bright is that the facial expression based on client during business service is analyzed, and the client analyzed is to business service process Scoring be to be substantially eliminated on client's factor and individual subjective factor based on objective factor (facial expression that client objectively occurs) Influence, so as to think that the obtained scoring of the present invention can reflect the true feelings of business service process from objective angle Condition objectively embodies the client to the attitude and evaluation of the business service process.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below by institute in the description to the embodiment of the present invention Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the invention Example, for those of ordinary skill in the art, without any creative labor, can also be according to these attached drawings Obtain other attached drawings.
Fig. 1 is an application environment schematic diagram of the service evaluation method in one embodiment of the invention based on Expression analysis;
Fig. 2 is a flow chart of the service evaluation method in one embodiment of the invention based on Expression analysis;
Fig. 3 is service evaluation method and step 102 in one embodiment of the invention based on Expression analysis in an application scenarios Under flow diagram;
Fig. 4 is that the service evaluation method in one embodiment of the invention based on Expression analysis is instructed in advance under an application scenarios Practice the flow diagram of deep learning model;
Fig. 5 is that the service evaluation method in one embodiment of the invention based on Expression analysis is true in advance under an application scenarios Calibrate the flow diagram of quasi- characteristic value collection;
Fig. 6 be the service evaluation method in one embodiment of the invention based on Expression analysis score under an application scenarios turn It is changed to the flow diagram of customer evaluation;
Fig. 7 is the structural schematic diagram of the service evaluation device in one embodiment of the invention based on Expression analysis;
Fig. 8 is a schematic diagram of computer equipment in one embodiment of the invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts Example, shall fall within the protection scope of the present invention.
Service evaluation method provided by the present application based on Expression analysis, can be applicable in the application environment such as Fig. 1, In, client is communicated by network with server.Wherein, which can be, but not limited to various personal computers, pen Remember this computer, smart phone, tablet computer and portable wearable device.Server can be either more with independent server The server cluster of a server composition is realized.
In one embodiment, it as shown in Fig. 2, providing a kind of service evaluation method based on Expression analysis, answers in this way It is illustrated, includes the following steps: for the server in Fig. 1
101, the target video recorded in target service service process for the facial expression of target customer is obtained;
In the present embodiment, in carrying out target service service process, staff and target customer's face-to-face exchange.Here Described face-to-face exchange can refer to that staff is lower online and exchange with target customer, such as client is in counter window and sales counter Personnel transfer, alternatively, can refer to that staff is exchanged with target customer by long-distance video on line, such as existing Video consumer is opened video with client face-to-face and is exchanged, and both sides can see face and the facial expression etc. of other side.At this In the process, target video can be recorded for the facial expression of the target customer, so that server is easy to get target view Frequently.
102, each facial picture is extracted from the target video, the face picture includes the target customer Facial expression;
It is understood that due to the target video be record the target customer facial expression obtain, the mesh The middle video frame of mark video generally includes the facial expression of the target customer, extracts from the target video and regards one by one Frequency frame, these video frames can be used as described each facial picture.
Further, as shown in figure 3, the step 102 may include:
201, it is equally spacedly determined since the broadcast start time point of the target video according to prefixed time interval Each pumping frame time point;
202, the corresponding video frame of the pumping frame time point each in the target video is extracted, obtains each face Portion's picture.
For above-mentioned steps 201, server when extracting facial picture from the target video, specifically can using etc. The mode at interval extracts the video frame in the target video.Firstly, server can be according to prefixed time interval from the target The broadcast start time point of video starts equally spacedly to determine each pumping frame time point.The prefixed time interval can be according to reality Situation needs in border are configured, such as are set as 50 milliseconds, namely every 50 milliseconds of extractions, one video frame.For example, should Target video total duration is 1 minute, i.e. 60s, and broadcast start time point is 0, then server determines on the target video Each pumping frame time point be respectively 50ms, 100ms, 150ms, 200ms ..., and so on, the last one take out frame time point be Position where 60s.Therefore, the pumping frame time point totally 1200 determined in available 1 minute target video.
For above-mentioned steps 202, it is to be understood that after determining each pumping frame time point, be equivalent to and determined Which video frame this should extract as facial picture from the target video.Server can will be each in the target video The corresponding video frame of a pumping frame time point extracts, and obtains each facial picture.Accept the example above, that is, server From 50ms, 100ms on the target video, 150ms, 200ms ..., the video frame on time point of 60s extract, obtain Totally 1200 video frames are as each facial picture, namely totally 1200 facial pictures.
103, it is put into respectively using each Zhang Suoshu face picture as input to preparatory trained deep learning model, is obtained The deep learning model output, each corresponding facial characteristics value set of Zhang Suoshu face picture;
In the present embodiment, what a deep learning model server can train in advance, and the deep learning model is using more A sample face picture is as input, and each facial characteristics value extracted from these sample face pictures is as standard Output is trained to obtain.Specifically, which can be the model of convolutional neural networks.It is understood that After step 102 extraction obtains each facial picture, server can be using each facial picture as input point Facial picture that the deep learning model exports, each Tou Ru not be obtained to preparatory trained deep learning model Corresponding facial characteristics value set.That is, server is every using a facial picture as input investment to the deep learning mould Type, the corresponding facial characteristics value set of the deep learning mode input face picture.In general, the facial characteristics value set In include multiple facial characteristics values, these facial characteristics values are used to characterize feature or attribute in facial expression.
Further, as shown in figure 4, the deep learning model can training obtains in advance by following steps:
301, multiple first facial pictures as sample are obtained;
302, each first facial characteristic value on the first facial picture is extracted, the first facial picture is obtained First facial characteristic value collection;
303, using the first facial picture as input investment to deep learning model, the deep learning model is obtained Output;
304, using the output as the target of adjustment, the parameters in the deep learning model are adjusted, with minimum Change the error between the obtained output and the first facial characteristic value collection;
If 305, the error meets preset condition, it is determined that the current deep learning model is trained depth Learning model.
For above-mentioned steps 301, available multiple the first facial pictures as sample of server in general should The quantity of first facial picture is The more the better, skillfully more, better to the training effect of the deep learning model.
For above-mentioned steps 302, it is to be understood that server can be using existing Facial Feature Extraction Technology to this First facial picture carries out the extraction of face characteristic, thus facial characteristics thereon, the facial characteristics numerical value that these are extracted Each first facial characteristic value can be obtained after change.Form of each first facial characteristic value that these extractions are obtained to gather Record, obtains the first facial characteristic value collection.
For above-mentioned steps 303, in the training deep learning model, server can make the first facial picture It is input investment to deep learning model, obtains the output of the deep learning model.Wherein, the output of the deep learning model For the set of multiple numerical value, in order to which the subsequent first facial characteristic value collection compares.
It, can be by constantly adjusting the parameters in the deep learning model, so that described for above-mentioned steps 304 Export the first facial characteristic value collection between minimizing the error, to deep learning model successively exercise supervision study and Training.It is understood that the deep learning model may include N number of hidden layer, wherein N >=1, the specific value of N can root It is set according to actual conditions., can be by adjusting the parameter of this N number of hidden layer when being trained to deep learning model, realization pair The value of the output of current depth learning model is adjusted, itself and the first facial characteristic value collection are compared during adjustment Error between conjunction makes error minimum as far as possible.Error is smaller, then it represents that current deep learning model training effect is got over It is good, conversely, then training effect is poorer.
For above-mentioned steps 305, it is to be understood that above-mentioned steps 301~304 can be executed repeatedly, using a large amount of Sample is trained deep learning model.After training, when the error meets preset condition, it may be considered that current depth Learning model has been completed to train.The preset condition for example may is that be trained using M sample, and wherein K sample is corresponding Output and first facial characteristic value collection between error less than 10%, and K/M is more than or equal to 50%, namely meets certain The ratio of the corresponding sample of the error of condition is more than the threshold value of setting, it may be considered that error meets preset condition.
104, for every facial picture, the corresponding facial characteristics value set of the facial picture and each is calculated separately It is corresponding each to obtain the facial picture for set registration between the corresponding standard feature value set of a default expression classification Gather registration;
It it is understood that server is preset with each expression classification, for example may include happy group's property, mastery, active Property etc. 16 classification, it is as shown in table 1 below:
Table 1
Each expression classification shown in upper table 1 is preset with corresponding standard feature value set, the Standard Eigenvalue collection Closing includes multiple Standard Eigenvalues, these Standard Eigenvalues in each set, which characterize a kind of expression classification human face, to be had Some facial characteristics.For example, the corresponding standard feature value set of happy group's property includes 120 Standard Eigenvalues, these standards are special Value indicative is the numerical value after the quantization of facial characteristics possessed by the face of the client with happy group's property.In this example, server can be with The standard feature value set of each expression classification is obtained by preparatory trained deep learning model.For ease of understanding, into One step, as shown in figure 5, the default corresponding standard feature value set of expression classification can be predefined by following steps:
401, multiple the second facial pictures for belonging to the default expression classification are obtained, multiple second facial pictures Quantity is more than preset amount threshold;
402, it is put into respectively using multiple described second facial pictures as input to the deep learning model, is obtained described Multiple output of deep learning model, described second corresponding second facial characteristics value sets of facial picture;
403, the second facial characteristics value in the obtained multiple second facial characteristics value sets of comparison, from all described Each common facial characteristics value is filtered out in second facial characteristics value, the common facial characteristics value refers to multiple described second faces Existing second facial characteristics value in the corresponding second facial characteristics value set of portion's picture;
404, the set of each common facial characteristics value is determined as the corresponding standard spy of the default expression classification Value indicative set.
For step 401, firstly, server is needed for the default expression classification that currently determine standard feature value set Multiple second facial pictures are collected, the quantity of these the second facial pictures should be more than preset amount threshold.It is understood that Be, server obtain these belong to the second facial picture of the default expression classification quantity it is more, then finally by depth The standard feature value set for the default expression classification that degree learning model obtains is more accurate.Specifically, server, which can be used, climbs Worm tool swashes acquirement to a large amount of facial picture from network, and then staff manually sorts out these face picture networks to each Under a default expression classification, so that server is easy to get each second facial picture of some default expression classification.
For step 402, server, can be by multiple described second faces after getting multiple second facial pictures Picture is put into respectively as input to the deep learning model, obtain deep learning model output, it is described multiple the The two corresponding second facial characteristics value sets of facial picture.
For above-mentioned steps 403, it is to be understood that multiple second facial pictures are due to belonging to same preset table mutual affection Under class, thus between the facial characteristics in these second facial pictures should characteristic attribute having the same, these identical spies Attribute is levied to exist in the form of facial characteristics value in these the second facial characteristics value sets obtained with step 402.For this purpose, service The second facial characteristics value in multiple second facial characteristics value sets that device can compare, from all second faces Each common facial characteristics value is filtered out in portion's characteristic value, the common facial characteristics value refers to multiple described second facial pictures Existing second facial characteristics value in corresponding second facial characteristics value set, it is known that, common face mentioned here Common trait attribute under the characteristic value default expression classification.
For step 404, as shown in the above, after filtering out each common facial characteristics value, server can be with The set of each common facial characteristics value is determined as the corresponding standard feature value set of the default expression classification.
For above-mentioned steps 104, for face picture described in every, server can calculate separately the facial picture pair Set registration between the facial characteristics value set answered standard feature value set corresponding with each default expression classification.It can be with Understand, it is similar to the facial characteristics in default expression classification that set registration characterizes the facial characteristics in facial picture Degree, when gathering registration is 100%, then it represents that the facial characteristics in the face picture complies fully with the default expression classification Feature belongs to the default expression classification.Under practical application scene, the facial characteristics that target customer shows can equally embody Diversified characteristic, for example can have happy group's property, mastery simultaneously and be non-property, therefore, server can calculate the face Set between the corresponding facial characteristics value set of picture standard feature value set corresponding with each default expression classification is overlapped Degree obtains the corresponding each set registration of the facial picture, is embodied in the face picture by these set registrations Facial expression possessed by these default expression classifications ratio or degree of the characteristic that have.For example, it is assumed that some face figure The relatively happy group's property of piece, mastery and be 3 set registrations that non-property is calculated be respectively 50%, 60% and 40%, this 3 Set registration embodies facial expression of the target customer in the face picture with 50% pleasure group property, 60% domination Property and 40% is non-property.
105, it for every facial picture, is determined respectively according to the corresponding each set registration of the face picture Each score value of the face picture;
After obtaining the corresponding each set registration of the facial picture, for face picture described in every, service Device can determine each score value of the facial picture respectively according to the corresponding each set registration of the face picture.Tool Body, can score value directly be converted according to preset ratio by each set registration.For example, the example above is accepted, The relatively happy group's property of some facial picture, mastery and be 3 set registrations that non-property is calculated be respectively 50%, 60% and 40%, it is determined that the score value gone out is respectively 50,60 and 40.
106, corresponding pre- according to each score value and each default expression classification for every facial picture If weight computing obtains the expression score value of the facial picture;
It is understood that server is also provided with corresponding preset weights for each default expression classification, these are pre- If weight represents the corresponding default expression classification specific gravity shared when assessing evaluation of the client to service process.Such as at certain Under a concrete application scene, weight setting is as shown in table 2 below:
Table 2
Refering to above-mentioned table 2, weight of some default expression classification, which is negative, to be represented the default expression classification and appears in client Facial expression in it is brought influence to be negative, conversely, weight of some default expression classification is positive, to represent this default It is positive that expression classification, which appears in brought influence in the facial expression of client, and the absolute value of weight is bigger, then it represents that It is also bigger that the default expression classification appears in brought influence degree in the facial expression of client.
For above-mentioned steps 106, for face picture described in every, server is in each scoring for obtaining the face picture After value, the face can be calculated according to each score value and the corresponding preset weights of each default expression classification The expression score value of picture.Specifically, the example above is accepted, it is assumed that some the relatively happy group's property of face picture, mastery and right and wrong The score value of property is respectively 50,60 and 40, and the score value of relatively other expression classifications is 0.2 are tabled look-up it is found that happy group's property, domination Property be the corresponding weight of non-property be respectively 1,0 and 0.3, then can calculate 50*1+60*0+40*0.3=62, that is, be calculated The expression score value of the face picture is 62.
107, after the corresponding expression score value of each Zhang Suoshu face picture being calculated, according to each described Scoring mean value is calculated in expression score value, the scoring as the target customer to the target service service process.
In the present embodiment, after the corresponding expression score value of each facial picture being calculated, service Scoring mean value can be calculated according to each expression score value in device, take as the target customer to the target service The scoring of business process.Specifically, the formula for calculating the scoring mean value can be with are as follows:
Wherein, X indicates scoring mean value, xiIndicate that expression score value, n are the quantity of each facial picture.Assuming that altogether 1200 facial pictures, wherein the expression score value that the expression score value of 600 facial pictures is 62,600 facial pictures is 58, then it is calculated by step 107, obtaining the scoring mean value is 60, i.e., the target customer is to the target service service process Scoring is 60.
In view of scoring of the target customer to the target service service process can not intuitively embody the target Specific evaluation of the client to the target service service process is difficult to for staff for example, it is assumed that the scoring is 61 points Learn whether the target customer meets.For this purpose, the scoring further can also be converted to target customer to institute by the present embodiment State the evaluation of target service service process.Further, after step 107, this method can also include:
501, the score value section that the scoring mean value is fallen into is selected from preset each score value section;
502, the corresponding visitor in one score value section selected is determined according to preset section satisfaction corresponding relationship Family satisfaction evaluation, the section satisfaction corresponding relationship have recorded between each score value section and the evaluation of each customer satisfaction Corresponding relationship.
503, customer satisfaction evaluation is determined as the target customer to the target service service process most Final review valence.
For above-mentioned steps 501, in the present embodiment, server can preset several score value sections, these score value sections with There are corresponding relationships between each customer satisfaction evaluation.For example, it is as shown in table 3 below that score value section can be set:
Table 3
Therefore, which score value section is the scoring mean value that server may determine that step 107 obtains drop into.In undertaking State citing, the scoring 60 fall within score value section (50,100] in, so as to select the score value section (50,100].
For step 502, server, can basis after selecting the score value section that the scoring mean value is fallen into Preset section satisfaction corresponding relationship determines the corresponding customer satisfaction evaluation in the one score value section selected, described Section satisfaction corresponding relationship has recorded the corresponding relationship between each score value section and the evaluation of each customer satisfaction.Wherein, Under some concrete application scene, shown in the section satisfaction corresponding relationship table 3 as above, details are not described herein again.
For above-mentioned steps 503, it is to be understood that customer satisfaction evaluation can be determined as described by server Final evaluation of the target customer to the target service service process.For example, accepting the example above, which falls within score value Section (50,100] it is corresponding be evaluated as " very satisfied ", therefore the target customer is to the final of the target service service process It is evaluated as " very satisfied ".
In the embodiment of the present invention, recorded in target service service process for the facial expression of target customer firstly, obtaining The target video of system;Then, each facial picture is extracted from the target video, the face picture includes the mesh Mark the facial expression of client;Then, it is put into respectively using each facial picture as input to preparatory trained depth Model is practised, the deep learning model output, the corresponding facial characteristics value set of each facial picture are obtained;Needle To face picture described in every, the facial corresponding facial characteristics value set of picture and each default expression classification are calculated separately Set registration between corresponding standard feature value set obtains the corresponding each set registration of the facial picture;Needle To face picture described in every, the facial picture is determined according to the corresponding each set registration of the face picture respectively Each score value;It is corresponding pre- according to each score value and each default expression classification for face picture described in every If weight computing obtains the expression score value of the facial picture;It is corresponding in each facial picture being calculated After expression score value, scoring mean value is calculated according to each expression score value, as the target customer to described The scoring of target service service process.As it can be seen that the present invention is that the facial expression based on client during business service is divided Analysis, the client analyzed are based on objective factor (the facial table that client objectively occurs to the scoring of business service process Feelings), the influence on client's factor and individual subjective factor is substantially eliminated, the scoring so as to think that the present invention obtains can be from visitor The angle of sight reflects the truth of business service process, objectively embodies the client to the attitude of the business service process And evaluation.
It should be understood that the size of the serial number of each step is not meant that the order of the execution order in above-described embodiment, each process Execution sequence should be determined by its function and internal logic, the implementation process without coping with the embodiment of the present invention constitutes any limit It is fixed.
In one embodiment, a kind of service evaluation device based on Expression analysis is provided, it should the service based on Expression analysis Service evaluation method in evaluating apparatus and above-described embodiment based on Expression analysis corresponds.As shown in fig. 7, expression should be based on The service evaluation device of analysis includes that target video obtains module 601, facial picture extraction module 602, model identification module 603, registration computing module 604, score value determining module 605, score value computing module 606 and scoring mean value computation module 607.Detailed description are as follows for each functional module:
Target video obtains module 601, for obtaining the facial table for being directed to target customer in target service service process The target video that feelings are recorded;
Facial picture extraction module 602, for extracting each facial picture, the face figure from the target video Piece includes the facial expression of the target customer;
Model identification module 603, for being put into respectively using each Zhang Suoshu face picture as input to trained in advance Deep learning model obtains corresponding facial characteristics value of Zhang Suoshu face picture that the deep learning model exports, each Set;
Registration computing module 604, for it is corresponding to calculate separately the facial picture for every facial picture Set registration between facial characteristics value set standard feature value set corresponding with each default expression classification, obtains described The corresponding each set registration of facial picture;
Score value determining module 605, it is corresponding each according to the facial picture for being directed to every facial picture Set registration determines each score value of the facial picture respectively;
Score value computing module 606, for for every facial picture, according to each score value and each pre- If the expression score value of the facial picture is calculated in the corresponding preset weights of expression classification;
Score mean value computation module 607, for commenting in each corresponding expression of Zhang Suoshu face picture being calculated After score value, scoring mean value is calculated according to each expression score value, as the target customer to the target industry The scoring for service process of being engaged in.
Further, the deep learning model can be by the way that with lower module, training is obtained in advance:
First picture obtains module 608, for obtaining multiple first facial pictures as sample;
The First Eigenvalue extraction module 609, for extracting each first facial characteristic value on the first facial picture, Obtain the first facial characteristic value collection of the first facial picture;
First investment model module 610, for putting into the first facial picture as input to deep learning model, Obtain the output of the deep learning model;
Model parameter adjusts module 611, for adjusting the deep learning model using the output as the target of adjustment In parameters, to minimize the error between the obtained output and the first facial characteristic value collection;
Determine that module 612 is completed in training, if meeting preset condition for the error, it is determined that the current depth Habit model is trained deep learning model.
Further, preset the corresponding standard feature value set of expression classification can be by being predefined with lower module:
Second picture obtains module 613, for obtaining multiple the second facial pictures for belonging to the default expression classification, institute The quantity for stating multiple the second facial pictures is more than preset amount threshold;
Second investment model module 614, for being put into respectively using multiple described second facial pictures as input to described Deep learning model obtains multiple second corresponding second faces of facial picture that the deep learning model exports, described Portion's characteristic value collection;
Characteristic value contrast module 615, for comparing the second face in obtained multiple second facial characteristics value sets Portion's characteristic value filters out each common facial characteristics value, the common facial characteristics from all second facial characteristics values Value refers to existing second facial characteristics in multiple described second corresponding second facial characteristics value sets of facial picture Value;
Characteristic value collection determining module 616, it is described pre- for the set of each common facial characteristics value to be determined as If the corresponding standard feature value set of expression classification.
Further, the facial picture extraction module may include:
Frame time point determination unit is taken out, for the broadcast start time point according to prefixed time interval from the target video Start equally spacedly to determine each pumping frame time point;
Video frame extracting unit, for by the corresponding video frame extraction of the pumpings frame time point each in the target video Out, each facial picture is obtained.
Further, the service evaluation device based on Expression analysis can also include:
Module is chosen in score value section, one fallen into for selecting the scoring mean value from preset each score value section A score value section;
Satisfaction evaluation determining module, for determining select described one according to preset section satisfaction corresponding relationship The corresponding customer satisfaction evaluation in a score value section, the section satisfaction corresponding relationship have recorded each score value section with it is each Corresponding relationship between customer satisfaction evaluation.
Final evaluation determining module, for customer satisfaction evaluation to be determined as the target customer to the target The final evaluation of business service process.
Specific restriction about the service evaluation device based on Expression analysis may refer to above for based on expression point The restriction of the service evaluation method of analysis, details are not described herein.Each mould in the above-mentioned service evaluation device based on Expression analysis Block can be realized fully or partially through software, hardware and combinations thereof.Above-mentioned each module can be embedded in the form of hardware or independence In processor in computer equipment, it can also be stored in a software form in the memory in computer equipment, in order to Processor, which calls, executes the corresponding operation of the above modules.
In one embodiment, a kind of computer equipment is provided, which can be server, internal junction Composition can be as shown in Figure 8.The computer equipment include by system bus connect processor, memory, network interface and Database.Wherein, the processor of the computer equipment is for providing calculating and control ability.The memory packet of the computer equipment Include non-volatile memory medium, built-in storage.The non-volatile memory medium is stored with operating system, computer program and data Library.The built-in storage provides environment for the operation of operating system and computer program in non-volatile memory medium.The calculating The database of machine equipment is for storing the data being related in the service evaluation method based on Expression analysis.The computer equipment Network interface is used to communicate with external terminal by network connection.To realize one kind when the computer program is executed by processor Service evaluation method based on Expression analysis.
In one embodiment, a kind of computer equipment is provided, including memory, processor and storage are on a memory And the computer program that can be run on a processor, processor are realized in above-described embodiment when executing computer program based on expression The step of service evaluation method of analysis, such as step 101 shown in Fig. 2 is to step 107.Alternatively, processor executes computer The function of each module/unit of the service evaluation device in above-described embodiment based on Expression analysis, such as Fig. 7 institute are realized when program Show the function of module 601 to module 607.To avoid repeating, which is not described herein again.
In one embodiment, a kind of computer readable storage medium is provided, computer program is stored thereon with, is calculated Machine program realizes the step of service evaluation method in above-described embodiment based on Expression analysis, such as Fig. 2 when being executed by processor Shown step 101 is to step 107.Alternatively, being realized when computer program is executed by processor in above-described embodiment based on expression The function of each module/unit of the service evaluation device of analysis, such as module 601 shown in Fig. 7 is to the function of module 607.To keep away Exempt to repeat, which is not described herein again.
Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with Relevant hardware is instructed to complete by computer program, the computer program can be stored in a non-volatile computer In read/write memory medium, the computer program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, To any reference of memory, storage, database or other media used in each embodiment provided herein, Including non-volatile and/or volatile memory.Nonvolatile memory may include read-only memory (ROM), programming ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include Random access memory (RAM) or external cache.By way of illustration and not limitation, RAM is available in many forms, Such as static state RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDRSDRAM), enhancing Type SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM) etc..
It is apparent to those skilled in the art that for convenience of description and succinctly, only with above-mentioned each function Can unit, module division progress for example, in practical application, can according to need and by above-mentioned function distribution by different Functional unit, module are completed, i.e., the internal structure of described device is divided into different functional unit or module, more than completing The all or part of function of description.
Embodiment described above is merely illustrative of the technical solution of the present invention, rather than its limitations;Although referring to aforementioned reality Applying example, invention is explained in detail, those skilled in the art should understand that: it still can be to aforementioned each Technical solution documented by embodiment is modified or equivalent replacement of some of the technical features;And these are modified Or replacement, the spirit and scope for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution should all It is included within protection scope of the present invention.

Claims (10)

1. a kind of service evaluation method based on Expression analysis characterized by comprising
Obtain the target video recorded in target service service process for the facial expression of target customer;
Each facial picture is extracted from the target video, the face picture includes the facial table of the target customer Feelings;
It is put into respectively using each Zhang Suoshu face picture as input to preparatory trained deep learning model, obtains the depth Learning model output, each corresponding facial characteristics value set of Zhang Suoshu face picture;
For face picture described in every, the facial corresponding facial characteristics value set of picture and each preset table are calculated separately Set registration between the corresponding standard feature value set of mutual affection class obtains the corresponding each set of the facial picture and is overlapped Degree;
For face picture described in every, the face is determined according to the corresponding each set registration of the face picture respectively Each score value of picture;
For face picture described in every, according to each score value and the corresponding preset weights meter of each default expression classification It calculates and obtains the expression score value of the facial picture;
After the corresponding expression score value of each Zhang Suoshu face picture being calculated, scored according to each expression Scoring mean value is calculated in value, the scoring as the target customer to the target service service process.
2. the service evaluation method according to claim 1 based on Expression analysis, which is characterized in that the deep learning mould By following steps, training obtains type in advance:
Obtain multiple first facial pictures as sample;
Each first facial characteristic value on the first facial picture is extracted, the first facial of the first facial picture is obtained Characteristic value collection;
Using the first facial picture as input investment to deep learning model, the output of the deep learning model is obtained;
Using the output as the target of adjustment, the parameters in the deep learning model are adjusted, are obtained with minimum Error between the output and the first facial characteristic value collection;
If the error meets preset condition, it is determined that the current deep learning model is trained deep learning mould Type.
3. the service evaluation method according to claim 1 based on Expression analysis, which is characterized in that default expression classification pair The standard feature value set answered is predefined by following steps:
Multiple the second facial pictures for belonging to the default expression classification are obtained, the quantity of multiple second facial pictures is more than Preset amount threshold;
It is put into respectively using multiple described second facial pictures as input to the deep learning model, obtains the deep learning Multiple model output, described second corresponding second facial characteristics value sets of facial picture;
The second facial characteristics value in obtained multiple second facial characteristics value sets is compared, from all second faces Each common facial characteristics value is filtered out in characteristic value, the common facial characteristics value refers to that multiple described second facial pictures are each Existing second facial characteristics value in self-corresponding second facial characteristics value set;
The set of each common facial characteristics value is determined as the corresponding standard feature value set of the default expression classification.
4. the service evaluation method according to claim 1 based on Expression analysis, which is characterized in that described from the target Each facial picture is extracted in video includes:
Each pumping frame is equally spacedly determined since the broadcast start time point of the target video according to prefixed time interval Time point;
The corresponding video frame of the pumping frame time point each in the target video is extracted, each facial picture is obtained.
5. the service evaluation method according to any one of claim 1 to 4 based on Expression analysis, which is characterized in that Scoring mean value is calculated according to each expression score value, as the target customer to the target service service process Scoring after, further includes:
The score value section that the scoring mean value is fallen into is selected from preset each score value section;
The corresponding customer satisfaction in one score value section selected is determined according to preset section satisfaction corresponding relationship Evaluation, the section satisfaction corresponding relationship have recorded the corresponding pass between each score value section and the evaluation of each customer satisfaction System;
Customer satisfaction evaluation is determined as final evaluation of the target customer to the target service service process.
6. a kind of service evaluation device based on Expression analysis characterized by comprising
Target video obtains module, for obtaining in target service service process for the facial expression recording of target customer Target video;
Facial picture extraction module, for extracting each facial picture from the target video, the face picture includes There is the facial expression of the target customer;
Model identification module, for being put into respectively using each Zhang Suoshu face picture as input to preparatory trained deep learning Model obtains corresponding facial characteristics value set of Zhang Suoshu face picture that the deep learning model exports, each;
Registration computing module, for it is special to calculate separately the corresponding face of the facial picture for every facial picture Set registration between value indicative set standard feature value set corresponding with each default expression classification obtains the face figure The corresponding each set registration of piece;
Score value determining module, for being directed to every facial picture, according to the corresponding each set weight of the face picture The right each score value for determining the facial picture respectively;
Score value computing module, for being directed to every facial picture, according to each score value and each default expression The expression score value of the facial picture is calculated in corresponding preset weights of classifying;
Score mean value computation module, for the corresponding expression score value of each Zhang Suoshu face picture being calculated it Afterwards, scoring mean value is calculated according to each expression score value, as the target customer to the target service service The scoring of process.
7. the service evaluation device according to claim 6 based on Expression analysis, which is characterized in that the deep learning mould Type is by the way that with lower module, training is obtained in advance:
First picture obtains module, for obtaining multiple first facial pictures as sample;
The First Eigenvalue extraction module obtains institute for extracting each first facial characteristic value on the first facial picture State the first facial characteristic value collection of first facial picture;
First investment model module, for obtaining institute using the first facial picture as input investment to deep learning model State the output of deep learning model;
Model parameter adjusts module, for adjusting each in the deep learning model using the output as the target of adjustment A parameter, to minimize the error between the obtained output and the first facial characteristic value collection;
Determine that module is completed in training, if meeting preset condition for the error, it is determined that the current deep learning model For trained deep learning model.
8. the service evaluation device according to claim 6 or 7 based on Expression analysis, which is characterized in that preset table mutual affection The corresponding standard feature value set of class with lower module by being predefined:
Second picture obtain module, for obtains belong to the default expression classification multiple second face pictures, it is described multiple The quantity of second facial picture is more than preset amount threshold;
Second investment model module, for being put into respectively using multiple described second facial pictures as input to the deep learning Model obtains multiple second corresponding second facial characteristics values of facial picture that the deep learning model exports, described Set;
Characteristic value contrast module, for comparing the second facial characteristics in obtained multiple second facial characteristics value sets Value, filters out each common facial characteristics value, the common facial characteristics value refers to from all second facial characteristics values Existing second facial characteristics value in multiple described second corresponding second facial characteristics value sets of facial picture;
Characteristic value collection determining module, for the set of each common facial characteristics value to be determined as the preset table mutual affection The corresponding standard feature value set of class.
9. a kind of computer equipment, including memory, processor and storage are in the memory and can be in the processor The computer program of upper operation, which is characterized in that the processor realized when executing the computer program as claim 1 to The step of service evaluation method described in any one of 5 based on Expression analysis.
10. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, and feature exists In realization is as described in any one of claims 1 to 5 based on the clothes of Expression analysis when the computer program is executed by processor The step of evaluation method of being engaged in.
CN201910048303.3A 2019-01-18 2019-01-18 Service evaluation method, apparatus, equipment and storage medium based on Expression analysis Pending CN109858410A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910048303.3A CN109858410A (en) 2019-01-18 2019-01-18 Service evaluation method, apparatus, equipment and storage medium based on Expression analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910048303.3A CN109858410A (en) 2019-01-18 2019-01-18 Service evaluation method, apparatus, equipment and storage medium based on Expression analysis

Publications (1)

Publication Number Publication Date
CN109858410A true CN109858410A (en) 2019-06-07

Family

ID=66895224

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910048303.3A Pending CN109858410A (en) 2019-01-18 2019-01-18 Service evaluation method, apparatus, equipment and storage medium based on Expression analysis

Country Status (1)

Country Link
CN (1) CN109858410A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110458008A (en) * 2019-07-04 2019-11-15 深圳壹账通智能科技有限公司 Method for processing video frequency, device, computer equipment and storage medium
CN110458140A (en) * 2019-08-20 2019-11-15 中国工商银行股份有限公司 Site satisfaction evaluation method and apparatus based on Expression Recognition
CN110472592A (en) * 2019-08-20 2019-11-19 中国工商银行股份有限公司 Service satisfaction evaluation method and device based on Expression Recognition
CN111199210A (en) * 2019-12-31 2020-05-26 武汉星巡智能科技有限公司 Expression-based video generation method, device, equipment and storage medium
CN111260623A (en) * 2020-01-14 2020-06-09 广东小天才科技有限公司 Picture evaluation method, device, equipment and storage medium
CN112232101A (en) * 2019-07-15 2021-01-15 北京正和思齐数据科技有限公司 User communication state evaluation method, device and system
CN112308591A (en) * 2019-08-02 2021-02-02 中国移动通信有限公司研究院 User evaluation method, device, equipment and computer readable storage medium
WO2021054889A1 (en) * 2019-09-19 2021-03-25 Arctan Analytics Pte. Ltd. A system and method for assessing customer satisfaction from a physical gesture of a customer

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014127333A1 (en) * 2013-02-15 2014-08-21 Emotient Facial expression training using feedback from automatic facial expression recognition
CN106485227A (en) * 2016-10-14 2017-03-08 深圳市唯特视科技有限公司 A kind of Evaluation of Customer Satisfaction Degree method that is expressed one's feelings based on video face
WO2018033143A1 (en) * 2016-08-19 2018-02-22 北京市商汤科技开发有限公司 Video image processing method, apparatus and electronic device
CN108399376A (en) * 2018-02-07 2018-08-14 华中师范大学 Student classroom learning interest intelligent analysis method and system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014127333A1 (en) * 2013-02-15 2014-08-21 Emotient Facial expression training using feedback from automatic facial expression recognition
WO2018033143A1 (en) * 2016-08-19 2018-02-22 北京市商汤科技开发有限公司 Video image processing method, apparatus and electronic device
CN106485227A (en) * 2016-10-14 2017-03-08 深圳市唯特视科技有限公司 A kind of Evaluation of Customer Satisfaction Degree method that is expressed one's feelings based on video face
CN108399376A (en) * 2018-02-07 2018-08-14 华中师范大学 Student classroom learning interest intelligent analysis method and system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
刘志欣;黄旭;魏加项;于亮;叶晶晶;: "基于负面清单管理模式的电力客户服务评价方法", 电测与仪表, no. 17 *
周建国;唐东明;彭争;李闯;张加昌;: "基于卷积神经网络的课堂表情分析软件研究与实现", 成都信息工程大学学报, no. 05 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110458008A (en) * 2019-07-04 2019-11-15 深圳壹账通智能科技有限公司 Method for processing video frequency, device, computer equipment and storage medium
CN112232101A (en) * 2019-07-15 2021-01-15 北京正和思齐数据科技有限公司 User communication state evaluation method, device and system
CN112308591A (en) * 2019-08-02 2021-02-02 中国移动通信有限公司研究院 User evaluation method, device, equipment and computer readable storage medium
CN110458140A (en) * 2019-08-20 2019-11-15 中国工商银行股份有限公司 Site satisfaction evaluation method and apparatus based on Expression Recognition
CN110472592A (en) * 2019-08-20 2019-11-19 中国工商银行股份有限公司 Service satisfaction evaluation method and device based on Expression Recognition
WO2021054889A1 (en) * 2019-09-19 2021-03-25 Arctan Analytics Pte. Ltd. A system and method for assessing customer satisfaction from a physical gesture of a customer
CN111199210A (en) * 2019-12-31 2020-05-26 武汉星巡智能科技有限公司 Expression-based video generation method, device, equipment and storage medium
CN111199210B (en) * 2019-12-31 2023-05-30 武汉星巡智能科技有限公司 Expression-based video generation method, device, equipment and storage medium
CN111260623A (en) * 2020-01-14 2020-06-09 广东小天才科技有限公司 Picture evaluation method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN109858410A (en) Service evaluation method, apparatus, equipment and storage medium based on Expression analysis
CN109508638A (en) Face Emotion identification method, apparatus, computer equipment and storage medium
CN109472206A (en) Methods of risk assessment, device, equipment and medium based on micro- expression
CN109214298B (en) Asian female color value scoring model method based on deep convolutional network
CN109344682A (en) Classroom monitoring method, device, computer equipment and storage medium
CN105956150B (en) A kind of method and device generating user's hair style and dressing collocation suggestion
CN109241378A (en) Archives method for building up, device, equipment and storage medium
CN110135263A (en) Portrait attribute model construction method, device, computer equipment and storage medium
CN109241868A (en) Face identification method, device, computer equipment and storage medium
CN108288027A (en) A kind of detection method of picture quality, device and equipment
CN109697416A (en) A kind of video data handling procedure and relevant apparatus
CN109523290A (en) Evaluation method, device, equipment and medium are paid attention to the class based on the micro- expression of audience
CN110334570A (en) The automatic banking process of recognition of face, device, equipment and storage medium
CN109509087A (en) Intelligentized loan checking method, device, equipment and medium
CN108664569A (en) A kind of hair style recommends method, system, terminal and medium
CN103488756A (en) Picture classification method and terminal
CN109766754A (en) Human face five-sense-organ clustering method, device, computer equipment and storage medium
CN109872262A (en) A kind of attendance system and method
CN109376598A (en) Facial expression image processing method, device, computer equipment and storage medium
CN109472269A (en) Characteristics of image configuration and method of calibration, device, computer equipment and medium
CN107798027A (en) A kind of heatrate Forecasting Methodology, information recommendation method and device
CN111428662A (en) Advertisement playing change method and system based on crowd attributes
CN109461030A (en) A kind of O2O evaluation method and device based on block chain
CN109829706A (en) Transfer account method, device, computer equipment and storage medium based on recognition of face
CN110335666A (en) Medical image appraisal procedure, device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
AD01 Patent right deemed abandoned

Effective date of abandoning: 20240621