CN112329586A - Client return visit method and device based on emotion recognition and computer equipment - Google Patents

Client return visit method and device based on emotion recognition and computer equipment Download PDF

Info

Publication number
CN112329586A
CN112329586A CN202011188428.5A CN202011188428A CN112329586A CN 112329586 A CN112329586 A CN 112329586A CN 202011188428 A CN202011188428 A CN 202011188428A CN 112329586 A CN112329586 A CN 112329586A
Authority
CN
China
Prior art keywords
information
client
return visit
audio
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011188428.5A
Other languages
Chinese (zh)
Inventor
马亿凯
张凯
欧光礼
魏慕茹
周璇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Life Insurance Company of China Ltd
Original Assignee
Ping An Life Insurance Company of China Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Life Insurance Company of China Ltd filed Critical Ping An Life Insurance Company of China Ltd
Priority to CN202011188428.5A priority Critical patent/CN112329586A/en
Publication of CN112329586A publication Critical patent/CN112329586A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0282—Rating or review of business operators or products
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
    • G10L2015/025—Phonemes, fenemes or fenones being the recognition units

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Acoustics & Sound (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Business, Economics & Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Finance (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Strategic Management (AREA)
  • Evolutionary Biology (AREA)
  • Development Economics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Accounting & Taxation (AREA)
  • Psychiatry (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Hospice & Palliative Care (AREA)
  • Game Theory and Decision Science (AREA)
  • Child & Adolescent Psychology (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)

Abstract

The invention discloses a client return visit method and device based on emotion recognition, computer equipment and a storage medium. The method comprises the following steps: generating service return visit information according to the service record information, sending the service return visit information to the client and acquiring return visit video information, acquiring client image information and client audio information from the return visit video information, judging whether the client image information meets the detection condition, if so, acquiring a feature point set from the client image information and acquiring audio feature quantization information from the client audio information, and acquiring satisfaction degree information matched with the feature point set and the audio feature quantization information according to a satisfaction degree classification model. The invention is based on the biological recognition technology, belongs to the technical field of artificial intelligence, and obtains the satisfaction degree information of the client by combining the facial emotion characteristics of the client with the audio characteristic quantitative information, thereby greatly improving the authenticity of the obtained satisfaction degree information and realizing the accurate obtaining of the satisfaction degree of the client.

Description

Client return visit method and device based on emotion recognition and computer equipment
Technical Field
The invention relates to the technical field of artificial intelligence, belongs to an application scene of visiting back a client to obtain satisfaction degree information in a smart city, and particularly relates to a client visiting back method and device based on emotion recognition and computer equipment.
Background
After the enterprise transacts business for the client, the enterprise can return visit to the client to acquire the satisfaction degree of the client on the business transaction process, and the satisfaction degree of the client during return visit can be used as a basis for subsequently pushing related content to the client. Traditional customer return visit processes are all carried out in a telephone inquiry mode, enterprises dial the telephone of customers through a manual or intelligent voice platform to obtain the key information fed back by the customers as the corresponding satisfaction degree, however, the key information fed back by the customers possibly does not accord with the real ideas of the customers, the enterprises are difficult to obtain the real satisfaction degree of the customers for business handling, and the accuracy of the obtained customer satisfaction degree is low. Therefore, the existing return visit method has the problem that the satisfaction degree of the customer cannot be accurately obtained.
Disclosure of Invention
The embodiment of the invention provides a client return visit method and device based on emotion recognition, computer equipment and a storage medium, aiming at solving the problem that the satisfaction degree of a client cannot be accurately obtained in the conventional return visit method.
In a first aspect, an embodiment of the present invention provides a client return visit method based on emotion recognition, which includes:
generating service return visit information according to service record information of each client in a prestored service handling information table and sending the service return visit information to a client corresponding to the client;
receiving return visit video information fed back by the client according to the service return visit information, and intercepting client image information and client audio information from the return visit video information according to a preset interception rule;
judging whether the client image information meets a preset detection condition or not;
if the customer image information meets the detection condition, acquiring a corresponding feature point set from the customer image information according to a preset facial feature acquisition model;
acquiring corresponding audio characteristic quantization information from the client audio information according to a preset audio characteristic quantization model;
and inputting the feature point set and the audio feature quantitative information into the preset satisfaction degree classification model to acquire satisfaction degree information corresponding to the return visit video information.
In a second aspect, an embodiment of the present invention provides a client return visit device based on emotion recognition, including:
the service return visit information sending unit is used for generating service return visit information according to the service record information of each client in the pre-stored service handling information table and sending the service return visit information to the client corresponding to the client;
the return visit video information intercepting unit is used for receiving return visit video information fed back by the client according to the service return visit information and intercepting client image information and client audio information from the return visit video information according to a preset intercepting rule;
the detection condition judging unit is used for judging whether the client image information meets a preset detection condition or not;
a feature point set acquisition unit, configured to, if the client image information satisfies the detection condition, acquire a corresponding feature point set from the client image information according to a preset facial feature acquisition model;
the audio characteristic quantization information acquisition unit is used for acquiring corresponding audio characteristic quantization information from the client audio information according to a preset audio characteristic quantization model;
and the satisfaction degree information acquisition unit is used for inputting the feature point set and the audio feature quantitative information into the preset satisfaction degree classification model so as to acquire satisfaction degree information corresponding to the return visit video information.
In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor, when executing the computer program, implements the method for client return visit based on emotion recognition as described in the first aspect.
In a fourth aspect, the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, causes the processor to execute the method for customer return visit based on emotion recognition according to the first aspect.
The embodiment of the invention provides a client return visit method and device based on emotion recognition, computer equipment and a storage medium. Generating service return visit information according to the service record information, sending the service return visit information to the client and acquiring return visit video information, acquiring client image information and client audio information from the return visit video information, judging whether the client image information meets the detection condition, if so, acquiring a feature point set from the client image information and acquiring audio feature quantization information from the client audio information, and acquiring satisfaction degree information matched with the feature point set and the audio feature quantization information according to a satisfaction degree classification model. By the method, the characteristic point set containing the emotional characteristics of the face of the client is obtained from the image information of the client, the audio characteristic quantitative information is obtained through voice analysis, the satisfaction degree information of the client is obtained in a mode of combining the emotional characteristics of the face of the client and the audio characteristic quantitative information, the authenticity of the obtained satisfaction degree information can be greatly improved, and the satisfaction degree of the client can be accurately obtained.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart of a client return visit method based on emotion recognition according to an embodiment of the present invention;
fig. 2 is a schematic view of an application scenario of a client return visit method based on emotion recognition according to an embodiment of the present invention;
fig. 3 is a sub-flow diagram of a client return visit method based on emotion recognition according to an embodiment of the present invention;
FIG. 4 is a schematic view of another sub-flow of a customer return visit method based on emotion recognition according to an embodiment of the present invention;
FIG. 5 is a schematic view of another sub-flow of a method for customer return visit based on emotion recognition according to an embodiment of the present invention;
FIG. 6 is a schematic view of another sub-flow of a method for customer return visit based on emotion recognition according to an embodiment of the present invention;
fig. 7 is another schematic flow chart of a client return visit method based on emotion recognition according to an embodiment of the present invention;
fig. 8 is another schematic flow chart of a client return visit method based on emotion recognition according to an embodiment of the present invention;
FIG. 9 is a schematic block diagram of a customer return visit device based on emotion recognition provided by an embodiment of the present invention;
FIG. 10 is a schematic block diagram of a computer device provided by an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic flowchart of a client return visit method based on emotion recognition according to an embodiment of the present invention, fig. 2 is a schematic diagram of an application scenario of the client return visit method based on emotion recognition according to an embodiment of the present invention, the client return visit method based on emotion recognition is applied in a management server 10, the method is executed by application software installed in the management server 10, the management server 10 is connected with at least one client 20 through a network to implement data information transmission, the management server 10 is a server for executing the client return visit method based on emotion recognition and returning visit to the client to obtain satisfaction information, the management server may be an enterprise server, the client 20 is a terminal device connected with the management server 10 through a network to receive service return visit information and feed back visit video information, such as a desktop computer, a notebook computer, a tablet computer, or a mobile phone. Fig. 2 only illustrates the management server 10 and one client 20 for information transmission, and in practical applications, the management server 10 may also establish communication connections with multiple clients 20 at the same time to implement data information transmission. As shown in fig. 1, the method includes steps S110 to S160.
And S110, generating service return visit information according to the service record information of each client in the pre-stored service transaction information table, and sending the service return visit information to the client corresponding to the client.
The service handling information table is a data information table which is pre-stored in the management server and used for recording the service handling process of a client, the service recording information can be obtained by recording the handling process of the service when the client handles one service, a piece of service return visit information can be correspondingly generated according to one piece of service recording information of each client in the service handling information table, the service return visit information is information for returning visits to the client aiming at a certain piece of service recording information, and the service return visit information can be presented in a mode of character information, voice information or video information. Specifically, the service record information includes information such as a client name, a contact way, a service name, a service type, service handling time, and the like, and the client name, the service type, and the service handling time in the service record information can be acquired and filled in a return visit template to generate service return visit information corresponding to the client name, the service type, and the service handling time, and the generated service return visit information is sent to the client corresponding to the contact way according to the contact way.
For example, in the service record information, the client name is liu XX, the service name is a service a, the service type is a financial product, the service transaction time is 2020, 5, month and 1 day, the correspondingly generated service return visit information is "liu XX is good, you transact a financial product with the service name of a service a in 2020, 5, month and 1 day, and you ask for a suggestion or suggestion about the transaction flow of the service. The contact means can be a mobile phone number or an email address.
And S120, receiving return visit video information fed back by the client according to the service return visit information, and intercepting client image information and client audio information from the return visit video information according to a preset intercepting rule.
After receiving the service return visit information, the client can correspondingly input the return visit video information and feed the return visit video information back to the management server, and then the client image information and the client audio information can be intercepted and obtained from the received return visit video information according to a preset interception rule. The intercepting rule is rule information used for intercepting and obtaining client image information and client audio information from return visit video information, the client image information intercepted from the client video information fed back by the client comprises at least one client image, and the client audio information of each client comprises corresponding voice information in the video information. Specifically, the interception rule includes a plurality of image interception time points.
In an embodiment, as shown in fig. 3, step S120 includes substeps S121 and S122.
And S121, acquiring the voice information in the client video information as client audio information.
The client video information comprises a section of voice information and a corresponding image picture, and the voice information contained in the client video information is obtained to obtain the client audio information.
And S122, acquiring a corresponding client image from the client video information according to the image interception time point to obtain client image information.
The image capturing time point is time point information for capturing an image frame in the client video information, the capturing rule comprises a plurality of image capturing time points, each image capturing time point can correspondingly capture a client image from the client video information, and all captured client images are combined into the client image information of the client.
And S130, judging whether the client image information meets a preset detection condition.
Specifically, the detection condition may be living body detection and customer matching degree detection, and the detection condition includes portrait contour information and SIFT feature extraction rules. It is possible to determine whether each of the customer images in the customer image information contains a single living body and whether each of the customer images matches a pre-stored customer photograph.
In an embodiment, as shown in fig. 4, step S130 includes sub-steps S131, S132, S133 and S134.
S131, judging whether each client image of the client image information is matched with the portrait outline information or not so as to obtain a judgment result whether each client image contains a single living body or not.
It is possible to determine whether each of the customer images in the customer image information contains a single living body. Specifically, portrait contour information can be set in the detection condition, the portrait contour information can be a contour region corresponding to an external contour of the portrait, each customer image in the customer image information can be obtained, whether the customer contour in the customer image is matched with the portrait contour information or not is judged, that is, whether the coincidence degree of the customer contour and the contour region in the portrait contour information is greater than a preset coincidence degree threshold or not is judged, if so, the customer contour in the customer image is matched with the portrait contour information, and the customer image contains a single living body; if the image information is not larger than the preset value, the fact that the customer outline in the customer image is not matched with the portrait outline information is indicated, the customer image does not contain a single living body, and the judgment result of whether each customer image contains a single living body can be obtained according to the fact that whether each customer image in the customer image information contains a single living body is judged in sequence.
S132, if each client image contains a single living body, respectively extracting the client image characteristics of each client image according to the SIFT characteristic extraction rule; s133, acquiring photo features of customer photos matched with the customers in pre-stored customer data according to the SIFT feature extraction rule; and S34, judging whether each customer image feature is matched with the photo feature, and if so, judging that the customer image information meets the detection condition.
The method can judge whether each client image is matched with the pre-stored client photo, client data of all clients transacting business are stored in the management server, and the client data comprise the client photo, so that the client matching degree detection can be further carried out on the basis of living body detection. Specifically, the detection conditions further include an SIFT feature extraction rule, a matching degree calculation formula and a preset matching degree threshold, and a corresponding face feature vector can be obtained from each client image of the client image information according to the SIFT feature extraction rule. The face feature vector is an SIFT feature corresponding to a client image of the client, the SIFT feature is a local feature of the image extracted based on an SIFT (Scale-invariant feature transform) algorithm, a corresponding photo feature vector is obtained from a pre-stored client photo according to an SIFT feature extraction rule, the matching degree between the face feature vector and the photo feature vector is obtained according to a matching degree calculation formula, whether the calculated matching degree is larger than a preset matching degree threshold value is judged, and whether each client image is matched with the client photo can be judged. If each client image in the client image information contains a single living body and each client image is matched with the client photo, the client image information meets a preset detection condition; otherwise, the client image information does not meet the detection condition.
And S140, if the customer image information meets the detection condition, acquiring a corresponding feature point set from the customer image information according to a preset facial feature acquisition model.
The customer image information comprises a plurality of customer images of the customer, and in order to accurately analyze the satisfaction degree of the customer, a corresponding feature point set can be obtained from the customer image information according to a facial feature obtaining model, the feature point set can be used for embodying the overall features of the customer image information, and the feature point set can be a set for recording the relative positions of corresponding feature points in the customer image information, wherein the facial feature obtaining model comprises a pixel contrast obtaining rule, a pixel dissolution ratio value and a feature point collecting rule. The pixel contrast obtaining rule is a specific rule for obtaining the contrast of each pixel point in the client image, the contrast between each pixel point in the client image and a plurality of surrounding pixel points can be obtained according to the pixel contrast obtaining rule, the pixel dissolution proportion value is the proportion value of the outline information obtained by performing pixel dissolution according to the contrast of the pixel points, the outline information such as the left eye outline, the right eye outline, the left eyebrow outline, the right eyebrow outline, the nose outline, the lip outline and the like of one client image is obtained by combining the pixel contrast with the pixel dissolution method, the relative position information of corresponding facial feature points can be correspondingly obtained from the outline information according to the feature point collecting rule, a feature point set corresponding to the client image information is obtained, and the coordinate position information of a plurality of feature points is contained in the feature point set, the feature point set at least comprises eye feature points, lip feature points, eyebrow feature points and nose feature points.
In an embodiment, as shown in fig. 5, step S140 includes sub-steps S141, S142 and S143.
And S141, acquiring the pixel contrast information of each client image in the client image information according to the pixel contrast acquisition rule.
Specifically, a client image is obtained and one pixel point is determined as a target pixel point, eight pixel points of a first layer at the periphery of the target pixel point and sixteen pixel points of a second layer at the periphery are obtained, RGB values of the target pixel point (pixel values of the target pixel point corresponding to three channels of red, green and blue) and RGB values corresponding to eight pixel points of the first layer at the periphery and sixteen pixel points of the second layer at the periphery are obtained, a first difference value between the RGB values of the eight pixel points of the first layer at the periphery and the target pixel point is calculated, a second difference value between the RGB values of the sixteen pixel points of the second layer at the periphery and the target pixel point is calculated, the first difference value and the second difference value are weighted and added according to a contrast obtaining rule to obtain the contrast of the target pixel point, and the contrast of each pixel point in the client image can be obtained in sequence according to the obtaining rule, and obtaining the pixel point contrast of each client image in the client image information to obtain pixel contrast information.
For example, according to the pixel contrast obtaining rule, the contrast of a certain target pixel point can be calculated by using the following formula:
Figure BDA0002752008860000081
wherein, c1A weighted value of the first difference, c2Ru is the RGB value of the u-th pixel point in the peripheral first layer, R' v is the RGB value of the v-th pixel point in the peripheral second layer, R is the weighted value of the second difference value0The RGB value of the target pixel point is obtained.
And S142, performing pixel dissolution on each customer image according to the pixel contrast information and the pixel dissolution ratio value to acquire image contour information.
Specifically, the contrast of each pixel point in the client image is sequenced according to the contrast information matched with a certain client image in the pixel contrast information, the pixel point which is in the sequencing result and is matched with the pixel dissolution proportion value and is sequenced earlier is obtained as the dissolution pixel point of the client image, and the pixel dissolution is carried out on the client image according to the dissolution pixel point to obtain the image contour information of the client image. The image contour information of each client image comprises a left eye contour, a right eye contour, a left eyebrow contour, a right eyebrow contour, a nose contour and a lip contour.
For example, when the left eye contour is extracted, because the difference between the pixels at the junction of the left eye and the face is the largest, that is, the larger the contrast value of the pixel point at the left eye contour is, the pixel dissolution is sequentially performed on the regions with smaller pixel contrast according to the pixel dissolution proportion, and finally, a part of the pixels with the largest residual contrast is the contour of the left eye.
And S143, acquiring facial feature points from each facial feature image according to the feature point acquisition rule to combine to obtain a feature point set.
And obtaining facial feature points from each facial feature image according to a feature point acquisition rule, and combining the facial feature points to obtain a feature point set. The characteristic point acquisition rule comprises eye acquisition points corresponding to the eye contour, lip acquisition points corresponding to the lip contour, eyebrow acquisition points compared with the eyebrow contour and nose acquisition points corresponding to the nose contour; and respectively acquiring a plurality of facial feature points corresponding to the image contour information through a feature point acquisition rule, calculating the average value of each facial feature point in the plurality of image contour information, and combining the average value into a feature point set.
For example, the lip collection point is a leftmost pixel point, a rightmost pixel point and a central pixel point of the lip region image, and then the coordinate position of the leftmost pixel point in the lip region image in the facial feature image, the coordinate position of the rearmost pixel point and the coordinate position of the central pixel point are respectively obtained according to the lip collection point, and the three facial feature points corresponding to the lip region image are obtained.
And S150, acquiring corresponding audio characteristic quantization information from the client audio information according to a preset audio characteristic quantization model.
And acquiring a preset audio characteristic quantization model to acquire corresponding audio characteristic quantization information from the client audio information. The audio characteristic quantization model is a model for extracting the characteristics of the client audio information, and the obtained audio characteristic quantization information can be used for quantitatively expressing the characteristics in the client audio information in a numerical form. Specifically, the audio characteristic quantization model includes an acoustic model, a characteristic dictionary, and a volume characteristic calculation formula. The obtained audio characteristic quantization information comprises a speech speed characteristic value and a volume characteristic value, the speech speed characteristic value is used for performing quantization representation on speech speed characteristics in the client audio information, and the volume characteristic value is used for performing quantization representation on volume size characteristics in the client audio information.
In an embodiment, as shown in fig. 6, step S150 may further include sub-steps S151, S152, S153, and S154.
S151, segmenting the client audio information according to the acoustic model to obtain a plurality of phonemes contained in the client audio information.
And segmenting the client audio information according to the acoustic model to obtain a plurality of phonemes contained in the client audio information. Specifically, the client audio information is composed of phonemes of a plurality of character pronunciations, and the phoneme of one character comprises the frequency and tone of the character pronunciation. The acoustic model comprises phonemes of all character pronunciations, the phonemes of a single character in the voice information can be segmented by matching the phonemes contained in the client audio information with all the phonemes in the acoustic model, and a plurality of phonemes contained in the voice information are finally obtained through segmentation.
S152, matching the phonemes according to the feature dictionary to convert the phonemes into pinyin information.
And matching the phoneme according to the feature dictionary to convert the phoneme into pinyin information. The feature dictionary contains phoneme information corresponding to all character pinyins, and the obtained phonemes are matched with the phoneme information corresponding to the character pinyins, so that the phonemes of a single character can be converted into character pinyins matched with the phonemes in the feature dictionary, all the phonemes contained in the client audio information are converted into pinyin information, and the pinyin information corresponding to each phoneme contains pinyin letters and tones.
S153, calculating to obtain a speech speed characteristic value according to the number of character pinyins contained in the pinyin information and the duration of the client audio information.
And calculating to obtain a speech speed characteristic value according to the number of character pinyins contained in the pinyin information and the duration of the client audio information. Specifically, the number of the character pinyin is divided by the duration of the audio information of the client to obtain a speech speed characteristic value, the speech speed characteristic value can be represented by a numerical value, the larger the numerical value is, the faster the speech speed of the client of the instructor is, and the smaller the numerical value is, the slower the speech speed of the client of the instructor is.
For example, if the pinyin information includes 15 pinyin characters and the duration of the client audio information is 10 seconds, the corresponding speech rate characteristic value is calculated to be 15/10-1.5.
And S154, calculating to obtain a corresponding volume characteristic value according to the volume characteristic calculation formula and the average volume of the client audio information.
The satisfaction degree of the client can be indirectly reflected to be lower when the volume of the client is too large or too small, so that the average volume of the audio information of the client can be obtained firstly, the average volume of the client is calculated through a volume characteristic calculation formula to obtain a volume characteristic value, and the volume is measured by taking dB as a unit.
For example, the volume characteristic calculation formula may be expressed as
Figure BDA0002752008860000101
Where x is the average volume of the client audio information, and if the average volume of the client audio information is 55dB, the corresponding calculated volume characteristic value is 0.3775.
And S160, inputting the feature point set and the audio feature quantization information into the preset satisfaction classification model to acquire satisfaction information corresponding to the return visit video information.
And inputting the obtained characteristic point set and the audio characteristic quantitative information into a satisfaction degree classification model to obtain satisfaction degree information corresponding to the return visit video information. Specifically, the satisfaction degree classification model is a neural network for classifying satisfaction programs of clients in the return visit process, the satisfaction degree classification model comprises a plurality of input nodes, one or more middle layers and a plurality of output nodes, and the satisfaction degree corresponding to the return visit video information can be classified into five categories of aversion, apathy, normal, approval and excitation; each intermediate layer comprises a plurality of characteristic units, each characteristic unit is connected with an input node or other characteristic units through an input formula, each characteristic unit is also connected with an output node through an output formula, and the input formula or the output formula can be expressed as: a is as aX+ b; wherein, a and b are parameter values in a formula, y is a calculation value, and x is an input value; the calculation formula of the output value of any one output node can be expressed as:
Figure BDA0002752008860000102
wherein, ajThe weighted value h of the jth characteristic unit of the last middle layer in the full connection layerjThe calculated value of the jth characteristic unit of the last middle layer in the full connection layer is N, and N is the number of the characteristic units contained in the last middle layer in the full connection layer. Each input node corresponds to a feature point coordinate value in the feature point set or a feature value in the audio feature quantization information, one feature point coordinate value in the feature point set or one feature value in the audio feature quantization information is used as an input value corresponding to the input node and is input into the full connection layer, an output value corresponding to each output node can be calculated through an input formula, an output formula and a calculation formula of the output value, the number of the output nodes is matched with the number of the satisfaction degree categories, the output value is the matching degree between the revisit video information and the corresponding satisfaction degree categories, and the behavior satisfaction degree category corresponding to the output node with the highest matching degree is obtained and is used as the satisfaction degree information of the revisit video information.
In one embodiment, as shown in fig. 7, step S160 may be preceded by step S160 a.
And S160a, if the input training data set is received, performing iterative training on the satisfaction degree classification model according to a preset gradient descent training model and the training data set to obtain the trained satisfaction degree classification model.
And if the input training data set is received, performing iterative training on the satisfaction degree classification model according to a preset gradient descent training model and the training data set to obtain the trained satisfaction degree classification model. The input training data set can be an administrator of the management server, in order to enable the satisfaction degree classification model to have higher accuracy rate when the satisfaction degree classification model is used for performing satisfaction degree classification on the return visit video information, the satisfaction degree classification model can be subjected to iterative training before the satisfaction degree classification model is used, namely parameter values in an input formula and an output formula of the satisfaction degree classification model are adjusted, and the satisfaction degree classification model obtained after training can perform more accurate satisfaction degree classification on a client corresponding to the return visit video information. The gradient descent training model is a model for training the satisfaction degree classification model, the gradient descent training model comprises a loss value calculation formula and a gradient calculation formula, the training data set comprises a plurality of pieces of training data, each piece of training data corresponds to a sample client, and each piece of training data comprises a feature point set of the sample client, audio feature quantization information and a satisfaction degree classification label of the client; inputting a feature point set of a piece of training data and audio feature quantization information into a satisfaction degree classification model to obtain matching probabilities corresponding to a plurality of output nodes, calculating the matching probabilities corresponding to the output nodes according to a loss value calculation formula and satisfaction degree classification labels to obtain corresponding loss values, calculating an updated value corresponding to each parameter in the input formula and the output formula according to the loss values and a gradient calculation formula, updating a parameter value corresponding to each parameter through the updated value, and the process of updating the parameter values is a specific process of training the satisfaction degree classification model.
For example, the loss value calculation formula may be expressed as
Figure BDA0002752008860000111
Wherein f ispFor the matching probability of an output node corresponding to the satisfaction degree classification label in the satisfaction degree classification model, fnIs the matching probability of the nth output node, fpAnd fnAll values of (1) are [0, 1 ]]。
And calculating to obtain an updated value of each parameter in the satisfaction degree classification model according to the gradient calculation formula, the loss value and the calculated value of the satisfaction degree classification model. Specifically, a calculation value obtained by calculating the feature point set and the audio feature quantization information by using one parameter in the satisfaction degree classification model is input into a gradient calculation formula, and an update value corresponding to the parameter can be calculated by combining the loss value, and the calculation process is also gradient descent calculation.
Specifically, the gradient calculation formula can be expressed as:
Figure BDA0002752008860000112
wherein the content of the first and second substances,
Figure BDA0002752008860000113
for the calculated updated value of the parameter e, ωeIs the original parameter value of the parameter e, eta is the preset learning rate in the gradient calculation formula,
Figure BDA0002752008860000121
the partial derivative of the parameter e is calculated based on the loss value and the calculated value corresponding to the parameter e (the calculated value corresponding to the parameter is used in the calculation process).
And updating the parameter values of the corresponding parameters in the satisfaction degree classification model according to the update value of each parameter so as to train the satisfaction degree classification model. And correspondingly updating the parameter value of each parameter in the satisfaction degree classification model based on the calculated updated value, namely completing one training process of the satisfaction degree classification model. Calculating another piece of training data in the training data set again based on the satisfaction degree classification model obtained after the training for one time, and repeating the training process to realize iterative training of the satisfaction degree classification model; and when the calculated loss value is smaller than a preset loss threshold value or training data in the training data set are used for training, stopping the training process to obtain a trained satisfaction degree classification model.
In an embodiment, as shown in fig. 8, step S170 may be further included after step S160.
S170, uploading the satisfaction degree information and the client audio information as return visit preference information of the client to a block chain for storage.
The obtained satisfaction degree information and the client audio information are stored in the management server as return visit preference information of the client corresponding to the sent return visit video information, and corresponding summary information can be obtained based on the return visit preference information, specifically, the summary information is obtained by performing hash processing on a text detection result, for example, processing by using the sha256s algorithm. In the service expansion process, the enterprise can perform subsequent pushing of related content according to the stored return visit preference information matched with any client, specifically, the audio information of the client can be converted into text information through voice recognition, and the text information and the satisfaction degree information are combined and then stored as the return visit preference information of the client.
Uploading summary information to the blockchain can ensure the safety and the fair transparency of the user. The administrator can download the summary information from the blockchain through the management server so as to verify whether the text detection result is tampered. The blockchain referred to in this example is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
The technical method can be applied to application scenes such as intelligent government affairs, intelligent city management, intelligent community, intelligent security protection, intelligent logistics, intelligent medical treatment, intelligent education, intelligent environmental protection and intelligent traffic, and the like, wherein the application scenes comprise the return visit of customers to obtain satisfaction degree information, and therefore the construction of the intelligent city is promoted.
In the emotion recognition-based client return visit method provided by the embodiment of the invention, the service return visit information is generated according to the service record information and is sent to the client, the return visit video information is obtained, the client image information and the client audio information are obtained from the return visit video information, whether the client image information meets the detection condition or not is judged, if yes, the characteristic point set is obtained from the client image information, the audio characteristic quantization information is obtained from the client audio information, and the satisfaction degree information matched with the characteristic point set and the audio characteristic quantization information is obtained according to the satisfaction degree classification model. By the method, the characteristic point set containing the emotional characteristics of the face of the client is obtained from the image information of the client, the audio characteristic quantitative information is obtained through voice analysis, the satisfaction degree information of the client is obtained in a mode of combining the emotional characteristics of the face of the client and the audio characteristic quantitative information, the authenticity of the obtained satisfaction degree information can be greatly improved, and the satisfaction degree of the client can be accurately obtained.
The embodiment of the invention also provides a client return visit device based on emotion recognition, which is used for executing any embodiment of the client return visit method based on emotion recognition. Specifically, referring to fig. 9, fig. 9 is a schematic block diagram of a client return visit device based on emotion recognition according to an embodiment of the present invention. The emotion recognition-based client return visit apparatus may be configured in a management server.
As shown in fig. 9, the client revisit device 100 based on emotion recognition includes a service revisit information transmitting unit 110, a revisit video information intercepting unit 120, a detection condition judging unit 130, a feature point set acquiring unit 140, an audio feature quantization information acquiring unit 150, and a satisfaction degree information acquiring unit 160.
And a service return visit information sending unit 110, configured to generate service return visit information according to the service record information of each client in the pre-stored service transaction information table, and send the service return visit information to the client corresponding to the client.
And the return visit video information intercepting unit 120 is configured to receive return visit video information fed back by the client according to the service return visit information, and intercept the return visit video information according to a preset interception rule to obtain client image information and client audio information.
In one embodiment, the return visit video information intercepting unit 120 includes sub-units: a client audio information acquisition unit and a client image information acquisition unit.
The client audio information acquisition unit is used for acquiring the voice information in the client video information as client audio information; and the client image information acquisition unit is used for acquiring a corresponding client image from the client video information according to the image interception time point to obtain client image information.
A detection condition determining unit 130, configured to determine whether the client image information satisfies a preset detection condition.
In one embodiment, the detection condition determining unit 130 includes sub-units: the system comprises a living body judging unit, a client image characteristic acquiring unit, a photo characteristic acquiring unit and a matching judging unit.
The living body judging unit is used for judging whether each client image of the client image information is matched with the portrait outline information or not so as to obtain a judgment result whether each client image contains a single living body or not; the client image feature acquisition unit is used for respectively extracting the client image features of each client image according to the SIFT feature extraction rule if each client image contains a single living body; the photo feature acquisition unit is used for acquiring the photo features of the customer photos matched with the customers in the pre-stored customer data according to the SIFT feature extraction rule; and the matching judgment unit is used for judging whether each customer image characteristic is matched with the photo characteristic, and if so, judging that the customer image information meets the detection condition.
A feature point set obtaining unit 140, configured to, if the client image information satisfies the detection condition, obtain a corresponding feature point set from the client image information according to a preset facial feature obtaining model.
In one embodiment, the feature point set obtaining unit 140 includes sub-units: the device comprises a pixel contrast information acquisition unit, an image contour information acquisition unit and a facial feature point acquisition unit.
The pixel contrast information acquisition unit is used for acquiring the pixel contrast information of each client image in the client image information according to the pixel contrast acquisition rule; the image contour information acquisition unit is used for carrying out pixel dissolution on each customer image according to the pixel contrast information and the pixel dissolution proportion value so as to acquire image contour information; and the facial feature point acquisition unit is used for acquiring facial feature points from each facial feature image according to the feature point acquisition rule so as to combine the facial feature points to obtain a feature point set.
And an audio characteristic quantization information obtaining unit 150, configured to obtain corresponding audio characteristic quantization information from the client audio information according to a preset audio characteristic quantization model.
In one embodiment, the audio feature quantization information obtaining unit 150 includes sub-units: the device comprises a phoneme segmentation unit, a pinyin information acquisition unit, a speech speed characteristic value acquisition unit and a volume characteristic value acquisition unit.
The phoneme segmentation unit is used for segmenting the client audio information according to the acoustic model to obtain a plurality of phonemes contained in the client audio information; the pinyin information acquisition unit is used for matching the phonemes according to the feature dictionary so as to convert the phonemes into pinyin information; a speech rate characteristic value obtaining unit, configured to calculate a speech rate characteristic value according to the number of character pinyins included in the pinyin information and the duration of the client audio information; and the volume characteristic value acquisition unit is used for calculating to obtain a corresponding volume characteristic value according to the volume characteristic calculation formula and the average volume of the client audio information.
A satisfaction information obtaining unit 160, configured to input the feature point set and the audio feature quantization information into the preset satisfaction classification model to obtain satisfaction information corresponding to the return visit video information.
In an embodiment, the emotion recognition based client revisiting device 100 further comprises sub-units: and a classification model training unit.
And the classification model training unit is used for carrying out iterative training on the satisfaction degree classification model according to a preset gradient descent training model and the training data set to obtain the trained satisfaction degree classification model if the input training data set is received.
In an embodiment, the emotion recognition based client revisiting device 100 further comprises sub-units: and returning to the preference information storage unit.
And the return visit preference information storage unit is used for uploading the satisfaction degree information and the client audio information as return visit preference information of the client to a block chain for storage.
The client return visit device based on emotion recognition provided by the embodiment of the invention applies the client return visit method based on emotion recognition, generates service return visit information according to service record information, sends the service return visit information to a client and obtains return visit video information, obtains client image information and client audio information from the return visit video information, judges whether the client image information meets detection conditions, obtains a feature point set from the client image information and obtains audio feature quantization information from the client audio information if the client image information meets the detection conditions, and obtains satisfaction degree information matched with the feature point set and the audio feature quantization information according to a satisfaction degree classification model. By the method, the characteristic point set containing the emotional characteristics of the face of the client is obtained from the image information of the client, the audio characteristic quantitative information is obtained through voice analysis, the satisfaction degree information of the client is obtained in a mode of combining the emotional characteristics of the face of the client and the audio characteristic quantitative information, the authenticity of the obtained satisfaction degree information can be greatly improved, and the satisfaction degree of the client can be accurately obtained.
The above-described emotion recognition-based client return apparatus may be implemented in the form of a computer program that can be run on a computer device as shown in fig. 10.
Referring to fig. 10, fig. 10 is a schematic block diagram of a computer device according to an embodiment of the present invention. The computer device may be a management server for performing a client return visit method based on emotion recognition to return visit to a client to obtain satisfaction information.
Referring to fig. 10, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032, when executed, may cause the processor 502 to perform a method of customer return visit based on emotion recognition.
The processor 502 is used to provide computing and control capabilities that support the operation of the overall computer device 500.
The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503, which computer program 5032, when executed by the processor 502, causes the processor 502 to perform a client return access method based on emotion recognition.
The network interface 505 is used for network communication, such as providing transmission of data information. Those skilled in the art will appreciate that the configuration shown in fig. 10 is a block diagram of only a portion of the configuration associated with aspects of the present invention and is not intended to limit the computing device 500 to which aspects of the present invention may be applied, and that a particular computing device 500 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
The processor 502 is configured to run a computer program 5032 stored in the memory to implement the corresponding functions of the above-mentioned emotion recognition-based client return visit method.
Those skilled in the art will appreciate that the embodiment of a computer device illustrated in fig. 10 does not constitute a limitation on the specific construction of the computer device, and that in other embodiments a computer device may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components. For example, in some embodiments, the computer device may only include a memory and a processor, and in such embodiments, the structures and functions of the memory and the processor are consistent with those of the embodiment shown in fig. 10, and are not described herein again.
It should be understood that, in the embodiment of the present invention, the Processor 502 may be a Central Processing Unit (CPU), and the Processor 502 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In another embodiment of the invention, a computer-readable storage medium is provided. The computer readable storage medium may be a non-volatile computer readable storage medium. The computer-readable storage medium stores a computer program, wherein the computer program when executed by a processor implements the steps comprised in the above-mentioned emotion recognition-based client return access method.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, devices and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided by the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only a logical division, and there may be other divisions when the actual implementation is performed, or units having the same function may be grouped into one unit, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a computer-readable storage medium, which includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned computer-readable storage media comprise: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

1. A client return visit method based on emotion recognition is applied to a management server, the management server is in network connection with at least one client, and the method is characterized by comprising the following steps:
generating service return visit information according to service record information of each client in a prestored service handling information table and sending the service return visit information to a client corresponding to the client;
receiving return visit video information fed back by the client according to the service return visit information, and intercepting client image information and client audio information from the return visit video information according to a preset interception rule;
judging whether the client image information meets a preset detection condition or not;
if the customer image information meets the detection condition, acquiring a corresponding feature point set from the customer image information according to a preset facial feature acquisition model;
acquiring corresponding audio characteristic quantization information from the client audio information according to a preset audio characteristic quantization model;
and inputting the feature point set and the audio feature quantitative information into the preset satisfaction degree classification model to acquire satisfaction degree information corresponding to the return visit video information.
2. The client revisiting method based on emotion recognition as recited in claim 1, wherein said intercepting rule includes a plurality of image intercepting time points, and said intercepting client image information and client audio information from said revisiting video information according to a preset intercepting rule comprises:
acquiring voice information in the client video information as client audio information;
and acquiring a corresponding client image from the client video information according to the image interception time point to obtain client image information.
3. The customer return visit method based on emotion recognition according to claim 1, wherein the facial feature acquisition model includes a pixel contrast acquisition rule, a pixel dissolution ratio value, and a feature point acquisition rule, and the acquiring of the corresponding feature point set from the customer image information according to the preset facial feature acquisition model includes:
acquiring pixel contrast information of each customer image in the customer image information according to the pixel contrast acquisition rule;
performing pixel dissolution on each customer image according to the pixel contrast information and the pixel dissolution proportion value to obtain image contour information;
and obtaining facial feature points from each facial feature image according to a feature point acquisition rule so as to combine to obtain a feature point set.
4. The client revisiting method based on emotion recognition as recited in claim 1, wherein the audio characteristic quantization model includes an acoustic model, a characteristic dictionary and a volume characteristic calculation formula, and the obtaining of the corresponding audio characteristic quantization information from the client audio information according to a preset audio characteristic quantization model includes:
segmenting the client audio information according to the acoustic model to obtain a plurality of phonemes contained in the client audio information;
matching the phonemes according to the feature dictionary to convert the phonemes into pinyin information;
calculating to obtain a speech speed characteristic value according to the number of character pinyins contained in the pinyin information and the duration of the client audio information;
and calculating to obtain a corresponding volume characteristic value according to the volume characteristic calculation formula and the average volume of the client audio information.
5. The customer revisiting method based on emotion recognition as recited in claim 1, wherein before inputting the feature point set and the audio feature quantitative information into the preset satisfaction classification model to obtain satisfaction information corresponding to the revisit video information, further comprising:
and if the input training data set is received, performing iterative training on the satisfaction degree classification model according to a preset gradient descent training model and the training data set to obtain the trained satisfaction degree classification model.
6. The client revisit method based on emotion recognition as recited in claim 1, wherein the detection conditions include portrait outline information and SIFT feature extraction rules, and the determining whether the client image information satisfies preset detection conditions includes:
judging whether each client image of the client image information is matched with the portrait outline information or not so as to obtain a judgment result whether each client image contains a single living body or not;
if each client image contains a single living body, respectively extracting the client image characteristics of each client image according to the SIFT characteristic extraction rule;
acquiring photo features of customer photos matched with the customers in pre-stored customer data according to the SIFT feature extraction rule;
and judging whether each customer image feature is matched with the photo feature, and if so, judging that the customer image information meets the detection condition.
7. The customer revisiting method based on emotion recognition as recited in claim 1, wherein after inputting the feature point set and the audio feature quantitative information into the preset satisfaction classification model to obtain satisfaction information corresponding to the revisit video information, further comprising:
and uploading the satisfaction degree information and the client audio information as return visit preference information of the client to a block chain for storage.
8. A customer return visit device based on emotion recognition, comprising:
the service return visit information sending unit is used for generating service return visit information according to the service record information of each client in the pre-stored service handling information table and sending the service return visit information to the client corresponding to the client;
the return visit video information intercepting unit is used for receiving return visit video information fed back by the client according to the service return visit information and intercepting client image information and client audio information from the return visit video information according to a preset intercepting rule;
the detection condition judging unit is used for judging whether the client image information meets a preset detection condition or not;
a feature point set acquisition unit, configured to, if the client image information satisfies the detection condition, acquire a corresponding feature point set from the client image information according to a preset facial feature acquisition model;
the audio characteristic quantization information acquisition unit is used for acquiring corresponding audio characteristic quantization information from the client audio information according to a preset audio characteristic quantization model;
and the satisfaction degree information acquisition unit is used for inputting the feature point set and the audio feature quantitative information into the preset satisfaction degree classification model so as to acquire satisfaction degree information corresponding to the return visit video information.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the computer program implements a method of sentiment recognition based return visit to a client according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, causes the processor to carry out the emotion recognition based client revisit method as claimed in any one of claims 1 to 7.
CN202011188428.5A 2020-10-30 2020-10-30 Client return visit method and device based on emotion recognition and computer equipment Pending CN112329586A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011188428.5A CN112329586A (en) 2020-10-30 2020-10-30 Client return visit method and device based on emotion recognition and computer equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011188428.5A CN112329586A (en) 2020-10-30 2020-10-30 Client return visit method and device based on emotion recognition and computer equipment

Publications (1)

Publication Number Publication Date
CN112329586A true CN112329586A (en) 2021-02-05

Family

ID=74296418

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011188428.5A Pending CN112329586A (en) 2020-10-30 2020-10-30 Client return visit method and device based on emotion recognition and computer equipment

Country Status (1)

Country Link
CN (1) CN112329586A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112907306A (en) * 2021-05-06 2021-06-04 北京惠朗时代科技有限公司 Customer satisfaction judging method and device
CN112949495A (en) * 2021-03-04 2021-06-11 安徽师范大学 Intelligent identification system based on big data
CN113450124A (en) * 2021-06-24 2021-09-28 未鲲(上海)科技服务有限公司 Outbound method, device, electronic equipment and medium based on user behavior

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108694372A (en) * 2018-03-23 2018-10-23 广东亿迅科技有限公司 A kind of net cast customer service attitude evaluation method and device
CN108737763A (en) * 2018-05-21 2018-11-02 中国平安人寿保险股份有限公司 Video trace method, apparatus, computer equipment and storage medium
CN109766759A (en) * 2018-12-12 2019-05-17 成都云天励飞技术有限公司 Emotion identification method and Related product
CN109801105A (en) * 2019-01-17 2019-05-24 深圳壹账通智能科技有限公司 Service methods of marking, device, equipment and storage medium based on artificial intelligence
CN109961776A (en) * 2017-12-18 2019-07-02 上海智臻智能网络科技股份有限公司 Speech information processing apparatus
CN110610534A (en) * 2019-09-19 2019-12-24 电子科技大学 Automatic mouth shape animation generation method based on Actor-Critic algorithm
CN110688911A (en) * 2019-09-05 2020-01-14 深圳追一科技有限公司 Video processing method, device, system, terminal equipment and storage medium
CN110705349A (en) * 2019-08-26 2020-01-17 深圳壹账通智能科技有限公司 Customer satisfaction recognition method, device, terminal and medium based on micro expression
US20200043514A1 (en) * 2017-08-22 2020-02-06 Tencent Technology (Shenzhen) Company Limited Speech emotion detection method and apparatus, computer device, and storage medium
CN110991427A (en) * 2019-12-25 2020-04-10 北京百度网讯科技有限公司 Emotion recognition method and device for video and computer equipment

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200043514A1 (en) * 2017-08-22 2020-02-06 Tencent Technology (Shenzhen) Company Limited Speech emotion detection method and apparatus, computer device, and storage medium
CN109961776A (en) * 2017-12-18 2019-07-02 上海智臻智能网络科技股份有限公司 Speech information processing apparatus
CN108694372A (en) * 2018-03-23 2018-10-23 广东亿迅科技有限公司 A kind of net cast customer service attitude evaluation method and device
CN108737763A (en) * 2018-05-21 2018-11-02 中国平安人寿保险股份有限公司 Video trace method, apparatus, computer equipment and storage medium
CN109766759A (en) * 2018-12-12 2019-05-17 成都云天励飞技术有限公司 Emotion identification method and Related product
CN109801105A (en) * 2019-01-17 2019-05-24 深圳壹账通智能科技有限公司 Service methods of marking, device, equipment and storage medium based on artificial intelligence
CN110705349A (en) * 2019-08-26 2020-01-17 深圳壹账通智能科技有限公司 Customer satisfaction recognition method, device, terminal and medium based on micro expression
CN110688911A (en) * 2019-09-05 2020-01-14 深圳追一科技有限公司 Video processing method, device, system, terminal equipment and storage medium
CN110610534A (en) * 2019-09-19 2019-12-24 电子科技大学 Automatic mouth shape animation generation method based on Actor-Critic algorithm
CN110991427A (en) * 2019-12-25 2020-04-10 北京百度网讯科技有限公司 Emotion recognition method and device for video and computer equipment

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
手机技术资讯: "语音识别技术原理概述！", pages 1 - 10, Retrieved from the Internet <URL:https://xueqiu.com/3075605687/130174699> *
范佳慧: "基于视音频多特征融合的情感识别研究", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》, no. 7, pages 4 - 59 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112949495A (en) * 2021-03-04 2021-06-11 安徽师范大学 Intelligent identification system based on big data
CN112907306A (en) * 2021-05-06 2021-06-04 北京惠朗时代科技有限公司 Customer satisfaction judging method and device
CN112907306B (en) * 2021-05-06 2022-04-08 北京惠朗时代科技有限公司 Customer satisfaction judging method and device
CN113450124A (en) * 2021-06-24 2021-09-28 未鲲(上海)科技服务有限公司 Outbound method, device, electronic equipment and medium based on user behavior

Similar Documents

Publication Publication Date Title
CN112329586A (en) Client return visit method and device based on emotion recognition and computer equipment
CN111738357B (en) Junk picture identification method, device and equipment
CN110598019B (en) Repeated image identification method and device
CN110222728B (en) Training method and system of article identification model and article identification method and equipment
WO2021164232A1 (en) User identification method and apparatus, and device and storage medium
CN112163637B (en) Image classification model training method and device based on unbalanced data
CN108229375B (en) Method and device for detecting face image
CN111696029A (en) Virtual image video generation method and device, computer equipment and storage medium
WO2022257452A1 (en) Meme reply method and apparatus, and device and storage medium
CN110489659A (en) Data matching method and device
CN111784665B (en) OCT image quality evaluation method, system and device based on Fourier transform
CN113095333A (en) Unsupervised feature point detection method and unsupervised feature point detection device
CN111506798A (en) User screening method, device, equipment and storage medium
CN112651333B (en) Silence living body detection method, silence living body detection device, terminal equipment and storage medium
CN110717407A (en) Human face recognition method, device and storage medium based on lip language password
CN113327212B (en) Face driving method, face driving model training device, electronic equipment and storage medium
CN112967216B (en) Method, device, equipment and storage medium for detecting key points of face image
CN110348361B (en) Skin texture image verification method, electronic device, and recording medium
CN114066564A (en) Service recommendation time determination method and device, computer equipment and storage medium
CN113688206A (en) Text recognition-based trend analysis method, device, equipment and medium
CN113657293A (en) Living body detection method, living body detection device, electronic apparatus, medium, and program product
CN112819486A (en) Method and system for identity certification
CN112528140A (en) Information recommendation method, device, equipment, system and storage medium
CN115881304B (en) Risk assessment method, device, equipment and medium based on intelligent detection
CN111507743A (en) Designated driving recommendation method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination