CN120562967A - A method, device, equipment and medium for intelligent scoring of call recordings - Google Patents
A method, device, equipment and medium for intelligent scoring of call recordingsInfo
- Publication number
- CN120562967A CN120562967A CN202510659416.2A CN202510659416A CN120562967A CN 120562967 A CN120562967 A CN 120562967A CN 202510659416 A CN202510659416 A CN 202510659416A CN 120562967 A CN120562967 A CN 120562967A
- Authority
- CN
- China
- Prior art keywords
- scoring
- call
- scored
- prompt
- item
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
- G06N5/041—Abduction
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06395—Quality analysis or management
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/01—Customer relationship services
- G06Q30/015—Providing customer assistance, e.g. assisting a customer within a business location or via helpdesk
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Entrepreneurship & Innovation (AREA)
- Computational Linguistics (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Artificial Intelligence (AREA)
- Operations Research (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Tourism & Hospitality (AREA)
- Quality & Reliability (AREA)
- Mathematical Physics (AREA)
- Game Theory and Decision Science (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Finance (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Accounting & Taxation (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Machine Translation (AREA)
Abstract
The invention relates to the field of artificial intelligence, which can be applied to business system platforms of financial science and technology, medical health and the like, and discloses an intelligent scoring method, device, equipment and medium for call recording, comprising the steps of obtaining recording data to be scored; the method comprises the steps of carrying out voice recognition on recording data to be scored to obtain call text data of roles to be scored, obtaining scoring configuration files of current business scenes, respectively generating prompt languages of each scoring item according to the call text data and the scoring configuration files, carrying out scoring processing on the call text data according to corresponding scoring standards by prompting a large language model, inputting the prompt languages of each scoring item into the large language model to obtain scoring results of each scoring item, and summarizing the scoring results of each scoring item to obtain corresponding total scoring. And automatically identifying call text data of the role to be scored, and enabling the large language model to carry out automatic scoring processing according to unified scoring standards, so that the accuracy and reliability of call recording scoring are improved.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an intelligent scoring method, device, equipment and medium for call recording.
Background
In the operation process of enterprises or institutions, the quality of customer service is very important, the brand image and the customer satisfaction are directly related, and customer service dialogue quality inspection and evaluation are important links for improving the quality of customer service.
For example, in insurance business in the financial field, a collect professional is responsible for reminding customers to pay premium on time. Their communication skills, session compliance, and processing power for customer objections directly affect the client's renewal rate and satisfaction.
For another example, in the consultation business in the medical health field, the customer service personnel needs to answer the consultation call of the patient, and answer the problems about appointment registration, medical insurance reimbursement, hospital navigation and the like. Their communication ability and expertise directly affect the patient's experience of hospitalizing and the reputation of the hospital. At present, the conversation quality of customer service personnel mainly depends on manual spot check, and the problems of limited coverage range, non-uniform scoring standard, feedback lag and the like exist.
However, at present, any call quality evaluation for customer service mainly depends on manual spot check, and the problems of limited coverage range, easy missed check, non-uniform scoring standard and the like exist, so that the accuracy and reliability of call quality inspection scoring are reduced.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, the present invention aims to provide an intelligent scoring method, device, equipment and medium for call recording, which can be applied to the medical field, the financial science and technology or other related fields, and the main purpose of the present invention is to comprehensively and objectively score the call recording, and improve the accuracy and reliability of call recording scoring.
The technical scheme of the invention is as follows:
the first aspect of the present invention provides an intelligent scoring method for call recording, including:
Acquiring recording data to be scored;
Performing voice recognition on the recording data to be scored to acquire call text data of the roles to be scored;
obtaining a grading configuration file of a current business scene, wherein the grading configuration file comprises a plurality of grading items and corresponding grading standards;
Respectively generating a prompt for each scoring item according to the call text data and the scoring allocation file, wherein the prompt is used for prompting a large language model to score the call text data according to corresponding scoring standards;
And inputting the prompt of each scoring item into the large language model to obtain a scoring result of each scoring item, and collecting the scoring result of each scoring item to obtain a corresponding total scoring.
The second aspect of the present invention provides an intelligent scoring apparatus for call recording, including:
The acquisition module is used for acquiring recording data to be scored;
The voice recognition module is used for carrying out voice recognition on the recording data to be scored to acquire call text data of the roles to be scored;
the scoring standard acquisition module is used for acquiring a scoring configuration file of the current business scene, wherein the scoring configuration file comprises a plurality of scoring items and corresponding scoring standards;
The prompt generation module is used for respectively generating a prompt of each scoring item according to the call text data and the scoring allocation file, and the prompt is used for prompting a large language model to score the call text data according to corresponding scoring standards;
And the scoring module is used for inputting the prompt of each scoring item into the large language model, obtaining the scoring result of each scoring item, and obtaining the corresponding total scoring after the scoring result of each scoring item is summarized.
A third aspect of the invention provides a computer device comprising at least one processor, and
A memory communicatively coupled to the at least one processor, wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the intelligent scoring method for call recordings described above.
A fourth aspect of the present invention provides a non-volatile computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the intelligent scoring method for call recordings described above.
The method, the device and the medium have the beneficial effects that compared with the prior art, the method, the device and the medium for intelligent scoring of call records are disclosed, the method, the device and the medium for intelligent scoring of call records are characterized by acquiring call text data of roles to be scored by acquiring the to-be-scored record data, performing voice recognition on the to-be-scored record data, acquiring a scoring configuration file of a current business scene, wherein the scoring configuration file comprises a plurality of scoring items and corresponding scoring criteria, respectively generating a prompt of each scoring item according to the call text data and the scoring configuration file, wherein the prompt is used for prompting a large language model to score the call text data according to the corresponding scoring criteria, inputting the prompt of each scoring item into the large language model to obtain the scoring result of each scoring item, and summarizing the scoring result of each scoring item to obtain the corresponding total scoring item. And acquiring call text data of the role to be scored through automatic identification, and generating a prompt of each scoring item through a unified scoring configuration file to guide the large language model to carry out automatic scoring processing, so that comprehensive and objective scoring processing on call records is realized, and the accuracy and reliability of call record scoring are improved.
Drawings
In order to more clearly illustrate the solution of the present invention, a brief description will be given below of the drawings required for the description of the embodiments of the present invention, it being apparent that the drawings in the following description are some embodiments of the present invention, and that other drawings may be obtained from these drawings without the exercise of inventive effort for a person of ordinary skill in the art.
Fig. 1 is a schematic diagram of an application environment of an intelligent scoring method for call recording according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for intelligent scoring of call recordings according to an embodiment of the present invention;
fig. 3 is a flowchart of step S202 in the intelligent scoring method for call recording according to the embodiment of the present invention;
fig. 4 is a flowchart of step S203 in the intelligent scoring method for call recording according to the embodiment of the present invention;
fig. 5 is a flowchart of step S204 in the intelligent scoring method for call recording according to the embodiment of the present invention;
Fig. 6 is a schematic functional block diagram of an intelligent scoring device for call recording according to an embodiment of the present invention;
Fig. 7 is a schematic hardware structure of a computer device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail below in order to make the objects, technical solutions and effects of the present invention more clear and distinct. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. Embodiments of the present invention are described below with reference to the accompanying drawings.
The intelligent scoring method for call recording provided by the embodiment of the invention can be applied to an application environment as shown in fig. 1, and comprises a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is a medium used to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and/or wireless communication links, and the like.
The user may interact with the server 105 via the network 104 using the first terminal device 101, the second terminal device 102, the third terminal device 103, to receive or send messages etc. Various communication client applications, such as a knowledge reading class application, a web browser application, a search class application, an instant messaging tool, a mailbox client and/or social platform software, etc. (by way of example only) may be installed on the first terminal device 101, the second terminal device 102, the third terminal device 103.
The first terminal device 101, the second terminal device 102, the third terminal device 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server (merely an example) providing support for content browsed by the user with the first terminal device 101, the second terminal device 102, the third terminal device 103. The background server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device. The server 105 may be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of large management difficulty and weak service expansibility in the traditional physical hosts and VPS services ("Virtual PRIVATE SERVER" or "VPS" for short). The server 105 may also be a server of a distributed system or a server that incorporates a blockchain.
It should be noted that, the intelligent scoring method for call recording provided in the embodiment of the present application may be generally executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the intelligent scoring device for call recording provided in the embodiment of the present application may also be set in the first terminal device 101, the second terminal device 102 or the third terminal device 103. Or the intelligent scoring method for call records provided by the embodiment of the application can be generally executed by the server 105. Accordingly, the intelligent scoring device for call recording provided in the embodiment of the present application may be generally set in the server 105.
It should be understood that the above numbers of terminal devices, networks and servers are merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
As shown in fig. 2, the intelligent scoring method for call recording provided by the embodiment of the invention specifically includes the following steps:
s201, acquiring recording data to be scored.
In this embodiment, in the process of customer call, the call process is recorded by a telephone recording system, an online customer service recording tool, etc., for example, in the financial field (such as insurance sales call), the recording data is usually derived from the sales system or customer service platform of an insurance company, these systems usually have a recording function, and after the call is ended, the recording file is automatically stored in a background database (such as IOBS). And during scoring, the storage path and related attribute fields (such as recording duration, client information, conversation time and the like) of the recording file are acquired through an API interface and are used as recording data to be scored. Or the recording data stream can be directly obtained from the communication system in real time to serve as the recording data to be scored, so that scoring delay is reduced, and scoring timeliness is improved. Or in partial scenes, recording data to be scored can be obtained by manually uploading the recording file, and the method is low in data obtaining efficiency, but suitable for scenes with small data quantity, so that targeted scoring processing can be flexibly carried out.
Illustratively, in the financial field, insurance repayment personnel contact customers through the sales system of an insurance company to remind them of repayment of premium. After the call is ended, the recording file is automatically stored in a background database. The system obtains the storage path and the related attribute field of the record file through the API interface so as to prepare for the subsequent scoring processing.
In the medical health field, hospital staff answers patient consultations, such as appointment registration, medical insurance reimbursement, etc., by telephone. The call records are stored in a customer service system of a hospital, and the record files are transmitted from the customer service system to a scoring system through a data transmission tool, so that a reliable data basis is provided for subsequent voice recognition and scoring processing.
S202, performing voice recognition on the recording data to be scored to obtain call text data of the roles to be scored.
In this embodiment, the voice recognition engine performs voice recognition processing on the recording data, for example, models such as Paraformer, deepSpeech of an open source, and Paraformer is a voice recognition model based on deep learning, which can convert a voice signal into text. The method has high recognition accuracy and good performance, and is suitable for large-scale application. The audio data is converted into processable text data through a voice recognition process, and a basis is provided for subsequent text analysis and scoring. Meanwhile, different roles, such as customer service, clients, sales representatives, clients and the like, are usually involved in the conversation, so that speaking contents of the different roles in the conversation are distinguished during voice recognition, conversation text data of the roles to be scored are obtained, for example, when the customer service and the clients are in conversation, conversation text data of the customer service can be obtained in a separated mode through voice recognition processing of role separation, conversation contents of the clients are removed, and the accuracy of scoring is prevented from being influenced by speaking contents of the clients while the data amount of model processing is reduced through distinguishing processing of the roles.
In the financial field, for example, a call record of an insurance continued reception specialist and a customer is obtained, and a voice recognition section is performed on the call record through Paraformer models to distinguish the dialogue contents of the insurance continued reception specialist and the customer:
do you please ask for your own, is Zhang Sanzhen?
What are i am asking about what are i?
And (3) collecting personnel, namely, I are sales representatives of the XX insurance company, and the warranty of the user is about to expire, so that the fee is required to be collected.
The client is good and i know.
And extracting conversation text data of the role to be scored, namely the continuous receiving personnel, based on the identified conversation content to serve as a subsequent scoring basis.
In the medical health field, call records of medical services and patients are converted into texts through a voice recognition technology, and roles of the services and the patients are distinguished. For example:
do you ask you good, please ask for what is mr. Lithange?
What are i am asking about what are i am?
Customer service I are the customer service of XX hospitals, and check time reserved by you is up, please go to on time.
The patient is good and is thanks.
And extracting conversation text data of the role to be scored, namely the customer service, based on the identified conversation content as a subsequent scoring basis.
S203, obtaining a grading configuration file of the current business scene, wherein the grading configuration file comprises a plurality of grading items and corresponding grading standards.
In this embodiment, a plurality of scoring items and corresponding scoring criteria are predefined based on different business scenarios, and scoring configuration files are formed in JSON, XML and other formats and stored in a database, so as to facilitate reading and modification. During scoring processing, a scoring configuration file of the current business scene is obtained, and scoring items and scoring standards in the current business scene are obtained from the scoring configuration file, namely the scoring configuration file can be dynamically configured according to business requirements of different institutions, so that the scoring standards can be flexibly adjusted according to different business scenes.
Specifically, the financial field can formulate a scoring configuration file according to the supervision requirement of insurance sales and the internal standard of a company, the scoring items can include whether forbidden words are used (such as exaggeration of product benefits, misleading of customers and the like), whether product knowledge is accurate (such as term interpretation, rate calculation and the like), service attitudes (such as whether language is friendly, whether a customer problem is solved by tolerance and the like), the corresponding scoring standards can include forbidden words, namely, 2 points are deducted once, 20 points are deducted at the highest, 2 points are deducted once for product knowledge errors, 20 points are deducted at the highest time, 3 points are deducted once for each occurrence of service attitudes, and 30 points are deducted at the highest.
The medical field can formulate a grading configuration file according to service specifications of hospitals and communication requirements of patients, grading items can comprise whether to accurately answer patient questions (such as examination flow, expense description and the like), whether to use proper pacifying language (such as pacifying emotion of patients), service efficiency (such as whether to timely respond to patient questions and the like), and the like, and corresponding grading standards can comprise that each time the patient questions are not accurately answered, each time the grading item is buckled by 2 minutes, the highest grading item is buckled by 20 minutes, each time the grading item is buckled by 3 minutes, the highest grading item is buckled by 30 minutes, each time the grading item is buckled by 2 minutes, and the highest grading item is buckled by 20 minutes.
S204, according to the call text data and the grading allocation file, respectively generating a prompt of each grading item, wherein the prompt is used for prompting a large language model to grade the call text data according to corresponding grading standards.
In this embodiment, based on the call text data and the scoring configuration file of the role to be scored, a specific prompt is generated for each scoring item, where the format of the prompt may be a natural language description, and is used to guide a Large Language Model (LLM) to score the text data, so as to provide an explicit scoring task description for the large language model, so that the large language model can accurately score the call text data, and the accuracy and consistency of scoring are improved. Specifically, a prompt is generated for each score according to a structure of role definition-target description-task introduction-output specification, wherein the role definition is a role of defining a large language model, such as ' assume you are a supervisor, the main responsibility is to check the supervisor recording and ensure the continuous work quality of the supervisor ', the target description is a clear scoring target, such as ' the supervisor recording content, output score ', the task introduction is a detailed description scoring task, such as ' the following is a recording call between an insurance company customer service and a customer, score items are extracted one by one according to the scoring standard, and score is output according to the scoring standard ', and the output specification is an output format, such as ' the output format: score item 1 is scored 1.
Through constructing the prompt of each scoring item and the clear scoring guidance of different scoring items for the large language model based on the call text data and the scoring items and scoring standards in the scoring configuration file, the natural language processing capability of the large language model can be fully exerted, the large model can understand the input recording content and perform accurate scoring processing, and the accuracy of scoring processing is improved.
S205, inputting the prompt of each scoring item into the large language model, obtaining the scoring result of each scoring item, and obtaining the corresponding total scoring after summarizing the scoring result of each scoring item.
In this embodiment, the prompt language of each scoring item is sequentially input into a large language model, where the large language model may be an open source model, such as a GPT model, or may also use historical scoring data to fine tune the open source model, so that the open source model is more suitable for scoring tasks in different fields. The large language model analyzes the call text data according to the prompt of each scoring item, and outputs the score of each scoring item, for example:
Scoring the original text of the item 1, namely using forbidden words such as 'guarantee income', and scoring the original text of the item 1:18;
the scoring item 2 is a product knowledge error, and the score is 15;
score item 3, service attitudes are poor, score is 12;
The scoring results of the corresponding number can be obtained for the plurality of scoring items, the scoring results of each scoring item are summarized to obtain corresponding total scoring, the specific summarization can be direct addition or weighted summation, or the scoring results of all scoring items can be summarized through an Agent or other summarization algorithms, and the total scoring is calculated. The Agent summarizing means that scoring results from different scoring items are automatically received, processed and integrated by using one intelligent Agent (Agent), and the Agent can automatically complete summarizing and analyzing the scoring results according to preset rules and logic, for example, the Agent can perform weighted calculation on the different scoring items according to preset weights, or trigger specific condition judgment according to the scoring results, that is, if a certain scoring item is lower than a certain threshold value, additional alarm or processing logic may be triggered, and the like.
And generating a prompt of each scoring item through the unified scoring configuration file to guide the large language model to carry out automatic scoring processing, and realizing comprehensive and objective scoring processing on call records by utilizing the natural language processing capacity of the large language model, thereby improving the accuracy and scoring efficiency of call records scoring.
In the embodiment, the invention discloses an intelligent scoring method for call records, which comprises the steps of obtaining recording data to be scored, carrying out voice recognition on the recording data to be scored to obtain call text data of a role to be scored, obtaining a scoring configuration file of a current business scene, wherein the scoring configuration file comprises a plurality of scoring items and corresponding scoring standards, respectively generating a prompt of each scoring item according to the call text data and the scoring configuration file, wherein the prompt is used for prompting a large language model to score the call text data according to the corresponding scoring standards, inputting the prompt of each scoring item into the large language model to obtain scoring results of each scoring item, and summarizing the scoring results of each scoring item to obtain corresponding total scoring. And acquiring call text data of the role to be scored through automatic identification, and generating a prompt of each scoring item through a unified scoring configuration file to guide the large language model to carry out automatic scoring processing, so that comprehensive and objective scoring processing on call records is realized, and the accuracy and reliability of call record scoring are improved.
In one embodiment, as shown in fig. 3, step S202 includes:
s301, confirming the number of roles in the recording data to be scored;
S302, performing voice recognition processing on the recording data to be scored according to the number of the characters to obtain voice recognition texts and speaker characteristics of different characters;
s303, matching the speaker characteristics of different roles with preset speaker characteristics, and confirming the matched roles as roles to be scored;
s304, performing text optimization processing on the voice recognition text of the role to be scored to obtain call text data of the role to be scored.
In this embodiment, when performing voice recognition, the user sets the number of characters to be recognized in advance, for example, in an insurance sales call, generally only two characters of a sales representative and a customer are set, the number of characters is set to 2, if a multi-person call is performed, the number of other characters is set, and the accuracy of subsequent voice recognition and character separation is ensured by confirming the number of characters. According to the confirmed number of characters, voice recognition processing is performed on the recorded data, and model parameters (such as the number of characters) are assembled to realize character separation in the voice recognition process, for example, paraformer models can be set to recognize two characters (customer service and client) and output text content and speaker characteristics (such as tone, pitch, speed of speech and the like) of each character, for example:
character 1 (sales representative) do you, please be Mr. Zhang Sanzhen?
Character 2 (client): is me, ask what is?
Role 1 (sales representative) i am sales representative of XX insurance company, and your policy is about to expire, requiring renewal.
Role 2 (client), i know.
Based on the voice recognition result, the speaker characteristics of different roles are matched with the preset speaker characteristics, namely the speaker characteristics of the roles to be scored need to be predefined. For example, in a warranty sales call, speaker characteristics (e.g., tone, speech rate, etc.) of the sales representative may be obtained by extraction from historical recorded data, or in a medical customer service call, speaker characteristics of the customer service person may be obtained by extraction from customer service training data, etc. The speaker characteristics extracted in the voice recognition process are matched with preset speaker characteristics, and the matching algorithm can be calculated based on similarity, for example, euclidean distance, cosine similarity and other methods are used. If the matching is successful, the character is confirmed to be a character to be scored, for example, if the similarity between the characteristic of the speaker of a character and the characteristic of a preset sales representative is higher, the character is confirmed to be the sales representative, after the matching is completed, the text content of the character to be scored and the characteristic of the speaker are output, for example, the character to be scored (sales representative) is good, please ask for Zhang Sanzhun is the sales representative of XX insurance company, and the warranty of the user is about to expire, so that the fee is needed. The accuracy of the roles to be scored is ensured through feature matching, and scoring errors caused by role confusion are avoided.
Further, after matching the voice recognition text from which the character to be scored is extracted, problems may exist in the text generated by voice recognition, such as grammar errors, repeated content, incomplete sentences, etc., or the voice recognition result may include filling words such as "one's, two's, etc., or the sentence structure is incomplete, etc., which may affect the subsequent scoring process. Therefore, text optimization processing is carried out on the voice recognition text of the role to be scored, and the text optimization processing comprises grammar correction, denoising processing, integrity checking and other optimization processes, so that call text data of the role to be scored is obtained, the optimized data are clearer and more accurate, the quality and the readability of the text are improved, and the reliability of subsequent scoring is improved.
In one embodiment, step S304 includes:
performing text preprocessing on the voice recognition text of the role to be scored;
And confirming a corresponding proper noun word bank according to the current service scene, and carrying out proper noun correction on the text-preprocessed voice recognition text through the proper noun word bank to obtain call text data of the role to be scored.
In this embodiment, when performing text optimization processing on the speech recognition text of the character to be scored, text preprocessing is performed first, for example, text cleaning processing may be performed, non-language contents in the text, such as "one" or "one" filled words and possible background noise transcribed contents, in the text, or word segmentation processing may be performed, where continuous text is segmented into meaningful vocabulary units, and so on. The quality and the readability of the text are improved through text preprocessing, and subsequent scoring processing is facilitated.
And then confirming corresponding proper noun word libraries according to the current business scene, namely constructing proper noun word libraries in advance according to different field scenes, for example, the financial field can construct word libraries containing proper nouns such as insurance product names, company names, special terms and the like in advance, the word libraries can contain proper nouns such as XX insurance company, ping-Anfu, renewal and the like, the medical health field can construct word libraries containing proper nouns such as hospital names, examination item names, special terms and the like, and the word libraries can contain proper nouns such as XX hospital, CT examination, nuclear magnetic resonance and the like. And carrying out proper noun correction on the text-preprocessed voice recognition text through a pre-constructed proper noun word library.
Specifically, the text-preprocessed speech recognition text may be matched with a proper noun word library, and corrected if the vocabulary in the text is similar to but not completely identical to the proper noun in the word library, for example, "Pingfu" is corrected to "Pingfu", or corrected in combination with context information, for example, if "you need CT examination" appears in the text, but the context refers to "nuclear magnetic resonance", then "you need nuclear magnetic resonance examination". Proper nouns in the text are corrected in a matching and replacing mode, the accuracy and consistency of the text are improved, and scoring deviation caused by voice recognition errors is reduced.
In one embodiment, as shown in fig. 4, step S203 includes:
S401, acquiring a preset initial scoring configuration file, wherein the initial scoring configuration file comprises a plurality of general scoring items and corresponding scoring standards;
S402, performing service adaptation adjustment on the plurality of general scoring items and the corresponding scoring standards according to the evaluation requirements of the current service scene to obtain a scoring configuration file of the current service scene.
In this embodiment, when the scoring configuration file of the current business scenario is acquired, a preset initial scoring configuration file is acquired first, and a set of general scoring items and scoring criteria of teammates are designed according to common business requirements, where the general scoring items are applicable to multiple business scenarios, such as conversation normalization, information accuracy, service attitudes, and the like. By providing a set of common scoring terms and scoring criteria, objectivity and consistency of scoring is ensured. And then carrying out business adaptation adjustment on the initial scoring configuration file based on the evaluation requirements of different business scenes, for example, adding a specific scoring item based on the business requirements, adding a scoring item of whether a renewal reminder is timely in the financial field, or deleting the scoring item irrelevant to the current business scene, for example, deleting the scoring item of 'product accuracy' in the medical health field, and the like, and naturally adjusting the corresponding scoring standard according to the evaluation requirements of the business scenes so as to adapt to the evaluation emphasis of the different business scenes. Through business adaptation adjustment, the scoring configuration file is ensured to be suitable for the current business scene, and the accuracy and the practicability of scoring are improved.
In one embodiment, as shown in fig. 5, step S204 includes:
s501, calling a corresponding prompt template according to a current service scene, wherein the prompt template comprises a plurality of fields to be filled;
S502, generating prompt texts of all fields by using each scoring item and corresponding scoring standard in the call text data and the scoring allocation file;
s503, filling the prompt text into the field to be filled, and generating a prompt for each scoring item.
In this embodiment, a template library is pre-constructed, a prompting template of multiple service scenes is included in the template library, a corresponding prompting template is called in the template library according to the current service scene, and the template includes a plurality of fields to be filled for generating specific prompting languages, for example:
Prompting a template:
assuming you are a role, the main responsibility is to check the call recording of the business scenario, ensure business objectives.
The following is a section of conversation [ business scene ] [ conversation text ], please extract the scoring item texts one by one according to the scoring standard, and output the scoring according to the scoring standard.
The scoring criteria are as follows:
[ score item 1]: score criterion 1]
Output format:
Score 1 original:
Score 1:
The role, the business scene, the business target, the scoring item, the scoring standard and the like are the fields to be filled in the template. The method comprises the steps of obtaining prompt texts of various fields based on acquired call text data and a scoring configuration file containing scoring items and scoring standards, for example, confirming prompt texts of a role field, a business scene field and a business target field according to a business scene, for example, the role field is an insurance sales manager, the business scene field is an insurance sales call, the business target field is a sales call quality, generating the prompt texts of the call text fields according to the call text data, and generating the prompt texts of the scoring items and the scoring standards according to the scoring configuration file. And filling the text of each field into the corresponding field position in the prompt template, so as to generate the prompt of each scoring item. Through templatization and field filling, consistency of generated prompt structures and contents is ensured, meanwhile, prompts can be dynamically generated according to different business scenes and scoring items, and scoring adaptability is improved.
In one embodiment, after step S205, the method further comprises:
Comparing the total score of each record data with a preset score lower limit, screening record data with the total score lower than the preset score lower limit, and marking the record data as abnormal records;
Pushing the associated information of the abnormal record to a supervisor user for abnormal prompt.
In this embodiment, a score lower limit is set according to the service requirement and experience, and is used to determine whether the recording data is abnormal. For example, in the financial field, the lower score limit may be set to 60 points, in the medical health field, the lower score limit may be set to 70 points, or the like. Preferably, the scoring lower limit may be dynamically adjusted based on historical scoring data and business requirements, for example, using a machine learning model to automatically adjust the scoring lower limit based on historical data to accommodate changes in scoring patterns.
And comparing the total score of each record data with a preset score lower limit, and marking the record data as abnormal record if the total score is lower than the score lower limit. And generating an abnormal recording list containing paths, total scores, associated information and the like of the recording files by using all recording data marked as abnormal. And pushing the associated information of the abnormal sound recording, such as conversation time, client information, sales representative information and the like, to a supervisor user to carry out abnormal prompt, wherein the pushing mode can be mail, instant messaging tool, intra-system information and the like. Through automatic grading comparison, abnormal sound recordings are screened out rapidly, the working efficiency is improved, and meanwhile, the associated information of the abnormal sound recordings is timely notified to a supervisor user in an automatic pushing mode, so that the timeliness of problem processing is improved.
In one embodiment, the comparing the total score of each recording data with a preset score lower limit, screening recording data with a total score lower than the preset score lower limit, and after marking as abnormal recording, the method further includes:
carrying out statistical classification on abnormal sound recordings in a preset time period to obtain corresponding abnormal types;
and confirming the matched training courses according to the abnormal types, and generating a corresponding training plan.
In this embodiment, the abnormal records within the preset time period are statistically classified, for example, the past week, month or quarter, and the abnormal types are defined according to the scoring criteria and the service requirements, for example, the abnormal types may include "forbidden words are used", "product knowledge errors", "service attitudes are bad", and the like, the classified abnormal record data are statistically classified, the occurrence frequency of each abnormal type is counted, and the distribution situation of the abnormal records is intuitively displayed through statistical classification, so that the abnormal analysis and the coping process are facilitated.
A series of training courses are designed in advance according to business requirements and common problems. For example, courses such as "compliance training", "product knowledge training", and "service attitude training" may be designed, and a matching rule may be defined according to an association relationship between an anomaly type and a training course, for example, "rule matching" compliance training using forbidden words "," product knowledge error "matching" product knowledge training. The corresponding training courses are automatically matched according to the statistical classification result of the abnormal sound recordings, for example, if the statistical result shows that the use of forbidden words is a main problem, the training courses of the compliance art are matched, and the like. And generating a detailed training plan according to the matching result. The training program may include content such as training course name, training time, training object, training goal, etc., and is pushed to the relevant training user via mail, instant messaging tool, or in-system message. By matching corresponding training courses according to the abnormal types, the training plan making efficiency is improved, the training content is ensured to be closely related to the actual problems, and the training effect is improved.
It should be noted that, there is not necessarily a certain sequence between the steps, and those skilled in the art will understand that, in different embodiments, the steps may be performed in different orders, that is, may be performed in parallel, may be performed interchangeably, or the like.
With further reference to fig. 6, as an implementation of the method shown in fig. 2, the present invention provides an embodiment of an intelligent scoring apparatus for call recording, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the apparatus may be specifically applied to various electronic devices.
As shown in fig. 6, the intelligent scoring apparatus 60 for call recording according to the present embodiment includes:
the acquiring module 601 is configured to acquire recording data to be scored;
The voice recognition module 602 is configured to perform voice recognition on the recording data to be scored, and obtain call text data of the character to be scored;
The scoring standard obtaining module 603 is configured to obtain a scoring configuration file of the current business scenario, where the scoring configuration file includes a plurality of scoring items and corresponding scoring standards;
The prompt generation module 504 is configured to generate a prompt for each scoring item according to the call text data and the scoring allocation file, where the prompt is used to prompt the large language model to score the call text data according to the corresponding scoring standard;
and the scoring module 605 is configured to input the prompt for each scoring item into the large language model, obtain a scoring result of each scoring item, and aggregate the scoring result of each scoring item to obtain a corresponding total score. .
The modules referred to in the present invention refer to a series of computer program instruction segments capable of completing specific functions, and are more suitable for describing the intelligent scoring execution process of call recording than programs, and specific implementation manners of each module refer to the corresponding method embodiments and are not repeated herein.
In one embodiment, the speech recognition module 602 includes:
the number confirming unit is used for confirming the number of roles in the recording data to be scored;
The voice recognition unit is used for carrying out voice recognition processing on the recording data to be scored according to the number of the characters to obtain voice recognition texts and speaker characteristics of different characters;
The character matching unit is used for matching the speaker characteristics of different characters with preset speaker characteristics, and the matched characters are confirmed to be characters to be scored;
And the text optimization unit is used for performing text optimization processing on the voice recognition text of the role to be scored to obtain call text data of the role to be scored.
In one embodiment, the text optimization unit includes:
A text preprocessing unit, configured to perform text preprocessing on the speech recognition text of the role to be scored;
And the noun correction unit is used for confirming a corresponding proper noun word bank according to the current service scene, and carrying out proper noun correction on the voice recognition text subjected to text pretreatment through the proper noun word bank to obtain the call text data of the role to be scored.
In one embodiment, the scoring criteria acquisition module 603 includes:
the initial acquisition unit is used for acquiring a preset initial scoring configuration file, wherein the initial scoring configuration file comprises a plurality of general scoring items and corresponding scoring standards;
and the service adaptation unit is used for carrying out service adaptation adjustment on the plurality of universal scoring items and the corresponding scoring standards according to the evaluation requirements of the current service scene to obtain the scoring configuration file of the current service scene.
In one embodiment, the prompt generation module 504 includes:
The template acquisition unit is used for calling a corresponding prompting template according to the current service scene, wherein the prompting template comprises a plurality of fields to be filled;
the splicing unit is used for generating prompt texts of all fields by combining the call text data, each scoring item in the scoring allocation file and the corresponding scoring standard;
And the filling generation unit is used for filling the prompt text into the corresponding field to be filled and generating the prompt language of each scoring item.
In one embodiment, the apparatus 60 further comprises:
The abnormal recording screening module is used for comparing the total score of each recording data with a preset score lower limit, screening out recording data with the total score lower than the preset score lower limit, and marking the recording data as abnormal recording;
And the abnormality prompting module is used for pushing the associated information of the abnormal record to a supervisor user for abnormality prompting.
In one embodiment, the apparatus 60 further comprises:
the abnormality statistical module is used for carrying out statistical classification on abnormal records in a preset time period to obtain corresponding abnormality types;
and the training plan generation module is used for confirming the matched training courses according to the abnormal types and generating corresponding training plans.
In the embodiment, the invention discloses an intelligent scoring device for call recording, which is used for acquiring recording data to be scored, carrying out voice recognition on the recording data to be scored to acquire call text data of a role to be scored, acquiring a scoring configuration file of a current business scene, wherein the scoring configuration file comprises a plurality of scoring items and corresponding scoring standards, respectively generating a prompt of each scoring item according to the call text data and the scoring configuration file, wherein the prompt is used for prompting a large language model to score the call text data according to the corresponding scoring standards, inputting the prompt of each scoring item into the large language model to acquire the scoring result of each scoring item, and summarizing the scoring result of each scoring item to obtain a corresponding total scoring. And acquiring call text data of the role to be scored through automatic identification, and generating a prompt of each scoring item through a unified scoring configuration file to guide the large language model to carry out automatic scoring processing, so that comprehensive and objective scoring processing on call records is realized, and the accuracy and reliability of call record scoring are improved.
Another embodiment of the present invention provides a computer apparatus, as shown in fig. 7, a computer apparatus 70 includes:
One or more processors 701 and a memory 702, one processor 701 being illustrated in fig. 7, the processor 701 and the memory 702 being connected by a bus or other means, the connection being illustrated in fig. 7 by way of example.
The processor 701 is configured to implement various control logic of the computer device 70, which may be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a single chip microcomputer, an ARM (Acorn RISC MACHINE) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. Also, the processor 701 may be any conventional processor, microprocessor, or state machine. The processor 701 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP, and/or any other such configuration.
The memory 702 is used as a non-volatile computer readable storage medium for storing a non-volatile software program, a non-volatile computer executable program, and modules, such as program instructions corresponding to the intelligent scoring method for call recording in the embodiment of the present invention. The processor 701 executes various functional applications and data processing of the computer device 70 by running non-volatile software programs, instructions and units stored in the memory 702, i.e. implements the intelligent scoring method for call recordings in the method embodiment described above.
The memory 702 may include a storage program area that may store an operating system, application programs required for at least one function, and a storage data area that may store data created from the use of the computer device 70, etc. In addition, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 702 optionally includes memory remotely located relative to processor 701, which may be connected to computer device 70 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. One or more units are stored in memory 702 that, when executed by one or more processors 701, perform the steps of the intelligent scoring method for call recordings in any of the method embodiments described above.
In the embodiment, the invention discloses computer equipment, which is used for acquiring recording data to be scored, carrying out voice recognition on the recording data to be scored to acquire call text data of a role to be scored, acquiring a scoring configuration file of a current business scene, wherein the scoring configuration file comprises a plurality of scoring items and corresponding scoring standards, respectively generating a prompt of each scoring item according to the call text data and the scoring configuration file, wherein the prompt is used for prompting a large language model to score the call text data according to the corresponding scoring standards, inputting the prompt of each scoring item into the large language model to acquire the scoring result of each scoring item, and summarizing the scoring result of each scoring item to acquire the corresponding total scoring. And acquiring call text data of the role to be scored through automatic identification, and generating a prompt of each scoring item through a unified scoring configuration file to guide the large language model to carry out automatic scoring processing, so that comprehensive and objective scoring processing on call records is realized, and the accuracy and reliability of call record scoring are improved.
Embodiments of the present invention provide a non-transitory computer readable storage medium storing computer executable instructions that, when executed by one or more processors, perform the steps of the intelligent scoring method for call recording in any of the method embodiments described above.
In the embodiment, the invention discloses a non-volatile computer readable storage medium, which is used for acquiring recording data to be scored, carrying out voice recognition on the recording data to be scored to acquire call text data of a role to be scored, acquiring a scoring configuration file of a current business scene, wherein the scoring configuration file comprises a plurality of scoring items and corresponding scoring standards, respectively generating a prompt of each scoring item according to the call text data and the scoring configuration file, wherein the prompt is used for prompting a large language model to score the call text data according to the corresponding scoring standards, inputting the prompt of each scoring item into the large language model to acquire the scoring result of each scoring item, and summarizing the scoring result of each scoring item to obtain the corresponding total scoring. And acquiring call text data of the role to be scored through automatic identification, and generating a prompt of each scoring item through a unified scoring configuration file to guide the large language model to carry out automatic scoring processing, so that comprehensive and objective scoring processing on call records is realized, and the accuracy and reliability of call record scoring are improved.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The invention is operational with numerous general purpose or special purpose computer system environments or configurations. Such as a personal computer, a server computer, a hand-held or portable device, a tablet device, a multiprocessor system, a microprocessor-based system, a set top box, a programmable consumer electronics, a network PC, a minicomputer, a mainframe computer, a distributed computing environment that includes any of the above systems or devices, and the like. The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In summary, the intelligent scoring method, device, equipment and medium for call recording comprise the steps of obtaining recording data to be scored, carrying out voice recognition on the recording data to be scored to obtain call text data of roles to be scored, obtaining a scoring configuration file of a current service scene, wherein the scoring configuration file comprises a plurality of scoring items and corresponding scoring standards, respectively generating a prompt of each scoring item according to the call text data and the scoring configuration file, wherein the prompt is used for prompting a large language model to score the call text data according to the corresponding scoring standards, inputting the prompt of each scoring item into the large language model to obtain scoring results of each scoring item, and summarizing the scoring results of each scoring item to obtain corresponding total scoring. And acquiring call text data of the role to be scored through automatic identification, and generating a prompt of each scoring item through a unified scoring configuration file to guide the large language model to carry out automatic scoring processing, so that comprehensive and objective scoring processing on call records is realized, and the accuracy and reliability of call record scoring are improved.
Of course, those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-volatile computer readable storage medium, which when executed may comprise the steps of the above described method embodiments, to instruct related hardware (e.g., processors, controllers, etc.). The storage medium may be a memory, a magnetic disk, a floppy disk, a flash memory, an optical memory, etc.
It should be noted that, if a software tool or component other than the company appears in the embodiment of the present application, the embodiment is merely presented by way of example, and does not represent actual use. It is to be understood that the application is not limited in its application to the examples described above, but is capable of modification and variation in light of the above teachings by those skilled in the art, and that all such modifications and variations are intended to be included within the scope of the appended claims.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510659416.2A CN120562967A (en) | 2025-05-21 | 2025-05-21 | A method, device, equipment and medium for intelligent scoring of call recordings |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510659416.2A CN120562967A (en) | 2025-05-21 | 2025-05-21 | A method, device, equipment and medium for intelligent scoring of call recordings |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN120562967A true CN120562967A (en) | 2025-08-29 |
Family
ID=96820763
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202510659416.2A Pending CN120562967A (en) | 2025-05-21 | 2025-05-21 | A method, device, equipment and medium for intelligent scoring of call recordings |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN120562967A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN121214947A (en) * | 2025-11-28 | 2025-12-26 | 深圳民航凯亚有限公司 | Customer service voice intelligent quality inspection method |
-
2025
- 2025-05-21 CN CN202510659416.2A patent/CN120562967A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN121214947A (en) * | 2025-11-28 | 2025-12-26 | 深圳民航凯亚有限公司 | Customer service voice intelligent quality inspection method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11004013B2 (en) | Training of chatbots from corpus of human-to-human chats | |
| CN109783632B (en) | Customer service information pushing method and device, computer equipment and storage medium | |
| US10636047B2 (en) | System using automatically triggered analytics for feedback data | |
| US11120798B2 (en) | Voice interface system for facilitating anonymized team feedback for a team health monitor | |
| US12299694B2 (en) | Dynamic claims submission system | |
| CN112417158A (en) | Training method, classification method, device and equipment of text data classification model | |
| US12271706B2 (en) | System and method for incremental estimation of interlocutor intents and goals in turn-based electronic conversational flow | |
| CN105261362A (en) | Conversation voice monitoring method and system | |
| US20250117854A1 (en) | Generating portfolio changes based on upcoming life event | |
| US20250119494A1 (en) | Automated call list based on similar discussions | |
| US20250117856A1 (en) | Goal tracking and goal-based advice generation | |
| CN111782809A (en) | Data processing method, device, electronic device and storage medium for balance of payments network application | |
| US20250117629A1 (en) | Generating a call script based on conversation | |
| US20250119495A1 (en) | Real-time dynamic visualization of content | |
| CN110543550B (en) | Method and device for automatically generating test questions | |
| US20160283876A1 (en) | System and method for providing automomous contextual information life cycle management | |
| US20250117855A1 (en) | Predicting performance of a portfolio with asset of interest | |
| US20250117853A1 (en) | Portfolio generation based on missing asset | |
| CN110489747A (en) | A kind of image processing method, device, storage medium and electronic equipment | |
| CN112905745A (en) | Information processing method, information processing device, electronic equipment and computer readable storage medium | |
| CN120562967A (en) | A method, device, equipment and medium for intelligent scoring of call recordings | |
| CN119359252A (en) | A method, device and storage medium for implementing a business intelligent flow engine | |
| CN112131378B (en) | Method and device for identifying civil problem category and electronic equipment | |
| US20250308398A1 (en) | Content generation related policy drift | |
| CN119416785A (en) | A two-stage named entity recognition method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |