CN111309715B - Call scene identification method and device - Google Patents
Call scene identification method and device Download PDFInfo
- Publication number
- CN111309715B CN111309715B CN202010042390.4A CN202010042390A CN111309715B CN 111309715 B CN111309715 B CN 111309715B CN 202010042390 A CN202010042390 A CN 202010042390A CN 111309715 B CN111309715 B CN 111309715B
- Authority
- CN
- China
- Prior art keywords
- call
- scene
- belonging
- call set
- subset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a method and a device for identifying a call scene, wherein the method comprises the following steps: acquiring a marked call set and an unmarked call set belonging to a first scene from the call sets; determining a reliable negative sample which does not belong to a first scene from the unlabeled call set according to a first classifier obtained based on the call features of the labeled call set and the unlabeled call set; performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a first scene recognition model; a first subset of calls belonging to a first scenario and a second subset of calls not belonging to the first scenario are identified from the set of calls by a first scenario identification model. The scheme provided by the application can realize the recognition of the call scene.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for recognizing a call scenario, a computer-readable storage medium, and a computer device.
Background
With the rapid development of internet technology and intelligent terminals, various instant messaging tools have become popular and become a common communication mode among users. The user can establish a call group through the instant messaging tool, and multi-person communication among group members can be realized through the call group.
The scenes that a majority of users initiate multi-person calls in the call group can be roughly divided into a working scene, a chatting scene and other scenes, and the number of the initiated multi-person calls belonging to different call scenes can provide guidance for the functional design of the instant messaging tool. For example, besides supporting multi-person calls, the call group can provide functions of red envelope giving, expression giving, file giving, business card giving and the like, and the number of multi-person calls in different call scenes can provide reference for the development sequence of the functions. However, how to identify a scenario of a multi-person call initiated in a call group does not exist in the prior art.
Disclosure of Invention
Based on this, it is necessary to provide a call scenario identification method, apparatus, computer-readable storage medium, and computer device for solving the problem that a scenario of a multi-person call initiated in a call group cannot be identified in the prior art.
A method for recognizing call scenes comprises the following steps:
acquiring a marked call set and an unmarked call set belonging to a first scene from a call set;
determining a reliable negative sample which does not belong to a first scene from the unmarked call set according to a first classifier obtained based on the call features of the marked call set and the unmarked call set;
performing classification training according to the call characteristics of the labeled call set and the reliable negative sample to obtain a first scene recognition model;
a first subset of calls belonging to a first scenario and a second subset of calls not belonging to the first scenario are identified from the set of calls by the first scenario identification model.
An apparatus for identifying a call scenario, the apparatus comprising:
the acquiring module is used for acquiring a marked call set and an unmarked call set which belong to a first scene from the call set;
the reliable negative sample determining module is used for determining a reliable negative sample which does not belong to a first scene from the unmarked call set according to a first classifier obtained based on the call features of the marked call set and the unmarked call set;
the training module is used for carrying out classification training according to the call characteristics of the labeling call set and the reliable negative sample to obtain a first scene recognition model;
and the identification module is used for identifying a first call subset belonging to a first scene and a second call subset not belonging to the first scene from the call set through the first scene identification model.
A computer-readable storage medium, storing a computer program which, when executed by a processor, causes the processor to perform the steps of a method of identifying a call scenario.
A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to carry out the steps of a method of identification of a call scenario.
After a small number of marked call sets belonging to a first scene are determined from the call sets, an unlabelled call set is formed according to remaining calls in the call sets, then training is performed according to call characteristics of all calls in the current marked call set and the unlabelled call set to obtain a first classifier, a reliable negative sample which does not belong to the first scene is determined from the unlabelled call set according to the first classifier, and then classification training is performed according to the call characteristics of all calls in the marked call set and the reliable negative sample to obtain a first scene recognition model which can recognize whether the call scene is the first scene, so that a first call subset belonging to the first scene and a second call subset not belonging to the first scene can be recognized from the call sets through the first scene recognition model, and recognition of the call scene is realized.
Drawings
FIG. 1 is a diagram of an exemplary embodiment of a call scenario recognition method;
FIG. 2 is a block diagram of a method for identifying a call scenario in one embodiment;
FIG. 3 is a schematic diagram of feature engineering set up for a call in one embodiment;
FIG. 4 is a flowchart illustrating a method for identifying a call scenario in one embodiment;
FIG. 5 is a diagram illustrating a logarithmic transformation of feature data in one embodiment;
FIG. 6 is a flow diagram illustrating the determination of reliable negative examples that do not belong to a first scenario, under an embodiment;
FIG. 7 is a flow diagram illustrating the process of identifying calls belonging to a first scenario in an exemplary embodiment;
FIG. 8 is a flow diagram that illustrates the identification of a call belonging to a second scenario from a call set, in one embodiment;
FIG. 9 is a flow diagram illustrating the determination of reliable negative examples that do not belong to the second scenario in one embodiment;
FIG. 10 is a flow diagram illustrating the identification of a call belonging to a second scenario in an exemplary embodiment;
FIG. 11 is a general flow diagram illustrating a call scenario for identifying individual calls in a call set, according to one embodiment;
FIG. 12 is a block diagram showing the structure of a call scene recognition apparatus according to an embodiment;
FIG. 13 is a block diagram that illustrates the architecture of a computing device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Fig. 1 is an application environment diagram of a call scenario recognition method in one embodiment. Referring to fig. 1, the method for identifying a call scenario is applied to a system for identifying a call scenario. The call scenario recognition system includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected through a network. The terminal 110 may specifically be a desktop terminal or a mobile terminal, and the mobile terminal may specifically be at least one of a mobile phone, a tablet computer, a notebook computer, and the like. The server 120 may be implemented as a stand-alone server or a server cluster composed of a plurality of servers.
In one embodiment, the terminal 110 runs an instant messaging client, and a user can establish a talk group through the instant messaging client and initiate a multi-user instant messaging session in the talk group. The talk group may be at least one of a discussion group, a multi-person video talk group, and a multi-person audio talk group. The server 120 may provide support for an instant messaging service provided by an instant messaging client, and the server 120 may further record feature data corresponding to a multi-person call initiated by a user on the terminal 110, and obtain a call feature corresponding to each call according to the feature data.
In one embodiment, the server may obtain calls of a large number of users within a period of time as a call set, count feature data of each call in the call set within the period of time, and obtain call features corresponding to each call in the call set according to the feature data. The aforementioned period of time may be one week, one month, three months, etc., and may be selected according to specific needs. Further, the server can also acquire a marked call set and an unmarked call set belonging to the first scene from the call set; determining a reliable negative sample which does not belong to a first scene from the unlabeled call set according to a first classifier obtained based on the call features of the labeled call set and the unlabeled call set; performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a first scene recognition model; a first subset of calls belonging to a first scenario and a second subset of calls not belonging to the first scenario are identified from the set of calls by a first scenario identification model.
The call scene is a use scene of an initiated multi-person call, and can be divided into a work scene, a chat scene and other scenes. The work scenario is generally a usage scenario of a multi-person call initiated for efficient communication purposes, such as a work conference, an online course, an online discussion, an online training, and so on. The chat scene is generally a use scene of a multi-person call initiated for emotional communication, such as family communication, friend chat, friend-friend making, interactive entertainment, and the like. Other scenarios are usage scenarios of multi-person calls other than the two call scenarios, in which the activity of the call group in which the initiated multi-person call is located is low, or the call group in which the initiated multi-person call is located is a temporary call group, and may be defined as an accidentally initiated multi-person call. According to the method for identifying the call scene in the embodiment of the application, each call in the call set can be identified as belonging to one of a working scene, a chatting scene or other scenes.
The server can identify the call scene of each call according to the call characteristics of each call in the obtained call set. Of course, in some embodiments, the terminal may also obtain a marked call set and an unmarked call set belonging to the first scenario from the call set; determining a reliable negative sample which does not belong to a first scene from the unlabeled call set according to a first classifier obtained based on the call features of the labeled call set and the unlabeled call set; performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a first scene recognition model; a first subset of calls belonging to a first scenario and a second subset of calls not belonging to the first scenario are identified from the set of calls by a first scenario identification model.
Fig. 2 is a schematic diagram of a method for identifying a call scenario in an embodiment. Referring to fig. 3, the method mainly comprises four steps, wherein the first step is feature selection, namely, a feature project is set up, and feature data of each call in a call set is obtained, wherein the feature data comprises call duration features, call user portrait features, features of a call group where the call is located, and other behavior features during the call; the second step is data cleaning, namely preprocessing the characteristic data of each call in the call set to obtain call characteristics, including performing principal component analysis on the characteristic data and extracting principal component characteristics, and also including performing logarithmic transformation on the characteristic data to obtain uniformly distributed characteristic data; the third step is model training, which is used for selecting proper positive samples and negative samples from the call set, determining reliable negative samples and diffusing the positive samples in the selection process, and then performing model iterative training according to the determined positive samples and negative samples to obtain a call scene identification model for identifying call scenes; and the fourth step is total prediction, namely predicting and verifying all calls in the call set by using a trained call scene recognition model so as to classify the call scenes of all calls in the call set.
As shown in fig. 3, which is a schematic diagram of a feature engineering set up for multi-person conversation in an embodiment, the feature engineering takes multi-person conversation initiated each time as a statistical object, and divides characteristic data of conversation into four categories, including conversation duration characteristics, conversation user portrait characteristics, characteristics of a conversation group where the conversation is located, and other behavior characteristics during the conversation.
The call duration characteristics are characteristics related to call duration of a multi-person call initiated within a period of time, and include whether the call is a workday, a call hour, a call number, a call longest duration, a call duration variance, a call duration mean, a call duration sum, a video longest duration, a video duration median, a video duration mean, an audio longest duration, an audio duration median, an audio duration mean, an audio duration sum, and the like. It will be appreciated that since the call duration characteristic is statistically derived from the durations of a plurality of calls initiated over a period of time, the data in the dimension of the call duration characteristic is the same for each call initiated over the period of time.
The call user profile feature is a profile feature of a user participating in a multi-person call initiated at the current time, and includes the number of male users participating in the multi-person call at the current time, the male user ratio, the number of users using the iOS system, the user ratio using the iOS system, the number of users using WiFi, the user ratio using WiFi, the number of users having location information of a first-line city, the number of users having location information of a second-line city, the number of users having location information of a third-line city, the user ratio of a first-line city, the user ratio of a second-line city, the user ratio of a third-line city, the maximum age of a participating call member, the minimum age of a participating call member, the average age of a participating call member, and the like. It will be appreciated that the call user profile feature is a feature that is personalized to each call, with the data for each call differing in the dimension of the call user profile feature.
The call group in which the call is located is characterized by feature data of the call group in which a multi-person call initiated in a period of time belongs in the period of time, and mainly comprises the number of text messages sent in the call group in the period of time, the number of picture messages sent, the number of audio messages sent, the number of business card messages sent, the number of red packets sent, the number of text messages sent in working days, the number of picture messages sent in working days, the number of audio messages sent in working days, the number of business card messages sent in working days, the number of text messages sent in working time, the number of picture messages sent in working time, the number of audio messages sent in working time, the number of business card messages sent in working time, the number of video messages sent in working time and the like. It can be understood that, since the feature of the talk group in which the talk is located describes the feature of the talk group in which the talk is located, the feature data of the talk initiated in the same talk group in the feature dimension of the talk group is the same in this period of time.
Other behavior characteristics during the call are characteristics counted according to the user behavior in the current call process, and include the number of text messages sent during the call, the number of picture messages sent during the call, the number of audio messages sent during the call, the number of business card messages sent during the call, the number of video messages sent during the call, the minimum click or click of clicking during the call, the mute click or click of clicking during the call, the invitation of a new member to join the call during the call, and the like. It will be appreciated that the other behavioral characteristics during a call are characteristics that are personalized to each call, with the individual calls differing in data in the other behavioral characteristic dimensions during the call.
From the established feature engineering, the feature data corresponding to each call specifically includes feature data on a plurality of feature dimensions, including feature data with commonality and personalized feature data, and the feature dimensions can be divided into discrete features and continuous features according to different values of the features. In order to better represent each call by using the feature data, the computer device may perform preprocessing on the feature data of each call, obtain call features of each call, and implement recognition of a call scenario based on the call features.
As shown in fig. 4, in one embodiment, a method for identifying a call scenario is provided. The embodiment is mainly exemplified by applying the method to a computer device (such as the terminal 110 or the server 120 in fig. 1). Referring to fig. 4, the method for identifying a call scenario specifically includes the following steps:
s402, a marked call set and an unmarked call set belonging to a first scene are obtained from the call set.
The call set is a set formed by a plurality of calls, and the computer equipment can obtain the call set according to a plurality of calls initiated in the instant messaging client within a period of time by a user and obtain the call characteristics corresponding to each call in the call set. The computer device needs to obtain a scene recognition model for recognizing a call scene, and needs to obtain a positive sample and a negative sample for training the scene recognition model, so that the computer device can divide a call set into a marked call set and an unmarked call set belonging to a first scene.
Specifically, the computer device may obtain calls belonging to the first scene from the call set, and obtain a labeled call set belonging to the first scene after labeling the calls as belonging to the first scene, where the labeled call set is an initial positive sample used for training the first scene recognition model, and the initial positive sample is less in number, and is directly obtained from the call set, and is not necessarily completely accurate, and may perform diffusion and correction in a subsequent iteration process. The negative sample used for training the first scene recognition model in the call set is absent, and the call which does not belong to the first scene cannot be determined from the call set, so that the computer device can divide the call which is not divided into the marked call set in the call set into the unmarked call set, wherein the unmarked call set may include the call which belongs to the first scene, and may also include the call which does not belong to the first scene.
In one embodiment, the step of the computer device obtaining the marked call set marked as the first scene from the call set comprises: acquiring a first preset word corresponding to a first scene; acquiring a group name of a call group to which each call in a call set belongs; screening out a conversation group with the group name matched with a first preset word; and obtaining a marked call set belonging to the first scene according to the call initiated in the screened call group.
In this embodiment, the marked call set belonging to the first scenario may be determined according to the group name of the call group where the call is initiated. Specifically, the computer device may set a first preset word corresponding to a first scene, for example, the first scene is a work scene, and the first preset word may be at least one word of words such as "work", "company", "meeting", and the like. For each call in the call set, the computer device may obtain a group name of a call group to which each call belongs, screen out a call group whose group name matches a set first preset word from the call groups, and mark a call initiated in the screened call group as a call belonging to a first scene, thereby obtaining a marked call set belonging to the first scene. It can be seen that the marked call set belonging to the first scenario may include multiple calls initiated in the same call group.
S404, according to a first classifier obtained based on the call features of the marked call set and the unmarked call set, determining a reliable negative sample which does not belong to the first scene from the unmarked call set.
As mentioned earlier, the call feature is generated from feature data generated by the user based on various operations performed by the instant messaging client when initiating a multi-person call, the feature data including feature data in various feature dimensions. The computer equipment can acquire the feature data corresponding to each call in the call set according to the set feature dimensions, and preprocess the feature data to acquire the call features corresponding to each call. And the computer equipment trains a second classifier based on the call characteristics of all calls in the marked call set and the call characteristics of all calls in the unmarked call set to obtain a first classifier, and then adopts the trained first classifier to identify according to the call characteristics of all calls in the unmarked call set, so as to determine a reliable negative sample which does not belong to the first scene. The second classifier may be a naive bayes classifier or a classifier based on XGBoost (eXtreme Gradient Boosting model).
In one embodiment, the step of the computer device pre-processing the feature data comprises: carrying out normality test on the feature data under the continuous feature dimension; and when the test result indicates that the feature data do not conform to normal distribution, carrying out logarithmic transformation on the feature data under the feature dimensionality to obtain the transformed feature data.
The continuous characteristic dimension refers to a value of the characteristic being a continuous value, for example, a value of "the number of text messages sent during a call" may be any value, and the characteristic dimension is a continuous characteristic dimension. Specifically, the computer device may obtain feature data of each call in the call set in the continuous feature dimension, perform a normality test on the feature data, and when a test result indicates that the feature data does not conform to a normal distribution or that the data has a severe inclination, the computer device may perform a logarithmic transformation on the feature data of each call in the continuous feature dimension to obtain transformed feature data.
In one embodiment, the computer device may perform a logarithmic transformation on the feature data according to the following formula:
x' = ln (1 + θ X), where X ’ The feature data after transformation is represented, X represents the feature data before transformation, and θ is a transformation parameter and is usually set to 1.
Fig. 5 is a schematic diagram of transformed feature data obtained by performing logarithmic transformation on data in feature dimensions of a call duration mean value calltime _ mean, a number of sent text messages msg _ text, and a video duration mean value video _ mean in one embodiment. It can be seen that after logarithmic transformation, the data distribution is more biased to the normal distribution, and the influence of the inclination of the data distribution is reduced.
In this embodiment, by performing logarithmic transformation on the feature data, the distribution of the data can be made to approach a normal distribution and to be unrelated to the average value of the data, and the size relationship of the data itself is not changed, and the values of the originally densely distributed intervals are dispersed as much as possible, and the values of the originally dispersed intervals are aggregated as much as possible.
In one embodiment, the step of the computer device pre-processing the feature data comprises: acquiring feature data of various feature dimensions corresponding to each call in a call set; and extracting the principal component from the feature data to obtain the call feature of each call.
Specifically, principal component analysis extracts principal components from more feature dimensions through orthogonal transformation, obtains data with fewer dimensions, can eliminate mutual influence of data under different feature dimensions, can express original data as much as possible by using shorter data, and achieves dimension reduction of feature data.
In one embodiment, extracting the principal component from the feature data to obtain the call feature of each call includes: performing decentralized processing on the feature data of various feature dimensions, and calculating a covariance matrix; calculating the eigenvalue of the covariance matrix and the eigenvector corresponding to each eigenvalue; obtaining a principal component matrix according to the eigenvector corresponding to the selected eigenvalue; and performing linear transformation on the feature data according to the principal component matrix to obtain the call features of each call.
For example, the call set includes n calls, each call includes features in m feature dimensions, each feature has its own value, and there are m kinds of features, and data in each feature dimension is n dimensions, that is, the original feature data is an m × n matrix a. Characteristic data in the i-th characteristic dimension is Xi = (Z1, Z2, ..., zn), wherein i =1, 2, \8230, m, and the characteristic data in the characteristic dimension is subjected to a decentralization process to obtain Xi' = (Z1- μ, Z2- μ, ... zn- μ),then obtaining an m × n matrix B according to the feature data after centralization processing under each feature dimension, calculating a covariance matrix according to the matrix B to obtain an m × m matrix C, calculating an eigenvalue λ of the covariance matrix C and a corresponding eigenvector P by using matrix knowledge, wherein m eigenvalues are total, sorting the m eigenvalues according to a descending order, selecting the largest first k eigenvalues, obtaining a principal component matrix P according to the corresponding k eigenvectors, wherein P is a k × m matrix, and finally projecting the original feature data A onto the principal component matrix P to obtain a k × n matrix Y which comprises new call dimension reduction dataThe k-dimensional data of (2) can be used as call characteristics of each call.
In one embodiment, the computer device may perform a principal component analysis after performing a logarithmic transformation on the feature data in the continuous feature dimension.
In one embodiment, the step S404 of determining a reliable negative example that does not belong to the first scenario from the unlabeled call set according to a first classifier obtained based on call features of the labeled call set and the unlabeled call set includes: selecting a spy sample from a marked call set belonging to a first scene; after spy samples are removed from the marked call set, training positive samples are obtained; after the spy sample is added to the call set which is not marked, a training negative sample is obtained; obtaining a first classifier according to the call characteristics of the training positive sample and the training negative sample; obtaining the classification probability of each call in the call set which is not marked through a first classifier; determining a probability threshold value for judging the call as not belonging to the first scene according to the classification probability of the spy sample; and taking the calls of which the classification probability corresponding to the call sets which are not marked is smaller than the probability threshold as reliable negative samples which do not belong to the first scene.
Specifically, the computer device may set the call set as a, the marked call set belonging to the first scenario as P, the unmarked call set as U, the reliable negative sample not belonging to the first scenario as RN, and the initial RN as an empty set. Then, randomly selecting a part of the P as a spy sample S according to a preset proportion (for example, 15%), and after the spy sample S is removed from the marked call set P, obtaining a training positive sample PS, wherein PS = P-S, and the label of each call in the PS is 1; and after the spy sample S is added into the unlabeled call set U, obtaining a training negative sample US, wherein US = U + S, and the label of each call in US is-1. Then, according to the call characteristics of each call in the PS and the US, performing classification training to obtain a first classifier g1, and predicting each call in the U through the first classifier g1 to obtain the classification probability p1 (d) of each call d in the U. And finally, sorting the classification probabilities of all calls in the spy sample S, taking the classification probability corresponding to the call ranked later (for example, ranked 10% later) as a probability threshold value threshold not belonging to the first scene, and adding d to RN when p1 (d) < threshold, thereby obtaining a reliable negative sample not belonging to the first scene in U.
In one embodiment, the method further comprises: and after reliable negative samples which do not belong to the first scene are removed from the unlabelled call set in the iterative execution, obtaining an updated unlabelled call set, and determining the reliable negative samples which do not belong to the first scene from the updated unlabelled call set according to the call characteristics of the labeled call set and the updated unlabelled call set until the iteration number meets the preset number.
In this embodiment, in order to obtain more reliable negative samples that do not belong to the first scenario from the unlabeled call set, the computer device may repeat the iteration until the number of iterations satisfies a preset number or it is impossible to obtain more reliable negative samples from the unlabeled call set, for example, the iteration may be performed for 15 times. The computer equipment can remove the reliable negative samples which are determined at the previous time and do not belong to the first scene from the original unlabelled call set U to obtain an updated unlabelled call set, then carries out classification training according to the call features of all calls in the updated unlabelled call set and the labeled call set P, and determines the reliable negative samples which do not belong to the first scene from the updated unlabelled call set again through a classifier obtained through training.
Fig. 6 is a schematic flow chart of determining reliable negative examples that do not belong to the first scenario from the call set in one embodiment. Referring to fig. 6, first, a marked call set P belonging to a first scenario is obtained from the call sets, and other calls in the call sets are added to an unmarked call set U, for example, the first scenario is a working scenario, and a call whose group name of a call group in the call set includes "work" may be added to the marked call set P. Then, randomly selecting a call with a preset proportion from U as a spy sample S, and marking P-S as PS and 1,U + S as US and-1. And then, performing classification training by using the call characteristics of each call in the PS and the US to obtain a first classifier, and obtaining the classification probability of each call through the first classifier. And finally, determining a probability threshold value serving as a negative sample according to the classification probability of each call in the spy sample S, and adding the call of which the classification probability in the U is smaller than the probability threshold value as a reliable negative sample to the RN. And repeating the process, and retraining the classifier according to the remaining unlabeled samples and the labeled call set P in the U until the iteration times reach a preset number, and dividing the original call set into the labeled call set P belonging to the first scene, the reliable negative sample RN not belonging to the first scene and the remaining unlabeled call set U at the moment.
S406, performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a first scene recognition model.
Specifically, the first scene recognition model is a model having the capability of recognizing whether or not a call belongs to the first scene based on call characteristics of the call. The first scene recognition model is a classification model, and the recognition result of the call is that the call belongs to one of the first scene and the call does not belong to the first scene. After the computer device obtains the marked call set belonging to the first scene and the reliable negative sample not belonging to the first scene, classification training can be performed based on call characteristics of all calls in the set to obtain a first scene recognition model.
In one embodiment, in step S406, performing classification training according to the call features of the labeled call set and the reliable negative examples, and obtaining a first scene recognition model, including: dividing a training set and a verification set according to the marked call set and the reliable negative sample; training a second classifier according to the call characteristics of the training set to obtain a first scene recognition model; and verifying the obtained first scene recognition model according to the call characteristics of the verification set.
Specifically, when performing classification training according to a labeled call set belonging to a first scene and a reliable negative sample not belonging to the first scene, the computer device may divide the labeled call set and the reliable negative sample into a training set and a verification set according to a preset proportion, for example, 7. The second classifier may be a naive bayes classifier or an XGBoost based classifier.
The computer device can also verify the accuracy of the first scene recognition model obtained by training by using the call characteristics corresponding to the calls belonging to the first scene and the calls not belonging to the first scene in the verification set. When the verification fails, after step S408, dividing the call set into a new marked call set and an unmarked call set belonging to the first scenario according to the first call subset may be performed iteratively; determining a reliable negative sample which does not belong to a first scene from an unlabelled call set according to a first classifier obtained based on call features of the labeled call set and the unlabelled call set; performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a first scene recognition model; and identifying a first call subset belonging to the first scene and a second call subset not belonging to the first scene from the call set through the first scene identification model until the first scene identification model obtained after the iteration passes the verification or the iteration number meets the preset number.
S408, a first conversation subset belonging to the first scene and a second conversation subset not belonging to the first scene are identified from the conversation set through the first scene identification model.
Specifically, after obtaining the first scene recognition model, the computer device may perform full prediction on the call set by using the obtained first scene recognition model, that is, predict the call scene to which each call in the original call set belongs, thereby determining that the first call subset belonging to the first scene in the call set does not belong to the second call subset of the first scene.
In one embodiment, identifying a first subset of calls belonging to a first scenario and a second subset of calls not belonging to the first scenario from the set of calls by a first scenario identification model comprises: acquiring call characteristics of each call in a call set; inputting call characteristics of each call into a first scene recognition model; and obtaining classification probability corresponding to each call according to the call characteristics through the first scene recognition model, and dividing the call set into a first call subset belonging to the first scene and a second call subset not belonging to the first scene according to the classification probability.
The classification probability may be used to reflect the degree of possibility that the call belongs to the first scenario, and a higher classification probability may indicate that the call belongs to the first scenario with a higher possibility, whereas a lower possibility belongs to the first scenario. Therefore, the computer device can identify the call scene of the call according to the classification probability obtained by the first scene identification model, divide the call with the classification probability greater than the preset threshold into a first call subset belonging to the first scene, and divide the call with the classification probability less than the preset threshold into a second call subset not belonging to the first scene.
In one embodiment, after step S408, the method further comprises: the iteration execution divides the call set into a new marked call set and an unmarked call set which belong to a first scene according to the first call subset; determining a reliable negative sample which does not belong to a first scene from an unlabelled call set according to a first classifier obtained based on call features of the labeled call set and the unlabelled call set; performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a first scene recognition model; and identifying a first call subset belonging to the first scene and a second call subset not belonging to the first scene from the call set through a first scene identification model until an iteration stop condition is met.
In this embodiment, since the number of calls in the initial labeled call set belonging to the first scene obtained from the call set is small, after the call set is divided into the first call subset belonging to the first scene and the second call subset not belonging to the first scene by the first scene identification model, the computer device may use the first call subset as a new labeled call set belonging to the first scene, that is, correct and diffuse the initial labeled call set belonging to the first scene according to the prediction result of the first scene identification model for full prediction, so as to obtain more accurate and more reliable positive samples, and thus the call set is divided into the new call set belonging to the first scene and the unlabeled call set, and based on this, the above steps S402-S408 are iteratively executed until a new first scene identification model is obtained, and the full prediction is performed again by using the new first scene identification model until the number of iterations reaches a preset number or the prediction result of the first scene identification model obtained by the last iteration is expected. It can be understood that each iteration process realizes the correction and diffusion of the marked call set belonging to the first scenario.
The computer device may verify the recognition rate of the first scene recognition model obtained for the first time or during subsequent iterations. The verification method can include the following steps: the first method is to verify the first scene recognition model by using a verification set reserved during training of the first scene recognition model, namely, verify the obtained first scene recognition model by using call features of a call set and a reliable negative sample marked in the verification set, and if the recognition rate of the verification set does not meet a preset condition, continue iteration if the currently obtained first scene recognition model does not pass verification. The second is that the recognition rate of calls initiated in a call group whose group name includes the first preset word may be counted according to the full prediction result, for example, the recognition rate of calls initiated in a call group whose group name includes "training" may be counted, and the recognition rate of calls initiated in a call group whose group name includes "shipping" may be counted, and when the recognition rates of these calls do not satisfy the preset threshold, the first scene recognition model obtained currently does not meet the expectation, and the iteration may be continued. And thirdly, randomly extracting N calls, manually determining call scenes to which the N calls belong according to group names of call groups in which the N calls belong, and when the recognition rate of the N calls in the total prediction result does not meet a preset threshold value, obtaining a first scene recognition model at present, wherein the first scene recognition model does not meet the expectation, and the iteration can be continued.
According to the method for recognizing the call scene, after a small number of marked call sets belonging to a first scene are determined from the call sets, an unlabelled call set is formed according to remaining calls in the call sets, then training is carried out according to call characteristics of all calls in the current marked call set and the unlabelled call set to obtain a first classifier, a reliable negative sample which does not belong to the first scene is determined from the unlabelled call set according to the first classifier, then classification training is carried out according to the call characteristics of all calls in the marked call set and the reliable negative sample to obtain a first scene recognition model capable of recognizing whether the call scene is the first scene, and therefore a first call subset belonging to the first scene and a second call subset not belonging to the first scene can be recognized from the call sets through the first scene recognition model, and recognition of the call scene is achieved.
Fig. 7 is a flowchart illustrating a process of identifying a call belonging to a first scenario from a call set in a specific embodiment. Referring to fig. 7, the method specifically includes the following steps:
s702, acquiring a marked call set and an unmarked call set belonging to a first scene from the call sets;
s704, selecting a spy sample from a marked call set belonging to a first scene;
s706, after spy samples are removed from the marked call set, training positive samples are obtained;
s708, after the spy sample is added to the call set which is not marked, a training negative sample is obtained;
s710, training according to the call characteristics of the training positive sample and the training negative sample to obtain a first classifier;
s712, obtaining the classification probability of each call in the call set which is not marked through the first classifier;
s714, determining a probability threshold value for judging the call as not belonging to the first scene according to the classification probability of the spy samples;
and S716, taking the calls with the classification probability smaller than the probability threshold value corresponding to the call sets without labels as reliable negative samples which do not belong to the first scene.
In order to obtain more reliable negative examples, after reliable negative examples not belonging to the first scene are removed from the unlabeled call set, an updated unlabeled call set is obtained, and S704 to S716 are iteratively performed according to the labeled call set belonging to the first scene and the updated unlabeled call set. For example, it may be iterated 15 times.
S718, dividing a training set and a verification set according to the marked call set and the reliable negative sample;
s720, training a second classifier according to the call characteristics of the training set to obtain a first scene recognition model;
s722, acquiring call characteristics of each call in the call set;
s724, inputting the call characteristics of all calls into a first scene recognition model;
and S726, obtaining classification probability corresponding to each call according to the call characteristics through the first scene recognition model, and dividing the call set into a first call subset belonging to the first scene and a second call subset not belonging to the first scene according to the classification probability.
And S728, verifying the recognition rate of the first scene recognition model.
And when the verification fails, dividing the call set into a new marked call set and an unmarked call set which belong to the first scene according to the first call subset, and iteratively executing S702 to S728. When the verification is passed, the recognition rate of the first scene recognition model is proved to be in accordance with the expectation, and at the moment, the recognition of calls belonging to the first scene and calls not belonging to the first scene from the call set is also realized.
In this embodiment, after a small number of marked call sets belonging to a first scene are determined from a call set, an unlabeled call set is formed according to remaining calls in the call set, then training is performed according to call features of each call in the current marked call set and the unlabeled call set to obtain a first classifier, a reliable negative sample not belonging to the first scene is determined from the unlabeled call set according to the first classifier, the process can be iterated for multiple times to obtain more reliable negative samples, a verification set and a training set are divided according to the marked call set and the reliable negative samples, classification training is performed by using the call features of each call in the training set to obtain a first scene recognition model capable of recognizing whether a call scene is the first scene, so that the call set can be subjected to full prediction by the first scene recognition model to obtain a full prediction result, the recognition rate of the first scene recognition model is verified according to the full prediction result or the verification set, the initial call set is subjected to diffusion and correction according to the first call subset belonging to the first scene when verification fails, and the first call set is repeatedly subjected to the recognition process until the first scene recognition model is obtained.
In one embodiment, as shown in fig. 8, after identifying a first subset of calls belonging to a first scenario and a second subset of calls not belonging to the first scenario from the call set, the method for identifying a call scenario further includes a step of identifying calls belonging to a second scenario and calls not belonging to the second scenario from the remaining second subset of calls, and specifically includes the following steps S802-S808:
s802, a marked call set and an unmarked call set belonging to a second scene are obtained from the second call subset.
Specifically, the computer device needs to obtain a scene recognition model for recognizing a call scene belonging to the second scene, and needs to obtain a positive sample and a negative sample for training the second scene recognition model, so that the computer device may divide the second call subset recognized from the call set into a labeled call set and an unlabeled call set belonging to the second scene. The computer device may first obtain calls belonging to the second scene from the second call subset, and obtain a labeled call set belonging to the second scene after labeling the calls as belonging to the second scene, where the labeled call set is an initial positive sample used for training the second scene recognition model, and the initial positive sample is less in number, and is directly obtained from the second call subset, and is not necessarily completely accurate, and may be diffused and corrected in a subsequent iteration process. The negative samples used for training the second scene recognition model in the second call subset are missing, and calls not belonging to the second scene cannot be determined from the call set, so that the computer device can divide calls not divided into the labeled call set in the second call subset into the unlabeled call set, and the unlabeled call set may include calls belonging to the second scene or calls not belonging to the second scene.
In one embodiment, the step of obtaining the marked call set marked as the second scenario from the second call subset comprises: acquiring a second preset word corresponding to a second scene; acquiring the group name of a call group to which each call belongs in the second call subset; screening out a call group with the group name matched with a second preset word; and obtaining a marked call set belonging to the second scene according to the call initiated in the screened call group.
In this embodiment, the marked call set belonging to the second scenario may be determined according to the group name of the call group where the call is initiated. Specifically, the computer device may set a second preset word corresponding to a second scene, for example, the second scene is a chatting scene, and the second preset word may be at least one word of words such as "bagua", "chatting", and the like. For each call in the second call subset, the computer device may obtain a group name of a call group to which each call belongs, screen out a call group whose group name matches a set second preset word from the call groups, and label a call initiated in the screened call group as a call belonging to a second scene, thereby obtaining a labeled call set belonging to the second scene. It can be seen that the marked call set belonging to the second scenario may include multiple calls initiated in the same call group.
S804, according to a third classifier obtained based on the call features of the marked call set and the unmarked call set, reliable negative samples which do not belong to the second scene are determined from the unmarked call set.
Specifically, the computer device may obtain feature data corresponding to each call in the second call subset according to the set feature dimensions, and perform preprocessing on the feature data to obtain call features corresponding to each call. And the computer equipment trains a second classifier based on the call characteristics of all calls in the marked call set and the call characteristics of all calls in the unmarked call set belonging to the second scene to obtain a third classifier, and then adopts the trained third classifier to perform recognition according to the call characteristics of all calls in the unmarked call set, so as to determine a reliable negative sample which does not belong to the second scene.
In one embodiment, determining reliable negative examples that do not belong to the second scenario from the unlabeled call set according to a third classifier obtained based on call features of the labeled call set and the unlabeled call set comprises: selecting a spy sample from a marked call set belonging to a second scene; after spy samples are removed from the marked call set, training positive samples are obtained; after the spy sample is added to the call set which is not marked, a training negative sample is obtained; obtaining a third classifier according to the call characteristics of the training positive sample and the training negative sample; obtaining the classification probability of each call in the call set which is not marked through a third classifier; determining a probability threshold value for judging the call as not belonging to the second scene according to the classification probability of the spy samples; and taking the calls with the classification probability smaller than the probability threshold value corresponding to the calls in the call set which are not marked as the reliable negative samples which do not belong to the second scene.
Specifically, the computer device may set the second call subset to B, where the marked call set belonging to the second scenario is P, the unmarked call set is U, the reliable negative sample not belonging to the second scenario is RN, and the initial RN is set to be an empty set. Then, randomly selecting a part of the P as a spy sample S according to a preset proportion (for example, 15%), and after the spy sample S is removed from the marked call set P, obtaining a training positive sample PS, wherein PS = P-S, and the label of each call in the PS is 1; and after the spy sample S is added into the unlabeled call set U, obtaining a training negative sample US, wherein US = U + S, and the label of each call in US is-1. Then, according to the call characteristics of each call in the PS and the US, performing classification training to obtain a third classifier g3, and predicting each call in the U through the third classifier g3 to obtain a classification probability p1 (d) of each call d in the U. And finally, sorting the classification probabilities of all calls in the spy sample S, taking the classification probability corresponding to the call ranked later (for example, ranked 10% later) as a probability threshold value threshold not belonging to the second scene, and adding d to the RN when p1 (d) < threshold, so as to obtain a reliable negative sample not belonging to the second scene in the U.
In one embodiment, the method further comprises: and after iteratively removing the reliable negative samples which do not belong to the second scene from the second call subset, obtaining an updated unlabelled call set, and determining the reliable negative samples which do not belong to the second scene from the updated unlabelled call set according to the call characteristics of the labeled call set and the updated unlabelled call set until the iteration number meets the preset number.
In this embodiment, in order to obtain more reliable negative examples that do not belong to the second scenario from the unlabeled call set, the computer device may repeat the iteration until the number of iterations satisfies a preset number or it is impossible to obtain more reliable negative examples from the unlabeled call set, for example, the iteration may be performed for 15 times. The computer device can remove the reliable negative samples which are determined at the previous time and do not belong to the second scene from the original unlabeled call set U to obtain an updated unlabeled call set, then perform classification training according to call features of all calls in the updated unlabeled call set and the labeled call set P, and determine the reliable negative samples which do not belong to the second scene from the updated unlabeled call set again through a classifier obtained through training.
Fig. 9 is a schematic flow diagram illustrating the process of determining reliable negative examples that do not belong to the second scenario from the second communication subset in one embodiment. Referring to fig. 9, first, a marked call set P belonging to a second scenario is obtained from a second call subset, and other calls in the second call subset are added to an unmarked call set U, for example, the second scenario is a chat scenario, and calls whose group names of call groups in the second call subset include "chat" may be added to the marked call set P. Then, randomly selecting a call with a preset proportion from U as a spy sample S, and marking P-S as PS and 1,U + S as US and-1. And then, performing classification training by using the call characteristics of each call in the PS and the US to obtain a third classifier, and obtaining the classification probability of each call through the third classifier. And finally, determining a probability threshold value serving as a negative sample according to the classification probability of each call in the spy sample S, and adding the call of which the classification probability in the U is smaller than the probability threshold value as a reliable negative sample to the RN. And repeating the process, and retraining the classifier according to the remaining unmarked samples and the marked call set P in the U until the iteration times reach a preset number, and at the moment, dividing the second call subset into the marked call set P belonging to a second scene, the reliable negative sample RN not belonging to the second scene and the remaining unmarked call set U.
And S806, performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a second scene recognition model.
Specifically, the second scene recognition model is a model having the capability of recognizing whether or not a call belongs to the second scene based on the call characteristics of the call. The second scene recognition model is a classification model, and the recognition result of the call is that the call belongs to one of the second scene and the call does not belong to the second scene. After the computer device obtains the marked call set belonging to the second scene and the reliable negative sample not belonging to the second scene, classification training can be performed based on call characteristics of all calls in the set to obtain a second scene recognition model.
In one embodiment, in step S806, training the fourth classifier according to the call features of the labeled call set and the reliable negative examples, and obtaining the second scene recognition model includes: dividing a training set and a verification set according to the marked call set and the reliable negative sample; training the fourth classifier according to the call characteristics of the training set to obtain a second scene recognition model; and verifying the second scene recognition model according to the verification set.
Specifically, when performing the classification training according to the labeled call set belonging to the second scene and the reliable negative sample not belonging to the second scene, the computer device may divide the labeled call set and the reliable negative sample into a training set and a verification set according to a preset proportion, for example, 7.
The computer device may further verify the accuracy of the second scene recognition model obtained by training by using call features corresponding to calls belonging to the second scene and calls not belonging to the second scene in the verification set. When the verification fails, after step S808, dividing the second call subset into a new marked call set and a new unmarked call set belonging to the second scenario according to the third call subset may be performed iteratively; determining a reliable negative sample which does not belong to a second scene from the unmarked call set according to a third classifier obtained based on the call characteristics of the marked call set and the unmarked call set; performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a second scene recognition model; and identifying a third communication subset belonging to the second scene and a fourth communication subset not belonging to the second scene from the second communication subset through the second scene identification model until the second scene identification model obtained after the current iteration passes verification or the iteration number meets the preset number.
And S808, identifying a third communication subset belonging to the second scene and a fourth communication subset not belonging to the second scene from the second communication subset through the second scene identification model.
Specifically, after the computer device obtains the second scene recognition model, the obtained second scene recognition model may be used to perform full prediction on the second call subset, that is, to predict call scenes to which each call in the second call subset belongs, so as to determine a third call subset belonging to the second scene and a fourth call subset not belonging to the second scene in the second call subset.
In one embodiment, identifying, by the second scene identification model, from the second subset of calls, a third subset of calls belonging to the second scene and a fourth subset of calls not belonging to the second scene includes: acquiring the call characteristics of each call in the second call subset; inputting the call characteristics of each call into a second scene recognition model; and obtaining classification probability corresponding to each call through the second scene recognition model according to the call characteristics, and dividing the second call subset into a third call subset belonging to the second scene and a fourth call subset not belonging to the second scene according to the classification probability.
The classification probability may be used to reflect the possibility that the call belongs to the second scenario, and a higher classification probability may indicate that the call belongs to the second scenario with a higher possibility, whereas a lower possibility belongs to the second scenario. Therefore, the computer device can identify the call scene of the calls in the second call subset according to the classification probability obtained by the second scene identification model, divide the calls with the classification probability greater than the preset threshold into the third call subset belonging to the second scene, and divide the calls with the classification probability less than the preset threshold into the fourth call subset not belonging to the second scene.
In one embodiment, after step S808, the method further includes: the iterative execution divides the second call subset into a new marked call set and a new unmarked call set which belong to a second scene according to the third call subset; determining a reliable negative sample which does not belong to a second scene from the unmarked call set according to a third classifier obtained based on the call characteristics of the marked call set and the unmarked call set; performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a second scene recognition model; and identifying a third call subset belonging to the second scene and a fourth call subset not belonging to the second scene from the second call subset through a second scene identification model until an iteration stop condition is met.
In this embodiment, since the number of calls in the initial marked call set belonging to the second scene obtained from the second call subset is small, after the second call subset is divided into the third call subset belonging to the second scene and the fourth call subset not belonging to the second scene by the second scene identification model, the computer device may use the third call subset as a new marked call set belonging to the second scene, that is, modify and diffuse the initial marked call set belonging to the second scene according to the prediction result of the second scene identification model for full prediction, so as to obtain more accurate and more reliable positive samples, and thus the second call subset is divided into a new call set belonging to the second scene and a non-marked call set, and based on this, the above steps S802 to S808 are iteratively executed until a new second scene identification model is obtained, and the new second scene identification model is used for full prediction again until the number of iterations reaches a preset number or the prediction result of the second scene identification model obtained in the last iteration meets the expected scene identification model. It can be understood that each iteration process realizes the correction and diffusion of the marked call set belonging to the second scenario.
The computer device may verify the recognition rate of the second scene recognition model obtained for the first time or during a subsequent iteration. The verification method can include the following steps: the first method is to verify the second scene recognition model by using a verification set reserved during training the second scene recognition model, that is, to verify the obtained second scene recognition model by using the call features of a call set and a reliable negative sample marked in the verification set, and if the recognition rate of the verification set does not meet a preset condition, the currently obtained second scene recognition model is not verified, and then iteration can be continued. Secondly, the recognition rate of the calls initiated in the call group with the group name including the second preset word can be counted according to the full prediction result, for example, the recognition rate of the calls initiated in the call group with the group name including "chat" can be counted, the recognition rate of the calls initiated in the call group with the group name including "eight diagrams" can also be counted, and when the recognition rates of the calls do not meet the preset threshold value, the currently obtained second scene recognition model is not in accordance with the expectation, and the iteration can be continued. And thirdly, randomly extracting N calls, manually determining call scenes to which the N calls belong according to group names of call groups in which the N calls belong, and when the recognition rate of the N calls in the total prediction result does not meet a preset threshold value, obtaining a second scene recognition model at present, wherein the second scene recognition model does not meet the expectation, and the iteration can be continued.
According to the method for recognizing the call scene, after a small number of marked call sets belonging to the second scene are determined from the second call subset, an unlabelled call set is formed according to the remaining calls in the second call subset, then training is performed according to the call characteristics of each call in the current marked call set and the unlabelled call set to obtain a third classifier, a reliable negative sample which does not belong to the second scene is determined from the unlabelled call set according to the third classifier, then classification training is performed according to the call characteristics of each call in the marked call set and the reliable negative sample to obtain a second scene recognition model capable of recognizing whether the call scene is the second scene, so that a third call subset belonging to the second scene and a fourth call subset not belonging to the second scene can be recognized from the second call subset through the second scene recognition model, and after the calls belonging to the first scene are recognized from the original call set, the calls belonging to the second scene are continuously recognized from the remaining calls.
Fig. 10 is a flow chart illustrating a process of identifying a call belonging to the second scenario from the second subset of calls in one specific embodiment. Referring to fig. 10, the method specifically includes the following steps:
s1002, acquiring a marked call set and an unmarked call set belonging to a second scene from a second call subset;
s1004, selecting a spy sample from the marked call set belonging to the second scene;
s1006, after a spy sample is removed from the marked call set, a training positive sample is obtained;
s1008, after the spy sample is added to the call set which is not marked, a training negative sample is obtained;
s1010, training according to the call characteristics of the training positive sample and the training negative sample to obtain a third classifier;
s1012, obtaining the classification probability of each call in the call set which is not marked through a third classifier;
s1014, determining a probability threshold value for judging the call as not belonging to the second scene according to the classification probability of the spy sample;
and S1016, taking the calls of which the classification probability corresponding to the call sets which are not marked is smaller than the probability threshold as reliable negative samples which do not belong to the second scene.
In order to obtain more reliable negative samples, after reliable negative samples which do not belong to the second scene are removed from the unlabeled call set, an updated unlabeled call set is obtained, and S1004 to S1016 are iteratively executed according to the labeled call set belonging to the second scene and the updated unlabeled call set. For example, it may be iterated 15 times.
S1018, dividing a training set and a verification set according to the marked call set and the reliable negative sample;
s1020, training a fourth classifier according to the call characteristics of the training set to obtain a second scene recognition model;
s1022, obtaining the call characteristics of each call in the second call subset;
s1024, inputting the call characteristics of each call into a second scene recognition model;
and S1026, obtaining classification probabilities corresponding to all calls through the second scene recognition model according to the call characteristics, and dividing the second call subset into a third call subset belonging to the second scene and a fourth call subset not belonging to the second scene according to the classification probabilities.
S1028, verifying the recognition rate of the second scene recognition model.
And when the verification fails, dividing the second call subset into a new marked call set and an unmarked call set belonging to the second scene according to the third call subset, and iteratively executing S1002 to S1028. When the verification is passed, the recognition rate of the second scene recognition model is in accordance with the expectation, and at the moment, the recognition of the calls belonging to the second scene and the calls not belonging to the second scene from the second call subset is also realized.
In this embodiment, after a small number of marked call sets belonging to a second scene are determined from a second call subset, an unlabeled call set is formed according to remaining calls in the second call subset, then training is performed according to call features of each call in the current marked call set and the unlabeled call set to obtain a third classifier, a reliable negative sample not belonging to the second scene is determined from the unlabeled call set according to the third classifier, iteration can be performed for multiple times to obtain more reliable negative samples, then a verification set and a training set are divided according to the marked call set and the reliable negative samples, classification training is performed by using the call features of each call in the training set to obtain a second scene recognition model capable of recognizing whether a call scene is the second scene, so that full-volume prediction can be performed on the call set through the second scene recognition model to obtain a full-volume prediction result, the recognition rate of the second scene recognition model is verified according to the full-volume prediction result or the verification set, and the iterative iteration is performed on the initial call set according to the third call subset recognized that the second scene belongs to the second scene until the second scene recognition model is obtained, and the iterative iteration is performed after the verification process is verified to obtain a second scene.
Fig. 11 is a general flow diagram for identifying call scenarios for each call in a call set according to an embodiment. Referring to fig. 11, the method specifically includes the following steps:
determining a marked call set P1 and an unmarked call set U1 which belong to a first scene from the call set A; performing two-classification training according to the marked call set P1 and the unmarked call set U1 to obtain a first classifier g1; determining a reliable negative sample RN1 which does not belong to the first scene from U1 according to g1, wherein the process of obtaining the reliable negative sample RN1 can be iterated for multiple times; performing classification training by using the marked call set P1 and the reliable negative sample RN1 which does not belong to the first scene to obtain a second classifier g2, namely a first scene recognition model; and performing total prediction on the call set A by using the first scene recognition model g2, recognizing a first call subset belonging to a first scene and a second call subset B not belonging to the first scene from the call set A, and performing multiple iterations of the previous steps after correcting and diffusing the marked call set P1 according to the first call subset. Thus, the call belonging to the first scene is identified from the call set A.
The identification may also continue for a second subset of calls B that do not belong to the first scenario.
Determining a marked call set P2 and an unmarked call set U2 belonging to a second scene from the second call subset B; performing two-classification training according to the marked call set P2 and the unmarked call set U2 to obtain a third classifier g3; determining a reliable negative sample RN2 which does not belong to the second scene from the U2 according to the g2, wherein the process of obtaining the reliable negative sample RN2 can be iterated for multiple times; performing classification training by using the marked call set P2 and the reliable negative sample RN2 which does not belong to the second scene to obtain a fourth classifier g4, namely a second scene recognition model; and performing full prediction on the second call subset B by using the second scene recognition model g4, recognizing a third call subset belonging to the second scene and a fourth call subset C not belonging to the second scene from the second call subset B, and performing multiple iterations of the previous steps after correcting and diffusing the marked call set P2 according to the third call subset. Thus, the call belonging to the second scene is identified from the original call set A.
The calls divided into the fourth call subset C do not belong to the first scenario or the second scenario, and the call scenarios can be directly divided into other types, that is, calls initiated occasionally, which have fewer use scenarios.
It should be noted that, when the first scene is a working scene and the second scene is a chat scene, after the first scene identification model and the second scene identification model are obtained, the computer device may identify a usage scene of any multi-person call initiated in the call group, specifically, identify whether the usage scene belongs to the working scene through the first scene identification model, identify whether the usage scene belongs to the chat scene through the second scene, and if the usage scene does not belong to the working scene or the chat scene, may determine that the usage scene of the initiated multi-person call is another scene, which is a multi-person call initiated by accident.
It should be understood that, although the steps in the flowcharts of fig. 7 and 11 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 7 and 11 may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 12, there is provided a call scenario recognition apparatus 1200, which may be implemented by software, hardware, or a combination thereof as all or part of a computer device. The apparatus includes an acquisition module 1202, a reliable negative example determination module 1204, a training module 1206, and an identification module 1208, wherein:
an obtaining module 1202, configured to obtain a marked call set and an unmarked call set belonging to a first scene from a call set;
a reliable negative sample determining module 1204, configured to determine, according to a first classifier obtained based on call features of the labeled call set and the unlabeled call set, a reliable negative sample that does not belong to the first scene from the unlabeled call set;
the training module 1206 is used for performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a first scene recognition model;
an identifying module 1208, configured to identify, from the call set, a first call subset belonging to the first scenario and a second call subset not belonging to the first scenario through the first scenario identification model.
In one embodiment, the obtaining module 1202 is further configured to obtain a first preset word corresponding to a first scene; acquiring a group name of a call group to which each call in a call set belongs; screening out a conversation group with the group name matched with a first preset word; and obtaining a marked call set belonging to the first scene according to the call initiated in the screened call group.
In one embodiment, the device further comprises a feature data preprocessing module, configured to acquire feature data of various feature dimensions corresponding to each call in the call set; carrying out normality inspection on the feature data under the continuous feature dimension; and when the test result indicates that the feature data do not conform to normal distribution, carrying out logarithmic transformation on the feature data under the feature dimensionality to obtain the transformed feature data.
In one embodiment, the device further comprises a feature data preprocessing module, configured to acquire feature data of various feature dimensions corresponding to each call in the call set; and extracting the principal component from the feature data to obtain the call feature of each call.
In one embodiment, the feature data preprocessing module is further configured to calculate a covariance matrix after performing decentralized processing on feature data of each feature dimension; calculating eigenvalues of the covariance matrix and eigenvectors corresponding to the eigenvalues; obtaining a principal component matrix according to the eigenvector corresponding to the selected eigenvalue; and performing linear transformation on the characteristic data according to the principal component matrix to obtain the call characteristics of each call.
In one embodiment, the reliable negative example determining module 1204 is further configured to select a spy example from the callout collection belonging to the first scenario; after spy samples are removed from the marked call set, training positive samples are obtained; after the spy sample is added to the call set which is not marked, a training negative sample is obtained; obtaining a first classifier according to the call characteristics of the training positive sample and the training negative sample; obtaining the classification probability of each call in the call set which is not marked through a first classifier; determining a probability threshold value for judging the call as not belonging to the first scene according to the classification probability of the spy samples; and taking the calls of which the classification probability corresponding to the call sets which are not marked is smaller than the probability threshold as reliable negative samples which do not belong to the first scene.
In an embodiment, the reliable negative sample determining module 1204 is further configured to iteratively perform the step of removing the reliable negative sample that does not belong to the first scene from the unlabeled call set, obtaining an updated unlabeled call set, and determining the reliable negative sample that does not belong to the first scene from the updated unlabeled call set according to the call features of the labeled call set and the updated unlabeled call set until the iteration number satisfies the preset number.
In one embodiment, the training module 1206 is further configured to partition a training set and a verification set according to the labeled call set and the reliable negative samples; training a second classifier according to the call characteristics of the training set to obtain a first scene recognition model; wherein the verification set is used for verifying the first scene recognition model.
In one embodiment, the identifying module 1208 is further configured to obtain call characteristics of each call in the call set; inputting call characteristics of each call into a first scene recognition model; and obtaining classification probability corresponding to each call according to the call characteristics through the first scene recognition model, and dividing the call set into a first call subset belonging to the first scene and a second call subset not belonging to the first scene according to the classification probability.
In one embodiment, the apparatus is further configured to iteratively perform the dividing of the call set into a new marked call set and an unmarked call set belonging to the first scenario according to the first call subset; determining a reliable negative sample which does not belong to a first scene from the unlabeled call set according to a first classifier obtained based on the call features of the labeled call set and the unlabeled call set; performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a first scene recognition model; and identifying a first call subset belonging to the first scene and a second call subset not belonging to the first scene from the call set through a first scene identification model until an iteration stop condition is met.
In one embodiment, the apparatus further includes a second scene recognition module, where the second scene recognition module specifically includes an obtaining unit, a reliable negative example determining unit, a training unit, and a recognition unit.
The acquiring unit is used for acquiring a marked call set and an unmarked call set belonging to a second scene from the second call subset; determining a reliable negative sample which does not belong to a second scene from the unlabeled call set according to a third classifier obtained based on the call features of the labeled call set and the unlabeled call set; performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a second scene recognition model; a third subset of calls belonging to the second scenario and a fourth subset of calls not belonging to the second scenario are identified from the second subset of calls by the second scenario identification model.
In one embodiment, the obtaining unit is further configured to obtain a second preset word corresponding to a second scene; acquiring the group name of a call group to which each call belongs in the second call subset; screening out a call group with the group name matched with a second preset word; and obtaining a marked call set belonging to the second scene according to the call initiated in the screened call group.
In one embodiment, the reliable negative sample determination unit is further configured to select a spy sample from the callout collection belonging to the second scenario; after spy samples are removed from the marked call set, training positive samples are obtained; after the spy sample is added to the call set which is not marked, a training negative sample is obtained; obtaining a third classifier according to the call characteristics of the training positive sample and the training negative sample; obtaining the classification probability of each call in the call set which is not marked through a third classifier; determining a probability threshold value for judging the call as not belonging to the second scene according to the classification probability of the spy sample; and taking the calls with the classification probability smaller than the probability threshold value corresponding to the calls in the call set which are not marked as the reliable negative samples which do not belong to the second scene.
In an embodiment, the reliable negative sample determining unit is further configured to iteratively perform a step of removing reliable negative samples that do not belong to the second scene from the unlabeled call set, obtain an updated unlabeled call set, and determine reliable negative samples that do not belong to the second scene from the updated unlabeled call set according to call features of the labeled call set and the updated unlabeled call set until the iteration number satisfies a preset number.
In one embodiment, the training unit is further configured to partition a training set and a verification set according to the marked call set and the reliable negative samples; training a fourth classifier according to the call characteristics of the training set to obtain a second scene recognition model; wherein the verification set is used for verifying the second scene recognition model.
In one embodiment, the identifying unit is further configured to obtain call characteristics of each call in the second subset of calls; inputting the call characteristics of each call into a second scene recognition model; and obtaining classification probabilities corresponding to all calls through the second scene recognition model according to the call characteristics, and dividing the second call subset into a third call subset belonging to the second scene and a fourth call subset not belonging to the second scene according to the classification probabilities.
In one embodiment, the second scenario identification module is further configured to iteratively perform the dividing of the second call subset into a new marked call set and an unmarked call set belonging to the second scenario according to the third call subset; determining a reliable negative sample which does not belong to a second scene from the unlabeled call set according to a third classifier obtained based on the call features of the labeled call set and the unlabeled call set; performing classification training according to the call characteristics of the marked call set and the reliable negative sample to obtain a second scene recognition model; and identifying a third communication subset belonging to the second scene and a fourth communication subset not belonging to the second scene from the second communication subset through the second scene identification model until an iteration stop condition is met.
According to the recognition device for the call scenes, after a small number of marked call sets belonging to a first scene are determined from the call sets, an unlabelled call set is formed according to remaining calls in the call sets, then training is carried out according to call characteristics of all calls in the current marked call set and the unlabelled call set to obtain a first classifier, a reliable negative sample which does not belong to the first scene is determined from the unlabelled call set according to the first classifier, and then classification training is carried out according to the call characteristics of all calls in the marked call set and the reliable negative sample to obtain a first scene recognition model capable of recognizing whether the call scenes are the first scene, so that a first call subset belonging to the first scene and a second call subset not belonging to the first scene can be recognized from the call sets through the first scene recognition model, and recognition of the call scenes is achieved. Further, a third subset of calls belonging to the second scenario and a fourth subset of calls not belonging to the second scenario may also be identified from the second subset of calls in the same manner as described above.
FIG. 13 is a diagram illustrating an internal structure of a computer device in one embodiment. The computer device may specifically be the terminal 110 or the server 120 in fig. 1. As shown in fig. 13, the computer device includes a processor, a memory, a network interface connected by a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program, which, when executed by the processor, causes the processor to implement the recognition method of the call scenario. The internal memory may also store a computer program, and the computer program, when executed by the processor, may cause the processor to perform the method for recognizing a call scenario.
Those skilled in the art will appreciate that the architecture shown in fig. 13 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, the recognition apparatus 1200 for call scenario provided in the present application may be implemented in the form of a computer program, and the computer program may be run on a computer device as shown in fig. 13. The memory of the computer device may store various program modules that make up the recognition means of the call scenario, such as the acquisition module 1202, the reliable negative example determination module 1204, the training module 1206, and the recognition module 1208 shown in fig. 12. The computer program constituted by the respective program modules causes the processor to execute the steps in the call scene recognition method according to the respective embodiments of the present application described in the present specification.
For example, the computer device shown in fig. 13 may execute step S402 through the obtaining module 1202 in the recognition apparatus of the call scenario shown in fig. 12. The computer device may perform step S404 by the reliable negative example determination module 1204. The computer device may perform step S406 through the training module 1206. The computer device may perform step S408 through the identification module 1208.
In one embodiment, a computer device is provided, which includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method for identifying a call scenario. Here, the steps of the identification method of the call scenario may be steps in the identification method of the call scenario of the above embodiments.
In one embodiment, a computer-readable storage medium is provided, which stores a computer program, which, when executed by a processor, causes the processor to perform the steps of the above-described call scenario identification method. Here, the steps of the identification method of the call scenario may be steps in the identification method of the call scenario of the above embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (26)
1. A method for recognizing call scenes comprises the following steps:
acquiring a marked call set and an unmarked call set belonging to a first scene from a call set;
determining a reliable negative sample which does not belong to a first scene from the unmarked call set according to a first classifier obtained based on the call features of the marked call set and the unmarked call set;
performing classification training according to the call characteristics of the labeled call set and the reliable negative sample to obtain a first scene recognition model;
identifying a first subset of calls belonging to a first scenario and a second subset of calls not belonging to the first scenario from the call set through the first scenario identification model;
the method for acquiring the labeled call set belonging to the first scene from the call set comprises the following steps:
acquiring a first preset word corresponding to the first scene;
acquiring a group name of a call group to which each call in the call set belongs;
screening out a conversation group with a group name matched with the first preset word;
and obtaining a marked call set belonging to the first scene according to the call initiated in the screened call group.
2. The method of claim 1, further comprising:
acquiring feature data of various feature dimensions corresponding to each call in the call set;
carrying out normality inspection on the feature data under the continuous feature dimension;
when the test result indicates that the characteristic data does not conform to the normal distribution, then
And carrying out logarithmic transformation on the feature data under the feature dimension to obtain transformed feature data.
3. The method of claim 1, further comprising:
acquiring feature data of various feature dimensions corresponding to each call in a call set;
and extracting a main component from the feature data to obtain the call feature of each call.
4. The method according to claim 3, wherein the extracting a principal component from the feature data to obtain the call feature of each call comprises:
performing decentralized processing on the feature data of various feature dimensions, and calculating a covariance matrix;
calculating eigenvalues of the covariance matrix and eigenvectors corresponding to the eigenvalues;
obtaining a principal component matrix according to the eigenvector corresponding to the selected eigenvalue;
and performing linear transformation on the feature data according to the principal component matrix to obtain the call features of each call.
5. The method of claim 1, wherein the determining reliable negative examples that do not belong to a first scenario from the unlabeled call set according to a first classifier obtained based on call features of the labeled call set and the unlabeled call set comprises:
selecting a spy sample from the marked call set belonging to the first scene;
after spy samples are removed from the marked call set, training positive samples are obtained;
adding a spy sample into the unlabeled call set to obtain a training negative sample;
obtaining a first classifier according to the call characteristics of the training positive sample and the training negative sample;
obtaining the classification probability of each call in the call set which is not marked through a first classifier;
determining a probability threshold value for judging the call as not belonging to the first scene according to the classification probability of the spy samples;
and taking the calls of which the classification probability corresponding to the call sets not labeled is smaller than the probability threshold value as reliable negative samples which do not belong to the first scene.
6. The method of claim 5, further comprising:
and after reliable negative samples which do not belong to the first scene are removed from the unmarked call set in an iterative execution mode, obtaining an updated unmarked call set, and determining the reliable negative samples which do not belong to the first scene from the updated unmarked call set according to the call characteristics of the marked call set and the updated unmarked call set until the iteration times meet the preset number.
7. The method of claim 1, wherein performing classification training according to the call features of the labeled call set and the reliable negative examples to obtain a first scene recognition model comprises:
dividing a training set and a verification set according to the marked call set and the reliable negative sample;
training a second classifier according to the call characteristics of the training set to obtain a first scene recognition model;
wherein the verification set is used to verify the first scene recognition model.
8. The method of claim 1, wherein identifying a first subset of calls belonging to a first scenario and a second subset of calls not belonging to the first scenario from the call set through the first scenario identification model comprises:
acquiring call characteristics of each call in the call set;
inputting call characteristics of each call to the first scene recognition model;
and obtaining classification probability corresponding to each call according to the call characteristics through the first scene recognition model, and dividing the call set into a first call subset belonging to a first scene and a second call subset not belonging to the first scene according to the classification probability.
9. The method of claim 1, further comprising:
the iterative execution divides the call set into a new marked call set and an unmarked call set which belong to a first scene according to the first call subset; determining a reliable negative sample which does not belong to a first scene from the unmarked call set according to a first classifier obtained based on the call features of the marked call set and the unmarked call set; performing classification training according to the call characteristics of the labeled call set and the reliable negative sample to obtain a first scene recognition model; and identifying a first call subset belonging to a first scene and a second call subset not belonging to the first scene from the call set through the first scene identification model until an iteration stop condition is met.
10. The method of any one of claims 1 to 9, further comprising:
acquiring a marked call set and an unmarked call set belonging to a second scene from a second call subset;
determining a reliable negative sample which does not belong to a second scene from the unmarked call set according to a third classifier obtained based on the call features of the marked call set and the unmarked call set;
performing classification training according to the call characteristics of the labeled call set and the reliable negative sample to obtain a second scene recognition model;
identifying, by the second scene identification model, a third subset of calls belonging to the second scene and a fourth subset of calls not belonging to the second scene from the second subset of calls.
11. The method of claim 10, wherein the step of obtaining the marked call set marked as the second scenario from the second call subset comprises:
acquiring a second preset word corresponding to the second scene;
acquiring the group name of a call group to which each call belongs in the second call subset;
screening out a call group with the group name matched with the second preset word;
and obtaining a marked call set belonging to the second scene according to the call initiated in the screened call group.
12. The method of claim 10, wherein the determining reliable negative examples that do not belong to a second scenario from the unlabeled call set according to a third classifier obtained based on call features of the labeled call set and the unlabeled call set comprises:
selecting a spy sample from the marked call set belonging to the second scene;
after spy samples are removed from the marked call set, training positive samples are obtained;
adding a spy sample into the unlabeled call set to obtain a training negative sample;
obtaining a third classifier according to the call characteristics of the training positive sample and the training negative sample;
obtaining the classification probability of each call in the call set which is not marked through a third classifier;
determining a probability threshold value for judging the call as not belonging to the second scene according to the classification probability of the spy samples;
and taking the calls of which the classification probability corresponding to the call sets not labeled is smaller than the probability threshold value as reliable negative samples which do not belong to a second scene.
13. The method of claim 10, wherein the identifying, by the second scene identification model, from the second subset of calls, a third subset of calls belonging to the second scene and a fourth subset of calls not belonging to the second scene comprises:
acquiring the call characteristics of each call in the second call subset;
inputting call characteristics of each call into the second scene recognition model;
and obtaining classification probability corresponding to each call according to the call characteristics through the second scene recognition model, and dividing the second call subset into a third call subset belonging to the second scene and a fourth call subset not belonging to the second scene according to the classification probability.
14. An apparatus for identifying a call scenario, the apparatus comprising:
the acquisition module is used for acquiring a marked call set and an unmarked call set which belong to a first scene from the call set;
the reliable negative sample determining module is used for determining a reliable negative sample which does not belong to a first scene from the unlabeled call set according to a first classifier obtained based on the call features of the labeled call set and the unlabeled call set;
the training module is used for carrying out classification training according to the call characteristics of the labeling call set and the reliable negative sample to obtain a first scene recognition model;
the identification module is used for identifying a first call subset belonging to a first scene and a second call subset not belonging to the first scene from the call set through the first scene identification model;
the obtaining module is further configured to obtain a first preset word corresponding to the first scene, obtain a group name of a call group to which each call in the call set belongs, screen out a call group of which the group name matches the first preset word, and obtain a labeled call set belonging to the first scene according to a call initiated in the screened call group.
15. The device according to claim 14, further comprising a feature data preprocessing module, configured to obtain feature data of each feature dimension corresponding to each call in the call set; carrying out normality inspection on the feature data under the continuous feature dimension; and when the test result indicates that the characteristic data do not conform to normal distribution, carrying out logarithmic transformation on the characteristic data under the characteristic dimensionality to obtain transformed characteristic data.
16. The device according to claim 14, further comprising a feature data preprocessing module, configured to obtain feature data of each feature dimension corresponding to each call in the call set; and extracting principal components from the feature data to obtain the call features of the calls.
17. The apparatus according to claim 16, wherein the feature data preprocessing module is further configured to calculate a covariance matrix after performing a decentralization process on feature data of each feature dimension; calculating eigenvalues of the covariance matrix and eigenvectors corresponding to the eigenvalues; obtaining a principal component matrix according to the eigenvector corresponding to the selected eigenvalue; and carrying out linear transformation on the characteristic data according to the principal component matrix to obtain the call characteristics of each call.
18. The apparatus of claim 14, wherein the reliable negative sample determination module is further configured to select a spy sample from the callout collection belonging to the first scenario; after spy samples are removed from the marked call set, training positive samples are obtained; after a spy sample is added to the call set which is not marked, a training negative sample is obtained; obtaining a first classifier according to the call characteristics of the training positive sample and the training negative sample; obtaining the classification probability of each call in the call set which is not marked through a first classifier; determining a probability threshold value for judging the call as not belonging to the first scene according to the classification probability of the spy samples; and taking the calls of which the classification probability corresponding to the call sets not labeled is smaller than the probability threshold value as reliable negative samples which do not belong to the first scene.
19. The apparatus of claim 14, wherein the training module is further configured to partition a training set and a validation set according to the callout set and the reliable negative examples; training a second classifier according to the call characteristics of the training set to obtain a first scene recognition model; wherein the verification set is used to verify the first scene recognition model.
20. The apparatus of claim 14, wherein the identifying module is further configured to obtain call characteristics of each call in the call set; inputting call characteristics of each call to the first scene recognition model; and obtaining classification probability corresponding to each call according to the call characteristics through the first scene recognition model, and dividing the call set into a first call subset belonging to a first scene and a second call subset not belonging to the first scene according to the classification probability.
21. The apparatus according to any one of claims 14 to 20, further comprising a second scene recognition module, the second scene recognition module specifically comprising an acquisition unit, a reliable negative example determination unit, a training unit and a recognition unit;
the acquiring unit is used for acquiring a marked call set and an unmarked call set belonging to a second scene from a second call subset;
the reliable negative sample determining unit is used for determining a reliable negative sample which does not belong to a second scene from the unlabeled call set according to a third classifier obtained based on the call features of the labeled call set and the unlabeled call set;
the training unit is used for performing classification training according to the call characteristics of the labeling call set and the reliable negative sample to obtain a second scene recognition model;
the identifying unit is configured to identify, from the second subset of calls, a third subset of calls belonging to the second scenario and a fourth subset of calls not belonging to the second scenario by the second scenario identification model.
22. The apparatus according to claim 21, wherein the obtaining unit is further configured to obtain a second preset word corresponding to the second scene; acquiring the group name of a call group to which each call belongs in the second call subset; screening out a call group with the group name matched with the second preset word; and obtaining a marked call set belonging to the second scene according to the call initiated in the screened call group.
23. The apparatus of claim 21, wherein the reliable negative sample determination unit is further configured to select a spy sample from the callout collection belonging to the second scenario; after spy samples are removed from the marked call set, training positive samples are obtained; adding a spy sample into the unlabeled call set to obtain a training negative sample; obtaining a third classifier according to the call characteristics of the training positive sample and the training negative sample; obtaining the classification probability of each call in the call set which is not marked through a third classifier; determining a probability threshold value for judging the call as not belonging to the second scene according to the classification probability of the spy samples; and taking the calls of which the corresponding classification probability in the call set which is not marked is smaller than the probability threshold value as reliable negative samples which do not belong to a second scene.
24. The apparatus of claim 21, wherein the reliable negative example determining unit is further configured to obtain call characteristics of each call in the second subset of calls; inputting call characteristics of each call into the second scene recognition model; and obtaining classification probability corresponding to each call according to the call characteristics through the second scene recognition model, and dividing the second call subset into a third call subset belonging to the second scene and a fourth call subset not belonging to the second scene according to the classification probability.
25. A computer-readable storage medium, in which a computer program is stored which, when executed by a processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 13.
26. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method of any one of claims 1 to 13.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010042390.4A CN111309715B (en) | 2020-01-15 | 2020-01-15 | Call scene identification method and device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010042390.4A CN111309715B (en) | 2020-01-15 | 2020-01-15 | Call scene identification method and device |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111309715A CN111309715A (en) | 2020-06-19 |
CN111309715B true CN111309715B (en) | 2023-04-18 |
Family
ID=71150847
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010042390.4A Active CN111309715B (en) | 2020-01-15 | 2020-01-15 | Call scene identification method and device |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111309715B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112699285B (en) * | 2021-03-24 | 2021-06-18 | 平安科技(深圳)有限公司 | Data classification method and device, computer equipment and storage medium |
Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102158423A (en) * | 2010-01-28 | 2011-08-17 | 微软公司 | Following content item updates via chat groups |
CN103024170A (en) * | 2012-12-07 | 2013-04-03 | 北京百度网讯科技有限公司 | Management method and device of mobile terminal call records and mobile terminal |
CN106940805A (en) * | 2017-03-06 | 2017-07-11 | 江南大学 | A kind of group behavior analysis method based on mobile phone sensor |
CN107846516A (en) * | 2017-11-10 | 2018-03-27 | 维沃移动通信有限公司 | One kind call scene reminding method and mobile terminal |
WO2018145527A1 (en) * | 2017-02-09 | 2018-08-16 | 北京奇虎科技有限公司 | Cross-platform interaction method and device, program, and medium |
CN109359793A (en) * | 2018-08-03 | 2019-02-19 | 阿里巴巴集团控股有限公司 | A kind of prediction model training method and device for new scene |
CN109919244A (en) * | 2019-03-18 | 2019-06-21 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating scene Recognition model |
CN110019770A (en) * | 2017-07-24 | 2019-07-16 | 华为技术有限公司 | The method and apparatus of train classification models |
CN110048934A (en) * | 2019-04-04 | 2019-07-23 | 厦门小圈网络科技有限公司 | A kind of method of the internal information classification and display of circle in chat tool |
CN110225213A (en) * | 2019-06-17 | 2019-09-10 | 华为技术有限公司 | A kind of recognition methods of voice communication scene and audio policy service end |
CN110444229A (en) * | 2019-06-17 | 2019-11-12 | 深圳壹账通智能科技有限公司 | Communication service method, device, computer equipment and storage medium based on speech recognition |
CN110532542A (en) * | 2019-07-15 | 2019-12-03 | 西安交通大学 | It is a kind of that recognition methods and system are write out falsely with the invoice for not marking study based on positive example |
-
2020
- 2020-01-15 CN CN202010042390.4A patent/CN111309715B/en active Active
Patent Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102158423A (en) * | 2010-01-28 | 2011-08-17 | 微软公司 | Following content item updates via chat groups |
CN103024170A (en) * | 2012-12-07 | 2013-04-03 | 北京百度网讯科技有限公司 | Management method and device of mobile terminal call records and mobile terminal |
WO2018145527A1 (en) * | 2017-02-09 | 2018-08-16 | 北京奇虎科技有限公司 | Cross-platform interaction method and device, program, and medium |
CN106940805A (en) * | 2017-03-06 | 2017-07-11 | 江南大学 | A kind of group behavior analysis method based on mobile phone sensor |
CN110019770A (en) * | 2017-07-24 | 2019-07-16 | 华为技术有限公司 | The method and apparatus of train classification models |
CN107846516A (en) * | 2017-11-10 | 2018-03-27 | 维沃移动通信有限公司 | One kind call scene reminding method and mobile terminal |
CN109359793A (en) * | 2018-08-03 | 2019-02-19 | 阿里巴巴集团控股有限公司 | A kind of prediction model training method and device for new scene |
CN109919244A (en) * | 2019-03-18 | 2019-06-21 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating scene Recognition model |
CN110048934A (en) * | 2019-04-04 | 2019-07-23 | 厦门小圈网络科技有限公司 | A kind of method of the internal information classification and display of circle in chat tool |
CN110225213A (en) * | 2019-06-17 | 2019-09-10 | 华为技术有限公司 | A kind of recognition methods of voice communication scene and audio policy service end |
CN110444229A (en) * | 2019-06-17 | 2019-11-12 | 深圳壹账通智能科技有限公司 | Communication service method, device, computer equipment and storage medium based on speech recognition |
CN110532542A (en) * | 2019-07-15 | 2019-12-03 | 西安交通大学 | It is a kind of that recognition methods and system are write out falsely with the invoice for not marking study based on positive example |
Non-Patent Citations (2)
Title |
---|
戴昭华.统计数学在环境科学中的应用.《环境科学丛刊》.1987,(第01期),1-44. * |
李天彩 等.短文本信息流中的用户建模与应用.《信息工程大学学报》 .2016,第17卷(第17期),225-230. * |
Also Published As
Publication number | Publication date |
---|---|
CN111309715A (en) | 2020-06-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111506498B (en) | Automatic generation method and device of test case, computer equipment and storage medium | |
CN109522304B (en) | Abnormal object identification method and device and storage medium | |
CN110634471B (en) | Voice quality inspection method and device, electronic equipment and storage medium | |
US10089109B2 (en) | System and method for evaluating human resources in a software development environment | |
JP7391190B2 (en) | Generating training data for machine learning models | |
CN108363634B (en) | Method, device and equipment for identifying service processing failure reason | |
WO2021164205A1 (en) | Identity identification-based data auditing method and apparatus, and computer device | |
CN109325118B (en) | Unbalanced sample data preprocessing method and device and computer equipment | |
CN110888911A (en) | Sample data processing method and device, computer equipment and storage medium | |
CN111124878A (en) | Test report generation method and device, computer equipment and storage medium | |
CN112887371B (en) | Edge calculation method and device, computer equipment and storage medium | |
CN111309715B (en) | Call scene identification method and device | |
CN111126626A (en) | Training method, device, server, platform and storage medium | |
US10867255B2 (en) | Efficient annotation of large sample group | |
CN111817981A (en) | Network traffic classification method based on deep learning | |
CN111625342A (en) | Data tracing method, device and server | |
CN111367782B (en) | Regression testing data automatic generation method and device | |
CN111510566A (en) | Method and device for determining call label, computer equipment and storage medium | |
CN112764923B (en) | Computing resource allocation method, computing resource allocation device, computer equipment and storage medium | |
CN111091391B (en) | Method and device for identifying cheating user and electronic equipment | |
CN114662580A (en) | Training method, classification method, device, equipment and medium of data classification model | |
CN103106103B (en) | Solicited message sorting technique and device | |
US20240203424A1 (en) | Computerized-method and computerized-system for training and applying a machine learning (ml) textual behavioral identification model to authenticate an agent, in a digital multi-channel environment | |
CN112686718B (en) | Method and device for acquiring user loss reason, computer equipment and storage medium | |
US11921787B2 (en) | Identity-aware data management |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40024262 Country of ref document: HK |
|
GR01 | Patent grant | ||
GR01 | Patent grant |