CN111259985B

CN111259985B - Classification model training method and device based on business safety and storage medium

Info

Publication number: CN111259985B
Application number: CN202010103759.8A
Authority: CN
Inventors: 张戎
Original assignee: Tencent Cloud Computing Changsha Co Ltd
Current assignee: Tencent Cloud Computing Changsha Co Ltd
Priority date: 2020-02-19
Filing date: 2020-02-19
Publication date: 2023-06-30
Anticipated expiration: 2040-02-19
Also published as: CN111259985A

Abstract

The application relates to a classification model training method, a device and a storage medium based on business safety, wherein the method comprises the following steps: acquiring a full sample of a target service; performing abnormality detection on the full-volume sample by at least one abnormality detection mode, and determining an abnormal sample from the full-volume sample; screening malicious samples with sample content meeting malicious conditions from the abnormal samples; determining a normal sample according to samples of the total samples from which the malicious samples are removed; training an initial classification model based on the malicious sample and the normal sample to obtain a classification model for safely controlling the target service. The scheme provided by the application can reduce the cost of safety control.

Description

Classification model training method and device based on business safety and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method and apparatus for training a classification model based on business security, a computer readable storage medium, and a computer device.

Background

Along with the wide use of internet technology, there are some potential safety hazards while bringing convenience to people's life and work. For example, in some business scenarios, users may publish user-created content, such as UGC (User Generated Content ) content, etc., through a social network, while some malicious users use these methods to disseminate malicious content, such as spam messages, yellow violent information, or information violating legal regulations, etc., which has a very adverse impact on both life and work of the individual. It becomes very important how to screen for these malicious content.

Traditional screening and screening of malicious content in a network is generally based on keyword matching. For example, a table of black words can be maintained according to different business scenes, and malicious content can be screened according to the number of times of black words in the release content. However, for the traditional screening method, as time goes, more keywords need to be manually maintained are more and more, and the combination of words is more and more complex, so that the manual operation and maintenance cost is huge.

Disclosure of Invention

Based on this, it is necessary to provide a classification model training method, device, computer readable storage medium and computer equipment based on business security, aiming at the technical problem of high cost of the traditional malicious content screening method.

A classification model training method based on business safety comprises the following steps:

acquiring a full sample of a target service;

performing abnormality detection on the full-volume sample by at least one abnormality detection mode, and determining an abnormal sample from the full-volume sample;

screening malicious samples with sample content meeting malicious conditions from the abnormal samples;

determining a normal sample according to samples of the total samples from which the malicious samples are removed;

Training an initial classification model based on the malicious sample and the normal sample to obtain a classification model for safely controlling the target service.

A business security based classification model training apparatus, the apparatus comprising:

the acquisition module is used for acquiring a full sample of the target service;

the determining module is used for carrying out abnormality detection on the full-volume sample in at least one abnormality detection mode and determining an abnormal sample from the full-volume sample;

the screening module is used for screening malicious samples, the content of which meets malicious conditions, from the abnormal samples;

the determining module is further configured to determine a normal sample according to samples, from the total samples, from which the malicious samples are removed;

and the training module is used for training the initial classification model based on the malicious sample and the normal sample to obtain a classification model for safely controlling the target service.

A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:

acquiring a full sample of a target service;

A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of:

acquiring a full sample of a target service;

According to the classification model training method, the device, the computer readable storage medium and the computer equipment based on the business safety, the abnormality detection is carried out on the total samples of the target business by at least one abnormality detection mode so as to find abnormal samples from the total samples. And further, malicious samples with sample contents meeting malicious conditions can be screened from the abnormal samples, and normal samples are determined according to samples with malicious samples removed from the total samples. Therefore, positive and negative samples can be rapidly and accurately separated by combining an unsupervised abnormality detection mode with content screening, and the classification model is trained through the positive and negative samples. Therefore, the classification model obtained through training can carry out safety control on the target service on line, a large number of manual updating, maintenance, discrimination and screening rules are not needed, and the cost of safety control is greatly reduced.

Drawings

FIG. 1 is an application environment diagram of a business security based classification model training method in one embodiment;

FIG. 2 is a flow diagram of a business security based classification model training method in one embodiment;

FIG. 3 is an overall framework diagram of training and using an initial classification model in one embodiment;

FIG. 4 is a flowchart illustrating a step of determining an abnormal sample from a total sample by performing an abnormality detection on the total sample by at least one abnormality detection method according to an embodiment;

FIG. 5 is a diagram of a network architecture of a reconstruction model in one embodiment;

FIG. 6 is a flowchart illustrating a step of determining an abnormal sample from a total sample by performing an abnormality detection on the total sample by at least one abnormality detection method according to another embodiment;

FIG. 7 is a flowchart showing the steps of clustering samples in one embodiment;

FIG. 8 is a flowchart showing steps for security management of a target service in one embodiment;

FIG. 9 is a block diagram of a business security based classification model training apparatus in one embodiment;

FIG. 10 is a block diagram of a classification model training apparatus based on business security in another embodiment;

FIG. 11 is a block diagram of a computer device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

FIG. 1 is an application environment diagram of a business security based classification model training method in one embodiment. Referring to fig. 1, the classification model training method based on business safety is applied to a safety countermeasure system. The security countermeasure system includes a user terminal 110 and a computer device 120. The user terminal 110 and the computer device 120 are connected through a network. The user terminal 110 may be a desktop terminal or a mobile terminal, and the mobile terminal may be at least one of a mobile phone, a tablet computer, a notebook computer, and the like. The computer device 120 may be a terminal or a server, where the server may be implemented as a stand-alone server or as a server cluster composed of multiple servers.

It can be appreciated that the user may perform the operation based on the target service through the user terminal 110, the computer device 120 may obtain the log file generated based on the target service from different user terminals 110, further obtain the full sample of the target service from the log file, and the computer device 120 may obtain the classification model for performing security control on the target service by performing the classification model training method based on service security mentioned in the later embodiment of the present application, through which the security control on the target service may be performed on line, without a great number of manual real-time updating, maintenance, discrimination and screening rules, so as to greatly reduce the cost of performing security control on the target service.

As shown in FIG. 2, in one embodiment, a business security based classification model training method is provided. The present embodiment is mainly exemplified by the application of the method to the computer device 120 in fig. 1. Referring to fig. 2, the classification model training method based on business safety specifically includes the following steps:

s202, acquiring a full sample of the target service.

The target service is a service needing to be safely managed and controlled, and can be particularly an internet product in an internet service scene. The user may operate the internet product through which a series of events is generated, such as the user may add friends, post user-generated content, comment on, etc. through a social application, thereby generating a series of corresponding events. The security control of the target service is to perform security control on some series of events generated in the target service, namely to treat the situation that potential safety hazards exist in service scenes, such as treating malicious events such as pornography, junk messages, harassment behaviors, or user account theft.

The total number of samples is a batch of samples for training the classification model, and the total number of samples may be specifically available samples for training the classification model within a preset period, and the number of samples may be more or less, which is not limited in the embodiment of the present application. It will be appreciated that the training of the classification model is relatively better in cases where the number of samples is large. The corresponding full samples may be different in different traffic scenarios, and over time the number of full samples may increase or decrease. That is, new samples may be added to the full volume of samples to train the classification model, and unsuitable samples may be eliminated.

Specifically, different applications are run on the terminal, the computer device can determine the social application corresponding to the target service, acquire social data which is uploaded by the terminal and is generated by the social application, and determine a total sample based on the social data in a preset period. The social data is data generated when a user uses a social application, such as a user account number, object behavior data, user generated content, comment information and the like. The object behavior data is data reflecting social behavior of the user, such as login time, login location, login times, login terminals, and posting time or posting times of the user by the user account. The user-generated content is content authored and released by the user on the Internet platform, and the user-generated content can be text, pictures, videos, link addresses, symbols, expressions or the like.

In one embodiment, the full-scale sample for training the classification model includes at least one of a user account and user-generated content. When the full-quantity sample is the user account, the classification model obtained through training of the full-quantity sample is used for classifying the user account so as to find out the malicious user account. When the full sample is used for generating the content for the user, the classification model obtained through training of the full sample is used for classifying each piece of user generated content so as to find out malicious user generated content.

In one embodiment, the computer device may select the user account number or the user generated content from all social data within the preset period as the full sample, or may randomly or regularly select a part of the user account number or the user generated content as the full sample. The user account numbers with the same login locations or the user generated content with the same distribution locations are selected regularly, for example, the user account numbers with the same login locations or the user generated content with the same distribution locations are selected, for example, the user account numbers with the same login locations in a fixed time period or the user generated content published in the fixed time period are selected, and the embodiment of the present application is not limited herein.

In one embodiment, step S202, that is, the step of obtaining a full sample of the target service, specifically includes: acquiring a log file generated based on a target service in a preset period; determining user accounts appearing in the log file and user generated content corresponding to each user account; and taking all user accounts or all user generated contents as a full sample of the target service.

In particular, the computer device may obtain, from the terminal or the server, a log file generated based on the target service for a preset period. The log file is a historical log file, and a user account number with an operation, object behavior data corresponding to each user account number, user generated content and the like are recorded in the log file. It is understood that the preset period is a preset time period, such as one day, one week, one month, or the like.

Furthermore, the computer device may extract the user accounts appearing in the log file and the user generated content corresponding to each user account, and use all the user accounts or all the user generated content as the full sample of the target service.

It can be understood that, when the computer device takes all user accounts as the full-quantity samples, the classification model trained by the classification model training method based on the business security according to the embodiment of the application is executed based on the corresponding full-quantity samples, and the classification model can be used for classifying the user accounts collected on line. When the computer equipment takes all user-generated contents as full samples, the classification model trained by the classification model training method based on the business safety and provided by the embodiment of the application is executed based on the corresponding full samples, and the classification model can be used for classifying the user-generated contents collected on line.

In the above embodiment, all user accounts or all user generated contents in the log file generated based on the target service in the preset period may be used as the total sample of the target service, and further a classification model for processing the user accounts or the user generated contents may be obtained based on the corresponding total sample.

S204, carrying out abnormality detection on the total sample by at least one abnormality detection mode, and determining an abnormal sample from the total sample.

The anomaly detection mode is a mode for detecting anomaly data in a large amount of data, also called an outlier detection mode, and belongs to an unsupervised machine learning algorithm. Specifically, the computer device may determine at least one anomaly detection mode in advance, and then perform anomaly detection on the total samples by using each anomaly detection mode, so as to jointly form a set of anomaly samples detected by all anomaly detection modes, where elements in the set are anomaly samples. Thus, the abnormal sample can be prevented from being leaked.

In one embodiment, when the abnormality detection modes are multiple, the computer device may further perform an intersection operation on the abnormality samples detected by the multiple abnormality detection modes, and use the samples detected as the abnormality samples by the multiple abnormality detection modes as final abnormality samples. Thus, erroneous judgment on abnormal samples can be guaranteed to be reduced.

In one embodiment, the anomaly detection mode may specifically be statistical anomaly point detection, neural network anomaly point detection, cluster anomaly point detection, or PCA (principal components analysis, principal component analysis) anomaly point detection, or other anomaly detection modes, which are not limited in this embodiment. The abnormal point detection based on statistics may specifically be to construct a probability distribution model, calculate the probability that each sample accords with the model, and consider the sample with low probability as an abnormal point, that is, an abnormal sample. The abnormal point detection based on the neural network can be specifically based on One Class SVM (One-Class Support Vector Machine, a support vector machine) or Isolation Forest algorithm and the like. The cluster-based outlier detection may specifically be a flow-based clustering algorithm, a biKmeans (binary K-means clustering) algorithm, a Kmeans (K-means clustering) algorithm, or the like. In one embodiment, the computer device may construct sample features corresponding to each sample based on sample content corresponding to the sample, thereby constructing a first sample feature based on some or all of the sample features. The computer device can further perform abnormality detection on the full samples based on the first sample characteristics corresponding to each sample in the full samples, so that abnormal samples are determined from the full samples.

Wherein the sample feature is a feature vector constructed based on sample content of the sample for quantitatively representing the characteristics of the sample with a value. The first sample feature is a sample feature required for abnormality detection. The second sample feature mentioned later is a sample feature required for training the classification model. It may be appreciated that the first sample feature and the second sample feature may be the same sample feature, may be partially the same sample feature, or may be completely different sample features, etc., which may be specifically determined according to actual service requirements, and the embodiment of the present application is not limited to this.

In one embodiment, the computer device may construct a plurality of sample features of the sample according to the object behavior data corresponding to the user account, the user generated content, feedback information of other users, corresponding user description information, and the like, where the plurality of sample features may together form a feature library of the sample. Thus, the sample can be dataized, and the characteristics of the sample are reflected through a plurality of sample characteristics of the sample, so that the subsequent processing is convenient. There are various ways to construct the sample features, such as normalization (normalization), discretization (discretization), feature binarization (binarization), cross-over features, and the like, which are not limited by the embodiments of the present application. The user description information is a tag set for describing characteristics such as user preference, habit or attribute.

In one embodiment, referring to table 1 below, table 1 is a schematic table of a feature library of samples in one embodiment. The computer device may construct from the sample content a sample feature of 6 dimensions in the table below, and for each sample, may determine feature values in these 6 dimensions based on the corresponding sample content, respectively.

Feature 1

Feature 2

Feature 3

Feature 4

Feature 5

Feature 6

Sample 1

a1

b1

c1

d1

e1

f1

Sample 2

a2

b2

c2

d2

e2

f2

Sample 3

a3

b3

c3

d3

e3

f3

Sample 4

a4

b4

c4

d4

e4

f4

Table 1 sample feature library

In one embodiment, the computer device may screen some or all of the sample features from a feature library of the samples, and splice to form a first sample feature, such as [ feature 1, feature 2, feature 5, feature 6]. For the construction of the second sample feature, the computer device may select part or all of the sample features from the feature library of the sample according to the service requirement, and splice the sample features to form the second sample feature, for example [ feature 1, feature 2, feature 3, feature 4].

S206, screening malicious samples with sample contents meeting malicious conditions from the abnormal samples.

The content of the sample is the content corresponding to the sample, and may specifically be information for constructing the first sample feature. When the abnormal sample is a user account, the corresponding sample content can be specifically the object behavior data corresponding to the user account, the user generated content, feedback information of other users, corresponding user description information and the like. When the abnormal sample is content generated by a user, the corresponding sample content may specifically be information included in the content generated by the user, such as text, links, or pictures included in the content generated by the user.

Specifically, the computer device may screen out malicious samples satisfying the malicious conditions from the sample content of the abnormal samples. The sample content meets the malicious condition, and specifically may be sample content including malicious information. For example, when the abnormal sample is abnormal user generated content, the abnormal user generated content includes malicious links, pornography, characters violating legal regulations, or the repetition frequency of the user generated content exceeds a preset frequency, and the abnormal user generated content can be considered to be malicious user generated content, that is, malicious sample, if any one or more of the conditions are satisfied. For example, when the abnormal sample is an abnormal user account, the abnormal user account issues malicious user generated content, the posting frequency of the abnormal user account is higher than a preset frequency, the abnormal user account logs in a different place and performs funds transfer, and the like, and when any one or more conditions are satisfied, the abnormal user account is judged to be a malicious user account, that is, a malicious sample.

In one embodiment, the computer device may set the discrimination rules and screen out malicious samples from the abnormal samples by manually or mechanically employing the set discrimination rules.

In one embodiment, S206, that is, the step of screening the abnormal samples for the malicious samples whose content meets the malicious condition specifically includes: determining a malicious sample screening mode; according to sample contents of samples in the abnormal samples and in a malicious sample screening mode, screening abnormal samples of which the sample contents meet malicious conditions from the abnormal samples; and marking the screened abnormal samples as malicious samples.

The malicious sample screening method is a method for screening malicious samples, for example, the method can specifically be a method for screening malicious samples by means of experienced staff and judging sample contents of abnormal samples based on screening rules, so as to screen the abnormal samples, or the method can be a method for screening the abnormal samples by inputting the sample contents into a machine model through a machine model and pre-learning screening rules, or a method for screening the malicious samples from the abnormal samples by combining manual screening and machine screening, so that the malicious samples can be screened out more accurately, and the screening method is not limited.

Specifically, according to sample content of each sample in the abnormal samples, the computer equipment can screen the abnormal samples, of which the sample content meets the malicious conditions, from the abnormal samples according to a malicious sample screening mode. Further, the computer device may label the screened abnormal samples as malicious samples. The class label of the screened malicious sample is set as a malicious class label, and the malicious sample is a negative sample.

In the above embodiment, according to the malicious sample screening manner, the malicious samples are screened from the abnormal samples, so that a small number of screening rules can be maintained, the screening efficiency and accuracy are greatly improved, and the screening accuracy and efficiency can be improved no matter whether the screening is performed through manual auxiliary judgment or machine screening.

S208, determining a normal sample according to samples of the total samples, from which malicious samples are removed.

Specifically, the computer device may take all samples of the total sample except the marked malicious sample as normal samples. Alternatively, the computer device may also decimate a portion of the samples from the total number of samples after the malicious samples are removed as normal samples. The decimation method may be random sampling selection or directional decimation, which is not limited in the embodiment of the present application. Also, the number of partial samples decimated may be less than, equal to, or greater than the number of malicious samples, which is not limited by the embodiments of the present application.

In one embodiment, after the computer device determines the normal samples from the full number of samples, the class label of the normal sample may be set to be the normal class label, that is, the positive sample.

S210, training an initial classification model based on a malicious sample and a normal sample to obtain a classification model for safely controlling target service.

Specifically, the computer device may obtain an initial classification model, where the classification model may specifically be a mathematical model constructed by algorithms such as decision trees, logistic regression, naive bayes, or neural networks, which is not limited in this embodiment of the present application. Among them, neural network algorithms such as GBDT (Gradient Boosting Decision Tree, gradient descent tree) algorithm, random forest algorithm, or the like.

Furthermore, the computer device may train the initial classification model based on the malicious and normal samples to obtain a trained classification model for security management of the target traffic. It will be appreciated that training of the classification model at this time is a training approach to supervised machine learning algorithms. The classification model can classify the object to be processed in the target service on line to identify whether the object to be processed is a malicious object. Real-time online scoring and predictive classification can be performed through the trained classification model, so that the effect of judging malicious behaviors of the user is achieved.

In one embodiment, the initial classification model is constructed by an xgboost (extreme gradient boosting, extreme gradient lifting) algorithm, which is a variant of the GBDT algorithm. The classification model constructed by the xgboost algorithm has several advantages: (1) The fitting capability is superior, and the under fitting and over fitting can be effectively prevented; (2) The characteristic threshold is low, namely, the construction of excessive characteristic engineering is not needed; (3) The method has high speed, and the construction process of the tree can be accelerated in a multithreading mode; (4) Compared with GBDT, the method has the advantages that the precision is higher, compared with GBDT, the xgboost only carries out Taylor expansion on the objective function for the second time aiming at the first derivative information, and meanwhile, the first derivative and the second derivative are used, so that the classification accuracy is higher.

In one embodiment, step S210, that is, training the initial classification model based on the malicious sample and the normal sample, the step of obtaining the classification model for security management of the target service specifically includes: determining second sample characteristics and category labels corresponding to the malicious samples and the normal samples respectively; respectively taking the second sample characteristics corresponding to the malicious sample and the normal sample as input data of an initial classification model; using a class label corresponding to the input data as a training label; and training an initial classification model through input data and corresponding training labels to obtain a classification model for safety control of target service.

Wherein reference may be made to the manner mentioned in the previous embodiments with respect to the construction of the second sample feature. The second sample characteristic may be the same as or different from the first sample characteristic. The computer device may construct the second sample feature from all or part of the sample features.

In particular, the computer device may determine a second sample feature and a category label for each of the malicious and normal samples. For malicious samples, the corresponding class labels are malicious class labels; for a normal sample, its corresponding class label is a normal class label.

When the computer equipment trains the initial classification model, the second sample characteristics corresponding to the malicious sample and the normal sample can be respectively used as input data and sequentially input into the classification model to obtain prediction output corresponding to the input data. The computer device may construct a loss function based on the difference between the predicted output corresponding to the input data and the category label. During the training process, the model parameters are adjusted, and during the adjustment process, the model parameters which minimize the loss function are taken as the current model parameters, and the training process is continuously repeated. Stopping training until reaching the training stopping condition to obtain the trained classification model. The training stopping condition may specifically be that the preset iteration number is reached, or that the classification performance of the trained classification model reaches a preset index, or the like.

Thus, an initial classification model is trained through the normal sample and the malicious sample, and a classification model with good classification performance can be obtained, and the classification model is used for safely controlling target business.

Referring to FIG. 3, FIG. 3 is an overall frame diagram of training and using an initial classification model in one embodiment. As shown in FIG. 3, the overall frame diagram for training and using the initial classification model includes five components, positive and negative sample separations 301, constructed sample features 302, supervised machine learning algorithms 303, offline training classification model 304, and online usage classification model 305 as in FIG. 3. Specifically, the computer device obtains a full sample and performs positive and negative sample separation on the full sample. And constructing sample characteristics corresponding to the malicious sample and the normal sample by adopting the mode in the embodiment, wherein the second sample characteristic is used at the moment. And further, performing offline training on the initial classification model through a supervised machine learning algorithm to obtain a trained classification model. Finally, the computer equipment can use the trained classification model for online prediction, namely, classify the object to be processed in the target service on line so as to identify whether the object to be processed is a malicious object.

Wherein the positive and negative samples mentioned in the embodiments of the present application are separated, which is a safety countermeasure system different from the conventional recommendation system. For the recommendation system, when the supervised machine learning algorithm is used, the labeling of the sample is quite clear, and excessive extra work is not needed. In security countermeasure systems, however, sample separation is a very important link, since the target traffic is not data-tagged. Therefore, in practical application, a batch of malicious samples can be found by the anomaly detection method mentioned in the foregoing embodiment, that is, the anomaly detection method based on the unsupervised machine learning algorithm, and the experience of the staff. The normal sample is obtained by subtracting the malicious sample from the full sample. By the mode, sample separation can be achieved rapidly and accurately, and a large amount of labor cost is not required.

According to the classification model training method based on the business safety, the abnormality detection is carried out on the total samples of the target business in at least one abnormality detection mode, so that the abnormal samples are found out from the total samples. And further, malicious samples with sample contents meeting malicious conditions can be screened from the abnormal samples, and normal samples are determined according to samples with malicious samples removed from the total samples. In this way, the positive and negative samples can be rapidly and accurately separated by combining content screening in an unsupervised anomaly detection mode, so that the classification model is trained through the positive and negative samples. Therefore, the classification model obtained through training can carry out safety control on the target service on line, a large number of manual updating, maintenance, discrimination and screening rules are not needed, and the cost of safety control is greatly reduced. And the safety control of the target service is realized through the trained classification model, and the control efficiency and accuracy are also improved.

In one embodiment, step S202, that is, the step of obtaining a full sample of the target service, specifically includes: and acquiring a full-volume sample of the target service and first sample characteristics corresponding to each sample in the full-volume sample. Step S204, namely performing anomaly detection on the total sample by at least one anomaly detection method, wherein the step of determining the anomaly sample from the total sample specifically includes: performing anomaly detection on the total number of samples based on the first sample characteristics corresponding to each sample through at least one anomaly detection mode to obtain at least one group of candidate anomaly samples; and screening abnormal samples from the total samples according to the union set of at least one group of candidate abnormal samples.

Specifically, the computer device may obtain the full-volume sample of the target service and the first sample features respectively corresponding to each sample in the full-volume sample in the manner mentioned in the above embodiments. Furthermore, the computer device may perform abnormality detection on the total number of samples based on the first sample features corresponding to the samples, respectively, by at least one abnormality detection method, to obtain at least one group of candidate abnormal samples. Here, the candidate abnormal samples are abnormal samples detected by the respective abnormality detection methods. The computer device may then find a union of at least one set of candidate abnormal samples, i.e. when there are multiple sets of candidate abnormal samples, perform or operation on each set of candidate abnormal samples, and take all the candidate abnormal samples that occur as the current abnormal sample.

In the above embodiment, based on the first sample characteristics corresponding to each sample, the abnormality detection can be accurately and rapidly performed on the total number of samples. And then, the candidate abnormal samples detected by each abnormal detection mode are taken as abnormal samples, so that the abnormal samples can be prevented from being missed.

In one embodiment, the step of obtaining a full-scale sample of the target service and the first sample features corresponding to each sample in the full-scale sample specifically includes: acquiring a log file generated based on a target service in a preset period; the log file comprises user account numbers, object behavior data corresponding to each user account number and user generated content; taking all user account numbers appearing in the log file as full-quantity samples; acquiring user description information corresponding to each user account; and determining first sample characteristics corresponding to the user accounts respectively according to the object behavior data, the user generated content and the user description information corresponding to the user accounts respectively.

In one embodiment, the computer device may obtain a log file generated based on the target service in a preset period, where the log file includes a user account, object behavior data corresponding to each user account, and user generated content. Further, the computing device may take all user accounts present in the log file as full samples. The computer equipment can construct user description information corresponding to the user account number in advance from other service platforms or service data based on the local service platform. Furthermore, the computer device can perform feature construction according to the object behavior data, the user generated content, the user description information and other contents corresponding to the user account, so as to obtain corresponding sample features. It will be appreciated that the sample features a feature vector, each element in which may represent a feature value of the sample in a respective dimension. The computer equipment can screen part or all of characteristic values from sample characteristics corresponding to the user account, and splice the characteristic values to form corresponding first sample characteristics.

In one embodiment, the computer device may further obtain negative feedback information and positive feedback information of the user account of the other user, where the negative feedback information is, for example, report information, etc., and the positive feedback information is, for example, like information, etc. Furthermore, the computer device may perform feature construction based on the object behavior data corresponding to the user account, the user generated content, the user description information, and other feedback information of the user to the user account, so as to obtain corresponding sample features.

In the above embodiment, the corresponding first sample feature may be configured based on the object behavior data, the user generated content and the user description information corresponding to the user account, so that the configured first sample feature may reflect the characteristics corresponding to the user account comprehensively and accurately.

In one embodiment, the step of obtaining a full-scale sample of the target service and the first sample features corresponding to each sample in the full-scale sample specifically includes: acquiring a log file generated based on a target service in a preset period; taking all user-generated content appearing in the log file as a full sample; and determining first sample characteristics corresponding to the user generated contents respectively according to the malicious contents respectively included in the user generated contents.

In one embodiment, the computer device may obtain a log file generated based on the target service for a preset period and take all user-generated content present in the log file as a full sample. I.e. each piece of user generated content, i.e. a sample. Further, the computer device may obtain specific content corresponding to each piece of user-generated content to determine sample characteristics of the piece of user-generated content. For example, the computer device may calculate a dirty word score from dirty words that occur in the user-generated content, such as counting a score per dirty word that occurs. The computer device may also detect whether there is a malicious link in the user-generated content, or whether there is a malicious picture in the user-generated content, etc., counting a minute each time it occurs, etc. In this way, the computer device may determine, according to the score values corresponding to each of the plurality of dimensions described above, a feature value of the piece of user-generated content in the corresponding dimension, where the feature values of the plurality of dimensions together form the sample feature. Furthermore, the computer equipment can screen part or all of the characteristic values from the sample characteristics corresponding to the user generated content, and splice the characteristic values to form corresponding first sample characteristics.

In the above embodiment, the first sample characteristic corresponding to the user generated content may be determined based on the malicious content included in the user generated content, and thus the constructed first sample characteristic may reflect the characteristics corresponding to the user generated content comprehensively and accurately.

In one embodiment, step S204, that is, performing abnormality detection on the full-scale sample by at least one abnormality detection method, determines an abnormal sample from the full-scale sample, includes:

s402, acquiring first sample characteristics corresponding to all samples in the full-quantity samples.

S404, respectively inputting the corresponding first sample characteristics of each sample to the trained reconstruction model to obtain corresponding output vectors.

The reconstruction model is a mathematical model constructed based on a feedforward neural network. The reconstruction model may include a multi-layer feedforward neural network that simulates an identity map, the number of nodes of the input layer being equal to the number of nodes of the output layer, the number of nodes of the hidden layer typically being less than the number of nodes of the input layer. The reconstruction model can play a role in compressing data and recovering data, and input data is reconstructed through the reconstruction model.

Referring to fig. 5, fig. 5 is a schematic diagram of a network structure of a reconstruction model in one embodiment. The reconstruction model in fig. 5 includes an output layer, and three hidden layers. The number of nodes of the input layer and the output layer is 6, which represents that the sample has 6 characteristics. The first and third hidden layers have fewer nodes (4 nodes in fig. 5) than the input layer, and the second hidden layer has the smallest number of nodes (2 nodes in fig. 5). In the neural network transmission, a tanh function (hyperbolic tangent function) and a sigmoid function (S-type function) are used in between. Since the reconstruction model is to train an identity mapping (identity) The quality mapping), the data is compressed from the input layer at the time of transmission, and the data is decompressed after the second hidden layer. The objective function of training is to make the overall output error small enough, and the overall error is obtained by dividing the sum of all sample errors by the number of samples. Taking the 6 features shown in fig. 5 as an example, the error of the ith sample is:

wherein x is _ij Characteristic value of the j-th characteristic representing the i-th sample, r _ij A predicted feature value representing the j-th feature of the i-th sample output by the reconstruction model. In each training process, model parameters are generally updated by using a classical back propagation algorithm (back propagation), the model parameters of the reconstructed model are adjusted towards minimizing the overall error, and the trained reconstructed model is obtained after the training is finished. It will be appreciated that the network structure of the reconstruction model in fig. 5 is for illustration only and is not intended to limit the network structure of the reconstruction model.

Specifically, the computer device may obtain a trained reconstruction model, input the first sample features corresponding to each sample to the trained reconstruction model, perform data compression and decompression through a hidden layer of the reconstruction model, and output a corresponding output vector. The output vector is formed by splicing the predicted characteristic values of the characteristics corresponding to the sample.

S406, determining error values corresponding to the samples based on differences between the first sample characteristics corresponding to the samples and the output vectors.

In particular, the computer device may determine an error value corresponding to each sample based on a difference between the first sample feature and the output vector corresponding to each sample, which may also be considered a reconstruction score. Wherein the first sample feature is composed of feature values corresponding to the respective features. The output vector is composed of predicted feature values corresponding to the respective features. In one embodiment, the computer device may calculate the error value corresponding to each sample by the following formula:

wherein e _i An error value representing the i-th sample, n represents the total number of features possessed by the sample, x _ij Characteristic value of the j-th characteristic representing the i-th sample, r _ij A predicted feature value representing the j-th feature of the i-th sample output by the reconstruction model.

S408, taking a sample with the corresponding error value meeting the abnormal condition as an abnormal sample.

Specifically, the computer device may calculate an error value corresponding to each sample, and use a sample whose error value satisfies the abnormal condition as an abnormal sample. The error value meeting the abnormal condition may be that the error value is greater than a preset threshold, or that the ranking ranks of the error values after ranking from big to small are before the preset ranking. Thus, the computer device can take a batch of samples with larger error values as abnormal samples.

In the above embodiment, the reconstruction model is used to reconstruct the first sample feature corresponding to each sample, so as to determine the error value corresponding to each sample. Therefore, a batch of samples with larger error values can be used as abnormal samples quickly and accurately directly according to the error values.

s602, acquiring first sample characteristics corresponding to all samples in the full-scale samples.

S604, clustering is carried out on the total samples based on the first sample characteristics corresponding to the samples respectively, so as to obtain more than one type of clusters.

Specifically, the computer device may perform clustering on the total number of samples according to the first sample features corresponding to the samples, to obtain more than one type of clusters. The clustering algorithm adopted in the clustering process may be specifically a streaming clustering algorithm, a biKmeans algorithm, a Kmeans algorithm, or the like, which is not limited in the embodiment of the present application.

S606, determining characteristic average values corresponding to the clusters respectively according to the first sample characteristics of the samples respectively included in the clusters.

Specifically, the computer device may determine a centroid vector of each cluster according to the first sample characteristics of the samples included in each cluster, where the centroid vector may be used as a characteristic average value of the cluster. The computer device may determine the centroid vector of each cluster in various manners, for example, for each sample in a cluster, the computer device may calculate the sum of distances between the sample and other samples in the cluster, and use the sample with the shortest sum of distances as the centroid of the cluster, where the centroid vector of the corresponding cluster is the first sample characteristic of the centroid. The distance between different samples may specifically be calculated by using the first sample features corresponding to the samples, and the euclidean distance, the manhattan distance, or the chebyshev distance between the two samples. In addition, the computer device may further calculate an average vector based on the first sample features corresponding to all the samples in the cluster, where the average vector is a centroid vector. It will be appreciated that as new samples in a cluster increase, the centroid vector of the cluster is updated accordingly.

S608, based on the characteristic distribution of the characteristic mean value corresponding to each cluster, the abnormal clusters are screened out from the clusters, and samples in the abnormal clusters are used as abnormal samples.

Furthermore, the computer equipment can determine a distribution condition of the characteristic mean values of different clusters according to the characteristic mean values respectively corresponding to the clusters, the discrete characteristic mean values are screened out, the cluster corresponding to the discrete characteristic mean values can be regarded as an abnormal cluster, and samples included in the abnormal cluster are abnormal samples.

In the embodiment, the clustering processing can be performed on the total samples, and the samples in the abnormal clusters with larger differences with other clusters are used as the abnormal samples, so that the method is accurate and convenient.

In one embodiment, step S604, that is, the step of clustering the total number of samples to obtain more than one type of clusters, specifically includes: determining different clusters existing currently; for each sample, respectively calculating the distance between the sample and the different currently existing clusters according to the corresponding first sample characteristics; and when the minimum distance in the distances is smaller than or equal to the distance threshold value, dividing the samples into clusters corresponding to the minimum distance.

Specifically, when the computer equipment performs clustering processing on the total samples, the samples which are processed at the beginning can be directly used as a cluster, then the distances between other samples and the cluster are calculated, when the distances are smaller than or equal to a distance threshold value, the other samples are divided into the cluster, and otherwise, the other samples are self-formed into a cluster. Thus, as the number of processed samples increases, the clusters resulting from clustering the samples also increase.

Referring to fig. 7, fig. 7 is a flowchart of the steps of clustering samples in one embodiment. As shown in fig. 7, the flowchart of the step of clustering samples mainly includes: s702, separating positive and negative samples; s704, judging whether the new sample is similar to the existing certain class; if yes, go to step S706, insert the new sample into the class; if not, step S708 is entered where the new sample is self-organizing. In particular, when clustering unknown new samples, the computer device may determine whether the new samples are sufficiently similar to an existing class (i.e., cluster). If yes, the new sample is inserted into the cluster, and if not, the new sample is self-classified. The determining whether the new sample is similar to the existing class is specifically determined by comparing whether the distance between the new sample and the class is smaller than a distance threshold, if so, the new sample is similar to the class, otherwise, the new sample is dissimilar.

In one embodiment, step S604 further comprises: when the minimum distance in the distances is greater than the distance threshold, determining the number of currently existing clusters; when the number is smaller than the preset number, creating a new cluster, and dividing the samples into the new cluster; and when the number is equal to the preset number, dividing the samples into clusters corresponding to the minimum distance.

In one embodiment, when a minimum distance from among distances of a certain sample to an existing cluster is equal to or less than a distance threshold, the computer device may divide the sample into clusters corresponding to the minimum distance. When the minimum distance of all distances is greater than the distance threshold, the computer device may determine the number of currently existing clusters, and if the number of currently existing clusters is less than a preset number, divide the sample into new clusters. And if the number of the current existing clusters is equal to the preset number, dividing the sample into clusters corresponding to the minimum distance. In this way, the total number of clusters can be guaranteed to be a preset number.

The process of clustering samples is described in detail below, by way of example:

the computer device may construct a data matrix based on the first eigenvectors corresponding to the different samples, respectively. For example, when there are m samples, each sample has n features, that is, the first feature vector has n elements, it can form a matrix dataMat of m rows and n columns. Each row represents a first sample feature and columns represent the dimensions of the feature. In other words, there are m points in the n-dimensional euclidean space that need to be clustered. In this way, clusters of samples can be converted into clusters of points.

Let dataMat be a matrix of m rows and n columns, each row representing a vector and n representing the dimension of the vector. K represents the maximum number of clusters allowed to form during clustering; d represents a distance threshold. Where the distance between two points may use L ¹ 、L ² Or L ^∞ Norms. The centroid of a cluster is defined as the average of all points in the class. For example, a cluster j includes A0]，A[1]，…，A[n-1]Then the centroid of cluster j is

And, the number of elements in the j-th cluster is num [ j ]]And (3) representing.

Step (1), for dataMat [0], forming a cluster. The centroid of the cluster is itself C0=datamat 0, the number of elements of the cluster is num 0=1, and the number of all clusters is k=1.

Step (2), for each sample i,1<＝i<=m-1, the computer devices all perform the following loop operations: suppose there is currently K ^′ The centroid of the jth cluster is C [ j ]]The number of elements of the jth cluster is num [ j ]]Wherein 0 is<＝j<＝K ^′ -1 then, the computer device may be operated by the following schemeCalculating the minimum distance d=min of the distances of the samples i from the respective clusters _{0≤j≤K′-} ₁ Distance(dataMat[i],C[j]). Wherein the Distance may be L of Euclidean space ¹ 、L ² Or L ^∞ Norms. The cluster corresponding to the minimum distance is marked as cluster j ^′ 。

If the current K ^′ If K or D is less than or equal to D, dataMat [ i ]]Add to j ^′ And clustering. That is to say: the centroid is updated to C [ j ] ^′ ]<-(C[j ^′ ]*num[j]+dataMat[i])/(num[j]+1), the cluster j ^′ The number of the elements is updated to num [ j ] ^′ ]<-num[j ^′ ]+1. Otherwise, dataMat [ i ]]Self-organization is required. Meaning K ^′ <-K ^′ +1，num[K ^′ +1]＝1，C[K ^′ +1]＝dataMat[i]。

In the above embodiment, the computer device may perform distance on the sample according to the distance between the sample and the existing cluster, thereby obtaining a plurality of clusters. Samples belonging to the same cluster have certain similarity, and samples among different clusters have specific differences, so that abnormal samples can be conveniently screened.

In one embodiment, after the computer device clusters the full number of samples to obtain a plurality of clusters, a preset number of samples may be extracted from each cluster. And judging the extracted samples, and when the sample content of the extracted samples meets the malicious conditions, taking all the samples in the corresponding clusters as malicious samples. Therefore, the sample content of the whole sample is not required to be judged, and only the sample content of part of samples in each cluster is required to be judged, so that the screening efficiency of malicious samples is greatly improved.

In one embodiment, the classification model training method based on business safety further includes a step of performing safety control on the object to be processed, and the step specifically includes: acquiring an object to be processed belonging to a target service; classifying the object to be processed through the classification model obtained through training to obtain a class label of the object to be processed; and when the class label is a malicious class label, carrying out safety control on the object to be processed.

Specifically, the classification model obtained through training of the normal sample and the abnormal sample can be used for online safety control. The computer device may obtain the object to be processed belonging to the target service. The object to be processed can be any user account number or any piece of user production content generated in the target service, and also can be abnormal user account numbers or user production content screened out in other previous modes.

Furthermore, the computer equipment can input the feature vector corresponding to the object to be processed into the classification model, and the classification model is used for processing the feature vector to obtain a class label for classifying the object to be processed. When the class label is a malicious class label, the computer equipment can conduct safety control on the object to be processed. For example, the computer device may send warning information to a terminal corresponding to the malicious user account, limit the frequency with which the malicious user account issues information, limit the frequency with which the malicious user account adds friends, or freeze the malicious user account. When the computer device identifies the presence of malicious user-generated content, the malicious user-generated content may be deleted or closed in the background to prevent the malicious user-generated content from spreading.

In the embodiment, the online identification of the malicious content can be realized through the trained classification model, so that the identification efficiency and accuracy of the malicious content are greatly improved.

In one embodiment, the method for training a classification model based on business safety further comprises a step of updating the classification model, and the step specifically comprises the following steps: screening malicious objects with corresponding contents meeting malicious conditions from objects to be processed, wherein the class labels are malicious class labels; adding a malicious object into an existing malicious sample to update the malicious sample; training the classification model according to the updated malicious sample and the normal sample to update the classification model.

Specifically, the computer equipment can update the classification model based on the control condition while performing safety control on the target service through the classification model after training the classification model. The computer equipment classifies the object to be processed through the trained classification model to obtain the class label to which the object to be processed belongs. When the class label of the object to be processed is a malicious class label, the computer equipment can assist a machine to review or manually review and detect whether the classification result is accurate. When the classification result is accurate, that is, the content of the object to be processed does meet the malicious condition, the computer device can add the malicious object into the existing malicious sample to update the malicious sample. The computer device may then train the classification model based on the updated malicious and normal samples to update the classification model. In this way, malicious accounts or malicious user-generated content identified by the classification model may accumulate as historical malicious samples over time for retraining the updated classification model.

Referring to fig. 8, fig. 8 is a flowchart of the steps for security management of a target service in one embodiment. As shown in fig. 8, the step of performing security control on the target service specifically includes: s802, extracting features; s804, performing anomaly detection through an unsupervised machine learning algorithm; s806, outputting an abnormal sample; s808, manually auditing and feeding back a result; s810, a more positive and negative sample library; s812; training a classification model through a supervised machine learning algorithm; s814, safety control is carried out on the target service. Specifically, the computer device may determine a full sample from the original log of the target service, and further perform feature extraction on the full sample. Next, the computer device may provide an abnormal user account or abnormal UGC by anomaly detection of the full volume sample through an unsupervised machine learning algorithm. The staff can perform malicious recognition on the abnormal user account number or the abnormal UGC, and find out malicious user account numbers and malicious UGC in the abnormal user account numbers or the abnormal UGC as malicious samples. And subtracting the malicious samples from the total samples to obtain normal samples, so that positive and negative samples are separated, and a positive and negative sample library is constructed. And training the initial classification model based on the malicious sample and the normal sample by using a supervised machine learning algorithm to obtain a trained classification model. And applying the classification model to the target service to realize the closed loop of on-line identification, namely, safely controlling the target service. The computer device may also manually review the online identified malicious objects to add the actual malicious objects to the positive and negative sample libraries, updating the positive and negative sample libraries, so that the classification model may be retrained based on the updated positive and negative sample libraries to update the classification model. Therefore, the classification model can be continuously trained, used and updated, the manual operation and maintenance cost can be reduced by means of a small number of rule systems, and the accuracy and coverage rate can be effectively improved in the long term.

In a specific application scenario, the security countermeasure system can be applied to various internet products, such as instant messaging application, interested clans, or internet forums, and pornography information, spam messages, harassment behaviors, and theft accounts and the like existing in the specific service scenario can be effectively identified. By being able to achieve an index of 99% accuracy and 70% coverage even higher.

Fig. 2 is a flow diagram of a classification model training method based on business security in one embodiment. It should be understood that, although the steps in the flowchart of fig. 2 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 2 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the sub-steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of the sub-steps or stages of other steps or other steps.

As shown in fig. 9, in one embodiment, a classification model training apparatus 900 based on business safety is provided, including an acquisition module 901, a determination module 902, a screening module 903, and a training module 904, wherein:

an acquiring module 901, configured to acquire a full sample of a target service.

The determining module 902 is configured to perform anomaly detection on the full-scale sample by at least one anomaly detection method, and determine an anomaly sample from the full-scale sample.

A screening module 903, configured to screen the abnormal samples from malicious samples, where the content of the samples meets the malicious condition.

The determining module 902 is further configured to determine a normal sample according to samples from the total number of samples after the malicious samples are removed.

The training module 904 is configured to train the initial classification model based on the malicious sample and the normal sample, and obtain a classification model for performing security management and control on the target service.

In one embodiment, the obtaining module 901 is further configured to obtain a log file generated based on the target service in a preset period; determining user accounts appearing in the log file and user generated content corresponding to each user account; and taking all user accounts or all user generated contents as a full sample of the target service.

In one embodiment, the obtaining module 901 is further configured to obtain a full-scale sample of the target service, and a first sample feature corresponding to each sample in the full-scale sample. The determining module 902 is further configured to perform anomaly detection on the total number of samples according to at least one anomaly detection method, and obtain at least one group of candidate anomaly samples; and screening abnormal samples from the total samples according to the union set of at least one group of candidate abnormal samples.

In one embodiment, the obtaining module 901 is further configured to obtain a log file generated based on the target service in a preset period; the log file comprises user account numbers, object behavior data corresponding to each user account number and user generated content; taking all user account numbers appearing in the log file as full-quantity samples; acquiring user description information corresponding to each user account; and determining first sample characteristics corresponding to the user accounts respectively according to the object behavior data, the user generated content and the user description information corresponding to the user accounts respectively.

In one embodiment, the obtaining module 901 is further configured to obtain a log file generated based on the target service in a preset period; taking all user-generated content appearing in the log file as a full sample; and determining first sample characteristics corresponding to the user generated contents respectively according to the malicious contents respectively included in the user generated contents.

In one embodiment, the determining module 902 is further configured to obtain a first sample feature corresponding to each sample in the full-scale sample; respectively inputting the corresponding first sample characteristics of each sample into a trained reconstruction model to obtain corresponding output vectors; determining an error value corresponding to each sample based on the difference between the first sample characteristic and the output vector corresponding to each sample; and taking the sample with the corresponding error value meeting the abnormal condition as an abnormal sample.

In one embodiment, the determining module 902 is further configured to obtain a first sample feature corresponding to each sample in the full-scale sample; clustering the total samples based on the first sample characteristics corresponding to the samples respectively to obtain more than one type of clusters; according to the first sample characteristics of the samples respectively included in each cluster, determining characteristic average values respectively corresponding to each cluster; and screening out abnormal clusters from the clusters based on the characteristic distribution of the characteristic mean value corresponding to each cluster, and taking samples in the abnormal clusters as abnormal samples.

In one embodiment, the determining module 902 is further configured to determine a different cluster currently already present; for each sample, respectively calculating the distance between the sample and the different currently existing clusters according to the corresponding first sample characteristics; and when the minimum distance in the distances is smaller than or equal to the distance threshold value, dividing the samples into clusters corresponding to the minimum distance.

In one embodiment, the determining module 902 is further configured to determine the number of currently existing clusters when a minimum distance of the distances is greater than a distance threshold; when the number is smaller than the preset number, creating a new cluster, and dividing the samples into the new cluster; and when the number is equal to the preset number, dividing the samples into clusters corresponding to the minimum distance.

In one embodiment, the screening module 903 is further configured to determine a malicious sample screening manner; according to sample contents of samples in the abnormal samples and in a malicious sample screening mode, screening abnormal samples of which the sample contents meet malicious conditions from the abnormal samples; and marking the screened abnormal samples as malicious samples.

In one embodiment, training module 904 is further configured to determine a second sample feature and a category label for each of the malicious sample and the normal sample; respectively taking the second sample characteristics corresponding to the malicious sample and the normal sample as input data of an initial classification model; using a class label corresponding to the input data as a training label; and training an initial classification model through input data and corresponding training labels to obtain a classification model for safety control of target service.

In one embodiment, the business security based classification model training 900 further comprises a classification module 905 and a security management module 906, wherein:

the acquiring module 901 is further configured to acquire an object to be processed belonging to the target service.

The classification module 905 is configured to perform classification processing on the object to be processed by training the obtained classification model, so as to obtain a class label of the object to be processed.

The security management and control module 906 is configured to perform security management and control on the object to be processed when the class label is a malicious class label.

Referring to fig. 10, in one embodiment, the business security based classification model training apparatus 900 further comprises an update module 907, wherein:

the screening module 903 is further configured to screen, from objects to be processed whose class labels are malicious class labels, malicious objects whose corresponding content satisfies a malicious condition.

An updating module 907 is configured to add the malicious object to the existing malicious sample to update the malicious sample.

The updating module 907 is further configured to train the classification model according to the updated malicious samples and the normal samples, so as to update the classification model.

According to the classification model training device based on the business safety, the abnormality detection is carried out on the total samples of the target business by at least one abnormality detection mode, so that the abnormal samples are found out from the total samples. And further, malicious samples with sample contents meeting malicious conditions can be screened from the abnormal samples, and normal samples are determined according to samples with malicious samples removed from the total samples. Therefore, positive and negative samples can be rapidly and accurately separated by combining an unsupervised abnormality detection mode with content screening, and the classification model is trained through the positive and negative samples. Therefore, the classification model obtained through training can carry out safety control on the target service on line, a large number of manual updating, maintenance, discrimination and screening rules are not needed, and the cost of safety control is greatly reduced.

Fig. 11 shows an internal structural diagram of a computer device in one embodiment. The computer device may in particular be the computer device in fig. 1. As shown in fig. 11, the computer device includes a processor, a memory, and a network interface connected by a system bus. The memory includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system, and may also store a computer program that, when executed by a processor, causes the processor to implement a business security based classification model training method. The internal memory may also have stored therein a computer program which, when executed by the processor, causes the processor to perform a business security based classification model training method.

It will be appreciated by those skilled in the art that the structure shown in fig. 11 is merely a block diagram of a portion of the structure associated with the present application and is not limiting of the computer device to which the present application applies, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In one embodiment, the classification model training apparatus based on business security provided in the present application may be implemented as a computer program, which may be executed on a computer device as shown in fig. 11. The memory of the computer device may store various program modules constituting the traffic safety-based classification model training apparatus, such as the acquisition module, the determination module, the screening module, and the training module shown in fig. 9. The computer program of each program module is configured to cause a processor to perform the steps in the business security based classification model training method of each embodiment of the present application described in the present specification.

For example, the computer device shown in fig. 11 may perform step S202 through the acquisition module in the traffic safety-based classification model training apparatus as shown in fig. 9. The computer device may perform steps S204 and S208 by the determination module. The computer device may perform step S206 through the screening module. The computer device may perform step S210 through the training module.

In one embodiment, a computer device is provided that includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the business safety-based classification model training method described above. The step of the business safety-based classification model training method herein may be a step in the business safety-based classification model training method of the above-described respective embodiments.

In one embodiment, a computer readable storage medium is provided, storing a computer program which, when executed by a processor, causes the processor to perform the steps of the business security based classification model training method described above. The step of the business safety-based classification model training method herein may be a step in the business safety-based classification model training method of the above-described respective embodiments.

Those skilled in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, where the program may be stored in a non-volatile computer readable storage medium, and where the program, when executed, may include processes in the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.

Claims

1. A classification model training method based on business safety comprises the following steps:

acquiring a full sample of a target service;

constructing a plurality of sample characteristics of each sample according to the sample content of each sample to obtain a characteristic library of the sample, and screening all or part of sample characteristics from the characteristic library of the sample to construct first sample characteristics of each sample;

performing anomaly detection on the full samples based on the first sample characteristics corresponding to each sample through a plurality of anomaly detection modes to obtain a plurality of groups of candidate anomaly samples;

Determining an abnormal sample from the full samples according to the intersection or union of the plurality of groups of candidate abnormal samples;

and screening all or part of sample characteristics from a characteristic library of the samples according to service requirements, constructing second sample characteristics of the malicious samples and the normal samples, and further training an initial classification model to obtain a classification model for safely controlling the target service.

2. The method of claim 1, wherein the obtaining a full sample of the target traffic comprises:

acquiring a log file generated based on a target service in a preset period;

determining user accounts appearing in the log file and user generated contents corresponding to the user accounts;

and taking all user accounts or all user generated contents as a full sample of the target service.

3. The method of claim 1, wherein the obtaining a full sample of the target traffic comprises:

Acquiring a log file generated based on a target service in a preset period; the log file comprises user account numbers, object behavior data corresponding to each user account number and user generated content;

taking all user account numbers appearing in the log file as full samples;

the constructing a plurality of sample characteristics of each sample according to sample content of each sample comprises:

acquiring user description information corresponding to each user account;

and constructing corresponding sample characteristics according to the object behavior data, the user generated content and the user description information respectively corresponding to the user account numbers.

4. The method of claim 1, wherein the obtaining a full sample of the target traffic comprises:

acquiring a log file generated based on a target service in a preset period;

taking all user-generated content appearing in the log file as a full sample;

and constructing corresponding sample characteristics according to malicious content respectively included in each user generated content.

5. The method of claim 1, wherein one of the plurality of anomaly detection methods comprises:

Respectively inputting the corresponding first sample characteristics of each sample into a trained reconstruction model to obtain corresponding output vectors;

determining an error value corresponding to each sample based on the difference between the first sample characteristic and the output vector corresponding to each sample;

and taking the sample with the corresponding error value meeting the abnormal condition as a candidate abnormal sample.

6. The method of claim 1, wherein one of the plurality of anomaly detection methods comprises:

clustering the total number of samples based on the first sample characteristics corresponding to each sample to obtain more than one type of clusters;

according to the first sample characteristics of the samples respectively included in each cluster, determining characteristic average values respectively corresponding to each cluster;

and screening out abnormal clusters from the clusters based on the characteristic distribution of the characteristic mean value corresponding to each cluster, and taking samples in the abnormal clusters as candidate abnormal samples.

7. The method of claim 6, wherein clustering the full-scale samples based on the first sample features corresponding to each sample to obtain clusters of more than one type, comprises:

determining different clusters existing currently;

For each sample, respectively calculating the distance between the sample and different currently existing clusters according to the corresponding first sample characteristics;

and dividing the samples into clusters corresponding to the minimum distance when the minimum distance in the distances is smaller than or equal to a distance threshold.

8. The method of claim 7, wherein the method further comprises:

when the minimum distance in the distances is greater than the distance threshold, determining the number of currently existing clusters;

when the number is smaller than a preset number, creating a new cluster, and dividing the sample into the new cluster;

and dividing the samples into clusters corresponding to the minimum distance when the number is equal to the preset number.

9. The method according to claim 1, wherein the screening the abnormal samples for malicious samples whose sample contents satisfy a malicious condition comprises:

determining a malicious sample screening mode;

according to the sample content of each sample in the abnormal samples and the malicious sample screening mode, screening abnormal samples of which the sample content meets malicious conditions from the abnormal samples;

and marking the screened abnormal samples as malicious samples.

10. The method of claim 1, wherein training the initial classification model to obtain a classification model for security management of the target traffic comprises:

determining second sample characteristics and category labels corresponding to the malicious samples and the normal samples respectively;

respectively taking the second sample characteristics corresponding to the malicious sample and the normal sample as input data of an initial classification model;

using a class label corresponding to the input data as a training label;

and training the initial classification model through the input data and the corresponding training label to obtain a classification model for carrying out safety control on the target service.

11. The method according to any one of claims 1-10, further comprising:

acquiring an object to be processed belonging to the target service;

classifying the object to be processed through the classification model obtained through training to obtain a class label of the object to be processed;

and when the class label is a malicious class label, carrying out safety control on the object to be processed.

12. The method of claim 11, wherein the method further comprises:

Screening malicious objects with corresponding contents meeting the malicious conditions from objects to be processed, wherein the class labels are malicious class labels;

adding the malicious object into an existing malicious sample to update the malicious sample;

training the classification model according to the updated malicious sample and the normal sample to update the classification model.

13. A business security-based classification model training apparatus, the apparatus comprising:

the acquisition module is further used for constructing a plurality of sample characteristics of each sample according to the sample content of each sample, obtaining a characteristic library of the sample, and screening all or part of sample characteristics from the characteristic library of the sample to construct first sample characteristics of each sample;

the determining module is used for carrying out anomaly detection on the total samples based on the first sample characteristics corresponding to each sample respectively through a plurality of anomaly detection modes to obtain a plurality of groups of candidate anomaly samples; determining an abnormal sample from the full samples according to the intersection or union of the plurality of groups of candidate abnormal samples;

and the training module is used for screening all or part of sample characteristics from the characteristic library of the samples according to service requirements, constructing second sample characteristics of the malicious samples and the normal samples, and further training an initial classification model to obtain a classification model for safely controlling the target service.

14. The apparatus of claim 13, wherein the obtaining module is further configured to obtain a log file generated based on the target service in a preset period; determining user accounts appearing in the log file and user generated contents corresponding to the user accounts; and taking all user accounts or all user generated contents as a full sample of the target service.

15. The apparatus of claim 13, wherein the obtaining module is further configured to obtain a log file generated based on the target service in a preset period; the log file comprises user account numbers, object behavior data corresponding to each user account number and user generated content; taking all user account numbers appearing in the log file as full samples; acquiring user description information corresponding to each user account; and constructing corresponding sample characteristics according to the object behavior data, the user generated content and the user description information respectively corresponding to the user account numbers.

16. The apparatus of claim 13, wherein the obtaining module is further configured to obtain a log file generated based on the target service in a preset period; taking all user-generated content appearing in the log file as a full sample; and constructing corresponding sample characteristics according to malicious content respectively included in each user generated content.

17. The apparatus of claim 13, wherein the determining module is further configured to input the first sample feature corresponding to each sample to the trained reconstruction model to obtain a corresponding output vector; determining an error value corresponding to each sample based on the difference between the first sample characteristic and the output vector corresponding to each sample; and taking the sample with the corresponding error value meeting the abnormal condition as a candidate abnormal sample.

18. The apparatus of claim 13, wherein the determining module is further configured to cluster the full number of samples based on a first sample feature corresponding to each sample to obtain more than one type of cluster; according to the first sample characteristics of the samples respectively included in each cluster, determining characteristic average values respectively corresponding to each cluster; and screening out abnormal clusters from the clusters based on the characteristic distribution of the characteristic mean value corresponding to each cluster, and taking samples in the abnormal clusters as candidate abnormal samples.

19. The apparatus of claim 18, wherein the means for determining is further configured to determine a different cluster currently already present; for each sample, respectively calculating the distance between the sample and different currently existing clusters according to the corresponding first sample characteristics; and dividing the samples into clusters corresponding to the minimum distance when the minimum distance in the distances is smaller than or equal to a distance threshold.

20. The apparatus of claim 19, wherein the means for determining is further for determining a number of currently existing clusters when a minimum of the distances is greater than the distance threshold; when the number is smaller than a preset number, creating a new cluster, and dividing the sample into the new cluster; and dividing the samples into clusters corresponding to the minimum distance when the number is equal to the preset number.

21. The apparatus of claim 13, wherein the screening module is further configured to determine a malicious sample screening method; according to the sample content of each sample in the abnormal samples and the malicious sample screening mode, screening abnormal samples of which the sample content meets malicious conditions from the abnormal samples; and marking the screened abnormal samples as malicious samples.

22. The apparatus of claim 13, wherein the training module is further configured to determine a second sample feature and a category label for each of the malicious sample and the normal sample; respectively taking the second sample characteristics corresponding to the malicious sample and the normal sample as input data of an initial classification model; using a class label corresponding to the input data as a training label; and training the initial classification model through the input data and the corresponding training label to obtain a classification model for carrying out safety control on the target service.

23. The apparatus of any one of claims 13-22, further comprising a classification module and a security management module, wherein:

the acquisition module is used for acquiring an object to be processed belonging to the target service;

the classification module is used for carrying out classification treatment on the object to be treated through the classification model obtained through training to obtain a class label of the object to be treated;

and the security management and control module is used for performing security management and control on the object to be processed when the class label is a malicious class label.

24. The apparatus of claim 23, further comprising a screening module and an updating module, wherein:

the screening module is used for screening malicious objects with corresponding contents meeting the malicious conditions from objects to be processed, the class labels of which are malicious class labels;

the updating module is used for adding the malicious object into an existing malicious sample so as to update the malicious sample;

the updating module is further configured to train the classification model according to the updated malicious sample and the normal sample, so as to update the classification model.

25. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the method of any one of claims 1 to 12.

26. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method of any of claims 1 to 12.