CN111259985A

CN111259985A - Classification model training method and device based on business safety and storage medium

Info

Publication number: CN111259985A
Application number: CN202010103759.8A
Authority: CN
Inventors: 张戎
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Cloud Computing Changsha Co Ltd
Priority date: 2020-02-19
Filing date: 2020-02-19
Publication date: 2020-06-09
Anticipated expiration: 2040-02-19
Also published as: CN111259985B

Abstract

The application relates to a classification model training method, a device and a storage medium based on business safety, wherein the method comprises the following steps: acquiring a full sample of a target service; carrying out anomaly detection on the full-scale samples by at least one anomaly detection mode, and determining abnormal samples from the full-scale samples; screening a malicious sample of which the sample content meets a malicious condition from the abnormal sample; determining a normal sample according to the sample from which the malicious sample is removed from the full amount of samples; training an initial classification model based on the malicious sample and the normal sample to obtain a classification model for performing security control on the target service. The scheme provided by the application can reduce the cost of safety control.

Description

Classification model training method and device based on business safety and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a classification model training method and apparatus based on business security, a computer-readable storage medium, and a computer device.

Background

With the wide use of the internet technology, the potential safety hazards exist while convenience is brought to life and work of people. For example, in some business scenarios, users may publish, through a social network, User-created Content, such as UGC (User Generated Content) Content, and some malicious users may use these ways to disseminate malicious Content, such as spam messages, yellow violent information, or information violating laws and regulations, which may have a very adverse effect on life and work of people. Therefore, how to screen and screen the malicious content becomes very important.

The traditional way of screening and screening malicious content in a network is usually based on keyword matching. For example, a table about black words may be maintained according to different service scenarios, and malicious content may be screened according to the number of times of occurrence of black words in the published content. However, for the traditional screening mode, along with the migration of time, the number of keywords needing manual maintenance is increased, and the combination of words is also more and more complex, so that the manual operation and maintenance cost is huge.

Disclosure of Invention

Based on this, it is necessary to provide a classification model training method and apparatus based on business security, a computer-readable storage medium, and a computer device, for solving the technical problem that the conventional malicious content screening method is high in cost.

A classification model training method based on business safety comprises the following steps:

acquiring a full sample of a target service;

carrying out anomaly detection on the full-scale samples by at least one anomaly detection mode, and determining abnormal samples from the full-scale samples;

screening a malicious sample of which the sample content meets a malicious condition from the abnormal sample;

determining a normal sample according to the sample from which the malicious sample is removed from the full amount of samples;

training an initial classification model based on the malicious sample and the normal sample to obtain a classification model for performing security control on the target service.

A classification model training apparatus based on business security, the apparatus comprising:

the acquisition module is used for acquiring a full sample of the target service;

the determining module is used for carrying out anomaly detection on the full-scale samples through at least one anomaly detection mode and determining an abnormal sample from the full-scale samples;

the screening module is used for screening a malicious sample of which the sample content meets malicious conditions from the abnormal sample;

the determining module is further used for determining a normal sample according to the sample from which the malicious sample is removed from the full amount of samples;

and the training module is used for training an initial classification model based on the malicious sample and the normal sample to obtain a classification model for performing safety control on the target service.

A computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:

acquiring a full sample of a target service;

A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of:

acquiring a full sample of a target service;

According to the classification model training method based on the business safety, the device, the computer readable storage medium and the computer equipment, the anomaly detection is performed on the full-scale samples of the target business through at least one anomaly detection mode, so that the abnormal samples can be found from the full-scale samples. And then, malicious samples with sample contents meeting malicious conditions can be screened from the abnormal samples, and normal samples are determined according to the samples with the malicious samples removed from the total samples. Therefore, the positive and negative samples can be quickly and accurately separated by combining an unsupervised anomaly detection mode with content screening, and the classification model is trained through the positive and negative samples. Therefore, the trained classification model can perform safety control on the target service on line, a large amount of manual real-time updating, maintenance, discrimination and screening rules are not needed, and the safety control cost is greatly reduced.

Drawings

FIG. 1 is a diagram of an exemplary implementation of a classification model training method based on business security;

FIG. 2 is a flowchart illustrating a classification model training method based on business security in an embodiment;

FIG. 3 is an overall framework diagram for training and using an initial classification model in one embodiment;

FIG. 4 is a flowchart illustrating the steps of performing anomaly detection on a full-scale sample by at least one anomaly detection method and determining an anomalous sample from the full-scale sample according to one embodiment;

FIG. 5 is a schematic diagram of a network architecture of a reconstructed model in one embodiment;

FIG. 6 is a schematic flowchart illustrating the steps of performing anomaly detection on a full-scale sample by at least one anomaly detection method and determining an anomalous sample from the full-scale sample according to another embodiment;

FIG. 7 is a flow diagram of the steps of clustering samples in one embodiment;

FIG. 8 is a flowchart illustrating steps for performing security management and control on a target service in one embodiment;

FIG. 9 is a block diagram of an embodiment of a classification model training apparatus based on business security;

FIG. 10 is a block diagram showing a configuration of a classification model training apparatus based on business security according to another embodiment;

FIG. 11 is a block diagram of a computer device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

FIG. 1 is a diagram of an application environment of a classification model training method based on business security in an embodiment. Referring to fig. 1, the classification model training method based on business safety is applied to a security countermeasure system. The secure countermeasure system includes a user terminal 110 and a computer device 120. The user terminal 110 and the computer device 120 are connected via a network. The user terminal 110 may be a desktop terminal or a mobile terminal, and the mobile terminal may be at least one of a mobile phone, a tablet computer, a notebook computer, and the like. The computer device 120 may specifically be a terminal or a server, wherein the server may be implemented by an independent server or a server cluster composed of a plurality of servers.

It can be understood that, a user may execute an operation based on a target service through the user terminal 110, the computer device 120 may obtain log files generated based on the target service from different user terminals 110, and further obtain a full sample of the target service from the log files, the computer device 120 may obtain a classification model for performing security control on the target service by executing a classification model training method based on service security mentioned in the later embodiments of the present application, and may perform security control on the target service online through the classification model, without a large amount of manual real-time updates of the discrimination screening rules, thereby greatly reducing the cost of performing security control on the target service.

As shown in FIG. 2, in one embodiment, a classification model training method based on business security is provided. The embodiment is mainly illustrated by applying the method to the computer device 120 in fig. 1. Referring to fig. 2, the classification model training method based on business safety specifically includes the following steps:

s202, acquiring a full sample of the target service.

The target service is a service that needs to be managed and controlled safely, and specifically may be an internet product in an internet service scene. The user may operate the internet product and generate a series of events through the internet product, for example, the user may add friends through a social application, post posts, post user generated content, or make comments, etc., thereby generating a series of corresponding events. The security control of the target service is to perform security control on a series of events generated in the target service, that is, to handle the situation that potential safety hazards exist in a service scene, such as handling malicious events like pornographic content, spam messages, harassment behaviors, or user account stealing.

The full samples are a batch of samples for performing classification model training, and the full samples may be specifically samples which can be obtained currently and are used for performing classification model training in a preset period, and the number of the samples may be more or less. It can be understood that, in the case of a large number of samples, the training effect on the classification model is relatively better. In different traffic scenarios, the corresponding full size samples may be different, and the number of full size samples may increase or decrease over time. That is, new samples may be added to the full sample to train the classification model, or inappropriate samples may be eliminated.

Specifically, different applications run on the terminal, the computer device may determine a social application corresponding to the target service, acquire social data uploaded by the terminal and generated by the social application, and determine a full sample based on the social data in a preset period. The social data is data generated when a user uses a social application, such as a user account, user behavior data, user generated content, comment information, or the like. The user behavior data is data reflecting social behaviors of the user, such as login time, login location, login times, login terminal, posting time or posting times when the user posts through a user account, and the like. The user generated content is content created and issued by a user on an internet platform, and the user generated content specifically can be text, pictures, videos, link addresses, symbols, expressions and the like.

In one embodiment, the full-scale samples used to train the classification model include at least one of user accounts and user-generated content. When the full sample is the user account, the classification model obtained through the training of the full sample is used for classifying the user account so as to find out a malicious user account. When the full-scale samples are used for generating contents for the users, the classification model obtained through full-scale sample training is used for classifying each piece of user generated content so as to find out malicious user generated content.

In one embodiment, the computer device may select a user account or user-generated content from all social data in a preset period as a full sample, or may randomly or regularly select a part of the user account or user-generated content as a full sample. For example, the user account with the same login location or the user generated content with the same distribution location is selected regularly, or for example, the user account logged in within a fixed time period or the user generated content released within a fixed time period is selected, and the like.

In an embodiment, the step S202, that is, the step of obtaining the full sample of the target service specifically includes: acquiring a log file generated based on a target service in a preset period; determining user accounts appearing in the log file and user generated content corresponding to each user account; and taking all user accounts or all user generated contents as a full sample of the target service.

Specifically, the computer device may obtain, from the terminal or the server, a log file generated based on the target service for a preset period. The log file is a historical log file, in which user accounts where operations occur, user behavior data and user generated content corresponding to each user account, and the like are recorded. It is understood that the preset period is a preset period of time, such as a day, a week, a month, etc.

Furthermore, the computer device can extract the user accounts appearing in the log file and the user generated content corresponding to each user account, and take all the user accounts or all the user generated content as the full sample of the target service.

It can be understood that, when the computer device takes all the user accounts as full samples, the classification model obtained by executing the classification model training method based on the service security according to the embodiment of the present application is used for classifying the user accounts collected on line based on the corresponding full samples. When the computer device takes all the user-generated content as the full-scale samples, the classification model obtained by the classification model training method based on the business safety mentioned in the embodiment of the application is executed based on the corresponding full-scale samples, and can be used for classifying the user-generated content collected on line.

In the above embodiment, all user accounts or all user generated contents in a log file generated based on the target service in a preset period may be used as a full-scale sample of the target service, and a classification model for processing the user accounts or the user generated contents may be obtained based on the corresponding full-scale sample in a trainable manner.

S204, carrying out anomaly detection on the full-scale samples through at least one anomaly detection mode, and determining the anomalous samples from the full-scale samples.

The anomaly detection mode is a mode for detecting anomalous data in a large amount of data, is also an outlier detection mode, and belongs to an unsupervised machine learning algorithm. Specifically, the computer device may determine at least one abnormality detection mode in advance, perform abnormality detection on the entire samples in each abnormality detection mode, and form a set of the abnormality samples detected by all the abnormality detection modes, where elements in the set are all the abnormality samples. Thus, the abnormal sample can be prevented from being leaked.

In one embodiment, when there are multiple abnormality detection manners, the computer device may further perform an intersection operation on the abnormality samples detected by the multiple abnormality detection manners, and use a sample detected as an abnormality sample by all of the multiple abnormality detection manners as a final abnormality sample. Therefore, misjudgment of the abnormal sample can be reduced.

In an embodiment, the anomaly detection method may specifically be anomaly detection based on statistics, anomaly detection based on a neural network, anomaly detection based on clustering, or anomaly detection based on PCA (principal component analysis), or other anomaly detection methods, which are not limited in this embodiment of the present invention. The abnormal point detection based on statistics may be specifically to construct a probability distribution model, calculate the probability that each sample meets the model, and regard the sample with low probability as an abnormal point, that is, an abnormal sample. The abnormal point detection based on the neural network can be based on One Class Support Vector Machine (SVM) or Isolation Forest algorithm. The cluster-based outlier detection may be specifically a streaming clustering algorithm, biKmeans (binary K-means clustering), or Kmeans (K-means clustering) algorithm, etc. In one embodiment, the computer device may construct sample features corresponding to each sample based on sample content corresponding to the sample, thereby constructing the first sample feature based on some or all of the sample features. And the computer device can perform anomaly detection on the full-scale samples based on the first sample characteristics corresponding to each sample in the full-scale samples, so as to determine the anomalous samples from the full-scale samples.

The sample features are feature vectors constructed based on sample contents of the samples, and are used for quantitatively representing characteristics of the samples by using numerical values. The first sample characteristic is a sample characteristic required for abnormality detection. The second sample feature mentioned later is a sample feature required when training the classification model. It is to be understood that the first sample feature and the second sample feature may be the same sample feature, may also be partially the same sample feature, or may be completely different sample features, and the like, which may be determined according to actual business requirements, and this is not limited in this embodiment of the application.

In one embodiment, the computer device may construct a plurality of sample features of the sample according to user behavior data corresponding to the user account, user generated content, feedback information of other users, and corresponding user images, and the plurality of sample features may collectively form a feature library of the sample. Therefore, the sample can be converted into the sample, and the characteristics of the sample are reflected by a plurality of sample characteristics of the sample, so that the subsequent processing is facilitated. There are various ways to construct the sample features, such as normalization (normalization), discretization (discretization), binarization of the features (binning), cross-feature, etc., which are not limited in this embodiment. The user profile is a set of tags that describe characteristics such as user preferences, habits, or attributes.

In one embodiment, referring to Table 1 below, Table 1 is a schematic table of a library of features for a sample in one embodiment. The computer device may construct sample features for 6 dimensions from the sample content as in the table below, for each sample, feature values in these 6 dimensions may be determined based on the respective sample content, respectively.

Feature 1

Feature 2

Feature 3

Feature 4

Feature 5

Feature 6

Sample 1

a1

b1

c1

d1

e1

f1

Sample 2

a2

b2

c2

d2

e2

f2

Sample 3

a3

b3

c3

d3

e3

f3

Sample 4

a4

b4

c4

d4

e4

f4

TABLE 1 library of characteristics of samples

In one embodiment, the computer device may screen some or all of the sample features from a library of features of the sample, and concatenate to form a first sample feature, such as [ feature 1, feature 2, feature 5, feature 6 ]. For the construction of the second sample characteristics, the computer device can screen part or all of the sample characteristics from the characteristic library of the sample according to business requirements, and the second sample characteristics are spliced and formed, such as [ characteristics 1, 2, 3 and 4 ].

S206, screening malicious samples with sample contents meeting malicious conditions from the abnormal samples.

The sample content is content corresponding to the sample, and may specifically be information for constructing the first sample feature. When the abnormal sample is a user account, the corresponding sample content may specifically be user behavior data, user generated content, feedback information of other users, a corresponding user portrait, and the like corresponding to the user account. When the abnormal sample is the user-generated content, the corresponding sample content may specifically be information included in the user-generated content, such as information of a text, a link, or a picture included in the piece of user-generated content.

Specifically, the computer device can screen out a malicious sample meeting the malicious condition according to the sample content of the abnormal sample. The sample content satisfies a malicious condition, and specifically, the sample content may include malicious information. For example, when the abnormal sample is the abnormal user biographical content, the abnormal user biographical content includes a malicious link, a pornographic picture, a character violating laws and regulations, or the repetition frequency of the user generated content exceeds a preset frequency, and the like, and any one or more of the above conditions is satisfied, that is, the abnormal user generated content is the malicious user generated content, that is, the malicious sample. For example, when the abnormal sample is an abnormal user account, the abnormal user account issues malicious user generated content, the posting frequency of the abnormal user account is higher than the preset frequency, the abnormal user account logs in a different place and performs fund transfer, and the like, and when the abnormal user account is in any one or more of the above situations, the abnormal user account can be determined to be a malicious user account, that is, the abnormal sample.

In one embodiment, the computer device can set a judgment rule, and the set judgment rule is adopted manually or mechanically to screen out the malicious samples from the abnormal samples.

In one embodiment, the step S206, namely, the step of screening the abnormal samples for malicious samples whose sample contents satisfy the malicious conditions specifically includes: determining a malicious sample screening mode; according to the sample content of each sample in the abnormal samples, screening the abnormal samples of which the sample content meets the malicious conditions from the abnormal samples according to a malicious sample screening mode; and marking the screened abnormal sample as a malicious sample.

The malicious sample screening method is a method for screening a malicious sample, and for example, specifically, the method may be a method for screening a malicious sample by using an experienced worker to judge the sample content of an abnormal sample based on a screening rule, so as to perform screening, or the method may include learning the screening rule in advance through a machine model, inputting the sample content into the machine model to screen the abnormal sample, and may further combine manual screening and machine screening to more accurately screen the malicious sample from the abnormal sample, and the screening method is not limited, and the embodiment of the present application does not limit this.

Specifically, the computer device may screen, according to the sample content of each sample in the abnormal samples, the abnormal samples whose sample content satisfies the malicious condition from the abnormal samples in the malicious sample screening manner. Further, the computer device may label the screened abnormal sample as a malicious sample. Namely, the category label of the screened malicious sample is set as a malicious category label, and the malicious sample is a negative sample.

In the above embodiment, according to the malicious sample screening mode, the malicious sample is screened from the abnormal sample, a small amount of screening rules can be maintained, the screening efficiency and accuracy are greatly improved, and the accuracy and efficiency of screening can be improved regardless of whether the screening is judged by manual assistance or machine screening.

And S208, determining a normal sample according to the sample without the malicious sample in the full amount of samples.

Specifically, the computer device may take all the samples of the full amount of samples, from which the labeled malicious samples have been removed, as normal samples. Alternatively, the computer device may also decimate a portion of the samples from the full number of samples after removing the malicious samples as normal samples. The method of the decimation may be random sampling selection or directional decimation, which is not limited in the embodiment of the present application. The number of the decimated partial samples may be less than, equal to, or more than the number of the malicious samples, which is not limited in the embodiments of the present application.

In one embodiment, after the computer device determines the normal sample from the full amount of samples, the class label of the normal sample may be set as the normal class label, and the normal sample is the positive sample.

S210, training the initial classification model based on the malicious sample and the normal sample to obtain a classification model for performing security control on the target service.

Specifically, the computer device may obtain an initial classification model, where the classification model may be a mathematical model constructed by algorithms such as decision tree, logistic regression, naive bayes, or neural networks, and the embodiment of the present application is not limited thereto. The neural network algorithm may be, for example, a GBDT (Gradient Boosting Decision Tree) algorithm, or a random forest algorithm.

Furthermore, the computer device can train the initial classification model based on the malicious sample and the normal sample to obtain a trained classification model for performing security control on the target service. It is understood that the training of the classification model at this time is the training mode of the supervised machine learning algorithm. The classification model can classify the object to be processed in the target service on line so as to identify whether the object to be processed is a malicious object. The online real-time scoring and prediction classification can be carried out through the trained classification model, so that the effect of judging the malicious behavior of the user is achieved.

In one embodiment, the initial classification model is constructed by an xgboost (extreme gradient boosting) algorithm, which is a variation of the GBDT algorithm. The classification model constructed by the xgboost algorithm has several advantages: (1) the fitting capability is excellent, so that under-fitting and over-fitting can be effectively prevented; (2) the characteristic threshold is low, namely excessive characteristic engineering construction is not needed; (3) the speed is high, and the tree construction process can be accelerated in a multi-thread mode; (4) compared with GBDT which only aims at the first derivative information, the xgboost carries out second-order Taylor expansion on the target function, and uses the first derivative and the second derivative simultaneously, so that the classification accuracy is higher.

In an embodiment, the step S210, that is, training the initial classification model based on the malicious sample and the normal sample, and obtaining the classification model for performing security control on the target service specifically includes: determining second sample characteristics and class labels corresponding to the malicious sample and the normal sample respectively; respectively taking second sample characteristics corresponding to the malicious sample and the normal sample as input data of an initial classification model; taking a class label corresponding to the input data as a training label; and training an initial classification model through input data and corresponding training labels to obtain a classification model for performing safety control on the target service.

Reference may be made to the above-mentioned embodiments for the construction of the second sample feature. The second sample characteristic may be the same as or different from the first sample characteristic. The computer device may construct a second sample characteristic based on all or a portion of the sample characteristic.

In particular, the computer device may determine second sample characteristics and class labels that correspond to each of the malicious sample and the normal sample. For a malicious sample, the corresponding class label is a malicious class label; for a normal sample, its corresponding class label is a normal class label.

When the computer device trains the initial classification model, the second sample characteristics corresponding to the malicious sample and the normal sample can be respectively used as input data and are sequentially input into the classification model, and the prediction output corresponding to the input data is obtained. The computer device can construct a loss function based on the difference between the predicted output corresponding to the input data and the category label. And in the training process, adjusting model parameters, taking the model parameters when the loss function is minimized as current model parameters, and continuously repeating the training process. And stopping training until the training stopping condition is reached to obtain the trained classification model. The training stopping condition may specifically be that a preset iteration number is reached, or the classification performance of the trained classification model reaches a preset index, and the like.

Therefore, an initial classification model is trained through the normal sample and the malicious sample, a classification model with good classification performance can be obtained, and the classification model is used for carrying out safety control on the target service.

Referring to FIG. 3, FIG. 3 is an overall framework diagram for training and using an initial classification model in one embodiment. As shown in FIG. 3, the overall framework diagram for training and using the initial classification model includes five parts, such as positive and negative sample separation 301, constructed sample feature 302, supervised machine learning algorithm 303, off-line training classification model 304, and on-line using classification model 305 in FIG. 3. Specifically, the computer device takes a full sample and performs positive and negative sample separation on the full sample. By adopting the method mentioned in the previous embodiment, the corresponding sample characteristics of the malicious sample and the normal sample are constructed, and the second sample characteristic is used at this time. And then performing off-line training on the initial classification model through a supervised machine learning algorithm to obtain a trained classification model. And finally, the computer equipment can use the trained classification model for online prediction, namely, online classification of the object to be processed in the target service is carried out so as to identify whether the object to be processed is a malicious object.

In this case, the positive and negative samples mentioned in the embodiments of the present application are separated, which is different from the conventional recommendation system. For the recommendation system, when the supervised machine learning algorithm is used, the labeling of the samples is very clear, and excessive extra work is not needed. However, in the security countermeasure system, sample separation is a very important link because the target service is data-annotated. Therefore, in practical applications, a batch of malicious samples can be found by detecting the abnormal samples through the abnormal detection method mentioned in the foregoing embodiment, that is, the abnormal detection method based on the unsupervised machine learning algorithm, and by adding the experience of the worker. And subtracting the malicious sample from the full sample to obtain a normal sample. Through such a mode, can accurately accomplish sample separation fast, need not a large amount of human costs.

According to the classification model training method based on the business safety, the full-scale samples of the target business are subjected to anomaly detection through at least one anomaly detection mode, so that the anomalous samples are found from the full-scale samples. And then, malicious samples with sample contents meeting malicious conditions can be screened from the abnormal samples, and normal samples are determined according to the samples with the malicious samples removed from the total samples. Therefore, the positive and negative samples can be quickly and accurately separated by combining an unsupervised anomaly detection mode with content screening, and the classification model is trained through the positive and negative samples. Therefore, the trained classification model can perform safety control on the target service on line, a large amount of manual real-time updating, maintenance, discrimination and screening rules are not needed, and the safety control cost is greatly reduced. Moreover, the safety control of the target service is realized through the trained classification model, and the control efficiency and accuracy are also improved.

In an embodiment, the step S202, that is, the step of obtaining the full sample of the target service specifically includes: and acquiring a full-scale sample of the target service and first sample characteristics corresponding to each sample in the full-scale sample. Step S204, namely, performing anomaly detection on the full-scale samples by at least one anomaly detection method, and the step of determining the anomalous samples from the full-scale samples specifically includes: performing anomaly detection on the full amount of samples respectively based on the first sample characteristics corresponding to the samples through at least one anomaly detection mode to obtain at least one group of candidate anomaly samples; and screening abnormal samples from the total amount of samples according to the union set of the candidate abnormal samples of at least one group.

Specifically, the computer device may obtain, in the manner mentioned in the above embodiment, the full-size sample of the target service and the first sample feature corresponding to each sample in the full-size sample. Furthermore, the computer device can perform anomaly detection on the total number of samples respectively based on the first sample characteristics corresponding to the samples through at least one anomaly detection mode to obtain at least one group of candidate anomaly samples. The candidate abnormal samples here are abnormal samples detected by the abnormal family detection methods. And then the computer equipment can obtain a union of at least one group of candidate abnormal samples, namely when a plurality of groups of candidate abnormal samples exist, performing OR operation on each group of candidate abnormal samples, and taking all the candidate abnormal samples which appear as the abnormal samples at this time.

In the above embodiment, based on the first sample characteristics corresponding to each sample, the abnormality detection can be accurately and quickly performed on the whole number of samples. And then the candidate abnormal samples detected by each abnormal detection mode are used as the abnormal samples, so that the abnormal samples can be ensured not to be leaked.

In one embodiment, the step of obtaining the full-size samples of the target service and the first sample characteristics respectively corresponding to each sample in the full-size samples specifically includes: acquiring a log file generated based on a target service in a preset period; the log file comprises user accounts, user behavior data corresponding to the user accounts and user generated content; taking all user accounts appearing in the log file as a full sample; acquiring user figures corresponding to the user accounts respectively; and determining first sample characteristics respectively corresponding to the user accounts according to the user behavior data, the user generated content and the user portrait respectively corresponding to the user accounts.

In one embodiment, the computer device may obtain a log file generated based on the target service in a preset period, where the log file includes user accounts, user behavior data corresponding to the user accounts, and user-generated content. Further, the computing device may take as a full sample all of the user accounts present in the log file. The computer equipment can construct a user portrait corresponding to the user account in advance from other service platforms or based on service data of a local service platform. Furthermore, the computer device can perform feature construction according to the user behavior data corresponding to the user account, the user generated content, the user portrait and other contents to obtain corresponding sample features. It will be appreciated that the sample feature is a feature vector, each element in the feature vector representing a feature value of the sample in a respective dimension. The computer equipment can screen partial or all characteristic values from the sample characteristics corresponding to the user account, and the partial or all characteristic values are spliced to form corresponding first sample characteristics.

In one embodiment, the computer device may further obtain negative feedback information and positive feedback information of the other users for the user account, where the negative feedback information is, for example, reporting information and the like, and the positive feedback information is, for example, approval information and the like. Furthermore, the computer device can perform feature construction based on user behavior data corresponding to the user account, user generated content, user portrait, and other user feedback information to the user account, so as to obtain corresponding sample features.

In the embodiment, the corresponding first sample characteristic can be constructed based on the user behavior data, the user generated content and the user portrait corresponding to the user account, and the constructed first sample characteristic can comprehensively and accurately reflect the characteristics corresponding to the user account.

In one embodiment, the step of obtaining the full-size samples of the target service and the first sample characteristics respectively corresponding to each sample in the full-size samples specifically includes: acquiring a log file generated based on a target service in a preset period; taking all user generated contents appearing in the log file as a full sample; and determining first sample characteristics respectively corresponding to the user generated contents according to the malicious contents respectively included in the user generated contents.

In one embodiment, the computer device may obtain a log file generated based on the target service within a preset period, and take all user-generated content appearing in the log file as a full-size sample. I.e. each user generated content, is a sample. Further, the computer device may obtain specific content corresponding to each piece of user-generated content to determine sample characteristics of the piece of user-generated content. For example, the computer device may calculate a dirty word score based on the dirty words that occur in the user-generated content, such as a score for each occurrence of a dirty word. The computer device can also detect whether a malicious link exists in the user generated content or whether a malicious picture exists in the user generated content, and the like, and the score is counted once each occurrence. In this way, the computer device may determine feature values of the piece of user-generated content in the corresponding dimension according to the score values corresponding to the plurality of dimensions described above, where the feature values of the plurality of dimensions collectively constitute the sample feature. Furthermore, the computer device can filter partial or all characteristic values from the sample characteristics corresponding to the user generated content, and the partial or all characteristic values are spliced to form corresponding first sample characteristics.

In the above embodiment, the first sample feature corresponding to the user-generated content may be determined based on malicious content included in the user-generated content, and the first sample feature constructed in this way may comprehensively and accurately reflect the characteristics corresponding to the user-generated content.

In one embodiment, the step S204, namely performing anomaly detection on the full-scale samples by at least one anomaly detection method, and determining the anomalous samples from the full-scale samples, includes:

s402, acquiring first sample characteristics corresponding to all samples in the full amount of samples.

S404, inputting the first sample characteristics corresponding to each sample into the trained reconstruction model respectively to obtain corresponding output vectors.

The reconstruction model is a mathematical model constructed based on a feedforward neural network. The reconstruction model can comprise a multi-layer feedforward neural network, the feedforward neural network simulates an identity mapping, the number of nodes of an input layer is equal to that of nodes of an output layer, and the number of nodes of a hidden layer is generally smaller than that of nodes of the input layer. The reconstruction model can play a role in compressing data and recovering data, and input data is reconstructed through the reconstruction model.

Referring to fig. 5, fig. 5 is a schematic diagram of a network structure of a reconstructed model in an embodiment. The reconstructed model in fig. 5 includes an output layer, an output layer and three hidden layers. The number of nodes of the input layer and the output layer is 6, which represents that the sample has 6 characteristics. The number of nodes of the first and third hidden layers (4 nodes in fig. 5) is less than that of the input layer, and the number of nodes of the second hidden layer is the smallest (2 nodes in fig. 5). In the case of neural network transmission, a tanh function (hyperbolic tangent function) and a sigmoid function (sigmoid function) are used in the middle. Since the reconstruction model trains an identity mapping (identity mapping) from the input layer to the output layer, data compression starts from the input layer during transmission and data decompression starts after the second hidden layer. The training objective function is such that the overall output error is sufficiently small, which is obtained by dividing the sum of all sample errors by the number of samples. Taking the 6 features shown in fig. 5 as an example, the error of the ith sample is:

wherein x is_ijCharacteristic value r of j-th characteristic of i-th sample_ijAnd a predicted feature value representing the jth feature of the ith sample output by the reconstruction model. In each training process, a classical back propagation algorithm (back propagation) is generally used to update the model parameters, the model parameters of the reconstructed model are adjusted towards minimizing the overall error, and the trained reconstructed model is obtained after the training is finished. It is to be understood that the network structure of the reconstructed model in fig. 5 is for illustration only, and is not intended to limit the network structure of the reconstructed model.

Specifically, the computer device may obtain a trained reconstruction model, input the first sample characteristics corresponding to each sample to the trained reconstruction model, respectively, perform data compression and decompression through a hidden layer of the reconstruction model, and output a corresponding output vector. The output vector is formed by splicing the predicted eigenvalues of the characteristics corresponding to the sample.

S406, determining an error value corresponding to each sample based on the difference between the first sample feature and the output vector corresponding to each sample.

Specifically, the computer device may determine an error value for each sample based on a difference between the first sample feature and the output vector for each sample, which may also be considered a reconstruction score. Wherein the first sample feature is composed of feature values corresponding to the respective features. The output vector is composed of predicted feature values corresponding to the respective features. In one embodiment, the computer device may calculate the error value corresponding to each sample by the following formula:

wherein e is_iRepresents the error value of the ith sample, n represents the total number of features of the sample, x_ijCharacteristic value r of j-th characteristic of i-th sample_ijAnd a predicted feature value representing the jth feature of the ith sample output by the reconstruction model.

And S408, taking the sample with the corresponding error value meeting the abnormal condition as an abnormal sample.

Specifically, the computer device may calculate an error value corresponding to each sample, and use the sample whose error value satisfies the abnormal condition as the abnormal sample. The error value satisfying the abnormal condition may be specifically that the error value is greater than a preset threshold, or each error value is ranked before a preset ranking according to a ranking from large to small. In this way, the computer device can take a batch of samples with larger error values as outlier samples.

In the above embodiment, the reconstruction model is used to perform reconstruction processing on the first sample feature corresponding to each sample, so as to determine the error value corresponding to each sample. Therefore, a batch of samples with larger error values can be directly used as abnormal samples quickly and accurately according to the error values.

s602, acquiring first sample characteristics corresponding to each sample in the full amount of samples.

And S604, based on the first sample characteristics corresponding to the samples respectively, clustering the whole samples to obtain more than one type of clusters.

Specifically, the computer device may perform clustering processing on the full number of samples according to the first sample characteristics corresponding to each sample, respectively, to obtain more than one type of clusters. The clustering algorithm used in the clustering process may specifically be a streaming clustering algorithm, biKmeans, or Kmeans algorithm, and the like, which is not limited in the embodiment of the present application.

And S606, determining the feature mean value corresponding to each cluster according to the first sample feature of the sample included in each cluster.

Specifically, the computer device may determine a centroid vector of each cluster according to a first sample feature of a sample included in each cluster, and the centroid vector may be used as a feature mean of the cluster. For example, for each sample in the cluster, the computer device may calculate a sum of distances between the sample and other samples in the cluster, and use the sample with the shortest distance sum as the centroid of the cluster, where the corresponding centroid vector of the cluster is the first sample characteristic of the centroid. Specifically, the distance between different samples may be calculated by using the first sample characteristics corresponding to the samples, such as the euclidean distance, the manhattan distance, or the chebyshev distance between two samples, which is not limited in this embodiment of the present application. In addition, the computer device may further calculate an average vector based on the first sample features corresponding to all the samples in the cluster, where the average vector is the centroid vector. It will be appreciated that as new samples in a cluster increase, the centroid vector for that cluster will also be updated accordingly.

S608, based on the feature distribution of the feature mean value corresponding to each cluster, abnormal clusters are screened out from the clusters, and samples in the abnormal clusters are used as abnormal samples.

Furthermore, the computer device may determine a distribution of feature mean values of different clusters according to the feature mean values respectively corresponding to the clusters, and screen out the discrete feature mean values, where the cluster corresponding to the discrete feature mean value may be regarded as an abnormal cluster, and a sample included in the abnormal cluster is an abnormal sample.

In the embodiment, the clustering processing can be performed on the full amount of samples, and the samples in the abnormal clusters which are different from other clusters greatly are used as the abnormal samples, so that the method is accurate and convenient.

In an embodiment, the step S604, that is, the step of clustering the full samples to obtain more than one type of clusters specifically includes: determining different clusters which exist currently; for each sample, respectively calculating the distance between the sample and different clusters which currently exist according to the corresponding first sample characteristic; and when the minimum distance in the distances is smaller than or equal to the distance threshold, dividing the samples into clusters corresponding to the minimum distance.

Specifically, when the computer device performs clustering processing on a full number of samples, the sample which is processed at the beginning can be directly used as a cluster, then the distance between other samples and the cluster is calculated, when the distance is smaller than or equal to a distance threshold value, the other samples are divided into the cluster, otherwise, the other samples are self-clustered. Thus, as the number of processed samples increases, the clusters obtained by clustering the samples also increase.

Referring to FIG. 7, FIG. 7 is a flow diagram of the steps of clustering samples in one embodiment. As shown in fig. 7, the flowchart of the step of clustering the samples mainly includes: s702, separating positive and negative samples; s704, judging whether the new sample is similar to a certain existing class or not; if yes, go to step S706, insert the new sample into the class; if not, step S708 is entered, and the new sample is classified. In particular, when clustering an unknown new sample, the computer device can determine whether the new sample is sufficiently similar to an existing class (i.e., cluster). If so, the new sample is inserted into the cluster, and if not, the cluster is self-classified. The determination of whether the new sample is sufficiently similar to an existing class may be specifically determined by comparing whether the distance between the new sample and the class is smaller than a distance threshold, where if the distance is smaller than the distance threshold, it indicates that the new sample is sufficiently similar to the class, and otherwise, it is not similar.

In one embodiment, step S604 further comprises: when the minimum distance in the distances is larger than a distance threshold value, determining the number of clusters which exist currently; when the number is smaller than the preset number, creating a new cluster, and dividing the samples into the new cluster; and when the number is equal to the preset number, dividing the samples into clusters corresponding to the minimum distance.

In one embodiment, when the minimum distance of the distances between a certain sample and the existing clusters is less than or equal to the distance threshold, the computer device may divide the sample into the clusters corresponding to the minimum distance. When the minimum distance among all the distances is greater than the distance threshold, the computer device may determine the number of clusters that currently exist, and if the number of clusters that currently exist is less than a preset number, divide the sample into new clusters. And if the number of the clusters existing currently is equal to the preset number, dividing the sample into the cluster corresponding to the minimum distance. In this way, the total number of clusters can be guaranteed to be a preset number.

The process of clustering samples is detailed below by way of example:

the computer equipment can construct a data matrix according to the first eigenvectors respectively corresponding to different samples. For example, when there are m samples each having n features, i.e., the first feature vector has n elements, a matrix dataMat of m rows and n columns is formed. Each row represents a first sample feature and the columns represent the dimensions of the feature. In other words, there are m points inside the n-dimensional euclidean space that need to be clustered. In this way, the clustering of samples can be converted into clustering of points.

Let dataMat be a matrix of m rows and n columns, each row representing a vector, and n representing the dimensions of the vector. K represents the maximum number of clusters allowed to be formed in the clustering process; d represents a distance threshold. The distance between two points can here be used as L¹、L²Or L^∞Norm. The centroid of a cluster is defined as the average of all points in the class. For example, a cluster j includes A [0]]，A[1]，…，A[n-1]Then the centroid of the cluster j is

And the number of elements in the jth cluster is num [ j ]]And (4) showing.

Step (1), for dataMat [0], self-clustering. The centroid of the cluster is C [0] ═ dataMat [0], the number of elements in the cluster is num [0] ═ 1, and the number of all clusters is K ═ 1.

Step (2), for each sample i, 1 < i < m-1, the computer equipment performs the following cyclic operation: assuming that there are K' clusters currently, the centroid of the jth cluster is C [ j]The number of elements in the jth cluster is num [ j ]]Wherein 0 < ═ j < ═ K' -1, the computer apparatus may calculate a minimum distance d ═ min among distances of the sample i from the respective clusters by the following formula_{0≤j≤K′-1}Distance(dataMat[i]，C[j]). Where Distance may be L of Euclidean space¹、L²Or L^∞And (4) norm. The cluster corresponding to the minimum distance is denoted as cluster j'.

If either current K' is K or D ≦ D, dataMat [ i ] is added to the jth cluster. That is to say: the centroid is updated to C [ j ' ] < (C [ j ' ] × hum [ j ] + dataMat [ i ])/(num [ j ] +1), and the number of elements in the cluster j ' is updated to num [ j ' ] < -num [ j ' ] + 1. Otherwise, dataMat [ i ] needs to be self-categorizing. This means that K '< -K' +1, num [ K '+ 1] ═ 1, and C [ K' +1] ═ dataMat [ i ].

In the above embodiment, the computer device may perform distance processing on the sample according to the distance between the sample and the existing cluster, so as to obtain the plurality of clusters. Samples belonging to the same cluster have certain similarity, and samples in different clusters are different, so that abnormal samples can be screened conveniently.

In one embodiment, after the computer device clusters the full amount of samples to obtain a plurality of clusters, a preset number of samples can be extracted from each cluster respectively. And judging the extracted samples, and when the sample content of the extracted samples meets malicious conditions, all samples in the corresponding clusters can be used as malicious samples. Therefore, the sample content of the whole sample does not need to be judged, and the sample content of part of samples in each cluster only needs to be judged, so that the screening efficiency of malicious samples is greatly improved.

In one embodiment, the classification model training method based on business security further includes a step of performing security control on the object to be processed, where the step specifically includes: acquiring an object to be processed belonging to a target service; classifying the object to be processed through the classification model obtained through training to obtain a class label of the object to be processed; and when the category label is a malicious category label, performing safety control on the object to be processed.

Specifically, the classification model obtained through training of the normal sample and the abnormal sample can be used for online safety control. The computer device may obtain an object to be processed belonging to the target service. The object to be processed may be any user account or any user production content generated in the target service, or may be an abnormal user account or user generated content screened in other manners before.

Furthermore, the computer device can input the feature vector corresponding to the object to be processed into the classification model, and the feature vector is processed through the classification model to obtain the class label for classifying the object to be processed. When the category label is a malicious category label, the computer device can perform security control on the object to be processed. For example, the computer device may send warning information to a terminal corresponding to the malicious user account, limit the frequency of issuing information by the malicious user account, limit the frequency of adding friends to the malicious user account, or freeze the malicious user account. When the computer device identifies that the malicious user generated content exists, the malicious user generated content can be deleted or closed in the background so as to prevent the malicious user generated content from spreading.

In the embodiment, the online identification and attack of the malicious content can be realized through the trained classification model, and the identification efficiency and accuracy of the malicious content are greatly improved.

In one embodiment, the method for training a classification model based on business safety further includes a step of updating the classification model, where the step specifically includes: screening malicious objects of which corresponding contents meet malicious conditions from the objects to be processed of which the category labels are malicious category labels; adding the malicious object into an existing malicious sample to update the malicious sample; and training the classification model according to the updated malicious sample and the updated normal sample so as to update the classification model.

Specifically, after the classification model is trained, the computer device can update the classification model based on the condition of management and control while performing security management and control on the target service through the classification model. And the computer equipment classifies the object to be processed through the trained classification model to obtain the class label of the object to be processed. When the class label of the object to be processed is a malicious class label, the computer equipment can assist a machine to review or manually review to detect whether the classification result is accurate. When the classification result is accurate, that is, the content of the object to be processed does meet the malicious condition, the computer device may add the malicious object to the existing malicious sample to update the malicious sample. And then the computer equipment can train the classification model according to the updated malicious sample and the normal sample so as to update the classification model. In this way, over time, malicious accounts or malicious user-generated content hit by the classification model may be accumulated as historical malicious samples for retraining the updated classification model.

Referring to fig. 8, fig. 8 is a flowchart illustrating steps of performing security management on a target service in an embodiment. As shown in fig. 8, the step of performing security control on the target service specifically includes: s802, feature extraction; s804, carrying out anomaly detection through an unsupervised machine learning algorithm; s806, outputting an abnormal sample; s808, manually checking and feeding back results; s810, a more positive and negative sample library is obtained; s812; training the classification model by a supervised machine learning algorithm; s814, the target service is managed and controlled safely. Specifically, the computer device may determine a full sample from an original log of the target service, and then perform feature extraction on the full sample. Next, the computer device may perform anomaly detection on the full sample through an unsupervised machine learning algorithm, providing an abnormal user account or an abnormal UGC. The staff can carry out malicious identification on the abnormal user accounts or the abnormal UGC, and find out the malicious user accounts and the malicious UGC as malicious samples. And subtracting the malicious sample from the full sample to obtain a normal sample, so that positive and negative samples are separated, and a positive and negative sample library is constructed. And training the initial classification model based on the malicious sample and the normal sample by a supervised machine learning algorithm to obtain a trained classification model. The classification model is applied to the target service, so that the closed loop of online striking is realized, namely the target service is safely controlled. The computer equipment can also manually recheck the malicious objects identified on line so as to add the real malicious objects into the positive and negative sample libraries and update the positive and negative sample libraries, so that the classification model can be retrained based on the updated positive and negative sample libraries to update the classification model. Therefore, the classification model can be continuously trained, used and updated, the manual operation and maintenance cost can be reduced only by a small amount of rule systems, and the accuracy and the coverage rate can be effectively improved in the long term.

In a specific application scenario, the security countermeasure system can be applied to various internet products, such as instant messaging applications, interest departments, or internet forums, and the like, and pornographic information, spam messages, harassment behaviors, account stealing and the like existing in the specific service scenario can be effectively attacked. The accuracy can reach 99%, and the coverage rate is 70% or even higher.

FIG. 2 is a flowchart illustrating a classification model training method based on business security in an embodiment. It should be understood that, although the steps in the flowchart of fig. 2 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 2 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

As shown in fig. 9, in an embodiment, a classification model training apparatus 900 based on business security is provided, which includes an obtaining module 901, a determining module 902, a screening module 903, and a training module 904, where:

an obtaining module 901, configured to obtain a full sample of the target service.

A determining module 902, configured to perform anomaly detection on the full-scale samples through at least one anomaly detection method, and determine an anomalous sample from the full-scale samples.

And the screening module 903 is used for screening the malicious samples of which the sample contents meet the malicious conditions from the abnormal samples.

The determining module 902 is further configured to determine a normal sample according to the sample from which the malicious sample is removed from the full amount of samples.

And a training module 904, configured to train the initial classification model based on the malicious sample and the normal sample, so as to obtain a classification model for performing security control on the target service.

In an embodiment, the obtaining module 901 is further configured to obtain a log file generated based on a target service in a preset period; determining user accounts appearing in the log file and user generated content corresponding to each user account; and taking all user accounts or all user generated contents as a full sample of the target service.

In one embodiment, the obtaining module 901 is further configured to obtain a full sample of the target service and first sample features corresponding to respective samples in the full sample. The determining module 902 is further configured to perform anomaly detection on the full amount of samples respectively based on the first sample features corresponding to the samples through at least one anomaly detection manner, so as to obtain at least one group of candidate anomaly samples; and screening abnormal samples from the total amount of samples according to the union set of the candidate abnormal samples of at least one group.

In an embodiment, the obtaining module 901 is further configured to obtain a log file generated based on a target service in a preset period; the log file comprises user accounts, user behavior data corresponding to the user accounts and user generated content; taking all user accounts appearing in the log file as a full sample; acquiring user figures corresponding to the user accounts respectively; and determining first sample characteristics respectively corresponding to the user accounts according to the user behavior data, the user generated content and the user portrait respectively corresponding to the user accounts.

In an embodiment, the obtaining module 901 is further configured to obtain a log file generated based on a target service in a preset period; taking all user generated contents appearing in the log file as a full sample; and determining first sample characteristics respectively corresponding to the user generated contents according to the malicious contents respectively included in the user generated contents.

In one embodiment, the determining module 902 is further configured to obtain first sample features corresponding to respective samples in the full amount of samples; respectively inputting the first sample characteristics corresponding to each sample into the trained reconstruction model to obtain corresponding output vectors; determining an error value corresponding to each sample based on the difference between the first sample characteristic and the output vector corresponding to each sample; and taking the sample of which the corresponding error value meets the abnormal condition as an abnormal sample.

In one embodiment, the determining module 902 is further configured to obtain first sample features corresponding to respective samples in the full amount of samples; based on the first sample characteristics corresponding to each sample, clustering the full amount of samples to obtain more than one type of clusters; determining a characteristic mean value corresponding to each cluster according to the first sample characteristic of the sample included in each cluster; and screening abnormal clusters from the clusters based on the characteristic distribution of the characteristic mean value corresponding to each cluster, and taking the samples in the abnormal clusters as abnormal samples.

In one embodiment, the determining module 902 is further configured to determine a different cluster that currently exists; for each sample, respectively calculating the distance between the sample and different clusters which currently exist according to the corresponding first sample characteristic; and when the minimum distance in the distances is smaller than or equal to the distance threshold, dividing the samples into clusters corresponding to the minimum distance.

In one embodiment, the determining module 902 is further configured to determine the number of clusters that currently exist when the minimum distance of the distances is greater than a distance threshold; when the number is smaller than the preset number, creating a new cluster, and dividing the samples into the new cluster; and when the number is equal to the preset number, dividing the samples into clusters corresponding to the minimum distance.

In one embodiment, the screening module 903 is further configured to determine a malicious sample screening manner; according to the sample content of each sample in the abnormal samples, screening the abnormal samples of which the sample content meets the malicious conditions from the abnormal samples according to a malicious sample screening mode; and marking the screened abnormal sample as a malicious sample.

In one embodiment, the training module 904 is further configured to determine second sample features and class labels corresponding to the malicious sample and the normal sample, respectively; respectively taking second sample characteristics corresponding to the malicious sample and the normal sample as input data of an initial classification model; taking a class label corresponding to the input data as a training label; and training an initial classification model through input data and corresponding training labels to obtain a classification model for performing safety control on the target service.

In one embodiment, the traffic security-based classification model training 900 further includes a classification module 905 and a security management and control module 906, wherein:

the obtaining module 901 is further configured to obtain an object to be processed belonging to a target service.

The classification module 905 is configured to perform classification processing on the object to be processed through the trained classification model to obtain a class label of the object to be processed.

And the security management and control module 906 is configured to perform security management and control on the object to be processed when the category tag is the malicious category tag.

Referring to fig. 10, in one embodiment, the apparatus 900 for training classification model based on business safety further includes an update module 907, wherein:

the screening module 903 is further configured to screen a malicious object whose corresponding content meets a malicious condition from the objects to be processed whose category labels are malicious category labels.

An updating module 907, configured to add the malicious object to the existing malicious sample to update the malicious sample.

The updating module 907 is further configured to train the classification model according to the updated malicious samples and normal samples to update the classification model.

The classification model training device based on the business safety detects the abnormality of the full-scale samples of the target business through at least one abnormality detection mode so as to find the abnormal samples from the full-scale samples. And then, malicious samples with sample contents meeting malicious conditions can be screened from the abnormal samples, and normal samples are determined according to the samples with the malicious samples removed from the total samples. Therefore, the positive and negative samples can be quickly and accurately separated by combining an unsupervised anomaly detection mode with content screening, and the classification model is trained through the positive and negative samples. Therefore, the trained classification model can perform safety control on the target service on line, a large amount of manual real-time updating, maintenance, discrimination and screening rules are not needed, and the safety control cost is greatly reduced.

FIG. 11 is a diagram illustrating an internal structure of a computer device in one embodiment. The computer device may specifically be the computer device of fig. 1. As shown in fig. 11, the computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program, which, when executed by the processor, causes the processor to implement a classification model training method based on business security. The internal memory may also have stored therein a computer program that, when executed by the processor, causes the processor to perform a classification model training method based on business security.

Those skilled in the art will appreciate that the architecture shown in fig. 11 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, the classification model training apparatus based on business security provided by the present application may be implemented in the form of a computer program, and the computer program may be run on a computer device as shown in fig. 11. The memory of the computer device may store various program modules constituting the service-security-based classification model training apparatus, such as the acquisition module, the determination module, the screening module, and the training module shown in fig. 9. The computer program of each program module causes the processor to execute the steps of the classification model training method based on business safety of each embodiment of the application described in the specification.

For example, the computer device shown in fig. 11 may execute step S202 through an obtaining module in the classification model training apparatus based on business security as shown in fig. 9. The computer device may perform steps S204 and S208 by the determination module. The computer device may perform step S206 through the screening module. The computer device may perform step S210 through the training module.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the above-described traffic safety-based classification model training method. The steps of the business safety-based classification model training method herein may be the steps in the business safety-based classification model training methods of the various embodiments described above.

In one embodiment, a computer-readable storage medium is provided, in which a computer program is stored, which, when executed by a processor, causes the processor to perform the steps of the above-described traffic safety-based classification model training method. The steps of the business safety-based classification model training method herein may be the steps in the business safety-based classification model training methods of the various embodiments described above.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A classification model training method based on business safety comprises the following steps:

acquiring a full sample of a target service;

2. The method of claim 1, wherein obtaining a full sample of the target traffic comprises:

acquiring a log file generated based on a target service in a preset period;

determining user accounts appearing in the log file and user generated content corresponding to each user account;

and taking all user accounts or all user generated contents as a full sample of the target service.

3. The method of claim 1, wherein obtaining a full sample of the target traffic comprises:

acquiring a full-scale sample of a target service and first sample characteristics respectively corresponding to each sample in the full-scale sample;

the performing anomaly detection on the full-scale sample through at least one anomaly detection mode to determine an abnormal sample from the full-scale sample includes:

performing anomaly detection on the full-scale samples respectively based on the first sample characteristics corresponding to the samples through at least one anomaly detection mode to obtain at least one group of candidate anomaly samples;

and screening abnormal samples from the full amount of samples according to the union set of the candidate abnormal samples of the at least one group.

4. The method according to claim 3, wherein the obtaining of the full-size samples of the target service and the first sample characteristics corresponding to each of the full-size samples comprises:

acquiring a log file generated based on a target service in a preset period; the log file comprises user accounts, user behavior data corresponding to the user accounts and user generated content;

taking all user accounts appearing in the log file as a full sample;

acquiring user figures corresponding to the user accounts respectively;

and determining first sample characteristics respectively corresponding to the user accounts according to the user behavior data, the user generated content and the user portrait respectively corresponding to the user accounts.

5. The method according to claim 3, wherein the obtaining of the full-size samples of the target service and the first sample characteristics corresponding to each of the full-size samples comprises:

acquiring a log file generated based on a target service in a preset period;

taking all user-generated content appearing in the log file as a full sample;

and determining first sample characteristics respectively corresponding to the user generated contents according to the malicious contents respectively included in the user generated contents.

6. The method of claim 1, wherein said detecting anomalies in said full-size samples by at least one anomaly detection method, and wherein determining anomalous samples from said full-size samples comprises:

acquiring first sample characteristics corresponding to all samples in the full-scale samples respectively;

respectively inputting the first sample characteristics corresponding to each sample into the trained reconstruction model to obtain corresponding output vectors;

determining an error value corresponding to each sample based on a difference between a first sample characteristic corresponding to each sample and an output vector;

and taking the sample of which the corresponding error value meets the abnormal condition as an abnormal sample.

7. The method of claim 1, wherein said detecting anomalies in said full-size samples by at least one anomaly detection method, and wherein determining anomalous samples from said full-size samples comprises:

based on the first sample characteristics corresponding to each sample, clustering the full amount of samples to obtain more than one type of clusters;

determining a characteristic mean value corresponding to each cluster according to the first sample characteristic of the sample included in each cluster;

and screening abnormal clusters from the clusters based on the characteristic distribution of the characteristic mean value respectively corresponding to each cluster, and taking the samples in the abnormal clusters as abnormal samples.

8. The method according to claim 7, wherein the clustering the full amount of samples based on the first sample characteristics corresponding to the respective samples to obtain more than one type of clusters comprises:

determining different clusters which exist currently;

for each sample, respectively calculating the distance between the sample and different clusters which currently exist according to the corresponding first sample characteristic;

and when the minimum distance in the distances is smaller than or equal to a distance threshold value, dividing the samples into clusters corresponding to the minimum distance.

9. The method of claim 8, further comprising:

when the minimum distance in the distances is larger than the distance threshold value, determining the number of the clusters which exist currently;

when the number is smaller than the preset number, creating a new cluster, and dividing the sample into the new cluster;

and when the number is equal to the preset number, dividing the samples into clusters corresponding to the minimum distance.

10. The method of claim 1, wherein the screening of the abnormal samples for malicious samples whose sample contents satisfy malicious conditions comprises:

determining a malicious sample screening mode;

according to the sample content of each sample in the abnormal samples, screening abnormal samples of which the sample content meets malicious conditions from the abnormal samples according to the malicious sample screening mode;

and marking the screened abnormal sample as a malicious sample.

11. The method of claim 1, wherein training an initial classification model based on the malicious samples and the normal samples to obtain a classification model for security control of the target traffic comprises:

determining second sample characteristics and class labels corresponding to the malicious sample and the normal sample respectively;

respectively taking second sample characteristics corresponding to the malicious sample and the normal sample as input data of an initial classification model;

taking a class label corresponding to the input data as a training label;

and training the initial classification model through the input data and the corresponding training labels to obtain a classification model for performing safety control on the target service.

12. The method according to any one of claims 1-11, further comprising:

acquiring an object to be processed belonging to the target service;

classifying the object to be processed through a classification model obtained through training to obtain a class label of the object to be processed;

and when the category label is a malicious category label, performing security control on the object to be processed.

13. The method of claim 12, further comprising:

screening malicious objects of which corresponding contents meet the malicious conditions from the objects to be processed of which the category labels are malicious category labels;

adding the malicious object into an existing malicious sample to update the malicious sample;

and training the classification model according to the updated malicious sample and the normal sample so as to update the classification model.

14. A classification model training device based on business safety is characterized in that the device comprises:

15. A computer-readable storage medium, storing a computer program which, when executed by a processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 13.