CN113313404B - Method and device for generating countermeasure sample - Google Patents
Method and device for generating countermeasure sample Download PDFInfo
- Publication number
- CN113313404B CN113313404B CN202110662087.9A CN202110662087A CN113313404B CN 113313404 B CN113313404 B CN 113313404B CN 202110662087 A CN202110662087 A CN 202110662087A CN 113313404 B CN113313404 B CN 113313404B
- Authority
- CN
- China
- Prior art keywords
- feature
- sample
- transformation
- risk
- characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0635—Risk analysis of enterprise or organisation activities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Human Resources & Organizations (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Economics (AREA)
- Molecular Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Strategic Management (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- General Business, Economics & Management (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Tourism & Hospitality (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the specification provides a method and a device for generating a countermeasure sample, wherein the method comprises the steps of firstly obtaining original sample characteristics of a target risk sample and a risk detection model. Determining a predetermined number T of reference directions in a predetermined feature space for the target risk sample; and sequentially executing a plurality of rounds of feature transformation according to different reference directions in the T reference directions by preset first step length until a first transformation feature is obtained, wherein the risk detection model aims at that the predicted value of the first transformation feature falls into a preset interval corresponding to the non-risk sample. Then, for the first transformation characteristic, executing several times of fallback operations with successively increasing fallback amplitudes in the reverse direction of the last characteristic transformation direction until a second transformation characteristic is obtained, wherein the risk detection model reaches the boundary of the predetermined interval for the predicted value of the second transformation characteristic. Then, a countermeasure sample corresponding to the target risk sample can be formed according to the second transformation characteristic.
Description
Technical Field
One or more embodiments of the present description relate to the field of machine learning, and more particularly, to methods and apparatus for generating confrontational samples.
Background
The rapid development of machine learning enables various machine learning models to be applied to various business scenes. In security and wind control scenarios, for example, some risk detection models have been trained through machine learning for identifying at-risk or potentially-compromised objects. For example, identifying spam account numbers, identifying high risk transactions, identifying high risk operations, etc. by a risk detection model. Such risk objects are often intercepted after they are identified to ensure the security of the system and the user.
In view of the fact that existing risk detection models are often insufficient in robustness, an improved scheme is expected, and the risk detection models can be optimized to improve the robustness of the risk detection models and are better suitable for the characteristics of attack and defense in a risk detection scene.
Disclosure of Invention
One or more embodiments of the present specification describe a method and an apparatus for generating a countermeasure sample, which can simulate a countermeasure sample generated by an attacker, so that a risk detection model can be optimized from the perspective of attack, defense and countermeasure by using such a countermeasure sample, and the robustness and the security of the risk detection model are enhanced.
According to a first aspect, there is provided a method of generating a challenge sample, comprising:
acquiring original sample characteristics and a risk detection model of a target risk sample;
determining a predetermined number T of reference directions in a predetermined feature space;
aiming at the original sample characteristics, sequentially performing a plurality of rounds of characteristic transformation according to different reference directions in the T reference directions by preset first step length until first transformation characteristics are obtained, wherein the risk detection model aims at that the predicted value of the first transformation characteristics falls into a preset interval corresponding to a non-risk sample;
aiming at the first transformation characteristic, executing a plurality of times of fallback operation with sequentially increased fallback amplitudes along the reverse direction of the last characteristic transformation direction in the plurality of times of characteristic transformations until a second transformation characteristic is obtained, wherein the risk detection model aims at that the predicted value of the second transformation characteristic reaches the boundary of the preset interval; the backoff amplitude is smaller than the first step size;
and forming a countermeasure sample corresponding to the target risk sample according to the second transformation characteristic.
In one embodiment, the T reference directions are orthogonal to each other in the feature space.
According to one embodiment, the tth round of feature transformation of the plurality of rounds of feature transformation comprises: acquiring a t-th reference direction; taking the features obtained in the previous round as current features, and performing feature transformation according to the tth reference direction by the first step length on the basis of the current features to obtain intermediate features; updating the current feature with the intermediate feature if the first predicted value for the intermediate feature by the risk detection model is further away from a risk label value than the second predicted value for the current feature.
In an embodiment of the foregoing implementation, performing feature transformation according to the tth reference direction to obtain an intermediate feature specifically includes: respectively taking the forward direction and the reverse direction as selected directions; and superposing disturbance transformation with the size of a first step length on the current feature along the selected direction of the t-th reference direction, and taking the obtained feature as the intermediate feature.
In another embodiment of the foregoing embodiment, performing feature transformation according to the tth reference direction to obtain an intermediate feature specifically includes: respectively taking the forward direction and the reverse direction as selected directions; superimposing disturbance transformation with a first step length on the current feature along the selected direction of the t-th reference direction to obtain a temporary feature; if the transformation size of the temporary feature relative to the original sample feature exceeds a preset transformation boundary, projecting the temporary feature into the preset transformation boundary through a preset projection function to serve as the intermediate feature.
In an embodiment of the foregoing embodiment, the t-th round feature transformation further includes: judging whether the predicted value of the risk detection model for the updated current feature falls into the preset interval or not; if the current time does not fall into the preset interval, entering the next round of feature transformation; and if the current feature falls into the preset interval, taking the current feature as the first transformation feature.
According to one embodiment, the ith backoff operation of the plurality of backoff operations comprises: determining the backspacing amplitude of the time; backspacing the backspacing amplitude based on the first transformation characteristic along the reverse direction of the last characteristic transformation direction to obtain an intermediate characteristic; updating the current feature with the intermediate feature if a first predicted value for the intermediate feature by the risk detection model is closer to a risk label value than a second predicted value for a current feature of a previous recording.
In an embodiment of the foregoing embodiment, determining the current backoff amplitude includes: and superposing a preset second step length on the backspacing amplitude of the last backspacing operation to obtain the current backspacing amplitude.
In another embodiment of the foregoing embodiment, determining the current backoff amplitude includes: multiplying the back-off coefficient of the last back-off operation by a preset attenuation coefficient to obtain the back-off coefficient of the current time; the attenuation coefficient is less than 1; and determining a difference value obtained by subtracting the current backspacing coefficient from 1, and taking the product of the difference value and the first step length as the current backspacing amplitude.
In an embodiment of the foregoing embodiment, backspacing the current backspacing amplitude based on the first transformed feature to obtain an intermediate feature specifically includes: reducing the backspacing amplitude of the current time on the first transformation characteristic to obtain a temporary characteristic; if the transformation size of the temporary feature relative to the original sample feature does not exceed a preset transformation boundary, taking the temporary feature as an intermediate feature; and if the temporary feature exceeds the preset transformation boundary, projecting the temporary feature into the preset transformation boundary through a preset projection function to serve as the intermediate feature.
According to one embodiment, the ith backoff operation of the number of backoff operations further comprises: judging whether a first predicted value of the risk detection model for the intermediate feature belongs to the preset interval or not; if the current time interval belongs to the preset interval, entering the next rollback operation; and if the current feature does not belong to the preset interval, taking the current feature recorded last time as the second transformation feature.
According to one embodiment, the predetermined feature space is a low-dimensional feature space obtained by performing dimension reduction mapping on a high-dimensional feature space corresponding to the original sample feature according to a predetermined mapping relationship; correspondingly, the forming of the countermeasure sample corresponding to the target risk sample according to the second transformation characteristic includes: and mapping the second transformation characteristic in the preset characteristic space back to the high-dimensional characteristic space according to the inverse mapping of the preset mapping relation to serve as the sample characteristic of the confrontation sample.
In various examples, the sample is one of: account number, transaction, text segment, user operation.
According to a second aspect, there is provided a method of optimizing a risk detection model, comprising:
acquiring a sample set formed by original samples, wherein the sample set comprises normal samples and risk samples;
obtaining a challenge sample generated for the risk sample according to the method of the first aspect;
and updating the risk detection model by using each original sample and each confrontation sample in the sample set.
According to a third aspect, there is provided a method of generating a challenge sample, comprising:
obtaining original sample characteristics and a classification model of a target sample, wherein the target sample corresponds to a first category;
determining a predetermined number T of reference directions in a predetermined feature space;
according to the original sample characteristics, sequentially according to different reference directions in the T reference directions, executing a plurality of rounds of characteristic transformation with a preset first step length until first transformation characteristics are obtained, wherein the classification model aims at that the predicted value of the first transformation characteristics falls into a preset interval corresponding to a non-first-class sample;
according to the first transformation characteristic, executing a plurality of times of fallback operation with successively increased fallback amplitudes along the reverse direction of the last characteristic transformation direction in the plurality of times of characteristic transformations until a second transformation characteristic is obtained, wherein the classification model reaches the boundary of the preset interval according to a predicted value of the second transformation characteristic; the backoff amplitude is smaller than the first step size;
and forming a confrontation sample corresponding to the target sample according to the second transformation characteristic.
According to a fourth aspect, there is provided an apparatus for generating a challenge sample, comprising:
the acquisition unit is configured to acquire original sample characteristics of a target risk sample and a risk detection model;
a reference direction determination unit configured to determine a predetermined number T of reference directions in a predetermined feature space;
the feature transformation unit is configured to execute a plurality of rounds of feature transformation according to the original sample features in sequence and different reference directions in the T reference directions by a preset first step length until a first transformation feature is obtained, and the risk detection model is configured to enable a predicted value of the first transformation feature to fall into a predetermined interval corresponding to a non-risk sample;
a fallback operation unit configured to perform, for the first transformation characteristic, several fallback operations with successively increasing fallback amplitudes along a reverse direction of a last characteristic transformation direction in the several feature transformations until a second transformation characteristic is obtained, where a predicted value of the risk detection model for the second transformation characteristic reaches a boundary of the predetermined interval; the backoff amplitude is smaller than the first step size;
and the sample forming unit is configured to form a countermeasure sample corresponding to the target risk sample according to the second transformation characteristic.
According to a fifth aspect, there is provided an apparatus for optimizing a risk detection model, comprising:
the original sample acquisition unit is configured to acquire a sample set formed by original samples, wherein the sample set comprises a normal sample and a risk sample;
a countermeasure sample acquisition unit configured to acquire a countermeasure sample generated for the risk sample by the apparatus according to the fourth aspect;
an updating unit configured to update the risk detection model using each original sample and each confrontation sample in the set of samples.
According to a sixth aspect, there is provided an apparatus for generating a challenge sample, comprising:
an obtaining unit configured to obtain original sample characteristics and a classification model of a target sample, the target sample corresponding to a first class;
a reference direction determination unit configured to determine a predetermined number T of reference directions in a predetermined feature space;
the feature transformation unit is configured to execute a plurality of rounds of feature transformation according to the original sample features in sequence and different reference directions in the T reference directions by preset first step lengths until first transformation features are obtained, and the classification model is used for enabling the predicted values of the first transformation features to fall into a predetermined interval corresponding to samples of a non-first category;
a back-off operation unit configured to perform, for the first transformation characteristic, several back-off operations with successively increasing back-off amplitudes in a reverse direction of a last characteristic transformation direction in the several times of characteristic transformations until a second transformation characteristic is obtained, where the classification model reaches a boundary of the predetermined interval for a predicted value of the second transformation characteristic; the backspacing amplitude is smaller than the first step size;
and the sample forming unit is configured to form a confrontation sample corresponding to the target sample according to the second transformation characteristic.
According to a seventh aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of any of the first to third aspects.
According to an eighth aspect, there is provided a computing device comprising a memory and a processor, wherein the memory has stored therein executable code, and the processor, when executing the executable code, implements the method of any of the first to third aspects.
According to the method and the device provided by the embodiment of the specification, in a countermeasure scene of risk detection, a method for simulating an attacker to generate a countermeasure sample aiming at a black sample is provided, the method adopts an iterative approximation mode, a decision boundary of a risk detection model is approximated through multi-directional search and feature transformation of a feature space and rollback operation, and the countermeasure sample conforming to an attack target is generated rapidly.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 illustrates a schematic diagram of the attack and defense between risk detection and attack transformation, according to one embodiment;
FIG. 2 illustrates a flow diagram of a method of generating a challenge sample according to one embodiment;
FIG. 3 illustrates a flow diagram of a feature transformation process according to one embodiment;
FIG. 4 illustrates a flow diagram of a rollback operation process, according to one embodiment;
FIG. 5 shows a schematic diagram of feature transformation and rollback operations;
FIG. 6 illustrates a flow diagram of a method of optimizing a risk detection model, according to one embodiment;
FIG. 7 illustrates a method of generating a challenge sample in one embodiment;
FIG. 8 shows a schematic diagram of a generating device according to one embodiment;
FIG. 9 shows a schematic diagram of an optimization apparatus according to one embodiment;
FIG. 10 shows a generating device schematic according to an embodiment.
Detailed Description
The scheme provided by the specification is described below with reference to the accompanying drawings.
As mentioned above, in security and risk control scenarios, in order to identify high-risk business objects, some risk detection models have been trained through machine learning for detecting risk objects, so as to intercept or further perform security processing.
Although various existing risk detection models achieve good effects in various aspects such as comprehensiveness of feature processing, prediction accuracy and the like, the inventor realizes that a risk detection scene is actually a game scene of attack and defense confrontation: on one hand, model algorithms attempt to perform comprehensive analysis on business objects to identify risk objects, and on the other hand, attempt to bypass the analysis algorithms of the models or attack the models to attempt to break through the identification of the models through the league of risk object profit-making. Therefore, the inventor proposes to optimize the risk detection process from the perspective of attack and defense to improve the robustness of the risk detection model.
From the viewpoint of attack and defense, if the attacker of the risk detection model is to be defended, the possible attack mode of the attacker needs to be researched first. Therefore, a countermeasure sample can be generated by simulating the attack mode of an attacker, and the countermeasure sample is used for attacking the risk detection model so that the risk detection model is identified by mistake. Based on the countermeasure sample, the potential model attack can be better resisted, and the attack and defense safety is improved.
FIG. 1 illustrates a schematic diagram of the attack and defense between risk detection and attack transformation, according to one embodiment. As shown in fig. 1, the risk detection model is trained based on a training sample set. For better training, in general, the training sample set includes some normal samples, or white samples, and some risk samples, or black samples, so as to learn from different angles of positive and negative samples. The sample may correspond to an object to be detected, such as an account number, a transaction, a user action, text, and the like. Based on such a sample set, a preliminarily trained risk detection model can be obtained.
In a countermeasure scene of risk detection, an attacker is assumed to adopt a generation algorithm to generate some disturbance transformations aiming at a sample to obtain a corresponding countermeasure sample, so that a risk detection model cannot identify the true category of the sample. If such countermeasures are available, a new sample set can be formed based on the original sample set and the countermeasures, and the risk detection model is updated again based on the sample set, so that the detection performance can be optimized, and the counter-measures can better resist the attack of an attacker.
Therefore, in order to optimize the performance of the risk detection model, it is necessary to simulate the manner in which an attacker generates countermeasure samples to obtain corresponding countermeasure samples. According to one embodiment, it is assumed that the attack target of the attacker is to make the risk detection model not detect it as a black sample, i.e., to misidentify it as a white sample, after applying the perturbation transformation to the black sample to obtain a corresponding countersample. The goal may be expressed as minimizing the following objective function:
wherein f is θ Representing an algorithm process of a risk detection model, and theta represents a model parameter in the risk detection model; rho s A generating algorithm for generating a perturbation transformation for the input sample features, the parameters of the generating algorithm being denoted as s; l is training risk detection model f θ The loss function used.
With equation (1), the attacker intends to make, for the input black sample x i By generating an algorithm ρ s After the disturbance transformation is generated, a confrontation sample x can be obtained i +ρ s (x i ) The risk detection model aims at the prediction result of the confrontation sample and is closer to the label value 1-y of the white sample i 。
However, the numerical solution of the above equation (1) is very difficult, and in the case of black box attack, the loss function L used for training the risk detection model cannot be obtained. Thus, it is difficult to obtain the above generation algorithm ρ by solving s And further, it is difficult to obtain a corresponding countermeasure sample.
In view of the above, the inventor provides an iterative approximation method, which approximates the decision boundary of the risk detection model by multi-directional search and feature transformation of the feature space and rollback operation, and quickly generates a countermeasure sample that meets the attack target. Specific implementations of the above concepts are described below.
FIG. 2 illustrates a flow diagram of a method of generating a challenge sample according to one embodiment. It is to be appreciated that the method can be performed by any computing, processing capable apparatus, device, platform, cluster of devices. As shown in fig. 2, the method includes the following steps.
First, in step 21, the original sample characteristics of the target risk sample and the risk detection model are obtained.
The target risk sample may be selected from a sample set, in which any ith sample is marked as (x) i ,y i ) Wherein x is i Sample characteristics, y, representing the i-th sample i A corresponding label value is represented to show whether the sample is a risk sample. In general, a normal sample and a risk sample are represented by two different tag values, i.e., a first tag value and a second tag value, respectively. Typically, in the scenario of identifying risk samples, a white sample is generally denoted by 0 and a black sample by 1.
In this embodiment, the risk sample obtained from the sample set is called a target risk sample, and the original sample feature thereof is represented as x 0 。
In different embodiments, the sample may be various business objects to be detected, such as an account number, a transaction, a text, a user operation, and the like.
In one specific example, the sample is an account number. Accordingly, the risk sample may be a spam account, a naval account, a stolen account, and so on. For account samples, sample characteristics may include, for example, the length of time the account was registered, registration information, the frequency of use over the last period of time, the frequency of comments made, and so forth.
In another example, the sample is a transaction. Accordingly, the risk sample may be a high risk transaction such as fraud, cash-out, etc. For a transaction sample, sample characteristics may include, for example, transaction amount, transaction time, payment channel, transaction party attribute information, and the like.
In yet another example, the sample is text. Accordingly, the risk sample may be spam/text messages, advertising mail/text messages, illegal content text, and the like. For text samples, the sample characteristics mainly include characters in the text, text release time, source, and the like.
In other examples, the sample may also be other business objects. And not described in detail herein.
In addition, the risk detection model obtained in this step is the model for confrontation. Similar to equation (1), the model can be expressed as f θ 。
For the above target risk sample, in step 22, a predetermined number T of reference directions are determined as subsequent feature transformation directions in a predetermined feature space.
In one embodiment, the feature space is a space corresponding to the original feature dimension of the sample. For example, assuming that the original sample features are N-dimensional, the feature space is an N-dimensional space.
For some traffic samples, the original feature dimension of the sample may be high, corresponding to a high-dimensional feature space. For example, an image sample, a text sample, tends to have high original feature dimensions, which may be thousands of dimensions. In another embodiment, the original high-dimensional (M-dimensional) feature space may be subjected to dimension reduction mapping according to a predetermined mapping relationship, and the obtained low-dimensional (N-dimensional) feature space may be used as the predetermined feature space. The mapping relationship of the dimension reduction mapping can be determined by various existing methods, such as a Principal Component Analysis (PCA) method, and the like.
Based on the feature space, T reference directions can be determined therefrom as subsequent feature transformation directions. In one embodiment, T mutually orthogonal directions are found in the feature space as the T reference directions. Practice has proved that the original samples can be transformed into the confrontation samples more quickly by respectively performing feature transformation in different directions orthogonal to each other.
Based on the above-mentioned T reference directions, a feature transformation process of step 23 is performed in which the original sample feature x is subjected to 0 In turn, areAnd according to different reference directions in the T reference directions, executing a plurality of rounds of feature transformation with a preset first step length until a first transformation feature is obtained, wherein the first transformation feature is a sample feature which enables a predicted value of the risk detection model to fall into a preset interval corresponding to a non-risk sample.
For example, in a typical sample labeling case, the label value of a white sample (normal sample) is often set to 0, and the label value of a black sample (risk sample) is often set to 1. Accordingly, a risk detection model f is used θ When predicting the input sample x, if the predicted value f is θ (x) If greater than 0.5, the input sample x is identified as a black sample and if f is predicted θ (x) Less than 0.5, the input sample is identified as a white sample. In such a case, the predetermined interval corresponding to the non-risk sample is the interval [0,0.5 ] with the predicted value less than 0.5. However, other ways of labeling are not excluded. For example, when the risk sample label value is set to 0, the predetermined section corresponding to the non-risk sample is (0.5, 1).
Regardless of how the label value is set, once the predicted value of the risk detection model for the transformation feature falls into a predetermined interval corresponding to the non-risk sample, it means that the non-risk sample is identified as the non-risk sample; for the target risk sample, it means that the transformation thereof results in the false identification of the risk detection model, and the attack succeeds.
Accordingly, in step 23, feature transformation is iteratively performed in different reference directions by a first step length until the obtained transformation feature falls into a predetermined interval corresponding to the non-risk sample, at which point the attack is successful, and the transformation feature at this point is recorded as the first transformation feature.
However, the first step size is not set too small due to search efficiency considerations. The first transform feature at this time, although it may play a role in successful attack, may not be the preferred countersample because of the large transform amplitude due to the long step size. Therefore, a fallback operation is then performed on the basis of the first transformation characteristic in order to find a countermeasure sample where the transformation amplitude is smaller, while still being able to attack the risk detection model.
Specifically, in step 24, for the aforementioned first transformation characteristic, in the reverse direction of the last transformation direction of the several transformations of the characteristic, several fallback operations are performed with successively increasing back-off magnitudes until a second transformation characteristic is obtained, where the second transformation characteristic is a transformation characteristic that: risk detection model f θ The predicted value of the non-risk sample reaches the boundary of a preset interval corresponding to the non-risk sample; if the rollback is continued, the device will leave the preset interval. It is understood that the back-off amplitude of each of the aforementioned back-off operations is smaller than the first step size.
In this way, the backoff is gradually attempted by sequentially increasing the backoff width in the reverse direction of the last feature transformation until the predicted value reaches the boundary of the predetermined interval. At this point, the decision boundary of the risk detection model may be considered to be approached. The transformation characteristic at this time is recorded as a second transformation characteristic.
Then, in step 25, a countermeasure sample corresponding to the target risk sample may be formed based on the second transformation characteristics.
Specifically, in the case that the feature space corresponding to the transformation operation is the original feature space of the sample, the second transformation feature may be directly used as the feature of the countermeasure sample to obtain the countermeasure sample. When the feature space is a low-dimensional feature space obtained by performing dimensionality reduction mapping on an original high-dimensional feature space, in the step, according to inverse mapping of the mapping relation, a second transformation feature in the low-dimensional feature space is mapped back to the high-dimensional feature space to serve as a sample feature of a countermeasure sample, so that the countermeasure sample is obtained.
Thus, according to the method of fig. 2, search and feature transformation are performed in a feature space along multiple directions, and a small step of backspacing is performed after a predicted value enters an interval of a non-risk sample, so that a decision boundary of a risk detection model is approached, and a countermeasure sample is rapidly generated through a search approximation mode.
Taking the label value of the risk sample as 1 and the predicted value interval corresponding to the non-risk sample as [0,0.5 ]), the feature transformation process of step 23 and the rollback process of step 24 will be specifically described below.
Fig. 3 shows a flow chart of a feature transformation process, i.e. the sub-steps of the aforementioned step 23, according to one embodiment.
As shown in FIG. 3, in step 231, the original sample characteristics x of the target risk sample are analyzed 0 As current feature x, i.e. current feature x = x 0 . Then, a feature transformation iteration process is entered.
For the T-th iteration, in step 232, the T-th reference direction is determined among the T reference directions and is recorded as h t 。
Then, in step 233, the current feature x is processed in the t-th reference direction h with the first step size t And carrying out feature transformation to obtain intermediate features.
It is understood that, in the case where the t-th iteration is the first iteration, t =1, and the current feature at this time is the original sample feature x set in step 231 0 . If the t-th iteration is not the first iteration, the current feature at the moment is the feature obtained in the previous iteration.
In a specific example, for feature transformation, forward and backward directions are respectively used as selected directions, and along the selected direction of the tth reference direction, a perturbation transformation with a large step length of a first step length λ is superimposed on the current feature x to obtain an intermediate feature.
Denote the selection of the forward and reverse directions by sgn ∈ [ -1, +1], then according to a specific example, the intermediate feature x' can be expressed as:
x′＝x+sgn*λ*h t (2)
in one example, the first step λ takes 1, then the intermediate feature x' can be expressed as:
x′＝x+sgn*h t (3)
according to one embodiment, a transformation boundary e is set for the feature transformation. In order to prevent the amplitude of the feature transformation from exceeding the transformation boundary after multiple iterations, the feature obtained by adding the perturbation transformation to the current feature x is used as a temporary feature. If the temporal feature is relative to the original sample feature x 0 Exceeds the predetermined transformation boundary e, the temporary transformation is performed by presetting the projection functionThe features are projected within the predetermined transformation boundary as intermediate features. According to this embodiment, the intermediate feature x' may be expressed as:
wherein, S represents a transformation space with the transformation boundary being ∈, all transformations δ in the space S need to be satisfied, and the infinite norm is not greater than the transformation boundary, that is: s = { δ: | delta | non-woven phosphor ∞ ≤∈}；
Is a projection function for mapping the transformation to the transformation space S if the transformation is too large beyond a predetermined transformation space S.
After the intermediate transformation x' is obtained in various ways, the risk detection model f is determined in step 234 θ First predicted value f for intermediate feature θ (x') is compared to the second predicted value f for the current feature x θ (x) Whether further away from the risk label value. In other words, judgment, the predicted value f of the intermediate feature θ (x') is closer to the predetermined interval for which the non-risk sample corresponds.
When the label value of the risk sample is 1 and the prediction value interval corresponding to the non-risk sample is [0,0.5 ], step 234 determines that the first prediction value f is θ (x') is less than the second predicted value f θ (x)。
If the judgment result is negative, the current reference direction h is indicated t The transformation performed above is invalid and proceeds directly to the next iteration, returning to step 232, and trying other reference directions.
If the judgment result is yes, the current reference direction h is indicated t The above transformation is valid and the predicted value of the current feature is closer to the interval of non-risk samples, so in step 235, the current feature is updated with an intermediate feature x ', i.e. the current feature x is updated to x = x ', where x ' can be the result of any one of equations (2) - (4).
Next, at step 236,and also judges the risk detection model f θ And judging whether the updated predicted value of the current feature falls into a predetermined interval corresponding to the non-risk sample. In the case where the predetermined interval is [0, 0.5), the step judges f θ (x) Whether less than 0.5.
If the result of the determination in step 236 is negative, that is, the current feature does not fall into the predetermined interval yet, the process returns to step 232 to enter the next iteration.
If the determination in step 236 is yes, then in step 237, the current feature is recorded as the first transformed feature x 1 。
By the process of iteratively performing the feature transformation as shown in fig. 3, the first transformation feature x may be obtained 1 . In addition to recording the first transformation feature, the reference direction h upon which the last feature transformation was based is also recorded t For rollback operations.
Fig. 4 shows a flow chart of a fallback operation procedure, i.e. the sub-steps of the aforementioned step 24, according to one embodiment.
As shown in FIG. 4, in step 241, the first transformation feature x is transformed 1 As current feature x, i.e. current feature x = x 1 . Then, a rollback operation procedure is entered.
For the ith backoff operation, in step 242, the current backoff amplitude K is determined.
In one example, a predetermined second step size α may be superimposed on the backoff width K of the previous backoff operation to obtain the current backoff width, i.e., K = K + α. At this time, the backoff width is increased in uniform steps.
In another example, the backoff coefficient back of the last backoff operation is multiplied by a preset attenuation coefficient escape, as the current backoff coefficient, i.e., back = back × escape. The attenuation coefficient is less than 1, and may be, for example, 0.95,0.9, etc. Then, determining a difference value obtained by subtracting the back coefficient back of the current time from 1, and taking the product of the difference value and the first step length as the back amplitude K of the current time, namely:
K＝(1-back)*λ (5)
since the attenuation coefficient is less than 1, the back coefficient back value is smaller and the back amplitude is larger compared with the previous back operation. In this embodiment, the back-off step size is smaller and smaller, rather than proceeding uniformly.
Then, at step 243, sgn h is followed in the last feature transformation direction t Based on the first transformation characteristic x 1 And returning the returning amplitude K to obtain an intermediate characteristic x'.
In one example, when the backoff amplitude is calculated according to equation (5), and the first step length λ is 1, and the selected direction sgn of the last feature transformation is represented as β, the obtained intermediate feature x ″ can be represented as:
x″＝x 1 -(β-back)*h t (6)
in one embodiment, it may also be ensured that the resulting features do not exceed the transform boundaries based on the projection function. Specifically, the current backspacing amplitude may be cancelled in the first transformation feature, and the obtained feature may be used as a temporary feature; if the temporal feature is relative to the original sample feature x 0 If the transformation size of the temporary feature does not exceed a preset transformation boundary epsilon, taking the temporary feature as an intermediate feature; if the temporary feature exceeds the predetermined transformation boundary, it is projected within the predetermined transformation boundary as an intermediate feature x' by a preset projection function.
Specifically, in one example, the intermediate feature x "may be represented as:
wherein S represents the transformation space with the transformation boundary being epsilon;
is a projection function.
After the intermediate features x' are obtained by various fallback, at step 244, the risk detection model f is judged θ First predicted value f for the intermediate feature θ (x ") whether it belongs to a predetermined interval corresponding to a non-risk sample, e.g., [0,0.5 ].
It is to be understood that the predicted value of the first transform characteristic before the fallback belongs to the predetermined interval. Thus, if the determination of step 244 is negative, indicating that the rollback operation will cause the sample to leave the predetermined interval corresponding to the non-risk sample that will not be a challenge sample, then such rollback operation is unacceptable. At this time, the process goes to step 247, and the previously recorded current feature x is used as the second transformation feature. If the i-th rollback operation is the first rollback operation, then the current feature at that time is the first transformation feature. If the ith rollback operation is not the first rollback operation, then the current signature at that time is the signature obtained from the previous rollback operation.
If the determination at step 244 is yes, the process proceeds to step 245. In step 245, the first predicted value f of the risk detection model for the intermediate feature is determined θ (x') compared to a second predicted value f for the current feature of the previous recording θ (x) Whether closer to the risk label value. If the risk label value is 1, the step 244 determines that the first predicted value f is θ (x') is greater than the second predicted value f θ (x)。
If the result of the determination is yes, it indicates that the rollback operation is performed so that the predicted value of the current feature is closer to the interval boundary of the non-risk sample, i.e., the decision boundary of the risk detection model, and therefore, in step 246, the current feature is updated by using the intermediate feature x ″, i.e., the current feature x is updated to x = x ″, where x ″ may be the result of any one of the formulas (6) or (7). Then, the process returns to step 242 to perform the next rollback operation.
If not, the rollback operation is indicated, so that the predicted value of the current feature starts to be far away from the interval boundary, and the process also jumps to step 247.
Through the process, the decision boundary of the risk detection model is approached by the sequentially increased backspacing amplitude, and the finally obtained second transformation characteristic is used as the corresponding characteristic of the countermeasure sample.
Fig. 5 shows a schematic diagram of feature transformation and rollback operations. Last round of feature transformation assuming first transformation featuresIs to characterize the sample from x t-1 Transformation to x t With a direction of transformation h t . Thus, it is possible to use the first transformation feature x as a basis t And performing rollback operation. During the rollback operation, along h t And (4) continuously increasing the backspacing amplitude until the boundary for distinguishing the risk sample from the non-risk sample is approached. Based on the transformation characteristics at this time, a countermeasure sample with a small transformation amplitude and an offensive power can be generated.
Challenge samples generated according to the method of fig. 2 may be used to optimize a risk detection model. FIG. 6 illustrates a flow diagram of a method of optimizing a risk detection model, according to one embodiment. As shown in fig. 6, the optimization for the risk detection model may include the following steps.
In step 61, a sample set of raw samples is obtained, including normal samples and risk samples.
At step 62, several challenge samples generated for the risk samples according to the method of fig. 2 are obtained.
At step 63, the risk detection model is updated with each original sample and each challenge sample in the set of samples.
It is to be understood that the goal of the risk detection model update is to correctly identify its original signature even for the challenge sample. Therefore, the total prediction loss of the risk detection model for the total set of samples including the confrontation samples can be determined by using the loss function L for the original labels of the samples, and the model parameters of the risk detection model can be further adjusted to realize model updating with the aim of minimizing the total prediction loss.
Specifically, in one embodiment, the total prediction Loss may be expressed as:
the first term in equation (8) is calculated for all the original samples in the original sample set, resulting in their predicted loss relative to the original label. Second, for the risk sample, corresponding countermeasures are calculatedSample A s (x i ) Predicted loss relative to the original tag (risk tag).
By minimizing the total prediction loss, the updated risk prediction model still can correctly identify the risk category of the confrontation sample subjected to disturbance transformation, so that the effect of improving robustness and safety is achieved.
Although described above in connection with two-class models of risk samples and non-risk samples, it is to be understood that the concepts of FIG. 2 described above can be applied to generate countermeasure samples for two-class models in other scenarios, as well as for multiple-class models.
FIG. 7 illustrates a method of generating a challenge sample in one embodiment. As shown in fig. 7, at step 71, raw sample features and a classification model of a target sample are obtained, the target sample corresponding to a first class. And the classification model may be a multi-classification model for identifying a plurality of classes including the first class.
At step 72, a predetermined number T of reference directions are determined in a predetermined feature space for the target sample.
In step 73, for original sample features, sequentially according to different reference directions in the T reference directions, with a preset first step length, performing a plurality of rounds of feature transformations until a first transformation feature is obtained, where the classification model is directed to a predetermined interval where a prediction value of the first transformation feature falls into a non-first class of samples. Here, the predetermined section is a predicted value section corresponding to a category other than the first category, and the category other than the first category may be a certain category or a plurality of categories other than the first category.
Then, in step 74, for the first transformation characteristic, performing several fallback operations with successively increasing fallback amplitudes in a reverse direction of a last characteristic transformation direction in the several characteristic transformations until a second transformation characteristic is obtained, where the classification model reaches a boundary of the predetermined interval for a predicted value of the second transformation characteristic; the backoff amplitude is smaller than the first step size.
Thus, in step 75, a confrontation sample corresponding to the target sample is formed according to the second transformation characteristic.
The specific implementation of steps 73 and 74 above may be similar to that described above in connection with fig. 3 and 4, with only a different setting for the predetermined interval. The person skilled in the art can set the above different intervals and extend the setting to the case of a multi-classification model when reading the present specification.
According to an embodiment of another aspect, there is also provided an apparatus for generating a challenge sample, which may be deployed on any device or platform having computing and processing capabilities. FIG. 8 shows a schematic diagram of a generating device according to one embodiment. As shown in fig. 8, the generating apparatus 800 includes:
an obtaining unit 81 configured to obtain an original sample characteristic of a target risk sample and a risk detection model;
a reference direction determining unit 82 configured to determine a predetermined number T of reference directions in a predetermined feature space;
a feature transformation unit 83, configured to perform, for the original sample feature, several rounds of feature transformation sequentially according to different reference directions in the T reference directions by a preset first step length until a first transformation feature is obtained, where the risk detection model falls into a predetermined interval corresponding to a non-risk sample for a predicted value of the first transformation feature;
a back-off operation unit 84 configured to perform, for the first transformed feature, several back-off operations with successively increasing back-off amplitudes in a reverse direction of a last feature transformation direction in the several feature transformations until a second transformed feature is obtained, where a predicted value of the risk detection model for the second transformed feature reaches a boundary of the predetermined interval; the backspacing amplitude is smaller than the first step size;
and the sample forming unit 85 is configured to form a countermeasure sample corresponding to the target risk sample according to the second transformation characteristic.
The various elements of the apparatus 800 described above are specifically configured to perform the method steps described in conjunction with fig. 2, 3, and 4.
According to an embodiment of another aspect, an apparatus for optimizing a risk detection model is also provided, and the apparatus may be deployed on any device or platform with computing and processing capabilities. FIG. 9 shows a schematic diagram of an optimization apparatus according to one embodiment. As shown in fig. 9, the generating apparatus 900 includes:
an original sample acquiring unit 91 configured to acquire a sample set composed of original samples, including normal samples and risk samples;
a confrontation sample acquisition unit 92 configured to acquire a confrontation sample generated by the apparatus 800 for the risk sample;
an updating unit 93 configured to update the risk detection model with each original sample and each confrontation sample in the set of samples.
According to an embodiment of a further aspect, there is also provided an apparatus for generating a challenge sample, which may be deployed on any device or platform having computing and processing capabilities. FIG. 10 shows a generating device schematic according to one embodiment. As shown in fig. 10, the generating apparatus 1000 includes:
an obtaining unit 101 configured to obtain an original sample feature and a classification model of a target sample, the target sample corresponding to a first class;
a reference direction determination unit 102 configured to determine a predetermined number T of reference directions in a predetermined feature space;
a feature transformation unit 103, configured to, for the original sample features, sequentially perform a plurality of rounds of feature transformations according to different reference directions in the T reference directions with a preset first step length until a first transformation feature is obtained, where the classification model is configured to determine that a predicted value of the first transformation feature falls into a predetermined interval corresponding to a non-first class of sample;
a back-off operation unit 104 configured to perform, for the first transformation characteristic, several back-off operations with successively increasing back-off amplitudes in a reverse direction of a last characteristic transformation direction in the several times of characteristic transformations until a second transformation characteristic is obtained, where the classification model reaches a boundary of the predetermined interval for a predicted value of the second transformation characteristic; the backspacing amplitude is smaller than the first step size;
a sample forming unit 105 configured to form a confrontation sample corresponding to the target sample according to the second transformation characteristic.
Through the device, a more effective countermeasure sample can be generated from the perspective of attack and defense countermeasures, and then the countermeasure sample is utilized to optimize the risk detection model, so that the robustness and the safety of the risk detection model are enhanced.
According to an embodiment of another aspect, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method described in connection with fig. 2.
According to an embodiment of another aspect, there is also provided a computing device, including a memory and a processor, where the memory stores executable code, and the processor executes the executable code to implement the method described in conjunction with fig. 2.
Those skilled in the art will recognize that, in one or more of the examples described above, the functions described in this invention may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium.
The above-mentioned embodiments, objects, technical solutions and advantages of the present invention are further described in detail, it should be understood that the above-mentioned embodiments are only examples of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made on the basis of the technical solutions of the present invention should be included in the scope of the present invention.
Claims (17)
1. A method of generating a challenge sample, comprising:
acquiring original sample characteristics of a target risk sample consisting of risk business objects and a risk detection model, wherein the risk detection model is used for identifying the business objects with risks or potential safety hazards, and the business objects comprise account numbers, transactions or texts; when the business object is an account, the characteristics of the business object comprise at least one of the following: the registration duration, registration information, the use frequency of a recent period of time and the frequency of making comments of the account; when the business object is a transaction, the characteristics of the business object comprise at least one of the following items: transaction amount, transaction time, payment channel and attribute information of both transaction parties; when the business object is a text, the characteristics of the business object comprise at least one of the following: characters in the text, text publishing time and source;
determining a predetermined number T of reference directions in a predetermined feature space;
aiming at the original sample characteristics, sequentially executing a plurality of rounds of characteristic transformation according to different reference directions in the T reference directions by using a preset first step length until first transformation characteristics are obtained, wherein the risk detection model aims at that the predicted value of the first transformation characteristics falls into a preset interval corresponding to a non-risk business object;
aiming at the first transformation characteristic, executing a plurality of times of fallback operation with successively increased fallback amplitudes along the reverse direction of the last characteristic transformation direction in the plurality of rounds of characteristic transformations until a second transformation characteristic is obtained, wherein the risk detection model aims at that the predicted value of the second transformation characteristic reaches the boundary of the preset interval; the backspacing amplitude is smaller than the first step size;
and forming a countermeasure sample corresponding to the target risk sample according to the second transformation characteristic, wherein the countermeasure sample is used for updating the risk detection model, so that the updated risk detection model still identifies the countermeasure sample as a risk business object.
2. The method of claim 1, wherein the T reference directions are orthogonal to each other in the feature space.
3. The method of claim 1, wherein a tth round of feature transforms of the number of rounds of feature transforms comprises:
acquiring a t-th reference direction;
taking the features obtained in the previous round as current features, and performing feature transformation according to the tth reference direction by the first step length on the basis of the current features to obtain intermediate features;
updating a current feature with the intermediate feature if the first predicted value of the risk detection model for the intermediate feature is further away from a risk label value than the second predicted value for the current feature.
4. The method of claim 3, wherein performing feature transformation according to the tth reference direction to obtain an intermediate feature comprises:
respectively taking the forward direction and the reverse direction as selected directions;
and superposing disturbance transformation with the first step length on the current feature along the selected direction of the tth reference direction, and taking the obtained feature as the intermediate feature.
5. The method of claim 3, wherein performing feature transformation according to the tth reference direction to obtain an intermediate feature comprises:
respectively taking the forward direction and the reverse direction as selected directions;
superimposing disturbance transformation with a first step length on the current feature along the selected direction of the t-th reference direction to obtain a temporary feature;
if the transformation size of the temporary feature relative to the original sample feature exceeds a preset transformation boundary, projecting the temporary feature into the preset transformation boundary through a preset projection function to serve as the intermediate feature.
6. The method of claim 3, wherein the t-th round feature transformation further comprises:
judging whether the predicted value of the risk detection model for the updated current feature falls into the preset interval or not;
if the current time does not fall into the preset interval, entering the next round of feature transformation;
and if the current feature falls into the preset interval, taking the current feature as the first transformation feature.
7. The method of claim 1, wherein an ith fallback operation of the number of fallback operations comprises:
determining the backspacing amplitude of the time;
backspacing the backspacing amplitude based on the first transformation characteristic along the reverse direction of the last characteristic transformation direction to obtain an intermediate characteristic;
updating the current feature with the intermediate feature if the first predicted value for the intermediate feature by the risk detection model is closer to a risk label value than the second predicted value for the current feature of the previous record.
8. The method of claim 7, wherein determining the current backoff amplitude comprises:
and superposing a preset second step length on the backspacing amplitude of the last backspacing operation to obtain the current backspacing amplitude.
9. The method of claim 7, wherein determining the current backoff amplitude comprises:
multiplying the back-off coefficient of the last back-off operation by a preset attenuation coefficient to obtain the back-off coefficient of the current time; the attenuation coefficient is less than 1;
and determining a difference value obtained by subtracting the current backspacing coefficient from 1, and taking the product of the difference value and the first step length as the current backspacing amplitude.
10. The method according to claim 7, wherein backspacing the current backspacing amplitude based on the first transformed feature to obtain an intermediate feature comprises:
reducing the backspacing amplitude of the current time on the first transformation characteristic to obtain a temporary characteristic;
if the transformation size of the temporary feature relative to the original sample feature does not exceed a preset transformation boundary, taking the temporary feature as an intermediate feature;
and if the temporary feature exceeds the preset transformation boundary, projecting the temporary feature into the preset transformation boundary as the intermediate feature through a preset projection function.
11. The method of claim 7, wherein an ith fallback operation of the number of fallback operations further comprises:
judging whether a first predicted value of the risk detection model for the intermediate feature belongs to the preset interval or not;
if the current time interval belongs to the preset interval, entering the next rollback operation;
and if the current feature does not belong to the preset interval, taking the current feature recorded last time as the second transformation feature.
12. The method according to claim 1, wherein the predetermined feature space is a low-dimensional feature space obtained by performing dimension reduction mapping on a high-dimensional feature space corresponding to the original sample feature according to a predetermined mapping relationship;
forming a countermeasure sample corresponding to the target risk sample according to the second transformation characteristic, including: and according to the inverse mapping of the preset mapping relation, mapping the second transformation characteristic in the preset characteristic space back to the high-dimensional characteristic space to serve as the sample characteristic of the confrontation sample.
13. A method of optimizing a risk detection model, comprising:
obtaining a sample set formed by original samples, wherein the sample set comprises normal samples and risk samples, the original samples comprise business objects, and the business objects comprise account numbers, transactions or texts; when the business object is an account, the characteristics of the business object comprise at least one of the following: the registration duration, registration information, the use frequency of a recent period of time and the frequency of making comments of the account; when the business object is a transaction, the characteristics of the business object comprise at least one of the following: transaction amount, transaction time, payment channel and attribute information of both transaction parties; when the business object is a text, the characteristics of the business object comprise at least one of the following: characters in the text, text publishing time and source;
obtaining a challenge sample generated for the risk sample according to the method of claim 1;
updating the risk detection model by utilizing each original sample and each confrontation sample in the sample set, so that the updated risk detection model still identifies the confrontation sample corresponding to each risk sample as a risk business object; and carrying out risk detection on the business object to be detected by using the updated risk detection model.
14. An apparatus for generating a challenge sample, comprising:
the system comprises an acquisition unit, a risk detection unit and a risk analysis unit, wherein the acquisition unit is configured to acquire original sample characteristics of a target risk sample consisting of risk business objects and a risk detection model, the risk detection model is used for identifying the business objects with risks or potential safety hazards, and the business objects comprise account numbers, transactions or texts; when the business object is an account, the characteristics of the business object comprise at least one of the following: the registration duration, the registration information, the use frequency of the account in the last period of time and the comment issuing frequency of the account; when the business object is a transaction, the characteristics of the business object comprise at least one of the following items: transaction amount, transaction time, payment channel and attribute information of both transaction parties; when the business object is a text, the characteristics of the business object comprise at least one of the following: characters in the text, text publishing time and source;
a reference direction determination unit configured to determine a predetermined number T of reference directions in a predetermined feature space;
the characteristic transformation unit is configured to execute a plurality of rounds of characteristic transformation according to different reference directions in the T reference directions in sequence and preset first step length aiming at the original sample characteristics until first transformation characteristics are obtained, and the risk detection model aims at that the predicted value of the first transformation characteristics falls into a preset interval corresponding to a non-risk business object;
a fallback operation unit configured to perform, for the first transformation characteristic, a number of fallback operations with successively increasing fallback amplitudes in a reverse direction of a last characteristic transformation direction in the number of rounds of characteristic transformations until a second transformation characteristic is obtained, where a predicted value of the risk detection model for the second transformation characteristic reaches a boundary of the predetermined interval; the backoff amplitude is smaller than the first step size;
and the sample forming unit is configured to form a countermeasure sample corresponding to the target risk sample according to the second transformation characteristic, wherein the countermeasure sample is used for updating the risk detection model, so that the updated risk detection model still identifies the countermeasure sample as a risk business object.
15. An apparatus to optimize a risk detection model, comprising:
the system comprises an original sample acquisition unit, a risk sample acquisition unit and a risk sample acquisition unit, wherein the original sample acquisition unit is configured to acquire a sample set formed by original samples, the original samples comprise business objects, and the business objects comprise accounts, transactions or texts; when the business object is an account, the characteristics of the business object comprise at least one of the following: the registration duration, the registration information, the use frequency of the account in the last period of time and the comment issuing frequency of the account; when the business object is a transaction, the characteristics of the business object comprise at least one of the following items: transaction amount, transaction time, payment channel and attribute information of both transaction parties; when the business object is a text, the characteristics of the business object comprise at least one of the following: characters in the text, text publishing time and source;
a confrontation sample acquisition unit configured to acquire a confrontation sample generated by the apparatus according to claim 14 for the risk sample;
the updating unit is configured to update the risk detection model by using each original sample and each confrontation sample in the sample set, so that the updated risk detection model still identifies the confrontation sample corresponding to each risk sample as a risk business object; and carrying out risk detection on the business object to be detected by using the updated risk detection model.
16. A computer-readable storage medium, on which a computer program is stored which, when executed in a computer, causes the computer to carry out the method of any one of claims 1-13.
17. A computing device comprising a memory and a processor, wherein the memory has stored therein executable code that, when executed by the processor, performs the method of any of claims 1-13.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110662087.9A CN113313404B (en) | 2021-06-15 | 2021-06-15 | Method and device for generating countermeasure sample |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110662087.9A CN113313404B (en) | 2021-06-15 | 2021-06-15 | Method and device for generating countermeasure sample |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113313404A CN113313404A (en) | 2021-08-27 |
| CN113313404B true CN113313404B (en) | 2022-12-06 |
Family
ID=77378858
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110662087.9A Active CN113313404B (en) | 2021-06-15 | 2021-06-15 | Method and device for generating countermeasure sample |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113313404B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109934253A (en) * | 2019-01-08 | 2019-06-25 | 阿里巴巴集团控股有限公司 | A kind of confrontation sample generating method and device |
| CN110764958A (en) * | 2019-09-24 | 2020-02-07 | 华中科技大学 | White box target attack method of brain-computer interface regression system based on EEG |
| CN111046379A (en) * | 2019-12-06 | 2020-04-21 | 支付宝(杭州)信息技术有限公司 | Anti-attack monitoring method and device |
| CN111275123A (en) * | 2020-02-10 | 2020-06-12 | 北京信息科技大学 | Method and system for generating large-batch confrontation samples |
| CN111737691A (en) * | 2020-07-24 | 2020-10-02 | 支付宝(杭州)信息技术有限公司 | Method and device for generating confrontation sample |
| CN112926678A (en) * | 2021-03-25 | 2021-06-08 | 支付宝(杭州)信息技术有限公司 | Model similarity determination method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11494591B2 (en) * | 2019-01-11 | 2022-11-08 | International Business Machines Corporation | Margin based adversarial computer program |
| US11606389B2 (en) * | 2019-08-29 | 2023-03-14 | Nec Corporation | Anomaly detection with graph adversarial training in computer systems |
-
2021
- 2021-06-15 CN CN202110662087.9A patent/CN113313404B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109934253A (en) * | 2019-01-08 | 2019-06-25 | 阿里巴巴集团控股有限公司 | A kind of confrontation sample generating method and device |
| CN110764958A (en) * | 2019-09-24 | 2020-02-07 | 华中科技大学 | White box target attack method of brain-computer interface regression system based on EEG |
| CN111046379A (en) * | 2019-12-06 | 2020-04-21 | 支付宝(杭州)信息技术有限公司 | Anti-attack monitoring method and device |
| CN111275123A (en) * | 2020-02-10 | 2020-06-12 | 北京信息科技大学 | Method and system for generating large-batch confrontation samples |
| CN111737691A (en) * | 2020-07-24 | 2020-10-02 | 支付宝(杭州)信息技术有限公司 | Method and device for generating confrontation sample |
| CN112926678A (en) * | 2021-03-25 | 2021-06-08 | 支付宝(杭州)信息技术有限公司 | Model similarity determination method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113313404A (en) | 2021-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10523697B2 (en) | Method and apparatus for detecting cyberthreats through correlation analysis | |
| CN111753881B (en) | Concept sensitivity-based quantitative recognition defending method against attacks | |
| CN110348475B (en) | Confrontation sample enhancement method and model based on spatial transformation | |
| CN110852363A (en) | Anti-sample defense method based on deception attacker | |
| JP2022141931A (en) | Method and device for training living body detection model, method and apparatus for living body detection, electronic apparatus, storage medium, and computer program | |
| CN111737691A (en) | Method and device for generating confrontation sample | |
| CN110659486A (en) | System and method for detecting malicious files using two-level file classification | |
| CN111652290A (en) | Detection method and device for confrontation sample | |
| CN112200380A (en) | Method and device for optimizing risk detection model | |
| CN113919497A (en) | Attack and defense method based on feature manipulation for continuous learning ability system | |
| CN115240280A (en) | Construction method of human face living body detection classification model, detection classification method and device | |
| CN110020593B (en) | Information processing method and device, medium and computing equipment | |
| CN113313404B (en) | Method and device for generating countermeasure sample | |
| CN113435264A (en) | Face recognition attack resisting method and device based on black box substitution model searching | |
| CN117857088A (en) | Network traffic abnormality detection method, system, equipment and medium | |
| Naqvi et al. | Adversarial attacks on visual objects using the fast gradient sign method | |
| WO2023093346A1 (en) | Exogenous feature-based model ownership verification method and apparatus | |
| CN113222480A (en) | Training method and device for confrontation sample generation model | |
| CN116188439A (en) | False face-changing image detection method and device based on identity recognition probability distribution | |
| CN112766430B (en) | Method, device and storage medium for resisting attack based on black box universal face detection | |
| Xu et al. | Lancex: A versatile and lightweight defense method against condensed adversarial attacks in image and audio recognition | |
| CN114219011A (en) | Universal back door attack detection method based on non-migratability of back door | |
| CN113052314B (en) | Authentication radius guide attack method, optimization training method and system | |
| CN113723215A (en) | Training method of living body detection network, living body detection method and device | |
| CN114140670B (en) | Method and device for verifying ownership of model based on exogenous characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |