WO2021098384A1 - 一种数据异常检测方法及装置 - Google Patents

一种数据异常检测方法及装置 Download PDF

Info

Publication number
WO2021098384A1
WO2021098384A1 PCT/CN2020/118430 CN2020118430W WO2021098384A1 WO 2021098384 A1 WO2021098384 A1 WO 2021098384A1 CN 2020118430 W CN2020118430 W CN 2020118430W WO 2021098384 A1 WO2021098384 A1 WO 2021098384A1
Authority
WO
WIPO (PCT)
Prior art keywords
historical
tested
current
hidden layer
training
Prior art date
Application number
PCT/CN2020/118430
Other languages
English (en)
French (fr)
Inventor
臧大卫
Original Assignee
中国银联股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国银联股份有限公司 filed Critical 中国银联股份有限公司
Publication of WO2021098384A1 publication Critical patent/WO2021098384A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/38—Payment protocols; Details thereof
    • G06Q20/40—Authorisation, e.g. identification of payer or payee, verification of customer or shop credentials; Review and approval of payers, e.g. check credit lines or negative lists
    • G06Q20/401—Transaction verification
    • G06Q20/4016—Transaction verification involving fraud or risk level assessment in transaction processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present invention relates to the technical field of data processing, in particular to a method and device for detecting data abnormalities.
  • the risk control system for deep learning uses only simple deep neural networks. Compared with complex neural networks, simple neural networks have poorer feature mining capabilities and poorer performance in identifying fraudulent transactions. However, in the face of tens of millions of daily transactions, an overly complex neural network will lead to a large amount of resource overhead, and it is difficult to meet real-time requirements.
  • This application provides a data anomaly detection method and device to reduce resource overhead and meet the requirements of real-time detection.
  • An embodiment of the present invention provides a data abnormality detection method, including:
  • the historical cell state is the abnormal detection of inputting the historical detection sample of the object to be tested into the object to be tested
  • the risk index is compared with the risk threshold to determine the abnormality determination result of the object to be tested.
  • the method further includes:
  • the current cell state and the current hidden layer state are stored.
  • the anomaly detection model is trained in the following manner:
  • the loss function is less than a preset threshold, determine the corresponding parameter as the abnormality Detect the parameters corresponding to the model to obtain the anomaly detection model.
  • the method before acquiring the current test sample of the object to be tested, the method further includes:
  • the candidate parameter is used as the parameter of the abnormality detection model of the object to be tested, and the output cell state of the object to be tested is used as the history Cell state, output hidden layer state as historical hidden layer state.
  • the selecting the training feature of the training object according to the historical sample data includes:
  • an optional feature is selected as the training feature.
  • the anomaly detection model is a neural network model
  • the inputting the current characteristic value, the historical cell state, and the historical hidden layer state of the object to be tested into the abnormality detection model to determine the risk index corresponding to the object to be tested includes:
  • the risk index is determined according to the bypass of the current cell and the current hidden layer state.
  • the embodiment of the present invention also provides a data abnormality detection device, including:
  • the acquisition module is used to acquire the current detection sample, historical cell state and historical hidden layer state of the object to be tested;
  • the historical cell state is determined by inputting the historical detection sample of the object to be tested into the abnormal detection model of the object to be tested
  • the output cell state of the historical hidden layer state is the output hidden layer state determined by inputting the historical detection sample of the object under test into the abnormal detection model of the object under test;
  • a determining module configured to determine the current characteristic value of the object to be tested according to the current test sample
  • a calculation module configured to input the current characteristic value, the historical cell state, and the historical hidden layer state of the object under test into the abnormality detection model to determine the risk index corresponding to the object under test;
  • the comparison module is used to compare the risk index with the risk threshold to determine the abnormal determination result of the object to be tested.
  • the calculation module is further used for:
  • the current cell state and the current hidden layer state are stored.
  • it further includes a training module for training the anomaly detection model in the following manner:
  • the loss function is less than a preset threshold, determine the corresponding parameter as the abnormality Detect the parameters corresponding to the model to obtain the anomaly detection model.
  • the training module is used to:
  • the candidate parameter is used as the parameter of the abnormality detection model of the object to be tested, and the output cell state of the object to be tested is used as the history Cell state, output hidden layer state as historical hidden layer state.
  • the training module is used to:
  • an optional feature is selected as the training feature.
  • the anomaly detection model is a neural network model
  • the calculation module is specifically used for:
  • the risk index is determined according to the bypass of the current cell and the current hidden layer state.
  • the embodiment of the present invention also provides an electronic device, including:
  • At least one processor and,
  • a memory communicatively connected with the at least one processor; wherein,
  • the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the method described above.
  • the embodiment of the present invention also provides a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium storing computer instructions, and the computer instructions are used to make the computer execute the method as described above.
  • the risk control system stores the historical cell state of the current test sample and the historical hidden layer state, where the historical cell state is the output cell determined by the abnormal detection model that inputs the historical test sample of the test object into the test object State, the historical hidden layer state is the output hidden layer state determined by inputting historical detection samples of the object under test into the anomaly detection model of the object under test.
  • the risk control system obtains the current detection sample of the object to be tested, and determines the current characteristic value of the object to be tested according to the current detection sample.
  • the risk control system inputs the current feature value, historical cell state and historical hidden layer state of the object to be tested into the anomaly detection model, determines the risk index corresponding to the object to be tested, and compares the risk index with the risk threshold to determine the object to be tested The abnormality judgment result of the object.
  • the current feature value, historical cell state and historical hidden layer state of the object to be tested need to be input, which reduces the amount of real-time data input and the number of real-time calculations of the model. Better meet the requirements of real-time risk control.
  • the characteristics of the historical transaction of the object to be tested and the characteristics of the current transaction are combined into the model, so that the model can identify new high-risk data in a timely manner, which increases the accuracy of anomaly recognition, and at the same time improves the sensitivity of detection, and ensures Stability of anomaly detection.
  • FIG. 1 is an architecture diagram of a data anomaly detection system provided by an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of a method for detecting data anomaly according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of a cell structure of an LSTM model provided by an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of the cell structure of a specific LSTM model provided by an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a data anomaly detection device provided by an embodiment of the present invention.
  • Fig. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.
  • FIG. 1 shows an architecture diagram of a data anomaly detection system provided by an embodiment of the present application, which includes 4 subsystems, namely, the underlying service subsystem, the data and control component, the service subsystem, and the operation subsystem.
  • the underlying service subsystem provides the database, cache, and communication middleware required by the system;
  • the data and control components provide the data interface and system control management required for data anomaly detection;
  • the service subsystem provides specific data anomaly detection services and reports Suspicious transactions;
  • the operation subsystem provides management of anomaly detection models and risky transactions.
  • the data anomaly detection system can be a network device such as a computer.
  • the data anomaly detection system can be an independent device or a server cluster formed by multiple servers.
  • the data anomaly detection system can use cloud computing technology for information processing.
  • Hot Encoding is a code system in which there are as many bits as there are states, and only one bit is 1, and the others are all 0. In the embodiment of the present invention, it is used to convert the current detection sample into the current feature value and then input the abnormal detection model.
  • LSTM Long Short Term Memory Network (Long Short Term Memory Network) is an improved recurrent neural network that can solve the problem that recurrent neural networks cannot handle long-distance dependencies.
  • TCP Transmission Control Protocol (Transmission Control Protocol), a connection-oriented, reliable, byte stream-based transport layer communication protocol.
  • SCP Secure Copy, a communication protocol used to copy files remotely.
  • an embodiment of the present invention provides a data anomaly detection method.
  • the data anomaly detection method provided by the embodiment of the present invention includes the following steps:
  • Step 201 Obtain the current detection sample, historical cell state, and historical hidden layer state of the object to be tested.
  • the historical cell state is the output cell state determined by inputting the historical detection sample of the object to be tested into the abnormal detection model of the object to be tested
  • the historical hidden layer state is the inputting the historical detection sample of the object to be tested into The output hidden layer state determined by the anomaly detection model of the object to be tested.
  • the current detection sample and the historical detection sample in the embodiment of the present invention may be a transaction sequence of the user. By inputting the user's current transaction sequence into the anomaly detection model, the risk situation of the current transaction can be predicted.
  • the historical time period is the time period before the current time point corresponding to the current test sample. For example, the current time point is at 10:00 am on June 3, 2019, and the historical time period is from 10:00 am on June 3, 2018 to June 2019. 10 am on the 3rd of the month.
  • the time length of the historical time period can be selected according to needs and accuracy. Among them, the longer the historical time period, the higher the detection accuracy, but the greater the amount of calculation required; the historical time period’s time length The shorter the segment, the smaller the amount of calculation required for detection, but the accuracy is lower.
  • Step 202 Determine the current characteristic value of the object to be tested according to the current test sample.
  • Step 203 Input the current characteristic value, the historical cell state and the historical hidden layer state of the object to be tested into the abnormality detection model, and determine the risk index corresponding to the object to be tested.
  • Step 204 Compare the risk index with a risk threshold, and determine an abnormal determination result of the object to be tested.
  • the risk index is greater than the risk threshold, it indicates that the risk is relatively high, that is, there is an abnormality in the current transaction. At this time, the corresponding personnel can be notified by mail, internal process documents of the company, etc.
  • the risk index is less than or equal to the risk threshold, it indicates that the current transaction is normal.
  • the output cell state and output hidden layer state corresponding to the current transaction can be stored as input parameters for the next abnormality detection. That is, the above step 203, after inputting the current feature value, the historical cell state, and the historical hidden layer state of the object to be tested into the abnormality detection model, further includes:
  • the current cell state and the current hidden layer state are stored.
  • the current cell state is taken as the historical cell state of the next abnormality detection
  • the current hidden layer state is taken as the historical hidden layer state of the next abnormality detection.
  • the risk control system stores the historical cell state of the current test sample and the historical hidden layer state, where the historical cell state is the output cell determined by the abnormal detection model that inputs the historical test sample of the test object into the test object State, the historical hidden layer state is the output hidden layer state determined by inputting historical detection samples of the object to be tested into the anomaly detection model of the object to be tested.
  • the risk control system obtains the current detection sample of the object to be tested, and determines the current characteristic value of the object to be tested according to the current detection sample.
  • the risk control system inputs the current feature value, historical cell state and historical hidden layer state of the object to be tested into the anomaly detection model, determines the risk index corresponding to the object to be tested, and compares the risk index with the risk threshold to determine the object to be tested The abnormality judgment result of the object.
  • the current feature value, historical cell state and historical hidden layer state of the object to be tested need to be input, which reduces the amount of real-time data input and the number of real-time calculations of the model. Better meet the requirements of real-time risk control.
  • the characteristics of the historical transaction of the object to be tested and the characteristics of the current transaction are combined into the model, so that the model can identify new high-risk data in a timely manner, which increases the accuracy of anomaly recognition, and at the same time improves the sensitivity of detection and ensures Stability of anomaly detection.
  • the anomaly detection model is a neural network model
  • the LSTM model is taken as an example for introduction in the embodiment of the present invention.
  • the inputting the current characteristic value, the historical cell state and the historical hidden layer state of the object to be tested into the abnormality detection model to determine the risk index corresponding to the object to be tested includes:
  • the risk index is determined according to the bypass of the current cell and the current hidden layer state.
  • the traditional LSTM model needs to take out the user’s recent transaction sequence and input the model to make predictions.
  • the prediction model needs to input all the user’s previous transaction sequences, the amount of data is very large, and it is difficult to meet real-time detection. Requirements.
  • the LSTM model is split and disassembled into a common acyclic neural network.
  • Fig. 3 shows the cell structure of an LSTM model in an embodiment of the present invention. As shown in Figure 3, there are three inputs and three outputs. The inputs are the historical cell state c, the historical hidden layer state h, and the current feature value x; the output is the current cell state c′, and the current hidden layer state h′ is output. Risk index y.
  • the calculation formula of the LSTM model in the embodiment of the present invention is as follows:
  • LSTM model compared to conventional, state of the cell model LSTM embodiment of the present invention increases the current bypass s t cells. After the bypass st and the historical hidden layer state h t are connected through two layers of fully connected layers using the ReLU activation function, finally the risk index y t is output through the output layer, and the output layer uses the Sigmoid activation function to limit the output range to [0, 1].
  • the current feature value of the current transaction is very important for prediction.
  • the judgment of risk comes from the interaction between the current feature and the historical feature, while the traditional LSTM does not have a bypass s t , and the model output comes from
  • the output hidden layer state h t of the previous cell is mainly derived from the previous cell state c t .
  • part of the transaction characteristics contained in c t has been lost, mainly storing long and short-term memory characteristics Therefore, adding a bypass st and h t connection, and obtaining interactive features through two fully connected layers can improve the prediction performance of the model.
  • the embodiment of the present invention adopts a strategy of updating the model daily, which not only limits the length of the cell state memory sequence, but also improves the sensitivity and long-term performance of the model.
  • the embodiment of the present invention uses the latest data for feature screening every day, and then trains the LSTM model.
  • the selecting the training feature of the training object according to the historical sample data includes:
  • an optional feature is selected as the training feature.
  • the training features in the embodiment of the present invention include core features and optional features.
  • the core feature is an important feature, which can be selected directly without participating in automatic screening, and can be set in advance, such as the specific transaction amount, transaction time, etc.
  • Optional features are features with general effects or obvious deviations over time. Such features have a relatively large PSI (Population Stability Index, stability index).
  • the performance value of each optional feature can be calculated, such as an IV (Information Value) value, and sorted according to the performance value, and the optional feature with a higher performance value is selected as the training feature.
  • IV Information Value
  • the optional feature with a higher performance value is selected as the training feature.
  • the anomaly detection model in the embodiment of the present invention is trained in the following manner:
  • the loss function is less than a preset threshold, determine the corresponding parameter as the abnormality Detect the parameters corresponding to the model to obtain the anomaly detection model.
  • the model training module will retrain the model.
  • One-Hot Encoding one-hot encoding
  • step 301 before obtaining the current test sample of the object to be tested, also includes:
  • the candidate parameter is used as the parameter of the abnormality detection model of the object to be tested, and the output cell state of the object to be tested is used as the history Cell state, output hidden layer state as historical hidden layer state.
  • the model parameters trained every day can be used as candidate parameters, that is, the model that is performing data anomaly detection as the main model; the new model obtained from the daily model training is used as the secondary model, and performance testing and switching are performed in the background Prepare, if the test meets the requirements, it will be converted to the main model for data anomaly detection.
  • the specific performance test method may be to calculate the training effect index of the historical test sample, such as calculating the accuracy rate, the recall rate, and F1. Calculate the training effect index of the main model and the auxiliary model separately.
  • the training effect index of the auxiliary model is better than the training effect index of the main model, replace the main model with the auxiliary model; if the training effect index of the main model is better than the training effect of the auxiliary model Indicators, continue to use the main model for data anomaly detection.
  • the specific embodiment uses the LSTM depth model to intelligently identify fraudulent transactions, intercept them, and reduce the property losses of the issuing bank and users.
  • the model training module starts to update the model after the daily cut; first traverses the wrongly divided transaction database, corrects the latest feedback of the wrongly divided transaction, and updates the training set database and the test set database;
  • the training library and the test library respectively store the One-Hot characteristics of each transaction of each user in the most recent year.
  • the maximum number of transactions is 200, which are stored in tables according to the hash value of the card number.
  • 5% of users are stored in the test library, 95% of users are stored in the training library; put the newly added transactions into the training library and the test library, and delete old transactions to ensure that the maximum number of transactions is 200; sample the training library, all transactions All normal users are marked for training according to a 50:1 ratio, and users with abnormal transactions and misclassifications are all marked for training.
  • the model control component delivers the new model to the deep model module cluster
  • the main and auxiliary model conversion process is as follows:
  • the in-depth model module After the in-depth model module receives the new model every day, it will start the model test and conversion preparation work; first load the new model;
  • the model control component summarizes the calculation results of all deep model nodes, and calculates the accuracy rate, recall rate and F1;
  • the model control component performs version management according to the test set indicators and the administrator configuration; if the administrator interferes with the model version, the previous version of the model or the updated model will be retained according to the administrator's configuration; if the administrator does not interfere with the model version, it will follow the rules Make a decision, if the accuracy rate is> 0.3 and the recall rate is> 0.2 and F1 ⁇ the previous version model, load the new model, otherwise load the old model and prepare for model conversion;
  • the deep model module performs the pre-conversion preparation work; reads the user’s new transactions on the day and the user’s cell state, hidden layer state cache, enters the LSTM model, updates the state cache, and keeps the cell state, hidden layer state in sync with the latest transaction , Waiting to be converted to the main model;
  • the model control component sends a model role change notification through Zookeeper, the main model process is terminated, and the secondary model is connected to the online system to process real-time transactions.
  • the data anomaly detection process is as follows:
  • the online system calls the UnionPay intelligent real-time risk control system through Magpie middleware, and sends transactions to the transaction collection component;
  • the transaction collection component performs preliminary filtering of transactions, and filters out low-risk transactions and channels that do not require risk control through key field comparison;
  • the transaction collection component sends transactions to the historical feature calculation component cluster and the deep engine module cluster through TCP socket (container) communication, and distributes them to different nodes according to the mantissa of the card number hash value;
  • the historical feature calculation component will update the context and statistics based on the transaction information.
  • the following information stores the information of the user’s last specific behavior; the statistical information includes card number, merchant, mobile phone number, acquirer, card issuer, ID number, IP, device number and other dimensions, from the time span It is divided into short-term statistics, current-day statistics and multi-day statistics;
  • the deep model module evaluates the risk of the received transaction; first, according to the received transaction, request the corresponding feature from the historical feature calculation component, calculate the core feature and all optional features, and perform One-Hot Encoding;
  • the embodiment of the present invention also provides a data abnormality detection device, as shown in FIG. 5, including:
  • the obtaining module 501 is used to obtain the current detection sample, the historical cell state and the historical hidden layer state of the object to be tested;
  • the historical cell state is an abnormality detection model that inputs the historical detection sample of the object to be tested into the object to be tested A determined output cell state, where the historical hidden layer state is an output hidden layer state determined by inputting historical detection samples of the object under test into the abnormality detection model of the object under test;
  • the determining module 502 is configured to determine the current characteristic value of the object to be tested according to the current test sample
  • a calculation module 503 configured to input the current feature value, the historical cell state, and the historical hidden layer state of the object to be tested into the abnormality detection model to determine the risk index corresponding to the object to be tested;
  • the comparison module 504 is configured to compare the risk index with a risk threshold to determine the abnormality determination result of the object to be tested.
  • calculation module 503 is further used for:
  • the current cell state and the current hidden layer state are stored.
  • it also includes a training module 505, which is used to train an anomaly detection model in the following ways:
  • the loss function is less than a preset threshold, determine the corresponding parameter as the abnormality Detect the parameters corresponding to the model to obtain the anomaly detection model.
  • the training module 505 is configured to:
  • the candidate parameter is used as the parameter of the abnormality detection model of the object to be tested, and the output cell state of the object to be tested is used as the history Cell state, output hidden layer state as historical hidden layer state.
  • the training module 505 is configured to:
  • an optional feature is selected as the training feature.
  • the abnormality detection model is a neural network model
  • the calculation module 503 is specifically used for:
  • the risk index is determined according to the bypass of the current cell and the current hidden layer state.
  • the present invention also provides an electronic device, as shown in FIG. 6, including:
  • It includes a processor 601, a memory 602, a transceiver 603, and a bus interface 604, wherein the processor 601, the memory 602 and the transceiver 603 are connected through the bus interface 604;
  • the processor 601 is configured to read the program in the memory 602 and execute the following method:
  • the historical cell state is the output cell state determined by inputting the historical detection sample of the object under test into the abnormality detection model of the object under test
  • the historical hidden layer state is the output hidden layer state determined by inputting historical detection samples of the object to be tested into the anomaly detection model of the object to be tested
  • the risk index is compared with the risk threshold to determine the abnormality determination result of the object to be tested.
  • These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
  • the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
  • the instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

Abstract

一种数据异常检测方法及装置，用以减少资源开销，满足实时检测的要求。该方法包括获取待测对象的当前检测样本、历史细胞状态和历史隐藏层状态；历史细胞状态为将待测对象的历史检测样本输入待测对象的异常检测模型确定的输出细胞状态，历史隐藏层状态为将待测对象的历史检测样本输入待测对象的异常检测模型确定的输出隐藏层状态；根据当前检测样本，确定待测对象的当前特征值；将待测对象的当前特征值、历史细胞状态和历史隐藏层状态输入异常检测模型中，确定待测对象对应的风险指数；将风险指数与风险阈值进行对比，确定待测对象的异常判定结果。

Description

一种数据异常检测方法及装置
相关申请的交叉引用
本申请要求在2019年11月18日提交中国专利局、申请号为201911126382.1、申请名称为“一种数据异常检测方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及数据处理技术领域，尤其涉及一种数据异常检测方法及装置。
背景技术
互联网和互联网金融的快速发展给风控系统带来了前所未有的挑战，欺诈交易的形式、方式越发多种多样，隐蔽性高，难以挖掘，传统的规则引擎风控方法越发乏力。近年深度学习的快速发展为解决这一问题提供了另一种思路，开发深度引擎，通过深度学习构建模型挖掘隐含信息，辨别欺诈交易，已有不错的成果。
现有技术中，深度学习的风控系统仅使用简单深度神经网络，简单神经网络相较复杂神经网络挖掘特征的能力更差，鉴别欺诈交易的性能也较差。但是面对千万级的日交易量，过于复杂的神经网络会导致大量的资源开销，难以满足实时性要求。
发明内容
本申请提供一种数据异常检测方法及装置，用以减少资源开销，满足实时检测的要求。
本发明实施例提供的一种数据异常检测方法，包括：
获取待测对象的当前检测样本，以及所述待测对象的历史细胞状态和历史隐藏层状态；所述历史细胞状态为将所述待测对象的历史检测样本输入所 述待测对象的异常检测模型确定的输出细胞状态，所述历史隐藏层状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出隐藏层状态；
根据所述当前检测样本，确定所述待测对象的当前特征值；
将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中，确定所述待测对象对应的风险指数；
将所述风险指数与风险阈值进行对比，确定所述待测对象的异常判定结果。
一种可选的实施例中，所述将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中之后，还包括：
确定所述待测对象对应的当前细胞状态以及当前隐藏层状态；
将所述当前细胞状态以及所述当前隐藏层状态进行存储。
一种可选的实施例中，所述异常检测模型利用以下方式进行训练：
获取历史时间段内的历史样本数据；
根据所述历史样本数据，选择训练对象的训练特征，并确定训练特征对应的训练特征值；
将所述训练特征值输入初始异常检测模型，并根据输出的风险指数及所述训练对象的异常判定结果计算损失函数，当所述损失函数小于预设阈值时，确定对应的参数为所述异常检测模型对应的参数，得到异常检测模型。
一种可选的实施例中，所述获取待测对象的当前检测样本之前，还包括：
获取所述待测对象的历史检测样本；
将所述待测对象的历史检测样本输入所述异常检测模型中，确定所述待测对象的备选参数、备选参数的训练效果指标、所述待测对象的输出细胞状态以及输出隐藏层状态；
当所述备选参数的训练效果指标大于存储的训练效果指标时，将所述备选参数作为所述待测对象的异常检测模型的参数，并将所述待测对象的输出细胞状态作为历史细胞状态，输出隐藏层状态作为历史隐藏层状态。
一种可选的实施例中，所述根据所述历史样本数据，选择训练对象的训练特征，包括：
确定所述训练对象的核心特征和可选特征，并将所述核心特征作为训练特征；
根据所述历史样本数据，计算每个可选特征的性能值；
根据所述性能值，选择可选特征作为所述训练特征。
一种可选的实施例中，所述异常检测模型为神经网络模型；
所述将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中，确定所述待测对象对应的风险指数，包括：
根据所述当前特征值、所述历史细胞状态和所述历史隐藏层状态，确定所述神经网络模型的当前隐藏层状态；
根据所述当前特征值确定所述神经网络模型的当前细胞的旁路；
根据所述当前细胞的旁路和当前隐藏层状态，确定所述风险指数。
本发明实施例还提供一种数据异常检测装置，包括：
获取模块，用于获取待测对象的当前检测样本、历史细胞状态和历史隐藏层状态；所述历史细胞状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出细胞状态，所述历史隐藏层状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出隐藏层状态；
确定模块，用于根据所述当前检测样本，确定所述待测对象的当前特征值；
计算模块，用于将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中，确定所述待测对象对应的风险指数；
对比模块，用于将所述风险指数与风险阈值进行对比，确定所述待测对象的异常判定结果。
一种可选的实施例中，所述计算模块，还用于：
确定所述待测对象对应的当前细胞状态以及当前隐藏层状态；
将所述当前细胞状态以及所述当前隐藏层状态进行存储。
一种可选的实施例中，还包括训练模块，用于利用以下方式训练异常检测模型：
获取历史时间段内的历史样本数据；
根据所述历史样本数据，选择训练对象的训练特征，并确定训练特征对应的训练特征值；
将所述训练特征值输入初始异常检测模型，并根据输出的风险指数及所述训练对象的异常判定结果计算损失函数，当所述损失函数小于预设阈值时，确定对应的参数为所述异常检测模型对应的参数，得到异常检测模型。
一种可选的实施例中，所述训练模块，用于：
获取所述待测对象的历史检测样本；
将所述待测对象的历史检测样本输入所述异常检测模型中，确定所述待测对象的备选参数、备选参数的训练效果指标、所述待测对象的输出细胞状态以及输出隐藏层状态；
当所述备选参数的训练效果指标大于存储的训练效果指标时，将所述备选参数作为所述待测对象的异常检测模型的参数，并将所述待测对象的输出细胞状态作为历史细胞状态，输出隐藏层状态作为历史隐藏层状态。
一种可选的实施例中，所述训练模块，用于：
确定所述训练对象的核心特征和可选特征，并将所述核心特征作为训练特征；
根据所述历史样本数据，计算每个可选特征的性能值；
根据所述性能值，选择可选特征作为所述训练特征。
一种可选的实施例中，所述异常检测模型为神经网络模型；
所述计算模块，具体用于：
根据所述当前特征值、所述历史细胞状态和所述历史隐藏层状态，确定 所述神经网络模型的当前隐藏层状态；
根据所述当前特征值确定所述神经网络模型的当前细胞的旁路；
根据所述当前细胞的旁路和当前隐藏层状态，确定所述风险指数。
本发明实施例还提供一种电子设备，包括：
至少一个处理器；以及，
与所述至少一个处理器通信连接的存储器；其中，
所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如上所述的方法。
本发明实施例还提供一种非暂态计算机可读存储介质，所述非暂态计算机可读存储介质存储计算机指令，所述计算机指令用于使所述计算机执行如上所述的方法。
本发明实施例中，风控系统中存储有当前检测样本的历史细胞状态以及历史隐藏层状态，其中历史细胞状态为将待测对象的历史检测样本输入待测对象的异常检测模型确定的输出细胞状态，历史隐藏层状态为将待测对象的历史检测样本输入待测对象的异常检测模型确定的输出隐藏层状态。当需要对待测对象进行检测时，风控系统获取待测对象的当前检测样本，并根据所述当前检测样本，确定所述待测对象的当前特征值。风控系统将待测对象的当前特征值、历史细胞状态和历史隐藏层状态，输入异常检测模型中，确定待测对象对应的风险指数，并将风险指数与风险阈值进行对比，从而确定待测对象的异常判定结果。本发明实施例中，对待测对象进行异常检测时，仅需输入待测对象的当前特征值、历史细胞状态和历史隐藏层状态，减少了实时数据输入量，也减少了模型的实时运算次数，更好地满足实时风控的要求。且将待测对象的历史交易的特征与当笔交易的特征相结合输入模型，使得模型可以及时对新型高风险数据进行鉴别，增加了异常识别的准确性，同时提高了检测的灵敏度，保证了异常检测的稳定性。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简要介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例提供的一种数据异常检测系统架构图；
图2为本发明实施例提供的一种数据异常检测方法的流程示意图；
图3为本发明实施例提供的一种LSTM模型的一个细胞结构示意图；
图4为本发明实施例提供的一种具体的LSTM模型的细胞结构示意图；
图5为本发明实施例提供的一种数据异常检测装置的结构示意图；
图6为本发明实施例提供的电子设备的结构示意图。
具体实施方式
为了使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明作进一步地详细描述，显然，所描述的实施例仅仅是本发明一部份实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本发明保护的范围。
请参考图1，其示出了本申请一个实施例提供的数据异常检测系统架构图，包括4个子系统，分别是底层服务子系统、数据及控制组件、服务子系统和运营子系统。其中，底层服务子系统提供了系统所需数据库、缓存以及与通信中间件；数据及控制组件提供数据异常检测所需的数据接口和系统控制管理；服务子系统提供具体的数据异常检测服务，报告可疑交易；运营子系统提供对异常检测模型和风险交易的管理。
数据异常检测系统可以是计算机等网络设备。数据异常检测系统可以是一个独立的设备，也可以是多个服务器所形成的服务器集群。优选地，数据异常检测系统可以采用云计算技术进行信息处理。
需要注意的是，上文提及的应用场景仅是为了便于理解本申请的精神 和原理而示出，本申请实施例在此方面不受任何限制。相反，本申请实施例可以应用于适用的任何场景。
下面对本申请实施例中涉及的部分概念进行介绍。
热独编码：One-Hot Encoding，是有多少个状态就有多少比特，而且只有一个比特为1，其他全为0的一种码制。本发明实施例中用于将当前检测样本转为当前特征值后输入异常检测模型。
LSTM：长短时记忆网络(Long Short Term Memory Network)，是一种改进之后的循环神经网络，可以解决循环神经网络无法处理长距离的依赖的问题。
TCP：传输控制协议(Transmission Control Protocol)，一种面向连接的、可靠的、基于字节流的传输层通信协议。
SCP：安全拷贝协议(Secure Copy)，一种用来进行远程文件拷贝的通信协议。
为了监控系统数据，并提高异常检测的准确性，本发明实施例提供了一种数据异常检测方法，如图2所示，本发明实施例提供的数据异常检测方法包括以下步骤：
步骤201、获取待测对象的当前检测样本、历史细胞状态和历史隐藏层状态。
其中，历史细胞状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出细胞状态，所述历史隐藏层状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出隐藏层状态。
本发明实施例中的当前检测样本和历史检测样本可以为用户的交易序列。通过将用户当前的交易序列输入异常检测模型，来预测当前交易的风险情况。
历史时间段为当前检测样本对应的当前时间点之前的时间段，例如，当前时间点为2019年6月3日上午10点，历史时间段为2018年6月3日上午10点至2019年6月3日上午10点。具体实施过程中，历史时间段的时间长 度可以根据需要和精确度进行选择，其中，历史时间段的时长越长，则检测准确度越高，但所需计算量越大；历史时间段的时长越段短，则检测所需计算量越小，但准确性较低。
步骤202、根据所述当前检测样本，确定所述待测对象的当前特征值。
步骤203、将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中，确定所述待测对象对应的风险指数。
步骤204、将所述风险指数与风险阈值进行对比，确定所述待测对象的异常判定结果。
其中，若风险指数大于风险阈值，则表明风险较大，即当前交易出现了异常。此时，可以通过邮件、公司内部流程单据等方式通知到对应人员。另一方面，若风险指数小于或等于风险阈值，则表明当前交易正常。进一步地，无论当前交易是否异常，都可以将当前交易对应的输出细胞状态以及输出隐藏层状态进行存储，作为下一次异常检测的输入参数。即，上述步骤203，将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中之后，还包括：
确定所述待测对象对应的当前细胞状态以及当前隐藏层状态；
将所述当前细胞状态以及所述当前隐藏层状态进行存储。
其中，将当前细胞状态作为下一次异常检测的历史细胞状态，当前隐藏层状态作为下一次异常检测的历史隐藏层状态。
本发明实施例中，风控系统中存储有当前检测样本的历史细胞状态以及历史隐藏层状态，其中历史细胞状态为将待测对象的历史检测样本输入待测对象的异常检测模型确定的输出细胞状态，历史隐藏层状态为将待测对象的历史检测样本输入待测对象的异常检测模型确定的输出隐藏层状态。当需要对待测对象进行检测时，风控系统获取待测对象的当前检测样本，并根据所述当前检测样本，确定所述待测对象的当前特征值。风控系统将待测对象的当前特征值、历史细胞状态和历史隐藏层状态，输入异常检测模型中，确定 待测对象对应的风险指数，并将风险指数与风险阈值进行对比，从而确定待测对象的异常判定结果。本发明实施例中，对待测对象进行异常检测时，仅需输入待测对象的当前特征值、历史细胞状态和历史隐藏层状态，减少了实时数据输入量，也减少了模型的实时运算次数，更好地满足实时风控的要求。且将待测对象的历史交易的特征与当笔交易的特征相结合输入模型，使得模型可以及时对新型高风险数据进行鉴别，增加了异常识别的准确性，同时提高了检测的灵敏度，保证了异常检测的稳定性。
进一步地，异常检测模型为神经网络模型，本发明实施例中以LSTM模型为例进行介绍。
所述将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中，确定所述待测对象对应的风险指数，包括：
根据所述当前特征值、所述历史细胞状态和所述历史隐藏层状态，确定所述神经网络模型的当前隐藏层状态；
根据所述当前特征值确定所述神经网络模型的当前细胞的旁路；
根据所述当前细胞的旁路和当前隐藏层状态，确定所述风险指数。
传统的LSTM模型在预测一笔交易风险时需要取出该用户最近的交易序列，输入模型进行预测，每次预测模型都需要将用户之前的所有交易序列全部输入，数据量十分庞大，难以满足实时检测的要求。
本发明实施例中，将LSTM模型进行拆分，拆解成普通非循环神经网络。图3示出了本发明实施例中的一种LSTM模型的细胞结构。如图3所示，有三个输入和三个输出，输入分别是历史细胞状态c，历史隐藏层状态h，当前特征值x；输出分别是当前细胞状态c′，输出当前隐藏层状态h′，风险指数y。
具体来说，如图4所示，本发明实施例中LSTM模型的计算公式如下：
f t＝σ(W f·[h t-1,x t]+b f)
i t＝σ(W i·[h t-1,x t]+b i)
c’ t＝tanh(W c·[h t-1,x t]+b c)
c t＝f t*c t-1+i t*c’ t
o t＝σ(W o·[h t-1,x t]+b o)
h t＝o t*tanh(c t)
s t＝σ(W i·x t+b i)*x t
y t＝σ(W 3ReLU(W 2ReLU(W 1·[h t,s t]+b 1)+b 2)+b 3)
相较于传统的LSTM模型，本发明实施例的LSTM模型的细胞状态中增加了当前细胞的旁路s t。旁路s t和历史隐藏层状态h t相连后通过两层使用ReLU激活函数的全连接层，最后通过输出层输出风险指数y t，输出层使用Sigmoid激活函数，将输出范围限制在[0,1]。
在交易的数据异常检测中，当笔交易的当前特征值对预测是非常重要的，对风险的判断来自于当笔特征与历史特征的交互，而传统LSTM没有旁路s t，模型输出来自于上一细胞的输出隐藏层状态h t，其主要来源于上一细胞状态c t，经过输入门和输出门，c t中包含的当笔交易特征已经损失了一部分，主要存储了长短时记忆特征，故增加旁路s t与h t连接，通过两层全连接层获取交互特征，可以提升模型的预测性能。
传统的LSTM模型，若存在频繁交易的用户，对其多日大量交易进行连续预测，序列过长，与训练集序列差异大，可能会导致长时记忆的紊乱。本发明实施例中，每次预测完后只需记录c′和h′，下次预测该用户的交易时，只需一次LSTM运算即可完成预测，满足实时风控的要求。另一方面，本发明实施例采取每日更新模型的策略，既限制了细胞状态记忆序列的长度，又提升了模型灵敏度和长期性能。
为了保证数据检测的准确性，本发明实施例每日均采用最新的数据进行特征筛选，再训练LSTM模型。所述根据所述历史样本数据，选择训练对象的训练特征，包括：
确定所述训练对象的核心特征和可选特征，并将所述核心特征作为训练特征；
根据所述历史样本数据，计算每个可选特征的性能值；
根据所述性能值，选择可选特征作为所述训练特征。
具体来说，本发明实施例中的训练特征包括核心特征和可选特征。其中，核心特征为重要特征，不参与自动筛选直接选中，可以预先设置，例如具体的交易金额、交易时间等。可选特征为效果一般或随时间偏移较为明显的特征，这类特征的PSI(Population Stability Index，稳定度指标)较大。具体的针对可选特征，可以计算每个可选特征的性能值，例如IV(Information Value，信息价值)值，并根据性能值进行排序，选择性能值较高的可选特征作为训练特征。对应的，利用LSTM模型进行数据异常检测时，也需要筛选出核心特征和可选特征，并计算出核心特征值和可选特征值，作为当前特征值，输入LSTM模型进行计算检测。
进一步地，本发明实施例中异常检测模型利用以下方式进行训练：
获取历史时间段内的历史样本数据；
根据所述历史样本数据，选择训练对象的训练特征，并确定训练特征对应的训练特征值；
将所述训练特征值输入初始异常检测模型，并根据输出的风险指数及所述训练对象的异常判定结果计算损失函数，当所述损失函数小于预设阈值时，确定对应的参数为所述异常检测模型对应的参数，得到异常检测模型。
具体来说，每天日切后，模型训练模块将会重新训练模型。首先将客服反馈的错分交易更新至数据库；维护训练库、测试库，该库存储了每个用户的定长交易序列，读取训练库中的历史交易及特征信息，进行特征自动筛选，选中后进行特征组合，再进行One-Hot Encoding(独热编码)，转换为LSTM模型可用的特征值。最后输入LSTM模型进行训练，得到训练好的模型，发往模型控制组件。
每天训练出的模型并不是直接用于异常检测，而是需要与原有的模型进行对比和评估。即上述步骤301，获取待测对象的当前检测样本之前，还包括：
获取所述待测对象的历史检测样本；
将所述待测对象的历史检测样本输入所述异常检测模型中，确定所述待 测对象的备选参数、备选参数的训练效果指标、所述待测对象的输出细胞状态以及输出隐藏层状态；
当所述备选参数的训练效果指标大于存储的训练效果指标时，将所述备选参数作为所述待测对象的异常检测模型的参数，并将所述待测对象的输出细胞状态作为历史细胞状态，输出隐藏层状态作为历史隐藏层状态。
具体实施过程中，可以将每天训练出的模型参数作为备选参数，即将正在执行数据异常检测的模型作为主模型；将每日模型训练得到的新模型作为副模型，在后台进行性能测试和切换准备，若测试符合要求，则转换成主模型，用于执行数据异常检测。具体的性能测试方式可以为计算历史检测样本的训练效果指标，例如计算精确率、召回率和F1等。分别计算主模型和副模型的训练效果指标，若副模型的训练效果指标优于主模型的训练效果指标，则用副模型替换主模型；若主模型的训练效果指标优于副模型的训练效果指标，则继续沿用主模型进行数据异常检测。
为了更清楚地理解本发明，以具体实施例对上述流程进行详细描述。具体实施例利用LSTM深度模型，智能识别欺诈交易，进行拦截，降低发卡行及用户的财产损失。
具体的训练流程如下：
1.模型训练模块日切后启动开始更新模型；首先遍历错分交易库，将最新反馈的错分交易更正，更新训练集库和测试集库；
2.维护训练库和测试库。训练库和测试库分别存储了每个用户最近一年每笔交易的One-Hot(独热)化后的特征，最大交易数量为200条，按照卡号哈希值分表存储。5％的用户存储在测试库，95％的用户存储在训练库；将新增加的交易放入训练库和测试库，同时删除旧交易保证最大交易数为200；对训练库进行采样，所有交易均正常的用户按照50：1的比例标记用于训练，存在异常交易和存在错分的用户全部进行标记用于训练。
3.计算所有可选特征在训练集中被标记用户的样本上的IV值；
4.将特征按照IV值排序，按照给定的规则选取前n个IV值大于a的特 征；
5.选中的可选特征和所有核心特征进行组合，得到本次迭代模型的所有特征；取出训练库中被标记的用户，作为训练集，输入分布式深度学习集群，训练LSTM模型；
6.模型练完成后触发Zookeeper监听，通知模型控制组件通过SCP获取新模型；
7.模型控制组件将新模型下发至深度模型模块集群；
8.向深度模型集群发送副模型重载指令，开始模型测试和转换准备。
主副模型转换流程如下：
1.深度模型模块接收到每日的新模型后，开始模型测试和转换准备工作；首先加载新模型；
2.读取训练集库中所有交易数量大于20的用户和测试集库中所有的用户，组合相应特征作为LSTM输入；
3.将样本输入LSTM神经网络，保存每个用户LSTM的最终细胞状态、隐藏层状态；统计测试集用户输出信息，通过Zookeeper发送至模型控制组件；
4.模型控制组件汇总所有深度模型节点计算结果，计算精确率、召回率和F1；
5.模型控制组件根据测试集指标和管理员配置进行版本管理；若管理员干预模型版本，则按照管理员的配置保留上一版本模型或更新模型；若管理员不干预模型版本，则按照规则决策，若精确率>0.3且召回率>0.2且F1≥上版本模型，载入新模型，否则载入旧模型，准备模型转换；
6.深度模型模块进行转换前准备工作；读取用户当日新发生的交易和用户的细胞状态、隐藏层状态缓存，输入LSTM模型，更新状态缓存，保持细胞状态、隐藏层状态与最新的交易同步，等待转换为主模型；
7.日切后，模型控制组件通过Zookeeper发送模型角色转换通知，主模型进程终止，副模型接入联机系统，处理实时交易。
数据异常检测流程具体如下：
1.联机系统通过Magpie中间件调用银联智能实时风控系统，发送交易至交易采集组件；
2.交易采集组件对交易进行初步过滤，通过关键字段比对过滤掉低风险交易和不需要风控的渠道；
3.交易采集组件通过TCP socket(容器)通信发送交易至历史特征计算组件集群和深度引擎模块集群，按照卡号哈希值尾数分发至不同节点；
4.历史特征计算组件将根据这笔交易的信息更新上下文和统计量。下文信息存储了该用户的上次特定行为的信息；统计量信息包含了卡号、商户、手机号、收单机构、发卡机构、身份证号、IP、设备号等多种维度，从时间跨度上又分为短时统计量、当日统计量和多日统计量；
5.深度模型模块评估接收到的交易的风险；首先根据接收到的交易向历史特征计算组件请求相应特征，计算核心特征和全部可选特征，进行One-Hot Encoding；
6.按照当日特征筛选的结果对特征进行组合，得到神经网络的输入；
7.读取用户LSTM神经网络细胞状态、隐藏层状态，载入LSTM模型；若无该用户的状态缓存，则说明该用户历史交易少于20笔，读取其所有历史交易，依次输入LSTM模型，得到细胞状态、隐藏层状态；
8.向LSTM模型输入交易特征，进行前向传播算法，得到该笔交易的风险指数；风险指数大于等于阈值，则判定为欺诈交易，反馈交易采集组件对交易进行拦截，并发送至分析结果处理组件，继续步骤9；若风险指数小于阈值则判定为正常交易，继续步骤9；
9.将该用户LSTM的最新细胞状态、隐藏层状态更新至缓存；
10.将该笔交易One-Hot化后的全部特征和风险评估结果按卡号存储至训练集表和测试集表，用于模型自动迭代。
本发明实施例还提供了一种数据异常检测装置，如图5所示，包括：
获取模块501，用于获取待测对象的当前检测样本、历史细胞状态和历史 隐藏层状态；所述历史细胞状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出细胞状态，所述历史隐藏层状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出隐藏层状态；
确定模块502，用于根据所述当前检测样本，确定所述待测对象的当前特征值；
计算模块503，用于将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中，确定所述待测对象对应的风险指数；
对比模块504，用于将所述风险指数与风险阈值进行对比，确定所述待测对象的异常判定结果。
可选的，所述计算模块503，还用于：
确定所述待测对象对应的当前细胞状态以及当前隐藏层状态；
将所述当前细胞状态以及所述当前隐藏层状态进行存储。
可选的，还包括训练模块505，用于利用以下方式训练异常检测模型：
获取历史时间段内的历史样本数据；
根据所述历史样本数据，选择训练对象的训练特征，并确定训练特征对应的训练特征值；
将所述训练特征值输入初始异常检测模型，并根据输出的风险指数及所述训练对象的异常判定结果计算损失函数，当所述损失函数小于预设阈值时，确定对应的参数为所述异常检测模型对应的参数，得到异常检测模型。
可选的，所述训练模块505，用于：
获取所述待测对象的历史检测样本；
将所述待测对象的历史检测样本输入所述异常检测模型中，确定所述待测对象的备选参数、备选参数的训练效果指标、所述待测对象的输出细胞状态以及输出隐藏层状态；
当所述备选参数的训练效果指标大于存储的训练效果指标时，将所述备 选参数作为所述待测对象的异常检测模型的参数，并将所述待测对象的输出细胞状态作为历史细胞状态，输出隐藏层状态作为历史隐藏层状态。
可选的，所述训练模块505，用于：
确定所述训练对象的核心特征和可选特征，并将所述核心特征作为训练特征；
根据所述历史样本数据，计算每个可选特征的性能值；
根据所述性能值，选择可选特征作为所述训练特征。
可选的，所述异常检测模型为神经网络模型；
所述计算模块503，具体用于：
根据所述当前特征值、所述历史细胞状态和所述历史隐藏层状态，确定所述神经网络模型的当前隐藏层状态；
根据所述当前特征值确定所述神经网络模型的当前细胞的旁路；
根据所述当前细胞的旁路和当前隐藏层状态，确定所述风险指数。
基于相同的原理，本发明还提供一种电子设备，如图6所示，包括：
包括处理器601、存储器602、收发机603、总线接口604，其中处理器601、存储器602与收发机603之间通过总线接口604连接；
所述处理器601，用于读取所述存储器602中的程序，执行下列方法：
获取待测对象的当前检测样本、历史细胞状态和历史隐藏层状态；所述历史细胞状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出细胞状态，所述历史隐藏层状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出隐藏层状态；
根据所述当前检测样本，确定所述待测对象的当前特征值；
将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中，确定所述待测对象对应的风险指数；
将所述风险指数与风险阈值进行对比，确定所述待测对象的异常判定结果。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产 品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本发明的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本发明范围的所有变更和修改。
显然，本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的精神和范围。这样，倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内，则本发明也意图包括这些改动和变型在内。

Claims (14)

  1. 一种数据异常检测方法，其特征在于，包括：
    获取待测对象的当前检测样本、历史细胞状态和历史隐藏层状态；所述历史细胞状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出细胞状态，所述历史隐藏层状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出隐藏层状态；
    根据所述当前检测样本，确定所述待测对象的当前特征值；
    将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中，确定所述待测对象对应的风险指数；
    将所述风险指数与风险阈值进行对比，确定所述待测对象的异常判定结果。
  2. 如权利要求1所述的方法，其特征在于，所述将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中之后，还包括：
    确定所述待测对象对应的当前细胞状态以及当前隐藏层状态；
    将所述当前细胞状态以及所述当前隐藏层状态进行存储。
  3. 如权利要求1所述的方法，其特征在于，所述异常检测模型利用以下方式进行训练：
    获取历史时间段内的历史样本数据；
    根据所述历史样本数据，选择训练对象的训练特征，并确定训练特征对应的训练特征值；
    将所述训练特征值输入初始异常检测模型，并根据输出的风险指数及所述训练对象的异常判定结果计算损失函数，当所述损失函数小于预设阈值时，确定对应的参数为所述异常检测模型对应的参数，得到异常检测模型。
  4. 如权利要求3所述的方法，其特征在于，所述获取待测对象的当前检测样本之前，还包括：
    获取所述待测对象的历史检测样本；
    将所述待测对象的历史检测样本输入所述异常检测模型中，确定所述待测对象的备选参数、备选参数的训练效果指标、所述待测对象的输出细胞状态以及输出隐藏层状态；
    当所述备选参数的训练效果指标大于存储的训练效果指标时，将所述备选参数作为所述待测对象的异常检测模型的参数，并将所述待测对象的输出细胞状态作为历史细胞状态，输出隐藏层状态作为历史隐藏层状态。
  5. 如权利要求3所述的方法，其特征在于，所述根据所述历史样本数据，选择训练对象的训练特征，包括：
    确定所述训练对象的核心特征和可选特征，并将所述核心特征作为训练特征；
    根据所述历史样本数据，计算每个可选特征的性能值；
    根据所述性能值，选择可选特征作为所述训练特征。
  6. 如权利要求1至5任一项所述的方法，其特征在于，所述异常检测模型为神经网络模型；
    所述将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中，确定所述待测对象对应的风险指数，包括：
    根据所述当前特征值、所述历史细胞状态和所述历史隐藏层状态，确定所述神经网络模型的当前隐藏层状态；
    根据所述当前特征值确定所述神经网络模型的当前细胞的旁路；
    根据所述当前细胞的旁路和当前隐藏层状态，确定所述风险指数。
  7. 一种数据异常检测装置，其特征在于，包括：
    获取模块，用于获取待测对象的当前检测样本、历史细胞状态和历史隐藏层状态；所述历史细胞状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出细胞状态，所述历史隐藏层状态为将所述待测对象的历史检测样本输入所述待测对象的异常检测模型确定的输出隐藏 层状态；
    确定模块，用于根据所述当前检测样本，确定所述待测对象的当前特征值；
    计算模块，用于将所述待测对象的所述当前特征值、所述历史细胞状态和所述历史隐藏层状态输入所述异常检测模型中，确定所述待测对象对应的风险指数；
    对比模块，用于将所述风险指数与风险阈值进行对比，确定所述待测对象的异常判定结果。
  8. 如权利要求7所述的装置，其特征在于，所述计算模块，还用于：
    确定所述待测对象对应的当前细胞状态以及当前隐藏层状态；
    将所述当前细胞状态以及所述当前隐藏层状态进行存储。
  9. 如权利要求7所述的装置，其特征在于，还包括训练模块，用于利用以下方式训练异常检测模型：
    获取历史时间段内的历史样本数据；
    根据所述历史样本数据，选择训练对象的训练特征，并确定训练特征对应的训练特征值；
    将所述训练特征值输入初始异常检测模型，并根据输出的风险指数及所述训练对象的异常判定结果计算损失函数，当所述损失函数小于预设阈值时，确定对应的参数为所述异常检测模型对应的参数，得到异常检测模型。
  10. 如权利要求9所述的装置，其特征在于，所述训练模块，用于：
    获取所述待测对象的历史检测样本；
    将所述待测对象的历史检测样本输入所述异常检测模型中，确定所述待测对象的备选参数、备选参数的训练效果指标、所述待测对象的输出细胞状态以及输出隐藏层状态；
    当所述备选参数的训练效果指标大于存储的训练效果指标时，将所述备选参数作为所述待测对象的异常检测模型的参数，并将所述待测对象的输出细胞状态作为历史细胞状态，输出隐藏层状态作为历史隐藏层状态。
  11. 如权利要求9所述的装置，其特征在于，所述训练模块，用于：
    确定所述训练对象的核心特征和可选特征，并将所述核心特征作为训练特征；
    根据所述历史样本数据，计算每个可选特征的性能值；
    根据所述性能值，选择可选特征作为所述训练特征。
  12. 如权利要求7至9任一项所述的装置，其特征在于，所述异常检测模型为神经网络模型；
    所述计算模块，具体用于：
    根据所述当前特征值、所述历史细胞状态和所述历史隐藏层状态，确定所述神经网络模型的当前隐藏层状态；
    根据所述当前特征值确定所述神经网络模型的当前细胞的旁路；
    根据所述当前细胞的旁路和当前隐藏层状态，确定所述风险指数。
  13. 一种电子设备，其特征在于，包括：
    至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1-6任一所述的方法。
  14. 一种非暂态计算机可读存储介质，其特征在于，所述非暂态计算机可读存储介质存储计算机指令，所述计算机指令用于使所述计算机执行权利要求1～6任一所述方法。
PCT/CN2020/118430 2019-11-18 2020-09-28 一种数据异常检测方法及装置 WO2021098384A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911126382.1A CN110874744B (zh) 2019-11-18 2019-11-18 一种数据异常检测方法及装置
CN201911126382.1 2019-11-18

Publications (1)

Publication Number Publication Date
WO2021098384A1 true WO2021098384A1 (zh) 2021-05-27

Family

ID=69717096

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/118430 WO2021098384A1 (zh) 2019-11-18 2020-09-28 一种数据异常检测方法及装置

Country Status (2)

Country Link
CN (1) CN110874744B (zh)
WO (1) WO2021098384A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113743607A (zh) * 2021-09-15 2021-12-03 京东科技信息技术有限公司 异常检测模型的训练方法、异常检测方法及装置
CN117668740A (zh) * 2024-02-02 2024-03-08 浙江晶盛机电股份有限公司 蓝宝石的长速异常检测方法、装置、电子装置和存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110874744B (zh) * 2019-11-18 2022-08-02 中国银联股份有限公司 一种数据异常检测方法及装置
CN112633655A (zh) * 2020-12-14 2021-04-09 中国电力科学研究院有限公司 一种省间现货市场风险预警方法及系统
CN114612887B (zh) * 2021-09-01 2023-01-10 腾讯科技（深圳）有限公司 单据异常检测方法、装置、设备及计算机可读存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107769972A (zh) * 2017-10-25 2018-03-06 武汉大学 一种基于改进的lstm的电力通信网设备故障预测方法
US20180365560A1 (en) * 2017-06-19 2018-12-20 International Business Machines Corporation Context aware sensitive information detection
CN109800858A (zh) * 2018-12-21 2019-05-24 东软集团股份有限公司 数据异常检测方法、装置、可读存储介质及电子设备
CN110414666A (zh) * 2019-05-28 2019-11-05 河海大学 基于改进的lstm网络的泵站机组故障识别方法
CN110441065A (zh) * 2019-07-04 2019-11-12 杭州华电江东热电有限公司 基于lstm的燃气轮机在线检测方法与装置
CN110874744A (zh) * 2019-11-18 2020-03-10 中国银联股份有限公司 一种数据异常检测方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101880907B1 (ko) * 2017-09-22 2018-08-16 펜타시큐리티시스템 주식회사 비정상 세션 감지 방법
CN108197845B (zh) * 2018-02-28 2022-03-18 四川新网银行股份有限公司 一种基于深度学习模型lstm的交易指标异常的监测方法
CN108900546A (zh) * 2018-08-13 2018-11-27 杭州安恒信息技术股份有限公司 基于lstm的时间序列网络异常检测的方法与装置
CN109410036A (zh) * 2018-10-09 2019-03-01 北京芯盾时代科技有限公司 一种欺诈检测模型训练方法和装置及欺诈检测方法和装置
CN109561084B (zh) * 2018-11-20 2020-03-17 四川长虹电器股份有限公司 基于lstm自编码网络的url参数异常值检测方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180365560A1 (en) * 2017-06-19 2018-12-20 International Business Machines Corporation Context aware sensitive information detection
CN107769972A (zh) * 2017-10-25 2018-03-06 武汉大学 一种基于改进的lstm的电力通信网设备故障预测方法
CN109800858A (zh) * 2018-12-21 2019-05-24 东软集团股份有限公司 数据异常检测方法、装置、可读存储介质及电子设备
CN110414666A (zh) * 2019-05-28 2019-11-05 河海大学 基于改进的lstm网络的泵站机组故障识别方法
CN110441065A (zh) * 2019-07-04 2019-11-12 杭州华电江东热电有限公司 基于lstm的燃气轮机在线检测方法与装置
CN110874744A (zh) * 2019-11-18 2020-03-10 中国银联股份有限公司 一种数据异常检测方法及装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113743607A (zh) * 2021-09-15 2021-12-03 京东科技信息技术有限公司 异常检测模型的训练方法、异常检测方法及装置
CN113743607B (zh) * 2021-09-15 2023-12-05 京东科技信息技术有限公司 异常检测模型的训练方法、异常检测方法及装置
CN117668740A (zh) * 2024-02-02 2024-03-08 浙江晶盛机电股份有限公司 蓝宝石的长速异常检测方法、装置、电子装置和存储介质

Also Published As

Publication number Publication date
CN110874744A (zh) 2020-03-10
CN110874744B (zh) 2022-08-02

Similar Documents

Publication Publication Date Title
WO2021098384A1 (zh) 一种数据异常检测方法及装置
US10943186B2 (en) Machine learning model training method and device, and electronic device
CN111178456B (zh) 异常指标检测方法、装置、计算机设备和存储介质
CN107633265B (zh) 用于优化信用评估模型的数据处理方法及装置
JP2018538587A (ja) リスク評価方法およびシステム
US9680830B2 (en) Evaluating security of data access statements
WO2021004324A1 (zh) 资源数据的处理方法、装置、计算机设备和存储介质
WO2021051529A1 (zh) 评估云主机资源的方法、装置、设备及存储介质
CN111327608B (zh) 基于级联深度神经网络的应用层恶意请求检测方法及系统
CN110334879A (zh) 电网母线无功负荷预测方法及装置
CN115378988B (zh) 基于知识图谱的数据访问异常检测及控制方法、装置
CN116307215A (zh) 一种电力系统的负荷预测方法、装置、设备及存储介质
CN116561542B (zh) 模型的优化训练系统、方法以及相关装置
CN113590807A (zh) 一种基于大数据挖掘的科技企业信用评价方法
CN112990583A (zh) 一种数据预测模型的入模特征确定方法及设备
WO2024027071A1 (zh) 一种数据监控方法及系统
CN117081941A (zh) 基于注意力机制的流量预测方法及其装置、电子设备
CN114511022B (zh) 特征筛选、行为识别模型训练、异常行为识别方法及装置
CN115601042A (zh) 信息识别方法、装置、电子设备及存储介质
CN115509784A (zh) 数据库实例的故障检测方法和装置
CN111654853B (zh) 一种基于用户信息的数据分析方法
CN114861800A (zh) 模型训练方法、概率确定方法、装置、设备、介质及产品
CN114187081A (zh) 估值表处理方法、装置、电子设备及计算机可读存储介质
CN111429257A (zh) 一种交易监控方法和装置
Miao et al. An innovative Bayesian sequential censored sampling inspection method and application to test design

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20889476

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20889476

Country of ref document: EP

Kind code of ref document: A1