CN107220892B - Intelligent preprocessing tool and method applied to massive P2P network loan financial data - Google Patents

Intelligent preprocessing tool and method applied to massive P2P network loan financial data Download PDF

Info

Publication number
CN107220892B
CN107220892B CN201710392181.0A CN201710392181A CN107220892B CN 107220892 B CN107220892 B CN 107220892B CN 201710392181 A CN201710392181 A CN 201710392181A CN 107220892 B CN107220892 B CN 107220892B
Authority
CN
China
Prior art keywords
data
check
format
operator
preprocessing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710392181.0A
Other languages
Chinese (zh)
Other versions
CN107220892A (en
Inventor
马秀娟
毛洪亮
王秀文
苏沐冉
张露晨
吴震
李焱余
唐积强
徐小磊
李传海
苏志坚
谢铭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Scistor Technologies Co ltd
National Computer Network and Information Security Management Center
Original Assignee
Beijing Scistor Technologies Co ltd
National Computer Network and Information Security Management Center
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Scistor Technologies Co ltd, National Computer Network and Information Security Management Center filed Critical Beijing Scistor Technologies Co ltd
Priority to CN201710392181.0A priority Critical patent/CN107220892B/en
Publication of CN107220892A publication Critical patent/CN107220892A/en
Application granted granted Critical
Publication of CN107220892B publication Critical patent/CN107220892B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/03Credit; Loans; Processing thereof

Landscapes

  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Engineering & Computer Science (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Technology Law (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)

Abstract

The invention discloses an intelligent preprocessing tool and method applied to massive P2P network credit financial data, which can accurately and efficiently preprocess massive P2P network credit financial data before storing the data in a standard financial service database, and ensure the accuracy and the effectiveness of the data, thereby establishing an effective monitoring mechanism of a P2P network credit platform and effectively strengthening the supervision of the P2P network credit platform. The preprocessing comprises establishing validity and integrity check rules to realize data check before warehousing; data classification, data deduplication, data correction, data conversion and state calculation are carried out, and data processing before warehousing is achieved; and finally loading the data into a standard financial service database.

Description

Intelligent preprocessing tool and method applied to massive P2P network loan financial data
Technical Field
The invention relates to an intelligent preprocessing method applied to massive P2P network loan financial data, and belongs to the field of preprocessing methods of massive data.
Background
In recent years, with the rapid development of domestic internet finance, P2P network loan type internet financial platforms are in the endlessly. According to statistics, the established P2P network loan platform reaches more than 5000 families by 2016 and 7 months nationwide, and the historical transaction amount of the P2P network loan industry exceeds 2 trillion.
However, while the development is fast, P2P network lending continuously causes high bad account rate, frequent running and other risk events, and it is estimated that approximately half of the platforms have related problems. Therefore, it is urgently needed to grasp the specific investment and loan conditions of each platform by means of a unified monitoring platform, form bidirectional monitoring of "information flow" and "fund flow", and perform big data analysis on platform transaction data, so as to establish an effective monitoring mechanism of a P2P network loan platform, and effectively strengthen the monitoring of the P2P network loan platform.
Effective monitoring and accurate analysis of real-time transaction data of a large-scale platform are uniformly realized on a single monitoring platform, enterprise data completeness verification is needed, and automatic identification of wrong, lack and other problem data is realized. Therefore, it is important to accurately and efficiently perform the preprocessing.
Disclosure of Invention
The invention aims to solve the problems and provides a tool and a method for intelligently preprocessing massive P2P network credit financial data, and the invention relates to an intelligent preprocessing method for massive P2P network credit financial data, which is a method for realizing the whole process from the intelligent preprocessing process to the storage of a service library for the accessed massive financial data.
The invention discloses a tool and a method for intelligently preprocessing massive P2P network loan financial data, wherein the intelligent preprocessing comprises data classification of various manufacturers, data deduplication, data correction, format processing, data conversion, state calculation and the like. Thereby assisting the user to establish a standard database of standard internet financial transaction data. The method mainly comprises the following steps:
1. establishing various data integrity and validity check rules to verify the data integrity and validity;
2. and (4) pre-treating before warehousing.
The integrity check rule mainly comprises: null check, format check, etc.:
1. checking a null value: checking whether the field is empty;
2. and (3) format checking: the format of the field is checked for compliance with the specification.
The validity check rule comprises value threshold check, date format check, self-definition and the like:
1. threshold check: checking whether the field value exceeds the range;
2. and (3) date format verification: checking whether the format of the date field satisfies a prescribed format;
3. self-defining: defining a new validity check rule and developing an operator.
The warehousing pretreatment comprises: data classification, data deduplication, data correction, data conversion, state calculation, data loading and the like:
1. data classification: classifying and screening the data according to enterprises, and inputting the data into corresponding channels to perform a data preprocessing flow;
2. data deduplication: filtering out repeated data;
3. and (3) data correction: modifying missing, format-wrong, abnormal and other data messages by means of removing, cutting and the like;
4. data conversion: the data types or transaction types of different platforms are not consistent with the central standard, and are converted into standard type data according to a comparison table provided by the platform, such as: the ant golden uniform 1 represents investment data; and the rule 2 in the standard library indicates investment data, in which case data conversion is required for adjustment.
5. And (3) state calculation: the product data (scattered bidding | financing) of the P2P network loan enterprise, the product investment and loan state change (full bidding, running bidding, overdue and repayment) need to be calculated through transaction flow data. Calculating the formula:
(1) full scale: the investment amount is greater than the bid amount;
(2) and (4) flow marking: no deposit record;
(3) overdue: open time +8 days + deadline < current time.
6. Loading data: and loading the data passing the pretreatment and entering a service library.
In order to ensure the high efficiency and expandability of intelligent pretreatment, the invention adopts the following technical means:
1. the data preprocessing adopts a distributed real-time processing framework, and by utilizing the batch consumption characteristic of a distributed message queue, the real-time checking and processing of data integrity and validity are realized by formulating and developing data integrity and validity check rules;
2. loose coupling among the modules is realized through the message queue, and the expandability and the robustness of the system are improved;
3. operator specification and operator extension. Operators are packages developed according to certain specifications to realize certain aspect preprocessing of data. Such as null check operator, date check operator, value range check operator, character string check operator, scrambling code check operator, etc.
In the invention, the integrity and validity check rules can be regarded as operators in the streaming task, and the integrity and validity check can be carried out on fields such as whether the transaction time meets the 'yyyy-MM-dd HH: MM: ss' format, whether the value of a hash state is in the range of 1, 2 and 3, or user information messy codes and the like. Meanwhile, the validity and integrity check of the data can be expanded by developing other check rule operators, and only certain development and packaging specifications need to be followed. And uploading the newly developed operator to a system through an uploading interface, and then using the newly developed operator. In the aspect of operators, the system has the following functions:
(1) presetting common operators, such as null value checking operators, date checking operators and the like;
(2) issuing operator development specifications and packaging specifications;
(3) and the uploading operator adds and expands data preprocessing capacity for the system.
The invention has the advantages that:
1. the tool has the distributed processing characteristic, can perform parallel computation and efficiently process; meanwhile, the system has the system characteristics of fault tolerance, expandability, durability and the like;
2. the tool has strong pertinence, can efficiently carry out intelligent preprocessing on massive P2P network credit financial data, and has wide application prospect.
Drawings
FIG. 1 is a flow of Web loan financial data preprocessing;
FIG. 2 is a real-time processing framework structure of SAMZA + KAFKA;
FIG. 3 is a YARN-based SAMZA resource scheduling flow;
FIG. 4 is a streaming application framework design diagram;
fig. 5 is an application example of the present invention.
Detailed Description
The present invention will be described in further detail below with reference to the accompanying drawings.
The intelligent preprocessing flow of the mass P2P network loan financial data is shown in figure 1. The data is firstly accessed to the message queue system, and during preprocessing, the data is consumed from the message queue in the role of a consumer, and integrity and validity verification is firstly carried out. If the verification fails, transferring to an abnormal data processing sub-process, and returning an error code to the network credit platform; and if the verification is passed, switching to a processing sub-process, carrying out operations such as data classification, data deduplication, data correction, data conversion, state calculation and the like, and finally loading the network credit financial data into an internet financial service library.
The data preprocessing adopts a distributed real-time processing framework. The real-time processing framework using SAMZA + KAFKA was selected by analytical studies analyzing SAMZA + KAFKA, STORM and SPARKSTREAMING. KAFKA, as a distributed message queuing system, has implemented many core infrastructures at the bottom of a streaming framework; while SAMZA, as a distributed streaming data processing framework, naturally integrates KAFKA distributed message queues, the default implementation of which is based on KAFKA.
As shown in fig. 2, the data integrity and validity check rules in the present invention are all used as a JOB in the SAMZA + KAFKA real-time processing framework. SAMZA's JOB, a basic process flow, is that a user task reads data from one or more input streams, after certain processing, outputs to one or more output streams, specifically mapping to KAFKA reads data from one or more TOPICs/PARTITIONs and writes it out to another one or more TOPICs/PARTITIONs; the streaming data processing flow is completed by connecting a plurality of JOBs in series.
This real-time processing mode of SAMZA + KAFKA is actually somewhat like a MapReduce process, where the STREAM input part determines the number of PARTITIONs and tasks (tasks) from TOPIC/PARTITION of KAFKA, similar to a Map process, and the user Task specifies TOPIC and PARTITIONs at the time of output (or the frame automatically determines PARTITIONs from Key), which is equivalent to a Shuffle process, and the next JOB reads a new STREAM, which can be considered as a Reduce or the start of the next Map process. The difference is that the concatenation between the JOBs does not have to wait for the end of the last JOB, and the real-time message distribution mechanism determines that the entire concatenated JOB is uninterrupted, i.e. streamed.
SAMZA uses YARN for resource allocation scheduling (the scheduling module is optional, YARN is used by default) as shown in fig. 3. SAMZA AM is responsible for JOB scheduling, and Task runner is responsible for the operation of user TASK, and by means of the help of KAFKA and YARN, SAMZA can realize the characteristics of distribution/fault tolerance/scalability/persistence and the like.
The tool of the invention enables a user to construct a preprocessing task through a graphical interface. One streaming task is formed by adding one or more preprocessing operators and configuring the operation parameters of each operator. FIG. 4 is a layout diagram of a streaming application framework, the operation flow of which is described as follows:
(1) operator 1 calls: the process data is sent as a producer to topic 1;
(2) operator 2 calls: send operator 3 as data after the consumer gets data processing from topic 1. The other operators and so on.
(3) And (3) abnormal data discovery: and sending the abnormal data to an abnormal topic, processing by an abnormal processing operator, and returning an error code to the network credit platform.
Basic flow of network loan platform data preprocessing:
(1) classifying the basic data according to platforms, and formulating a conversion rule according to the characteristics of each platform;
(2) data deduplication processing is carried out, and the phenomenon that an enterprise repeatedly reports data is avoided;
(3) calculating the data state of the network loan label scattering data, and calculating the actual state of the label scattering according to the dimensions of transaction flow, label base attributes and time;
(4) calculating and extracting final states of the scattered labels;
(5) classifying the network loan label-dispersing industry, setting rules according to the amount range and the field content, and marking an industry use label on the label;
(6) normalizing the transaction types, performing normalized calculation on the transaction states of different enterprises according to a set of established transaction state specifications (see table 1 in detail) with unified standards, and synchronizing the transaction states to a service library;
(7) filtering invalid data, namely deleting and filtering historical test data and data which is overdue and has no complete transaction process;
(8) dictionary data extraction, data reporting batch codes, country area codes, telephone numbers and the like.
1. The tool of the invention establishes validity and integrity check rules aiming at massive P2P network credit financial data, and realizes data check before warehousing; and data classification, data deduplication, data correction, data conversion and state calculation are carried out, and data processing before warehousing is realized.
2. The tool of the invention realizes real-time check and processing of data integrity and validity by formulating and developing data integrity and validity check rules through a distributed real-time processing framework and simultaneously utilizing the batch consumption characteristics of a distributed message queue, thereby ensuring the high efficiency, robustness and expandability of preprocessing.
3. The tool, operator definition and implementation of the invention are a big element for ensuring flexibility and expansibility. Formulating and releasing operator development and packaging specifications, presetting a default operator, and developing and expanding according to the operator specifications; in the invention, the integrity and validity check rules can be regarded as operators in the streaming task, and the distributed task is operated.
4. The tool of the present invention provides a visual modeling tool. The user may construct the pre-processing task through a graphical interface. By adding one or more preprocessing operators and configuring the operating parameters of each operator, each operator completes different preprocessing functions to form a streaming task. In the invention, operators can be dragged to the working area from the operator list in a dragging mode, the operator parameter configuration area is opened at the same time, and the operators are stored as a preprocessing task after being selected and configured, so that better user experience is provided.
5. According to the tool, basic data can be classified according to network credit platforms, and corresponding conversion rules are formulated according to the characteristics of different network credit platforms, so that data from different network credit platforms can be correctly converted into standard data. Meanwhile, repeated data can be screened, the phenomenon that the network credit platform data are repeatedly reported is prevented, and therefore accuracy of later-period data statistics and analysis results is guaranteed.
6. The tool of the invention can complete the calculation of the state of the loan and scatter data: calculating the actual state of the scattered label according to the dimensions of transaction flow, label bottom attributes and time; can complete the classification of the network loan label-dispersing industry: setting rules according to the amount range and the field content, and marking industrial purpose labels on the scattered labels;
application example:
as shown in fig. 5, each P2P web loan platform, such as ant dress, clap loan, yixin, etc., enters the KAFKA message queue through the data access layer. The data is preprocessed through massive P2P network credit financial data provided by the invention, and enters a standard financial service database through data loading.
Table 1-transaction state specification table:
Figure BDA0001307777610000061

Claims (2)

1. an intelligent preprocessing method applied to massive P2P network credit financial data is characterized in that data are accessed into a message queue system to be preprocessed, data are consumed from a message queue in the role of a consumer, integrity and validity verification is carried out, if the verification fails, an abnormal data processing sub-process is carried out, an error code is returned to a network credit platform, if the verification passes, the abnormal data processing sub-process is carried out, preprocessing is carried out, data classification, data deduplication, data correction, data conversion, state calculation and data loading are carried out, and finally the network credit financial data are loaded into an internet financial service library;
the integrity verification function comprises null value check and format check, wherein the null value check refers to checking whether a field is null, and the format check refers to checking whether the format of the field is in accordance with the specification;
the validity verification function comprises threshold value check, date format check and user-defined validity check, wherein the threshold value check is used for checking whether the value of a field exceeds a range, the date format check is used for checking whether the format of the date field meets a specified format, and the user-defined validity check is a user-defined validity check rule;
the pre-treatment before warehousing specifically comprises the following key steps:
(1) data classification: classifying and screening the data according to enterprises, and inputting the data into corresponding channels to perform a data preprocessing flow;
(2) data deduplication: filtering out repeated data;
(3) and (3) data correction: correcting missing, format-incorrect and abnormal data messages;
(4) data conversion: the data types or transaction types of different platforms are not consistent with the central standard, and are converted into standard type data according to a comparison table provided by the platform;
(5) and (3) state calculation: product data of a P2P network loan enterprise, namely, loose bid and financing, and product investment and loan state changes, namely, full bid, tape flow, overdue and repayment, need to be calculated through transaction flow data, and a calculation formula is as follows:
<1> full mark: the investment amount is greater than the bid amount;
<2> stream mark: no deposit record;
<3> overdue: open time +8 days + deadline < current time;
(6) loading data: loading the data passing the pretreatment and entering a service library;
data pre-processing employs a real-time processing framework of SAMZA + KAFKA.
2. An intelligent preprocessing tool applied to massive P2P network credit fused data constructs a task through a graphical interface, a plurality of operators are added, operation parameters of each operator are configured to form a streaming task, data preprocessing is realized through the operators, integrity and validity verification are carried out on the data, if the verification fails, abnormal data processing is carried out, an error code is returned to a network credit platform, if the verification passes, data classification, data deduplication, data correction, data conversion, state calculation and data loading are realized through an algorithm, and finally the network credit fused data are loaded into an Internet financial service library;
the operators comprise an empty check operator, a date check operator, a value range check operator, a character string check operator and a messy code check operator; meanwhile, a user can develop and apply a new operator according to the operator specification;
the integrity verification function comprises null value check and format check, wherein the null value check refers to checking whether a field is null or not; the format check refers to checking whether the format of the field is in a specification;
the validity verification function comprises threshold value check, date format check and user-defined validity check, wherein the threshold value check is to check whether the value of a field exceeds a range; the date format check refers to checking whether the format of the date field meets the specified format; the user-defined validity check refers to a user-defined validity check rule;
the data classification, data deduplication, data correction, data conversion, state calculation and data loading refer to the following steps:
(1) and data classification: classifying and screening the data according to enterprises, and inputting the data into corresponding channels to perform a data preprocessing flow;
(2) data deduplication: filtering out repeated data;
(3) and data correction: correcting missing, format-incorrect and abnormal data messages;
(4) and data conversion: the data types or transaction types of different platforms are not consistent with the central standard, and are converted into standard type data according to a comparison table provided by the platform;
(5) and state calculation: product data of a P2P network loan enterprise, namely, loose bid and financing, and product investment and loan state changes, namely, full bid, tape flow, overdue and repayment, need to be calculated through transaction flow data, and a calculation formula is as follows:
<1> full mark: the investment amount is greater than the bid amount;
<2> stream mark: no deposit record;
<3> overdue: open time +8 days + deadline < current time;
(6) and data loading: loading the data passing the pretreatment and entering a service library;
data pre-processing employs a real-time processing framework of SAMZA + KAFKA.
CN201710392181.0A 2017-05-27 2017-05-27 Intelligent preprocessing tool and method applied to massive P2P network loan financial data Active CN107220892B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710392181.0A CN107220892B (en) 2017-05-27 2017-05-27 Intelligent preprocessing tool and method applied to massive P2P network loan financial data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710392181.0A CN107220892B (en) 2017-05-27 2017-05-27 Intelligent preprocessing tool and method applied to massive P2P network loan financial data

Publications (2)

Publication Number Publication Date
CN107220892A CN107220892A (en) 2017-09-29
CN107220892B true CN107220892B (en) 2020-10-16

Family

ID=59946751

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710392181.0A Active CN107220892B (en) 2017-05-27 2017-05-27 Intelligent preprocessing tool and method applied to massive P2P network loan financial data

Country Status (1)

Country Link
CN (1) CN107220892B (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108154431B (en) * 2018-01-17 2021-07-06 北京网信云服信息科技有限公司 Target recruitment state processing method and device
CN108446973A (en) * 2018-02-28 2018-08-24 四川新网银行股份有限公司 Credit solution on a kind of conventional banking facilities line based on finance opening platform
CN109635162A (en) * 2018-12-18 2019-04-16 北京九章云极科技有限公司 A kind of data processing system and method
CN111724178A (en) * 2019-03-18 2020-09-29 河南省技术产权交易所有限公司 Intellectual property entrusting quotation method, system and computer readable storage medium
CN110020952A (en) * 2019-04-12 2019-07-16 李升东 A kind of finance data processing method and device
CN111047431A (en) * 2019-12-11 2020-04-21 深圳微众信用科技股份有限公司 Credit service processing device, method and equipment based on big data
CN111382579A (en) * 2020-01-13 2020-07-07 中船第九设计研究院工程有限公司 Data preprocessing verification platform of ship pipeline manufacturing execution system
CN112527820B (en) * 2020-12-09 2024-04-09 航天信息股份有限公司广州航天软件分公司 Method and system for uniformly checking various service application data
CN112632169B (en) * 2020-12-29 2023-03-28 永辉云金科技有限公司 Automatic financial data reporting method and device and computer equipment
CN113239188A (en) * 2021-04-21 2021-08-10 上海快确信息科技有限公司 Financial transaction conversation information analysis technical scheme
CN115242349B (en) * 2022-06-21 2023-11-14 苏州盈数智能科技有限公司 Enterprise-level data verification method, enterprise-level data verification device, computer equipment and storage medium
CN115391838B (en) * 2022-10-27 2023-02-28 湖南三湘银行股份有限公司 Data interaction service platform based on trusted prediction machine

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102693508A (en) * 2012-05-11 2012-09-26 杭州商友全球网信息技术有限公司 Data processing method for bank loan application management
CN103605512A (en) * 2013-11-05 2014-02-26 广东电网公司电力科学研究院 System and method for data verification based on GTechnology platform
CN106600369A (en) * 2016-12-09 2017-04-26 广东奡风科技股份有限公司 Real-time recommendation system and method of financial products of banks based on Naive Bayesian classification
EP3166065A1 (en) * 2015-11-04 2017-05-10 Validus Services (Bermuda) Ltd Portfolio optimization and evaluation tool

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9317885B2 (en) * 2011-06-17 2016-04-19 Chicago Mercantile Exchange Inc. Facilitation of payments between counterparties by a central counterparty
CN104463662A (en) * 2014-12-02 2015-03-25 山东中创软件工程股份有限公司 Financial data calculation method and device
CN105512937A (en) * 2015-12-18 2016-04-20 中国建设银行股份有限公司 Payment data processing method and deposit storage system
CN106254543A (en) * 2016-09-27 2016-12-21 盐城工学院 Distributed interconnection Network and Finance Network based on cloud computing framework borrows method and system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102693508A (en) * 2012-05-11 2012-09-26 杭州商友全球网信息技术有限公司 Data processing method for bank loan application management
CN103605512A (en) * 2013-11-05 2014-02-26 广东电网公司电力科学研究院 System and method for data verification based on GTechnology platform
EP3166065A1 (en) * 2015-11-04 2017-05-10 Validus Services (Bermuda) Ltd Portfolio optimization and evaluation tool
CN106600369A (en) * 2016-12-09 2017-04-26 广东奡风科技股份有限公司 Real-time recommendation system and method of financial products of banks based on Naive Bayesian classification

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
大并发、高吞吐量实时数据平台的研究;郑文俊 等;《电信快报》;20161231;第28页第1.2节至第32页第4节,第34页第5节 *

Also Published As

Publication number Publication date
CN107220892A (en) 2017-09-29

Similar Documents

Publication Publication Date Title
CN107220892B (en) Intelligent preprocessing tool and method applied to massive P2P network loan financial data
Yang et al. A system architecture for manufacturing process analysis based on big data and process mining techniques
CN101876938B (en) Message queue-based application software response time measuring method
US10116534B2 (en) Systems and methods for WebSphere MQ performance metrics analysis
CN110502426A (en) The test method and device of distributed data processing system
CN105162628B (en) Quickly determine the system and method for the reasonable alarm threshold of network
CN112395177A (en) Interactive processing method, device and equipment of service data and storage medium
CN111400011B (en) Real-time task scheduling method, system, equipment and readable storage medium
CN116664019B (en) Intelligent gas data timeliness management method, internet of things system, device and medium
CN110262975A (en) Test data management method, device, equipment and computer readable storage medium
CN111800292A (en) Early warning method and device based on historical flow, computer equipment and storage medium
CN111913824A (en) Method for determining data link fault reason and related equipment
CN106156170B (en) The analysis of public opinion method and device
US11016736B2 (en) Constraint programming using block-based workflows
CN102314631A (en) Event processing device of manufacturing execution system
CN110347741B (en) System for effectively improving output result data quality in big data processing process and control method thereof
CN101741624B (en) Internet composite service performance fault-tolerant system
CN114358910A (en) Abnormal financial data processing method, device, equipment and storage medium
CN114661571A (en) Model evaluation method, model evaluation device, electronic equipment and storage medium
US20200019910A1 (en) Block-based prediction for manufacturing environments
Dąbrowski et al. Manufacturing Line-Level Root Cause Analysis and Bottleneck Detection Using the Digital Shadow Concept and Cloud Computing
CN111737242A (en) Method for monitoring mass data processing process
WO2014184263A1 (en) Integration platform monitoring
CN113570333B (en) Process design method suitable for integration
CN116450305B (en) SOAR platform assembly execution method and device based on distributed task scheduling

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant