CN117911159A

CN117911159A - Real-time data processing method, device, equipment, storage medium and program product

Info

Publication number: CN117911159A
Application number: CN202410101801.0A
Authority: CN
Inventors: 徐云
Original assignee: China Construction Bank Corp; CCB Finetech Co Ltd
Current assignee: China Construction Bank Corp; CCB Finetech Co Ltd
Priority date: 2024-01-24
Filing date: 2024-01-24
Publication date: 2024-04-19

Abstract

The disclosure provides a real-time data processing method, a device, equipment, a storage medium and a program product, which can be applied to the technical field of data processing. The method comprises the following steps: acquiring transaction data sent from different service systems in real time; inputting transaction data into a pre-constructed rule analysis model, and automatically generating a monitoring rule corresponding to the transaction data, wherein the rule analysis model is trained by sample transaction data and a predictive monitoring rule corresponding to the sample transaction data; monitoring transaction data according to the monitoring rule to obtain a monitoring result corresponding to the transaction data; determining an abnormality level corresponding to the monitoring result under the condition that the monitoring result is abnormal; and processing the transaction data according to the abnormal grade and the monitoring result.

Description

Real-time data processing method, device, equipment, storage medium and program product

Technical Field

The present disclosure relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, a storage medium, and a program product for real-time data processing.

Background

At present, a centralized processing mode is generally adopted by a financial business taking a host as a core, and when one financial business is processed in real time, the existing method is to process transactions in daytime, and account checking can be performed only at the end of the day, so that batch financial account checking is performed.

However, when checking account at the end of day, the batch processing time is long, and when abnormal account occurs, the background batch processing is required to retry, so that account problems cannot be found in time, and the optimal processing time is missed.

Disclosure of Invention

In view of the foregoing, the present disclosure provides a real-time data processing method, apparatus, device, storage medium, and program product.

According to an aspect of the present disclosure, there is provided a real-time data processing method, including: acquiring transaction data sent from different service systems in real time; inputting transaction data into a pre-constructed rule analysis model, and automatically generating a monitoring rule corresponding to the transaction data, wherein the rule analysis model is trained by sample transaction data and a predictive monitoring rule corresponding to the sample transaction data; monitoring transaction data according to the monitoring rule to obtain a monitoring result corresponding to the transaction data; determining an abnormality level corresponding to the monitoring result under the condition that the monitoring result is abnormal; and processing the transaction data according to the abnormal grade and the monitoring result.

According to an embodiment of the present disclosure, inputting transaction data into a pre-constructed rule parsing model, automatically generating a monitoring rule corresponding to the transaction data, includes: acquiring a transaction identifier of transaction data; according to the transaction identification of the transaction data, carrying out field extraction on the transaction data to obtain target field information corresponding to the transaction data; and automatically generating a monitoring rule corresponding to the transaction data according to the target field information.

According to an embodiment of the present disclosure, according to a transaction identifier of transaction data, field extraction is performed on the transaction data to obtain target field information corresponding to the transaction data, including: according to the transaction identification of the transaction data, carrying out field extraction on the transaction data to obtain a plurality of field information corresponding to the transaction data; and determining target field information from the plurality of field information based on the field mapping rule, wherein the target field information comprises business scenario information related to the transaction data, and the business scenario information has a unique corresponding relationship with the transaction identification of the transaction data.

According to an embodiment of the present disclosure, the target field information further includes monitoring period information; according to the monitoring rule, monitoring the transaction data to obtain a monitoring result corresponding to the transaction data, including: calling an automatic monitoring thread, and generating a monitoring task corresponding to the transaction data according to the service scene information and the monitoring time period information; and executing the monitoring task corresponding to the transaction data according to the monitoring rule to obtain a monitoring result corresponding to the transaction data.

According to an embodiment of the present disclosure, according to a monitoring rule, a monitoring task corresponding to transaction data is executed to obtain a monitoring result corresponding to the transaction data, including: under the condition that the monitoring task monitors that field information in transaction data is changed, determining business scenario information related to the changed transaction data according to transaction identification of the transaction data; and according to the business scene information related to the changed transaction data, automatically matching the monitoring rule corresponding to the changed transaction data, and monitoring the transaction data to obtain a monitoring result corresponding to the transaction data.

According to an embodiment of the present disclosure, a rule parsing model is trained from sample transaction data and predictive monitoring rules corresponding to the sample transaction data, comprising: acquiring a plurality of sample transaction data; carrying out data preprocessing on the sample transaction data to obtain preprocessed sample transaction data; inputting the preprocessed sample transaction data into a rule analysis model to obtain a predictive monitoring rule corresponding to the sample transaction data; and carrying out parameter adjustment on the rule analysis model according to the true monitoring rule and the predicted monitoring rule corresponding to the sample transaction data to obtain a trained rule analysis model.

According to an embodiment of the present disclosure, after acquiring transaction data transmitted from different service systems in real time, further includes: screening the transaction data to obtain screened transaction data; and normalizing the screened transaction data to obtain normalized transaction data.

Another aspect of the present disclosure provides a real-time data processing apparatus, comprising: the acquisition module is used for acquiring transaction data sent from different service systems in real time; the generation module is used for inputting the transaction data into a pre-constructed rule analysis model and automatically generating a monitoring rule corresponding to the transaction data, wherein the rule analysis model is trained by sample transaction data and a predictive monitoring rule corresponding to the sample transaction data; the monitoring module is used for monitoring the transaction data according to the monitoring rule to obtain a monitoring result corresponding to the transaction data; the determining module is used for determining an abnormal grade corresponding to the monitoring result under the condition that the monitoring result is abnormal; and the processing module is used for processing the transaction data according to the abnormal grade and the monitoring result.

Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a storage device for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method described above.

Another aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above-described method.

Another aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the above method.

Methods, apparatus, devices, media, and program products for real-time data processing are provided in accordance with the present disclosure. Transaction data sent from different systems are obtained in real time, a rule analysis model is utilized to automatically generate a monitoring rule corresponding to the transaction data, the transaction data is monitored in real time, and abnormal monitoring transaction data is processed in real time. At least partially, the technical problems that when the general method is used for daily final account checking in the prior art, the account problem with specific characteristics can be hardly found out and the optimal treatment time can not be found out in time when the account abnormality occurs are solved, the comprehensive monitoring of transaction data is realized, and the technical effect that the problem of the transaction data can be solved in time when the problem occurs is found out are solved.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be more apparent from the following description of embodiments of the disclosure with reference to the accompanying drawings, in which:

FIG. 1 schematically illustrates an application scenario diagram of a real-time data processing method and apparatus according to an embodiment of the present disclosure;

FIG. 2 schematically illustrates a flow chart of a method of real-time data processing according to an embodiment of the present disclosure;

FIG. 3 schematically illustrates a flow chart of a method of automatically generating monitoring rules in accordance with an embodiment of the present disclosure;

FIG. 4 schematically illustrates a flow chart of a method of obtaining a monitoring result according to an embodiment of the disclosure;

FIG. 5 schematically illustrates a block diagram of a real-time data processing apparatus according to an embodiment of the present disclosure; and

Fig. 6 schematically illustrates a block diagram of an electronic device adapted to implement a real-time data processing method according to an embodiment of the disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a convention should be interpreted in accordance with the meaning of one of skill in the art having generally understood the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

In the technical scheme of the disclosure, the related data (such as including but not limited to personal information of a user) are collected, stored, used, processed, transmitted, provided, disclosed, applied and the like, all conform to the regulations of related laws and regulations, necessary security measures are adopted, and the public welcome is not violated.

The embodiment of the disclosure provides a real-time data processing method, which comprises the following steps: acquiring transaction data sent from different service systems in real time; inputting transaction data into a pre-constructed rule analysis model, and automatically generating a monitoring rule corresponding to the transaction data, wherein the rule analysis model is trained by sample transaction data and a predictive monitoring rule corresponding to the sample transaction data; monitoring transaction data according to the monitoring rule to obtain a monitoring result corresponding to the transaction data; determining an abnormality level corresponding to the monitoring result under the condition that the monitoring result is abnormal; and processing the transaction data according to the abnormal grade and the monitoring result.

Fig. 1 schematically illustrates an application scenario diagram of a real-time data processing method and apparatus according to an embodiment of the present disclosure.

As shown in fig. 1, an application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium used to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 through the network 104 using at least one of the first terminal device 101, the second terminal device 102, the third terminal device 103, to receive or send messages, etc. Various communication client applications, such as a shopping class application, a web browser application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only) may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.

The first terminal device 101, the second terminal device 102, the third terminal device 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.

It should be noted that the real-time data processing method provided by the embodiments of the present disclosure may be generally performed by the server 105. Accordingly, the real-time data processing apparatus provided by the embodiments of the present disclosure may be generally provided in the server 105. The real-time data processing method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105. Accordingly, the real-time data processing apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Fig. 2 schematically illustrates a flow chart of a real-time data processing method according to an embodiment of the present disclosure.

As shown in fig. 2, the method 200 includes operations S210 to S250.

Transaction data transmitted from different business systems is acquired in real time in operation S210.

According to embodiments of the present disclosure, transaction data may be obtained in real-time from various different business systems after the different business systems process transactions during the daytime. The transaction data may include transaction data for cross-system lending transactions, and may include transaction data for financial institution storage transactions. Here, the present disclosure is not particularly limited.

In operation S220, the transaction data is input into a pre-constructed rule analysis model, and a monitoring rule corresponding to the transaction data is automatically generated, wherein the rule analysis model is trained by sample transaction data and a predictive monitoring rule corresponding to the sample transaction data.

According to the embodiment of the invention, the rule analysis model can be a deep learning model and is used for extracting characteristics of transaction data and automatically generating monitoring rules corresponding to the transaction data based on the extracted data characteristics.

According to an embodiment of the invention, the monitoring rules are specific and have different monitoring rules according to different transaction data. The monitoring rule is used for monitoring whether the received transaction data are abnormal or not in real time, and if the received transaction data are abnormal, the transaction data are normal.

According to an embodiment of the present invention, the rule parsing model is trained from sample transaction data and predictive monitoring rules corresponding to the sample transaction data. The transaction data can be processed by using the trained rule analysis model to obtain the monitoring rule corresponding to the transaction data.

In operation S230, the transaction data is monitored according to the monitoring rule, and a monitoring result corresponding to the transaction data is obtained.

In operation S240, if the monitoring result is abnormal, an abnormality level corresponding to the monitoring result is determined.

In operation S250, transaction data is processed according to the abnormality level and the monitoring result.

According to the embodiment of the disclosure, based on the monitoring rule corresponding to the transaction data, the transaction data is monitored and processed, and a monitoring result aiming at the transaction data is obtained. The monitoring result may be indicative of whether the transaction data is normal data. If the monitoring result is normal data, the transaction data is normal.

According to the embodiment of the disclosure, if the monitoring result is abnormal data, the abnormal transaction data is displayed, and the abnormal grade of the abnormal transaction data is determined according to the data characteristics of the transaction data. For example, the abnormality level of the abnormality of the transaction data may be determined based on the priority of the transaction data, the service system to which the transaction data belongs, the amount of money, the type of currency, and the like.

According to an embodiment of the present disclosure, the anomaly level may be classified into A, B, C levels in order from high to low. And correspondingly processing transaction data according to different abnormality grades and abnormality monitoring results.

According to the embodiment of the disclosure, for example, for the abnormal grade corresponding to the monitoring result of the transaction data is grade a, the component blocking can be started, the transaction component is not called any more, so as to avoid reply reduction of the transaction success rate, and when the transaction component is detected to be recovered to be normal, the backlogged transaction is retried by outbound; for the abnormal grade corresponding to the monitoring result of the transaction data as grade B, the transaction flow limiting can be triggered to limit the calling amount of the transaction; and (3) carrying out alarm prompt on the abnormal grade corresponding to the monitoring result of the transaction data as the grade C.

According to the embodiment of the disclosure, the transaction data transmitted from different systems are obtained in real time, the rule analysis model is utilized to automatically generate the monitoring rule corresponding to the transaction data, the transaction data is monitored in real time, and the abnormal monitoring transaction data is processed in real time. At least partially solves the technical problems that when the general method is used for daily final account checking in the prior art, the transaction data with specific characteristics can be hardly found out when the transaction data is abnormal, but the transaction data can not be found out in time, so that the optimal treatment time is missed, realizes the comprehensive monitoring of the transaction data, and can timely solve the technical effects when the problem of the transaction data is found out.

According to the embodiment of the invention, the transaction data can be subjected to desensitization before being subjected to screening processing, so that the privacy and the safety of the transaction data are improved; the transaction data is desensitized and then subjected to a screening process, which may include: and automatically generating data affecting the accuracy of the monitoring rule in the transaction data, and clearing abnormal values of the transaction data to obtain screened transaction data. Normalizing the screened transaction data to unify the data format of the transaction data.

Fig. 3 schematically illustrates a flow chart of a method of automatically generating monitoring rules according to an embodiment of the disclosure.

As shown in fig. 3, the method 300 may include operations S310-S330.

In operation S310, a transaction identification of transaction data is acquired.

In operation S320, according to the transaction identifier of the transaction data, the field extraction is performed on the transaction data, so as to obtain the target field information corresponding to the transaction data.

In operation S330, a monitoring rule corresponding to the transaction data is automatically generated according to the target field information.

According to embodiments of the present disclosure, the transaction identification may be represented as a uniqueness of the type of service to which the piece of transaction data belongs. The transaction data may be field extracted based on a transaction identification of the transaction data.

According to the embodiment of the disclosure, the field extraction can be performed on the transaction data according to the transaction identification of the transaction data, and a plurality of field information contained in the transaction data can be obtained. The plurality of field information is unique to a transaction identification of the transaction data.

According to the embodiment of the disclosure, the obtained plurality of field information and the preset field in the rule analysis model can be matched according to the field mapping rule, and the field information with the matching degree larger than the matching threshold value is used as the target field information. For example, the match threshold may be 95%.

According to embodiments of the present disclosure, the target field information may include business scenario information related to the transaction data. The business scenario information may be a cross-system uneven business scenario, a business scenario in which financial institutions store transactions, a business scenario in which loans are being drawn across systems, etc. Here, the present disclosure is not particularly limited.

According to the embodiment of the disclosure, the business scenario corresponding to the transaction data can be determined according to the target field information, and the monitoring rule corresponding to the transaction data is automatically generated aiming at the specific business scenario.

According to the embodiment of the disclosure, the rule analysis model can be utilized to extract the fields of the transaction data to obtain a plurality of field information, the business scenario information with the unique relation with the transaction identifier is determined from the plurality of field information through the field mapping rule, and the monitoring rule corresponding to the transaction data is automatically generated based on the business scenario information. The method can realize comprehensive monitoring of the transaction data by having targeted monitoring rules on different types of transaction data, so that the problem of the transaction data can be found in time and effectively solved.

Fig. 4 schematically illustrates a flow chart of a method of obtaining a monitoring result according to an embodiment of the disclosure.

As shown in fig. 4, the method 400 may include operations S410-S420.

In operation S410, an automatic monitoring thread is invoked to generate a monitoring task corresponding to the transaction data according to the business scenario information and the monitoring time period information.

In operation S420, according to the monitoring rule, a monitoring task corresponding to the transaction data is executed, and a monitoring result corresponding to the transaction data is obtained.

According to embodiments of the present disclosure, the target field information may further include monitoring period information. And calling a resident automatic monitoring thread, and generating a monitoring task corresponding to the transaction data according to the business scene information and the monitoring time period information corresponding to each transaction data.

According to the embodiment of the disclosure, a monitoring task for monitoring the transaction data can be executed according to the automatically generated monitoring rule, and a monitoring result corresponding to the transaction data is correspondingly obtained, wherein the monitoring result can be abnormal data or normal data.

According to embodiments of the present disclosure, for example, the transaction scenario information of the transaction data may be determined to be cross-system unevenness according to the target field information in the transaction data, and a specified monitoring period is generated to perform a monitoring task of the cross-system unevenness in the specified monitoring period. Monitoring rules for the sum of debit amounts and sum of credit amounts in the transaction data corresponding to the transaction data can be automatically generated according to the business scenario of cross-system unevenness. Monitoring whether the sum of the debit amounts in the transaction data is equal to the sum of the credit amounts by using the monitoring rule, and if so, indicating that the transaction data belongs to normal data; if not, the transaction data is indicated as abnormal data.

According to an embodiment of the present disclosure, according to a monitoring rule, a monitoring task corresponding to transaction data is executed to obtain a monitoring result corresponding to the transaction data, including:

Under the condition that the monitoring task monitors that field information in transaction data is changed, determining business scenario information related to the changed transaction data according to transaction identification of the transaction data; and according to the business scene information related to the changed transaction data, automatically matching the monitoring rule corresponding to the changed transaction data, and monitoring the transaction data to obtain a monitoring result corresponding to the transaction data.

According to embodiments of the present disclosure, the field information change may be an addition of field information or a change of field information. When the automatic monitoring thread monitors that field information in transaction data of the same transaction identifier changes, the changed business scenario information related to the transaction data can be determined according to the transaction identifier.

According to the embodiment of the disclosure, it is noted that the transaction identifier and the service scenario information have a unique correspondence, and the transaction identifier of the transaction data is unchanged, so that the service scenario information related to the transaction data is unchanged. That is, the transaction scenario information related to the transaction data after the modification is the same as the transaction scenario information related to the transaction data before the modification.

According to the embodiment of the disclosure, according to the business scenario information related to the transaction data, the monitoring rule automatically generated by the rule analysis model can be automatically matched, the changed transaction data is monitored, and the monitoring result of the changed transaction data is obtained.

According to embodiments of the present disclosure, the sample transaction data may be historical real-time acquired transaction data. The data preprocessing can comprise desensitizing the sample transaction data to improve the privacy and safety of the data, screening the sample transaction data to screen out the data characteristics affecting the prediction monitoring rule, and cleaning the abnormal values in the sample transaction data, so that the accuracy of model parameter training is ensured, and the accuracy of a prediction result is improved.

According to the embodiment of the disclosure, the preprocessed sample transaction data is input into a rule analysis model, so that a predictive monitoring rule corresponding to the sample transaction data can be obtained. The predictive monitoring rule can be compared with a real monitoring rule, and parameter adjustment is performed on the rule analysis model according to a comparison result. And after multiple rounds of prediction and actual comparison, finding that the comparison result is in a relatively stable numerical value, ending the training of the rule analysis model, and obtaining a trained rule analysis model.

Based on the real-time data processing method, the disclosure also provides a real-time data processing device. The device will be described in detail below in connection with fig. 5.

Fig. 5 schematically illustrates a block diagram of a real-time data processing apparatus according to an embodiment of the present disclosure.

As shown in fig. 5, the apparatus 500 may include: the acquisition module 510, the generation module 520, the monitoring module 530, the determination module 540, and the processing module 550.

The acquiring module 510 is configured to acquire transaction data sent from different service systems in real time.

The generating module 520 is configured to input the transaction data into a pre-constructed rule analysis model, and automatically generate a monitoring rule corresponding to the transaction data, where the rule analysis model is trained by sample transaction data and a predictive monitoring rule corresponding to the sample transaction data.

The monitoring module 530 is configured to monitor the transaction data according to the monitoring rule, and obtain a monitoring result corresponding to the transaction data.

And a determining module 540, configured to determine an abnormality level corresponding to the monitoring result when the monitoring result is abnormal.

And the processing module 550 is used for processing the transaction data according to the abnormality level and the monitoring result.

According to an embodiment of the present disclosure, the generating module 520 may include: the device comprises a first acquisition sub-module, an extraction sub-module and a generation sub-module.

And the first acquisition sub-module is used for acquiring the transaction identification of the transaction data.

And the extraction sub-module is used for carrying out field extraction on the transaction data according to the transaction identification of the transaction data to obtain target field information corresponding to the transaction data.

And the generation sub-module is used for automatically generating the monitoring rule corresponding to the transaction data according to the target field information.

According to an embodiment of the present disclosure, the extraction sub-module may include: an extraction unit and a first determination unit.

And the extraction unit is used for carrying out field extraction on the transaction data according to the transaction identification of the transaction data to obtain a plurality of field information corresponding to the transaction data.

And the first determining unit is used for determining target field information from the plurality of field information based on the field mapping rule, wherein the target field information comprises business scene information related to the transaction data, and the business scene information has a unique corresponding relationship with the transaction identification of the transaction data.

According to an embodiment of the present disclosure, the target field information further includes monitoring period information. The monitoring module 530 may include: the calling sub-module and the executing sub-module.

And the calling sub-module is used for calling the automatic monitoring thread and generating a monitoring task corresponding to the transaction data according to the service scene information and the monitoring time period information.

And the execution sub-module is used for executing the monitoring task corresponding to the transaction data according to the monitoring rule to obtain a monitoring result corresponding to the transaction data.

According to an embodiment of the present disclosure, the execution sub-module may include: a second determining unit and a monitoring unit.

And the second determining unit is used for determining business scenario information related to the changed transaction data according to the transaction identification of the transaction data under the condition that the monitoring task monitors that the field information in the transaction data is changed.

And the monitoring unit is used for automatically matching the monitoring rule corresponding to the changed transaction data according to the business scene information related to the changed transaction data, and monitoring the transaction data to obtain a monitoring result corresponding to the transaction data.

According to an embodiment of the present disclosure, the generating module 520 may further include: the system comprises a second acquisition sub-module, a processing sub-module, an input sub-module and an adjustment sub-module.

And the second acquisition sub-module is used for acquiring a plurality of sample transaction data.

And the processing sub-module is used for carrying out data preprocessing on the sample transaction data to obtain preprocessed sample transaction data.

And the input sub-module is used for inputting the preprocessed sample transaction data into the rule analysis model to obtain a predictive monitoring rule corresponding to the sample transaction data.

And the adjustment sub-module is used for carrying out parameter adjustment on the rule analysis model according to the true monitoring rule and the predicted monitoring rule corresponding to the sample transaction data to obtain a trained rule analysis model.

According to an embodiment of the present disclosure, the apparatus 500 may further include: a screening module and a normalization module.

And the screening module is used for screening the transaction data to obtain screened transaction data.

And the normalization module is used for carrying out normalization processing on the screened transaction data to obtain normalized transaction data.

Any of the acquisition module 510, the generation module 520, the monitoring module 530, the determination module 540, and the processing module 550 may be combined in one module to be implemented, or any of the modules may be split into a plurality of modules, according to an embodiment of the present disclosure. Or at least some of the functionality of one or more of the modules may be combined with, and implemented in, at least some of the functionality of other modules. At least one of the acquisition module 510, the generation module 520, the monitoring module 530, the determination module 540, and the processing module 550 may be implemented, at least in part, as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable manner of integrating or packaging the circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Or at least one of the acquisition module 510, the generation module 520, the monitoring module 530, the determination module 540 and the processing module 550 may be at least partially implemented as a computer program module which, when executed, may perform the respective functions.

It should be noted that, in the embodiment of the present invention, the real-time data processing apparatus portion corresponds to the real-time data processing method portion in the embodiment of the present invention, and the description of the real-time data processing apparatus portion specifically refers to the real-time data processing method portion and is not repeated herein.

As shown in fig. 6, an electronic device 600 according to an embodiment of the present disclosure includes a processor 601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. The processor 601 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. Processor 601 may also include on-board memory for caching purposes. The processor 601 may comprise a single processing unit or a plurality of processing units for performing different actions of the method flows according to embodiments of the disclosure.

In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The processor 601 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 602 and/or the RAM 603. Note that the program may be stored in one or more memories other than the ROM 602 and the RAM 603. The processor 601 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

According to an embodiment of the present disclosure, the electronic device 600 may also include an input/output (I/O) interface 605, the input/output (I/O) interface 605 also being connected to the bus 604. The electronic device 600 may also include one or more of the following components connected to an input/output (I/O) interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to an input/output (I/O) interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.

The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 602 and/or RAM 603 and/or one or more memories other than ROM 602 and RAM 603 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. The program code means for causing a computer system to carry out the method for real-time data processing provided by the embodiments of the present disclosure when the computer program product is run on the computer system.

The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 601. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and/or installed from the removable medium 611. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 601. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure are described above. These examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims

1. A method of real-time data processing, comprising:

acquiring transaction data sent from different service systems in real time;

inputting the transaction data into a pre-constructed rule analysis model, and automatically generating a monitoring rule corresponding to the transaction data, wherein the rule analysis model is trained by sample transaction data and a predictive monitoring rule corresponding to the sample transaction data;

monitoring the transaction data according to the monitoring rule to obtain a monitoring result corresponding to the transaction data;

determining an abnormality level corresponding to the monitoring result under the condition that the monitoring result is abnormal;

And processing the transaction data according to the abnormal grade and the monitoring result.

2. The method of claim 1, wherein the inputting the transaction data into a pre-built rule parsing model automatically generates monitoring rules corresponding to the transaction data, comprising:

acquiring a transaction identifier of the transaction data;

According to the transaction identification of the transaction data, carrying out field extraction on the transaction data to obtain target field information corresponding to the transaction data;

And automatically generating a monitoring rule corresponding to the transaction data according to the target field information.

3. The method of claim 2, wherein the performing field extraction on the transaction data according to the transaction identifier of the transaction data to obtain target field information corresponding to the transaction data includes:

according to the transaction identification of the transaction data, carrying out field extraction on the transaction data to obtain a plurality of field information corresponding to the transaction data;

And determining the target field information from the plurality of field information based on a field mapping rule, wherein the target field information comprises business scenario information related to the transaction data, and the business scenario information has a unique corresponding relationship with a transaction identifier of the transaction data.

4. The method of claim 3, wherein the target field information further comprises monitoring period information;

the step of monitoring the transaction data according to the monitoring rule to obtain a monitoring result corresponding to the transaction data, comprises the following steps:

Calling an automatic monitoring thread, and generating a monitoring task corresponding to the transaction data according to the service scene information and the monitoring time period information;

And executing a monitoring task corresponding to the transaction data according to the monitoring rule to obtain a monitoring result corresponding to the transaction data.

5. The method of claim 4, wherein the performing the monitoring task corresponding to the transaction data according to the monitoring rule to obtain the monitoring result corresponding to the transaction data comprises:

Under the condition that the monitoring task monitors that field information in the transaction data is changed, determining business scenario information related to the changed transaction data according to the transaction identification of the transaction data;

And according to the business scene information related to the changed transaction data, automatically matching the monitoring rule corresponding to the changed transaction data, and monitoring the transaction data to obtain a monitoring result corresponding to the transaction data.

6. The method of claim 1, wherein the rule-parsing model is trained from sample transaction data and predictive monitoring rules corresponding to the sample transaction data, comprising:

Acquiring a plurality of sample transaction data;

performing data preprocessing on the sample transaction data to obtain preprocessed sample transaction data;

Inputting the preprocessed sample transaction data into the rule analysis model to obtain a predictive monitoring rule corresponding to the sample transaction data;

and carrying out parameter adjustment on the rule analysis model according to the true monitoring rule and the predictive monitoring rule corresponding to the sample transaction data to obtain a trained rule analysis model.

7. The method of claim 1, further comprising, after the acquiring in real time the transaction data sent from the different business systems:

Screening the transaction data to obtain screened transaction data;

And normalizing the screened transaction data to obtain normalized transaction data.

8. A real-time data processing apparatus comprising:

The acquisition module is used for acquiring transaction data sent from different service systems in real time;

the generation module is used for inputting the transaction data into a pre-constructed rule analysis model and automatically generating a monitoring rule corresponding to the transaction data, wherein the rule analysis model is trained by sample transaction data and a predictive monitoring rule corresponding to the sample transaction data;

the monitoring module is used for monitoring the transaction data according to the monitoring rule to obtain a monitoring result corresponding to the transaction data;

The determining module is used for determining an abnormal grade corresponding to the monitoring result when the monitoring result is abnormal;

and the processing module is used for processing the transaction data according to the abnormal grade and the monitoring result.

9. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs,

Wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-7.

10. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any of claims 1-7.

11. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.