CN109831478A - Rule-based and model distributed processing intelligent decision system and method in real time - Google Patents
Rule-based and model distributed processing intelligent decision system and method in real time Download PDFInfo
- Publication number
- CN109831478A CN109831478A CN201811556554.4A CN201811556554A CN109831478A CN 109831478 A CN109831478 A CN 109831478A CN 201811556554 A CN201811556554 A CN 201811556554A CN 109831478 A CN109831478 A CN 109831478A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- service
- machine learning
- message middleware
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000010801 machine learning Methods 0.000 claims abstract description 38
- 238000004458 analytical method Methods 0.000 claims abstract description 21
- 238000012544 monitoring process Methods 0.000 claims description 15
- 238000007781 pre-processing Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 238000005192 partition Methods 0.000 claims description 8
- 238000003860 storage Methods 0.000 claims description 7
- 238000013480 data collection Methods 0.000 claims 2
- 238000011161 development Methods 0.000 abstract description 9
- 238000007726 management method Methods 0.000 description 8
- 238000001514 detection method Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The present invention provides rule-based and model distributed processing intelligent decision system and method in real time.The described method includes: reading target service data to be processed and being pre-processed to it；It is loaded as the business rule and machine learning model of the target service data deployment；The target service data are analyzed according to the logical order between the business rule and the machine learning model, and export analysis result.Technical solution of the present invention has preferable complicated event processing capability in real time, and handling capacity is high, is not necessarily to secondary development in face of different business scenarios.
Description
Technical Field
The invention relates to the field of big data real-time calculation, in particular to a rule and model-based distributed real-time processing intelligent decision making system and method suitable for multiple scenes such as financial anti-fraud, IOT (Internet of things) equipment monitoring alarm and the like.
Background
In the big data era, data as a carrier of information has a great deal of value and is the most important production data. With the development of the internet industry, data is generated and explosively increases all the time, and how to timely and effectively process the data and mine valuable information to provide production services becomes important.
In the method of data processing, event processing is a method of monitoring, analyzing information flow, and deriving a decision therefrom. Complex event processing is the analysis of multiple streams of information and the identification of certain complex relationships that exist between particular patterns or events, such as the likelihood of profitability, potential threats, etc., and possibly responses thereto. In reality, there are many similar scenarios, such as:
1. the financial field is as follows: transaction analysis, fraud detection, risk management, etc.;
2. the medical field is as follows: complaint treatment, real-time detection of patient state, and the like;
3. the traffic industry: real-time road conditions and the like;
4. automation of the Internet of things: equipment state anomaly detection, process monitoring and the like.
In the traditional solution, a big data batch computation save-before-compute processing mode can effectively perform data analysis processing and state detection aiming at certain specific service scenes, but is not suitable for scenes with high real-time requirement; on the other hand, the quasi-real-time scheme based on the traditional Spring project combined with the rule engines such as Drools and the like can timely detect events and has low processing delay, but the throughput is not high, the real-time processing capability on massive data is limited, secondary development is needed for different service scenes, and the expansion performance is poor.
Disclosure of Invention
In view of the above drawbacks of the prior art, the present invention provides a rule and model based distributed real-time processing intelligent decision system and method, which are used to solve the technical problems of poor real-time processing capability for complex events, low throughput, secondary development requirement in multiple scenes, and the like in the prior art.
To achieve the above and other related objects, the present invention provides a rule and model based distributed real-time processing intelligent decision system, comprising: the data acquisition module is used for acquiring the service data to be processed and sending the service data to the first message middleware module; the first message middleware module is used for storing the received service data in a partition mode and issuing a message to the core decision module so as to provide target service data required to be analyzed to the core decision module; the service deployment module is used for deploying the business rules and the machine learning models required by analyzing the target business data and the logic sequence between the business rules and the machine learning models; the core decision module is used for reading the target service data from the first message middleware module and preprocessing the target service data; loading the service rule and the machine learning model deployed for the target service data by the service deployment module; analyzing the target business data according to the business rules and the logic sequence between the machine learning models, and sending an analysis result to a second message middleware module; and the second message middleware module is used for storing the analysis result and sending a message to the service party so as to provide the required analysis result for the service party.
In an embodiment of the present invention, the first message middleware module employs a Kafka message middleware cluster; the Kafka message middleware cluster creates specific topics according to different service types, and the partition number contained in each topic depends on a specific scene.
In an embodiment of the present invention, the service deployment module includes: a business rule engine for deploying business rules according to user input; a machine learning model engine to deploy a machine learning model according to user input; the monitoring management module is used for monitoring the resource use condition of the first message middleware module and the running state of the core decision module; and the authority management module is used for controlling the data range which can be accessed by the service party.
In an embodiment of the present invention, the core decision module includes: the stream processing module is used for reading the incremental target service data and carrying out real-time preprocessing on the incremental target service data; analyzing the incremental target service data in real time according to the service rule and the logic sequence between the machine learning models, and sending an analysis result to a second message middleware module; the batch processing module is used for reading the full-scale target service data in batches and preprocessing the full-scale target service data; performing batch analysis on the full-scale target service data according to the service rule and the logic sequence between the machine learning models, and sending an analysis result to a second message middleware module; and the basic service module is used for loading the service rules and the machine learning model deployed for the target service data by the service deployment module, supplying the service rules and the machine learning model to the stream processing module or the batch processing module, and providing system monitoring service and external third-party interface loading service.
In an embodiment of the present invention, the second message middleware module employs: hbase storage, Redis storage, or Kafka message middleware cluster.
In an embodiment of the present invention, the service data includes: transaction data, operational data, server log data, and/or buried point data.
In an embodiment of the present invention, for the buried point data and the server log data, the data acquisition module may collect the data through a flash component, and then send the data to the first message middleware module.
In an embodiment of the present invention, the data acquisition module converts the received service data into a preset data format, and then sends the preset data format to the first message middleware module.
To achieve the above and other related objects, the present invention provides a distributed real-time processing intelligent decision method based on rules and models, comprising: reading target service data to be processed and preprocessing the target service data; loading a service rule and a machine learning model deployed for the target service data; and analyzing the target business data according to the business rules and the logic sequence between the machine learning models, and outputting an analysis result.
To achieve the above and other related objects, the present invention provides an electronic device, comprising: a processor and a memory; wherein the memory is for storing a computer program; the processor is used for loading and executing the computer program so as to enable the electronic equipment to execute the rule and model based distributed real-time processing intelligent decision method.
As described above, the system and method for distributed real-time processing intelligent decision making based on rules and models of the present invention has the following features
Has the advantages that:
1. aiming at various application scenes, the dynamic configuration of the business rules and the machine learning model is automatically realized, the technical personnel do not need to be excessively relied on to carry out secondary development and service upgrade on the original system, and the development period is greatly shortened;
2. complex calculation and distributed calculation can be performed within the second level, and the requirements of various fields on high concurrency and low delay of big data processing are successfully met;
3. the combination of rules and models improves the accuracy of state detection.
Drawings
Fig. 1 is a schematic diagram of an architecture of a rule and model based distributed real-time processing intelligent decision making system according to an embodiment of the present invention.
Fig. 2 is a schematic workflow diagram of a rule and model based distributed real-time processing intelligent decision making system according to an embodiment of the present invention.
FIG. 3 is a flowchart of a method performed by the core decision module according to an embodiment of the invention.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
Referring to fig. 1, the embodiment provides an intelligent decision system based on a Spark Streaming distributed stream computing framework in combination with dynamically configurable business rules and a machine learning model, which can effectively handle the problems of multiple scene events, does not need to excessively depend on technicians to perform secondary development and service upgrade on the original system, greatly shortens the development period, and has many advantages of high throughput, high computing speed, and the like.
As shown in fig. 1, the architecture of the rule and model based distributed real-time processing intelligent decision system of the present embodiment is shown, which has good flexibility, adaptability and expansibility, and mainly includes: the system comprises a data acquisition module, a first message middleware module, a service deployment module, a core decision module and a second message middleware module, wherein a red line represents service data, a green line represents basic service data, and a blue line represents deployment management operation data.
The functions of the respective modules will be described in detail below:
1) data acquisition module
In this embodiment, the Data acquisition module mainly obtains transaction, operation type Data (Business Data), server LOG (LOG), buried Data (SDK), and the like from the Business department. For the service data, the service data is generally directly issued to a first message middleware module according to the processing requirement and a fixed format Json or Avro and the like; and collecting the buried point data and the server log data through a flash component, and then sending the data to the first message middleware module.
2) First message middleware module
In this embodiment, the Kafka message middleware cluster is adopted by the first message middleware module, and the distributed publish/subscribe message system has the characteristics of high throughput rate, data persistence, support of horizontal expansion, distributed consumption, ordered messages of each partition, and the like. The Kafka message middleware cluster creates specific topics according to different services, and the number of partitions contained in each topic depends on a specific scene. For scenes requiring sequential consumption, the number of partitions is set to 1, otherwise multiple partitions can be set for parallel consumption.
3) Service configuration module
In this embodiment, the service configuration module adopts a Web service form, and includes a configurable Drools business Rule engine and a machine learning Model engine (Rule and Model engine), a Monitoring management module (Monitoring), and an authority management module (ACL). Business personnel can deploy business rules and machine learning models required for processing target business data in a dynamic drag-type mode on a web page, and generate and manage rule flows and model flows. In addition, the monitoring management module is used for monitoring the resource use condition of the cluster, the running state of the project and the like; and the authority management module is used for controlling the data range which can be accessed by the service and the like.
4) Core decision module
In this embodiment, the core decision module mainly includes three parts: a stream processing module (spare streaming computer), a Batch processing module (spare Batch computer), and a Base Service module (Base Service).
The stream processing module is mainly used for processing online real-time incremental data, specifically, the stream processing module is used for reading topic data in Kafka in real time, preprocessing the data such as cleaning, analyzing the preprocessed data based on Drools rule streams loaded by the basic service module and a machine learning model algorithm, and sending an analysis result to the second message middleware module. Wherein the third party (3rd service) data relied on is provided by the basic service module. For example: and if the intelligent decision system aims at wind control fraud prevention detection, the third-party service provides blacklist data and the like.
The batch processing module mainly processes the full data of the business history, specifically, it uses Spark Sql engine to periodically perform index calculation based on the full data to generate corresponding business labels, for example: user portrait tagging, and updating machine learning models, etc.
The basic service module is used for providing a load service (Rule and model Loader) of a Drools business Rule and a machine learning model, a system monitoring service (Monitor Agent) and an external third-party interface load service (Access control).
5) Second message middleware module
In this embodiment, the second message middleware module is configured to store an analysis result of the core decision module and provide a service for outputting a message, and mainly employs external storage such as HBase and Redis or a Kafka message middleware cluster.
As shown in fig. 2, the work flow of the intelligent decision system of the embodiment is as follows:
s21: the data producer or the data sender sends data sources, such as: data of buried points, log data and the like are sent to a Kafka message queue appointed by a service party and a corresponding topic;
s22: the Kafka message queue stores and backups the received data;
s23: the business department configures the relevant information of the data source and the data output component on the web, and configures the needed business rules, machine learning models and logic of data processing, such as: the machine learning model depends on the execution result of the business rule, and the logic needing to configure data processing is the business rule- > machine learning model; the core decision module consumes the data in the Kafka message queue and sends the analysis result to a downstream Kafka message queue appointed by the service party;
s24: and the service party acquires the output processing result.
In addition, the business department can monitor the health state of the application in real time through the monitoring system, so that the intelligent decision system can operate stably.
Referring to fig. 3, the method steps performed by the core decision module in the intelligent decision system are shown, including:
s31: reading target service data to be processed and preprocessing the target service data;
s32: deploying and loading a service rule and a machine learning model which are set for processing the target service data;
s33: and analyzing the target business data according to the business rules and the logic sequence between the machine learning models, and outputting an analysis result.
Since the technical principle of the embodiment shown in fig. 3 corresponds to the core decision module of the embodiment shown in fig. 1, repeated description of the same technical details is omitted here.
All or part of the steps for implementing the above method embodiments may be performed by hardware associated with a computer program. Based upon such an understanding, the present invention also provides a computer program product comprising one or more computer instructions. The computer instructions may be stored in a computer readable storage medium. The computer-readable storage medium can be any available medium that a computer can store or a data storage device, such as a server, a data center, etc., that is integrated with one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
Referring to fig. 4, the present embodiment provides an electronic device, which may be a server, a desktop, a laptop, a smart phone, or the like. In detail, the electronic device comprises at least, connected by a bus 41: a memory 42 and a processor 43, wherein the memory 42 is used for storing a computer program, and the processor 43 is used for executing the computer program stored in the memory 42 to execute all or part of the steps in the foregoing method embodiments.
The above-mentioned system bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The communication interface is used for realizing communication between the database access device and other equipment (such as a client, a read-write library and a read-only library). The Memory may include a Random Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
In summary, the rule and model based distributed real-time processing intelligent decision making system and method of the invention have better real-time processing capability of complex events, high throughput, and no need of secondary development in different business scenes. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (10)
1. A distributed real-time processing intelligent decision making system based on rules and models is characterized by comprising the following components:
the data acquisition module is used for acquiring the service data to be processed and sending the service data to the first message middleware module;
the first message middleware module is used for storing the received service data in a partition mode and issuing a message to the core decision module so as to provide target service data required to be analyzed to the core decision module;
the service deployment module is used for deploying the business rules and the machine learning models required by analyzing the target business data and the logic sequence between the business rules and the machine learning models;
the core decision module is used for reading the target service data from the first message middleware module and preprocessing the target service data; loading the service rule and the machine learning model deployed for the target service data by the service deployment module; analyzing the target business data according to the business rules and the logic sequence between the machine learning models, and sending an analysis result to a second message middleware module;
and the second message middleware module is used for storing the analysis result and sending a message to the service party so as to provide the required analysis result for the service party.
2. The system of claim 1, wherein the first message middleware module employs a Kafka message middleware cluster; the Kafka message middleware cluster creates specific topics according to different service types, and the partition number contained in each topic depends on a specific scene.
3. The system of claim 1, wherein the service deployment module comprises:
a business rule engine for deploying business rules according to user input;
a machine learning model engine to deploy a machine learning model according to user input;
the monitoring management module is used for monitoring the resource use condition of the first message middleware module and the running state of the core decision module;
and the authority management module is used for controlling the data range which can be accessed by the service party.
4. The system of claim 1, wherein the core decision module comprises:
the stream processing module is used for reading the incremental target service data and carrying out real-time preprocessing on the incremental target service data; analyzing the incremental target service data in real time according to the service rule and the logic sequence between the machine learning models, and sending an analysis result to a second message middleware module;
the batch processing module is used for reading the full-scale target service data in batches and preprocessing the full-scale target service data; performing batch analysis on the full-scale target service data according to the service rule and the logic sequence between the machine learning models, and sending an analysis result to a second message middleware module;
and the basic service module is used for loading the service rules and the machine learning model deployed for the target service data by the service deployment module, supplying the service rules and the machine learning model to the stream processing module or the batch processing module, and providing system monitoring service and external third-party interface loading service.
5. The system of claim 1, wherein the second message middleware module employs: hbase storage, Redis storage, or Kafka message middleware cluster.
6. The system of claim 1, wherein the traffic data comprises: transaction data, operational data, server log data, and/or buried point data.
7. The system of claim 6, wherein the data collection module collects the landfill data and the server log data through a flash component and then sends the landfill data and the server log data to the first message middleware module.
8. The system of claim 1, wherein the data collection module converts the received service data into a predetermined data format before sending the predetermined data format to the first message middleware module.
9. A distributed real-time processing intelligent decision method based on rules and models is characterized by comprising the following steps:
reading target service data to be processed and preprocessing the target service data;
loading a service rule and a machine learning model deployed for the target service data;
and analyzing the target business data according to the business rules and the logic sequence between the machine learning models, and outputting an analysis result.
10. An electronic device, comprising: a processor and a memory; wherein,
the memory is used for storing a computer program;
the processor is configured to load and execute the computer program to cause the electronic device to execute the rule and model based distributed real-time processing intelligent decision method of claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811556554.4A CN109831478A (en) | 2018-12-19 | 2018-12-19 | Rule-based and model distributed processing intelligent decision system and method in real time |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811556554.4A CN109831478A (en) | 2018-12-19 | 2018-12-19 | Rule-based and model distributed processing intelligent decision system and method in real time |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109831478A true CN109831478A (en) | 2019-05-31 |
Family
ID=66858819
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811556554.4A Pending CN109831478A (en) | 2018-12-19 | 2018-12-19 | Rule-based and model distributed processing intelligent decision system and method in real time |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109831478A (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111555957A (en) * | 2020-03-26 | 2020-08-18 | 孩子王儿童用品股份有限公司 | Kafka-based synchronous message service system and implementation method |
| CN111695064A (en) * | 2020-04-29 | 2020-09-22 | 北京城市网邻信息技术有限公司 | Embedded point loading method and device |
| CN112036577A (en) * | 2020-08-20 | 2020-12-04 | 第四范式（北京）技术有限公司 | Method and device for application machine learning based on data form and electronic equipment |
| CN112104706A (en) * | 2020-08-24 | 2020-12-18 | 中国银联股份有限公司 | Method, device, equipment and storage medium for releasing model in distributed system |
| CN112540893A (en) * | 2020-12-16 | 2021-03-23 | 北京同有飞骥科技股份有限公司 | Performance test method for distributed storage |
| CN112583931A (en) * | 2020-12-25 | 2021-03-30 | 北京百度网讯科技有限公司 | Message processing method, message middleware, electronic device and storage medium |
| CN112685400A (en) * | 2021-01-22 | 2021-04-20 | 浪潮云信息技术股份公司 | Method and system for detecting quality of health medical data based on SDK rule engine |
| CN112698971A (en) * | 2020-12-30 | 2021-04-23 | 平安科技（深圳）有限公司 | Rule engine based parameter conversion method, device, equipment and medium |
| CN112783920A (en) * | 2021-02-05 | 2021-05-11 | 树根互联股份有限公司 | Industrial Internet of things data real-time computing method and system based on data arrangement |
| WO2021088400A1 (en) * | 2019-11-07 | 2021-05-14 | 达而观信息科技（上海）有限公司 | Document review method, apparatus and system, device, and storage medium |
| US20210350262A1 (en) * | 2020-05-08 | 2021-11-11 | Paypal, Inc. | Automated decision platform |
| CN113762688A (en) * | 2021-01-06 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Business analysis system, method and storage medium |
| CN114971527A (en) * | 2022-04-25 | 2022-08-30 | 北京云链金汇数字科技有限公司 | Multi-service scene rule routing strategy management system and method based on Spring |
| CN115035705A (en) * | 2021-03-03 | 2022-09-09 | 上海博泰悦臻网络技术服务有限公司 | Vehicle real-time condition monitoring method and device, electronic equipment and medium |
| CN115269208A (en) * | 2022-09-29 | 2022-11-01 | 北京中科江南信息技术股份有限公司 | Resource processing method and system based on formula configuration |
| CN116185782A (en) * | 2023-04-23 | 2023-05-30 | 智者四海（北京）技术有限公司 | Service monitoring method and device for social software |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103795791A (en) * | 2014-01-22 | 2014-05-14 | 北京交通大学 | Railway disaster prevention safety monitoring system based on wireless sensor network |
| CN105224445A (en) * | 2015-10-28 | 2016-01-06 | 北京汇商融通信息技术有限公司 | Distributed tracking system |
| CN106815338A (en) * | 2016-12-25 | 2017-06-09 | 北京中海投资管理有限公司 | A kind of real-time storage of big data, treatment and inquiry system |
-
2018
- 2018-12-19 CN CN201811556554.4A patent/CN109831478A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103795791A (en) * | 2014-01-22 | 2014-05-14 | 北京交通大学 | Railway disaster prevention safety monitoring system based on wireless sensor network |
| CN105224445A (en) * | 2015-10-28 | 2016-01-06 | 北京汇商融通信息技术有限公司 | Distributed tracking system |
| CN106815338A (en) * | 2016-12-25 | 2017-06-09 | 北京中海投资管理有限公司 | A kind of real-time storage of big data, treatment and inquiry system |
Non-Patent Citations (2)
| Title |
|---|
| 徐东泽: "基于STORM的网络流量实时分析系统的设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑（月刊）》 * |
| 王志泳: "分布式消息系统Kafka的性能建模与优化技术研究与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑（月刊）》 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021088400A1 (en) * | 2019-11-07 | 2021-05-14 | 达而观信息科技（上海）有限公司 | Document review method, apparatus and system, device, and storage medium |
| CN111555957A (en) * | 2020-03-26 | 2020-08-18 | 孩子王儿童用品股份有限公司 | Kafka-based synchronous message service system and implementation method |
| CN111695064A (en) * | 2020-04-29 | 2020-09-22 | 北京城市网邻信息技术有限公司 | Embedded point loading method and device |
| CN111695064B (en) * | 2020-04-29 | 2023-08-18 | 北京城市网邻信息技术有限公司 | Buried point loading method and device |
| WO2021223215A1 (en) * | 2020-05-08 | 2021-11-11 | Paypal, Inc. | Automated decision platform |
| US20210350262A1 (en) * | 2020-05-08 | 2021-11-11 | Paypal, Inc. | Automated decision platform |
| CN112036577A (en) * | 2020-08-20 | 2020-12-04 | 第四范式（北京）技术有限公司 | Method and device for application machine learning based on data form and electronic equipment |
| CN112036577B (en) * | 2020-08-20 | 2024-02-20 | 第四范式（北京）技术有限公司 | Method and device for applying machine learning based on data form and electronic equipment |
| CN112104706A (en) * | 2020-08-24 | 2020-12-18 | 中国银联股份有限公司 | Method, device, equipment and storage medium for releasing model in distributed system |
| CN112104706B (en) * | 2020-08-24 | 2022-12-20 | 中国银联股份有限公司 | Method, device, equipment and storage medium for releasing model in distributed system |
| CN112540893A (en) * | 2020-12-16 | 2021-03-23 | 北京同有飞骥科技股份有限公司 | Performance test method for distributed storage |
| CN112583931A (en) * | 2020-12-25 | 2021-03-30 | 北京百度网讯科技有限公司 | Message processing method, message middleware, electronic device and storage medium |
| CN112698971A (en) * | 2020-12-30 | 2021-04-23 | 平安科技（深圳）有限公司 | Rule engine based parameter conversion method, device, equipment and medium |
| CN113762688A (en) * | 2021-01-06 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Business analysis system, method and storage medium |
| CN112685400A (en) * | 2021-01-22 | 2021-04-20 | 浪潮云信息技术股份公司 | Method and system for detecting quality of health medical data based on SDK rule engine |
| CN112783920A (en) * | 2021-02-05 | 2021-05-11 | 树根互联股份有限公司 | Industrial Internet of things data real-time computing method and system based on data arrangement |
| CN115035705A (en) * | 2021-03-03 | 2022-09-09 | 上海博泰悦臻网络技术服务有限公司 | Vehicle real-time condition monitoring method and device, electronic equipment and medium |
| CN114971527A (en) * | 2022-04-25 | 2022-08-30 | 北京云链金汇数字科技有限公司 | Multi-service scene rule routing strategy management system and method based on Spring |
| CN115269208A (en) * | 2022-09-29 | 2022-11-01 | 北京中科江南信息技术股份有限公司 | Resource processing method and system based on formula configuration |
| CN116185782A (en) * | 2023-04-23 | 2023-05-30 | 智者四海（北京）技术有限公司 | Service monitoring method and device for social software |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109831478A (en) | Rule-based and model distributed processing intelligent decision system and method in real time | |
| CN107577805B (en) | Business service system for log big data analysis | |
| US11086687B2 (en) | Managing resource allocation in a stream processing framework | |
| US10311044B2 (en) | Distributed data variable analysis and hierarchical grouping system | |
| US11755452B2 (en) | Log data collection method based on log data generated by container in application container environment, log data collection device, storage medium, and log data collection system | |
| US9965330B2 (en) | Maintaining throughput of a stream processing framework while increasing processing load | |
| CN103513983B (en) | method and system for predictive alert threshold determination tool | |
| US20180300124A1 (en) | Edge Computing Platform | |
| US20180240062A1 (en) | Collaborative algorithm development, deployment, and tuning platform | |
| Cao et al. | Analytics everywhere: generating insights from the internet of things | |
| CN107103064B (en) | Data statistical method and device | |
| CN105183625A (en) | Log data processing method and apparatus | |
| CN110532152A (en) | A kind of monitoring alarm processing method and system based on Kapacitor computing engines | |
| US20180276508A1 (en) | Automated visual information context and meaning comprehension system | |
| CN109120428B (en) | Method and system for wind control analysis | |
| CN111414376A (en) | Data early warning method and device | |
| CN111240876A (en) | Fault positioning method and device for microservice, storage medium and terminal | |
| US20240220319A1 (en) | Automated visual information context and meaning comprehension system | |
| CN111340240A (en) | Method and device for realizing automatic machine learning | |
| CN115080275A (en) | Twin service assembly based on real-time data model and method thereof | |
| CN116820714A (en) | Scheduling method, device, equipment and storage medium of computing equipment | |
| CN113626869A (en) | Data processing method, system, electronic device and storage medium | |
| CN114756301B (en) | Log processing method, device and system | |
| CN110309206A (en) | Order information acquisition method and system | |
| CN114625763A (en) | Information analysis method and device for database, electronic equipment and readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190531 |