CN113610190A - Abnormal network behavior mining system based on big data - Google Patents
Abnormal network behavior mining system based on big data Download PDFInfo
- Publication number
- CN113610190A CN113610190A CN202110975586.3A CN202110975586A CN113610190A CN 113610190 A CN113610190 A CN 113610190A CN 202110975586 A CN202110975586 A CN 202110975586A CN 113610190 A CN113610190 A CN 113610190A
- Authority
- CN
- China
- Prior art keywords
- task
- operator
- data
- calculation
- network behavior
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000005065 mining Methods 0.000 title claims abstract description 35
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 29
- 238000004364 calculation method Methods 0.000 claims abstract description 30
- 238000007726 management method Methods 0.000 claims abstract description 12
- 238000013500 data storage Methods 0.000 claims abstract description 4
- 230000001419 dependent effect Effects 0.000 claims abstract description 4
- 238000004458 analytical method Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 10
- 238000000034 method Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000000354 decomposition reaction Methods 0.000 claims description 3
- 230000036541 health Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000007418 data mining Methods 0.000 abstract description 4
- 238000012544 monitoring process Methods 0.000 abstract description 3
- 230000006399 behavior Effects 0.000 description 28
- 206010000117 Abnormal behaviour Diseases 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000012549 training Methods 0.000 description 7
- 230000000007 visual effect Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06F18/24155—Bayesian classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/56—Computer malware detection or handling, e.g. anti-virus arrangements
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Hardware Design (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Virology (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses an abnormal network behavior mining system based on big data, which comprises a basic service layer, a calculation and storage layer and an application layer, wherein the basic service layer is dependent on a Hadoop cluster ecological environment and provides data calculation, data storage and task management capabilities for upper-layer services; the calculation and storage layer is used as the core of the abnormal network behavior mining system, supports the compiling and calculation tasks of the operator model submitted by the application layer, and stores the calculation result into the corresponding database; the application layer supports analysts to write operator models in a graphical page mode through a WEB system, submits tasks to the big data cluster for calculation after compiling is completed, and analyzes task results through data overview. The system can realize the compiling of the task execution flow, the monitoring of the task flow and the preview of the task result in one-stop flow, simplifies the submitting difficulty of the big data mining task, provides user authority hierarchy and better manages the user task and the user task data.
Description
Technical Field
The invention belongs to the technical field of networks, and particularly relates to an abnormal network behavior mining system based on big data.
Background
At present, network attack methods are more and more diversified, and mining models are also continuously complicated. Analysts urgently need to design attack mining models for different attacks independently and find suspicious clues from a large amount of multi-source heterogeneous data, at present, a plurality of relational databases are adopted to store multi-source heterogeneous data accessed from multiple channels, and operators are compiled, compiled and scheduled manually; carrying out data statistics, summarization and display by adopting traditional office and other tools;
however, the traditional relational database has limited storage data size, cannot store massive data, has high requirements on a data structure, and cannot meet analysis operation of each dimension under various requirements because analysis data come from different channels and most of the analysis data are heterogeneous data and the data structure is more diverse; operator compiling, compiling and scheduling are completely manually carried out, so that the working efficiency is low, and the processing capacity is limited; the prior art cannot meet business requirements, and therefore an abnormal network behavior mining system based on big data is provided.
Disclosure of Invention
The invention aims to solve the defects in the prior art, and provides the abnormal network behavior mining system based on the big data.
In order to achieve the purpose, the invention provides the following technical scheme:
the abnormal network behavior mining system based on big data comprises a basic service layer, a calculation and storage layer and an application layer, wherein the basic service layer is dependent on a Hadoop cluster ecological environment and provides data calculation, data storage and task management capabilities for upper-layer services;
the calculation and storage layer is used as the core of the abnormal network behavior mining system, supports an operator model compiling and calculating task submitted by the application layer, and stores a calculation result into a corresponding database;
the application layer supports analysts to compile operator models in a graphical page mode through a WEB system, submits tasks to a big data cluster for calculation after compiling is completed, and supports subsequent business decisions through data overview analysis task results.
Preferably, in the application layer, a specific method for an analyst to write the operator model is as follows:
1) an analyst creates operator configuration at an abnormal network behavior mining platform end according to specific service requirements, writes operator model codes based on the service requirements, submits the codes to a system Server end after the operator configuration is completed, and the Server end issues the complete model codes to a Hadoop cluster for compiling operation and waits for a compiling result to return to a WEB end;
2) after the operator model is successfully compiled, a computing task is established on the abnormal network behavior mining platform, the successfully compiled operator and task attributes are configured and then submitted to a system Server end, and after a computing engine of the Server end carries out preprocessing operations such as task decomposition, data connection and the like, the task is submitted to a Hadoop cluster to wait for a task queue to execute, and the task state is detected at regular time and returned to a WEB end;
3) after the task is executed, the task execution result can be viewed and analyzed through the data preview, and the subsequent decision of the service is guided according to the task execution result.
Preferably, the computation and storage layer adopts Spark as a computation basis of the operator tasks, Spark is a source-opening general-purpose large-scale data processing engine, and the operator tasks can be rapidly submitted to the distributed clusters for computation and processing through the framework.
Preferably, the operator model: mathematically, it can be interpreted as a function space to function space mapping O: x- > X is a processing unit which is a function, input and output are often generated when an operator is used, the operator completes conversion of corresponding data, in the project, the operator task is compiled and submitted through a mining platform, and finally the task is submitted to a big data platform to complete calculation and storage.
Preferably, the operator task is responsible for submitting a certain successfully compiled operator to a certain node of the cluster to complete subsequent calculation and output operations.
Preferably, an operator model is provided with an operator type for dividing the operator into business logics, so that analysts can classify and merge different operators conveniently.
Preferably, the user of the system firstly logs in through the administrator user, and can log in the system through the user name by adding a new role and a user through the user module;
after the login is successful, the operation overview data of the whole system is observed through the home page which is displayed in the form of a dashboard, the relevant technical indexes of the system and the health state of the target cluster.
Preferably, the operator management of the system is specifically as follows: clicking the newly-built operator, and a user can newly add operator codes according to operator constraint requirements and can compile, check, store, delete and update own operators;
the task management specifically comprises the following steps: clicking a created task, checking out an operator which passes compiling according to a system prompt, supporting the configuration of Crontab to carry out timing execution, starting the execution of the task after the task is released, and warehousing a task result, wherein the task can be stored, updated, deleted and released and executed; the task real-time status is supported to be checked, and the task execution result data can be checked by clicking the related task.
The invention has the technical effects and advantages that: compared with the traditional abnormal network behavior mining system, the abnormal network behavior mining system based on the big data supports the storage and retrieval of massive threat metadata by relying on mature and open-source big data related technology; by adopting a stable general basic operator, the platform submits operator tasks on line and compiles the operator tasks to the big data cluster, and a user can simply operate on the web to call the operator; and data, operators, tasks and the like in the analysis platform can be managed and displayed in a unified manner through visual operation, and visual construction and test of various independent professional analysis models and multi-dimensional display statistical analysis summary effect data can be displayed.
Drawings
FIG. 1 is a diagram of the big data-based abnormal network behavior mining system architecture according to the present invention;
FIG. 2 is a flow chart of the calculation process of the distributed cluster for submitting operator tasks.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides an abnormal network behavior mining system based on big data as shown in figures 1-2, which comprises a basic service layer, a calculation and storage layer and an application layer, and is characterized in that: the basic service layer is dependent on a Hadoop cluster ecological environment and provides data calculation, data storage and task management capabilities for upper-layer services;
the calculation and storage layer is used as the core of the abnormal network behavior mining system, supports an operator model compiling and calculating task submitted by the application layer, and stores a calculation result into a corresponding database;
the application layer supports analysts to compile operator models in a graphical page mode through a WEB system, submits tasks to a big data cluster for calculation after compiling is completed, and supports subsequent business decisions through data overview analysis task results;
the operator model is as follows: mathematically, it can be interpreted as a function space to function space mapping O: x- > X, which is a processing unit and often refers to a function, when an operator is used, input and output are often generated, the operator completes conversion of corresponding data, the operator is compiled and submitted to an operator task through a mining platform in the project, and finally the task is submitted to a big data platform to complete calculation and storage, the operator task is responsible for submitting an operator which is successfully compiled to a certain node of a cluster to complete subsequent calculation and output operations, and an operator type for dividing the operator into business logic is arranged in the operator model, so that analysts can classify and merge different operators;
supporting storage and retrieval of massive threat metadata by relying on a mature and open-source big data related technology; by adopting a stable general basic operator, the platform submits operator tasks on line and compiles the operator tasks to the big data cluster, and a user can simply operate on the web to call the operator; data, operators, tasks and the like in the analysis platform can be managed and displayed in a unified manner through visual operation, and visual construction and test of various independent professional analysis models and multi-dimensional display statistical analysis summary effect data are displayed;
in the application layer, the specific method for an analyst to write the operator model comprises the following steps:
1) an analyst creates operator configuration at an abnormal network behavior mining platform end according to specific service requirements, writes operator model codes based on the service requirements, submits the codes to a system Server end after the operator configuration is completed, and the Server end issues the complete model codes to a Hadoop cluster for compiling operation and waits for a compiling result to return to a WEB end;
2) after the operator model is successfully compiled, a computing task is established on the abnormal network behavior mining platform, the successfully compiled operator and task attributes are configured and then submitted to a system Server end, and after a computing engine of the Server end carries out preprocessing operations such as task decomposition, data connection and the like, the task is submitted to a Hadoop cluster to wait for a task queue to execute, and the task state is detected at regular time and returned to a WEB end;
3) after the task execution is finished, the task execution result can be viewed and analyzed through data preview, and the subsequent decision of the service is guided according to the task execution result;
the application layer has: operator configuration and management, task management, preview, registration and login and other functions, and the system has access control and supports multi-user authority distribution and control; the task execution and calculation are completed by matching with a support operator configuration, compiling and releasing to a big data platform, and the task execution state and the final data analysis and statistics summary of each dimension can be checked in a visual mode, so that a disaster recovery system is integrally equipped;
the computing and storage layer adopts Spark as a computing basis of the operator task, Spark is a general large-scale data processing engine for opening sources, and the operator task can be rapidly submitted to the distributed cluster for computing and processing through the framework;
the user of the system firstly logs in through the administrator user, and can log in the system through the user name through newly added roles of the user module and the user;
after the login is successful, displaying the related technical indexes of the system and the health state of the target cluster in a dashboard form through a home page, and observing the overall operation overview data of the system;
the operator management of the system specifically comprises the following steps: clicking the newly-built operator, and a user can newly add operator codes according to operator constraint requirements and can compile, check, store, delete and update own operators;
the task management specifically comprises the following steps: clicking a created task, checking out an operator which passes compiling according to a system prompt, supporting the configuration of Crontab to carry out timing execution, starting the execution of the task after the task is released, and warehousing a task result, wherein the task can be stored, updated, deleted and released and executed; the checking of the real-time state of the task is supported, and the task execution result data can be checked by clicking the related task; the system can realize compiling of a task execution flow, monitoring of the task flow, previewing of a one-stop flow of a task result, simplifying the submitting difficulty of a big data mining task, providing user permission layering and better managing of user tasks and user task data;
optionally, the system mixes a multi-classification naive Bayes algorithm and a two-step screening incremental learning method; firstly, scanning the current network behavior data by using a white list scanning engine to acquire normal behaviors for incremental learning; and obtaining the abnormal behaviors by utilizing the output of the known abnormal behavior feature matching engine. Thus obtaining an original incremental training set DT including abnormal behaviors and normal behaviors, then carrying out two-step screening, adding the two-step screening into the incremental training set, training the existing model, and mixing a multi-classification naive Bayes algorithm:
let X ═ X1, X2......, xk } be the data tuple, which is described by k attributes { a1, a 2...., Ak }; let D be the set of training tuples and associated class labels (training set). Assuming that for a given tuple X with n +1 class attribute values C ═ { C0, C1,. ·, Cn }, naive bayes classification predicts the probability that X belongs to class Ci under the highest probability condition, if and only if P (Ci | X) > P (Cj | X), (0 ≦ j ≦ n, i ≠ j) since it is a fixed constant for all classes, it only needs to determine that P (X | Ci) P (Ci) is the largest according to bayes theorem: in order to predict the class label of X, calculating P (X | Ci) P (Ci) for each class Ci;
the attribute values selected in the network request of the mobile internet industrial control network are independent, so the probability calculation can be carried out based on the independent probability values P (x1| Ci), P (x2| Ci), … and P (xk | Ci) of each attribute: if the malicious behaviors are classified by using a binary classification naive Bayes algorithm, n is equal to 1, the total number of classes is 2, namely the classes only have normal behaviors and abnormal behaviors;
because abnormal behaviors can be caused by various malicious programs and the behaviors are different, a mixed multi-classification naive Bayes algorithm is adopted for analysis;
adding behaviors of different classes of malicious programs into a training set D for multi-class training during modeling; during detection, the detection is carried out according to two categories;
for the n +1 classification set C, C0 is defined as a normal behavior class, C 'is defined as an abnormal behavior classification, and includes n subsets of malicious program behaviors C' ═ C1, C2.
When the network behavior X is classified and detected, for the network behavior X, when the class conditional probability P (C0| X) of the normal behavior class C0 is greater than the maximum value of the class conditional probability of the abnormal behavior class, judging that X is a normal behavior, otherwise, judging that X is an abnormal behavior;
the exception mining system is a big data service system for simplifying data mining capability, and can realize writing of task execution flow, monitoring of task flow, previewing of one-stop flow of task results, simplifying of submitting difficulty of big data mining tasks, providing user permission layering, and better managing of user tasks and user task data.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments or portions thereof without departing from the spirit and scope of the invention.
Claims (8)
1. The abnormal network behavior mining system based on big data comprises a basic service layer, a calculation and storage layer and an application layer, and is characterized in that: the basic service layer is dependent on a Hadoop cluster ecological environment and provides data calculation, data storage and task management capabilities for upper-layer services;
the calculation and storage layer is used as the core of the abnormal network behavior mining system, supports an operator model compiling and calculating task submitted by the application layer, and stores a calculation result into a corresponding database;
the application layer supports analysts to compile operator models in a graphical page mode through a WEB system, submits tasks to a big data cluster for calculation after compiling is completed, and supports subsequent business decisions through data overview analysis task results.
2. The big-data based abnormal network behavior mining system according to claim 1, wherein: in the application layer, the specific method for an analyst to write the operator model comprises the following steps:
1) an analyst creates operator configuration at an abnormal network behavior mining platform end according to specific service requirements, writes operator model codes based on the service requirements, submits the codes to a system Server end after the operator configuration is completed, and the Server end issues the complete model codes to a Hadoop cluster for compiling operation and waits for a compiling result to return to a WEB end;
2) after the operator model is successfully compiled, a computing task is established on the abnormal network behavior mining platform, the successfully compiled operator and task attributes are configured and then submitted to a system Server end, and after a computing engine of the Server end carries out preprocessing operations such as task decomposition, data connection and the like, the task is submitted to a Hadoop cluster to wait for a task queue to execute, and the task state is detected at regular time and returned to a WEB end;
3) after the task is executed, the task execution result can be viewed and analyzed through the data preview, and the subsequent decision of the service is guided according to the task execution result.
3. The big-data based abnormal network behavior mining system according to claim 1, wherein: the computing and storage layer adopts Spark as a computing basis of the operator tasks, Spark is a general large-scale data processing engine for opening sources, and the operator tasks can be rapidly submitted to the distributed clusters for computing and processing through the framework.
4. The big-data based abnormal network behavior mining system according to claim 1, wherein: the operator model is as follows: mathematically, it can be interpreted as a function space to function space mapping O: x- > X is a processing unit which is a function, input and output are often generated when an operator is used, the operator completes conversion of corresponding data, in the project, the operator task is compiled and submitted through a mining platform, and finally the task is submitted to a big data platform to complete calculation and storage.
5. The big-data based abnormal network behavior mining system according to claim 4, wherein: and the operator task is responsible for submitting a certain successfully compiled operator to a certain node of the cluster to complete subsequent calculation and output operation.
6. The big-data based abnormal network behavior mining system according to claim 4, wherein: the operator model is provided with an operator type for dividing the operator into service logics, so that analysts can classify and merge different operators conveniently.
7. The big-data based abnormal network behavior mining system according to claim 1, wherein: the user of the system firstly logs in through the administrator user, and can log in the system through the user name through newly added roles of the user module and the user;
after the login is successful, the operation overview data of the whole system is observed through the home page which is displayed in the form of a dashboard, the relevant technical indexes of the system and the health state of the target cluster.
8. The big-data based abnormal network behavior mining system according to claim 7, wherein: the operator management of the system specifically comprises the following steps: clicking the newly-built operator, and a user can newly add operator codes according to operator constraint requirements and can compile, check, store, delete and update own operators;
the task management specifically comprises the following steps: clicking a created task, checking out an operator which passes compiling according to a system prompt, supporting the configuration of Crontab to carry out timing execution, starting the execution of the task after the task is released, and warehousing a task result, wherein the task can be stored, updated, deleted and released and executed; the task real-time status is supported to be checked, and the task execution result data can be checked by clicking the related task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110975586.3A CN113610190B (en) | 2021-08-24 | 2021-08-24 | Abnormal network behavior mining system based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110975586.3A CN113610190B (en) | 2021-08-24 | 2021-08-24 | Abnormal network behavior mining system based on big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113610190A true CN113610190A (en) | 2021-11-05 |
| CN113610190B CN113610190B (en) | 2024-02-02 |
Family
ID=78341814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110975586.3A Active CN113610190B (en) | 2021-08-24 | 2021-08-24 | Abnormal network behavior mining system based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113610190B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116932335A (en) * | 2023-09-18 | 2023-10-24 | 中国电子科技集团公司第十五研究所 | Task flow monitoring method and system based on network transmission in domestic environment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050102292A1 (en) * | 2000-09-28 | 2005-05-12 | Pablo Tamayo | Enterprise web mining system and method |
| US7328192B1 (en) * | 2002-05-10 | 2008-02-05 | Oracle International Corporation | Asynchronous data mining system for database management system |
| CN106407472A (en) * | 2016-11-01 | 2017-02-15 | 广西电网有限责任公司电力科学研究院 | Visual editing and management system for big data analysis and calculation task of order model |
| CN106844385A (en) * | 2015-12-07 | 2017-06-13 | 北京航天长峰科技工业集团有限公司 | A kind of method of combination Spark technique constructions elastic traffic model |
| CN107526600A (en) * | 2017-09-05 | 2017-12-29 | 成都优易数据有限公司 | A kind of visual numeric simulation analysis platform and its data cleaning method based on hadoop and spark |
| CN112202736A (en) * | 2020-09-15 | 2021-01-08 | 浙江大学 | Industrial control system communication network abnormity classification method based on statistical learning and deep learning |
| CN113225359A (en) * | 2021-07-12 | 2021-08-06 | 深圳市永达电子信息股份有限公司 | Safety flow analysis system based on brain-like calculation |
-
2021
- 2021-08-24 CN CN202110975586.3A patent/CN113610190B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050102292A1 (en) * | 2000-09-28 | 2005-05-12 | Pablo Tamayo | Enterprise web mining system and method |
| US7328192B1 (en) * | 2002-05-10 | 2008-02-05 | Oracle International Corporation | Asynchronous data mining system for database management system |
| CN106844385A (en) * | 2015-12-07 | 2017-06-13 | 北京航天长峰科技工业集团有限公司 | A kind of method of combination Spark technique constructions elastic traffic model |
| CN106407472A (en) * | 2016-11-01 | 2017-02-15 | 广西电网有限责任公司电力科学研究院 | Visual editing and management system for big data analysis and calculation task of order model |
| CN107526600A (en) * | 2017-09-05 | 2017-12-29 | 成都优易数据有限公司 | A kind of visual numeric simulation analysis platform and its data cleaning method based on hadoop and spark |
| CN112202736A (en) * | 2020-09-15 | 2021-01-08 | 浙江大学 | Industrial control system communication network abnormity classification method based on statistical learning and deep learning |
| CN113225359A (en) * | 2021-07-12 | 2021-08-06 | 深圳市永达电子信息股份有限公司 | Safety flow analysis system based on brain-like calculation |
Non-Patent Citations (1)
| Title |
|---|
| 李艳，刘成龙: "基于Hadoop的大数据挖掘系统构建", 信息通信, no. 2019, pages 70 - 71 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116932335A (en) * | 2023-09-18 | 2023-10-24 | 中国电子科技集团公司第十五研究所 | Task flow monitoring method and system based on network transmission in domestic environment |
| CN116932335B (en) * | 2023-09-18 | 2024-02-13 | 中国电子科技集团公司第十五研究所 | Task flow monitoring method and system based on network transmission in domestic environment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113610190B (en) | 2024-02-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11755628B2 (en) | Data relationships storage platform | |
| Zhou et al. | Database meets artificial intelligence: A survey | |
| Abdelhamid et al. | Incremental frequent subgraph mining on large evolving graphs | |
| CN106778253A (en) | Threat context aware information security Initiative Defense model based on big data | |
| US10216782B2 (en) | Processing of updates in a database system using different scenarios | |
| CN111885040A (en) | Distributed network situation perception method, system, server and node equipment | |
| CN105183625A (en) | Log data processing method and apparatus | |
| CN112527774A (en) | Data center building method and system and storage medium | |
| CN115221337A (en) | Data weaving processing method and device, electronic equipment and readable storage medium | |
| CN113610190B (en) | Abnormal network behavior mining system based on big data | |
| CN111813870A (en) | Machine learning algorithm resource sharing method and system based on unified description expression | |
| Huang et al. | Survey on performance optimization for database systems | |
| Chen et al. | Information-based massive data retrieval method based on distributed decision tree algorithm | |
| CN111414355A (en) | Offshore wind farm data monitoring and storing system, method and device | |
| Rodríguez-Mazahua et al. | Active rule base development for dynamic vertical partitioning of multimedia databases | |
| CN111311352A (en) | Goods source matching method, system, equipment and storage medium | |
| Sheikh et al. | Provenance inference techniques: Taxonomy, comparative analysis and design challenges | |
| Shakhovska et al. | Big Data information technology and data space architecture | |
| CN109033196A (en) | A kind of distributed data scheduling system and method | |
| CN113886465A (en) | Big data analysis platform for automobile logistics | |
| CN114218216A (en) | Resource management method, device, equipment and storage medium | |
| Fan et al. | Design and implementation of scientific research big data service platform for experimental data managing | |
| US10909242B2 (en) | System and method for detecting security risks in a computer system | |
| Lin et al. | Approximate processing of massive continuous quantile queries over high-speed data streams | |
| Huang et al. | A web interface for XALT log data analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |