CN111949481A - Anomaly tracking detection system based on micro-service - Google Patents
Anomaly tracking detection system based on micro-service Download PDFInfo
- Publication number
- CN111949481A CN111949481A CN202010798815.4A CN202010798815A CN111949481A CN 111949481 A CN111949481 A CN 111949481A CN 202010798815 A CN202010798815 A CN 202010798815A CN 111949481 A CN111949481 A CN 111949481A
- Authority
- CN
- China
- Prior art keywords
- tracking
- data
- module
- log
- codes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3452—Performance evaluation by statistical analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Computer Hardware Design (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to the technical field of anomaly tracking and detection, in particular to an anomaly tracking detection system based on micro-services, which comprises a tracking code implantation module, a log acquisition module, a log storage module, an anomaly tracking module and an anomaly detection module, wherein the tracking code implantation module selects an AspectJ implantation mechanism according to requirements, directly embeds tracking codes into a target code module, the AspectJ implantation mechanism freely inserts the codes into any position of the tracking codes and automatically implants the codes according to configuration information, can monitor various scattered micro-services, mainly focuses on call requests among the micro-services, and can collect and collect monitoring information in time through tracking on each link. Meanwhile, the abnormity tracking system also can analyze and audit the generated monitoring information, extract the interactive characteristics among the microservices, find abnormity from the monitoring information according to the characteristics and send out an alarm in time.
Description
Technical Field
The invention relates to the technical field of anomaly tracking and detection, in particular to an anomaly tracking and detecting system based on micro-service.
Background
The micro-service architecture emphasizes more thorough componentization and service, the original single application is divided into a series of micro-services which are independently distributed according to services, and each micro-service can be independently deployed and expanded. These characteristics of the microservice architecture make microservice face more severe security issues. When the micro service system provides services to the outside, more interfaces need to be exposed to the outside, so that the possibility of the attack on the micro service is greatly increased. Micro-services are usually designed to be mutually trusted, and if an intruder intrudes into a certain micro-service and completely controls the micro-service, the intruder can send a request to other micro-services to acquire sensitive information, and even attack other micro-services to cause the whole system to be paralyzed.
The monitoring methods used in the current micro-service anomaly tracking mainly include two types: the monitoring method based on the black box and the monitoring method based on the label have the main disadvantages that the monitoring code needs to be implanted, and the black box method needs more detailed log record to achieve higher precision.
Disclosure of Invention
The invention aims to solve the technical problem of how to omit the implantation of monitoring codes and ensure the accuracy of exception tracking and detection.
In order to solve the technical problems, the invention provides an exception tracking detection system based on microservice, which comprises a tracking code implantation module, a log collection module, a log storage module, an exception tracking module and an exception detection module, wherein the tracking code implantation module selects an AspectJ implantation mechanism according to requirements, directly embeds tracking codes into a target code module, and the AspectJ implantation mechanism freely inserts codes into any position of the tracking codes and automatically implants the codes according to configuration information.
Preferably, the log collection module includes: the analysis collector adopts an implantable analysis collector and collects data information;
the analysis collector includes: the data extraction unit extracts data from a data source, the data storage unit and the data transmission unit.
Preferably, after receiving the log data, the log collection module extracts, merges and converts the attributes of the log data, and finally stores the log data in the log storage module.
Preferably, the log storage module acquires the tracking data, extracts the feature rules and filters through the abnormal feature extraction unit, verifies the tracking record of the abnormal tracking module in the extraction process, if the tracking record is missing or tampered, the module sends an alarm and writes corresponding information into the abnormal record database, marks the data as Next, and writes the data back into the log storage module.
Preferably, the abnormal feature extracting unit extracting step includes:
(1) acquiring abnormal information data in the log storage module through scan;
(2) data normalization processing;
(3) extracting characteristic information;
(4) and judging whether the calling relation, the calling sequence and the called role relation are respectively in the characteristic information sets R1, R2 and R3, if one does not exist, executing an alarm prompt and terminating the service process, otherwise, returning to execute the judging relation and tracking the abnormal dynamic state.
Preferably, the extraction method of the abnormal feature extraction unit is as follows: defining each abnormal feature ID, wherein each ID corresponds to an ID generated by each calling request in log tracking, the feature vectors are R1, R2 and R3, and the calling relationship R1, the calling sequence R2 and the calling role R3 of each calling in feature data are extracted.
The invention has the advantages that:
the anomaly tracking system of the microservice application can monitor each dispersed microservice, the monitoring is mainly focused on call requests among each microservice, and because the microservices are dispersed in each virtual machine of each node, a monitoring mechanism can collect and gather monitoring information in time through tracking on each link. Meanwhile, the abnormity tracking system also can analyze and audit the generated monitoring information, extract the interactive characteristics among the micro-services, find abnormity from the monitoring information according to the characteristics, and send out an alarm in time, so that the safety problem under the micro-service architecture is solved.
Drawings
FIG. 1 is a system framework diagram of the present invention;
FIG. 2 is a flow chart of the abnormal feature extraction unit according to the present invention.
Detailed Description
In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further described with the specific embodiments.
Referring to fig. 1-2, an exception tracking detection system based on microservice includes a tracking code implanting module, a log collecting module, a log storing module, an exception tracking module, and an exception detection module, where the tracking code implanting module selects an AspectJ implanting mechanism according to requirements, directly embeds tracking codes into a target code module, and the AspectJ implanting mechanism freely inserts codes into any position of the tracking codes and automatically implants the codes according to configuration information.
As a specific embodiment of the present invention, the log collection module includes: the analysis collector adopts an implantable analysis collector and collects data information;
the analysis collector includes: the data extraction unit extracts data from a data source, the data storage unit and the data transmission unit.
As a specific embodiment of the present invention, after receiving log data, the log collection module extracts, merges and converts each attribute of the log data, and finally stores the log data in the log storage module.
As a specific embodiment of the present invention, the log storage module obtains the trace data, and extracts the feature rules and filters through the abnormal feature extraction unit, and verifies the trace record of the abnormal trace module during the extraction process, and if the trace record is missing or tampered, the module will send an alarm and write the corresponding information into the abnormal record database, mark the data as Next, and write the data back into the log storage module.
As a specific embodiment of the present invention, the abnormal feature extracting unit extracting step includes:
(1) acquiring abnormal information data in the log storage module through scan;
(2) data normalization processing;
(3) extracting characteristic information;
(4) and judging whether the calling relation, the calling sequence and the called role relation are respectively in the characteristic information sets R1, R2 and R3, if one does not exist, executing an alarm prompt and terminating the service process, otherwise, returning to execute the judging relation and tracking the abnormal dynamic state.
As a specific embodiment of the present invention, the extraction method of the abnormal feature extraction unit is: defining each abnormal feature ID, wherein each ID corresponds to an ID generated by each calling request in log tracking, the feature vectors are R1, R2 and R3, and the calling relationship R1, the calling sequence R2 and the calling role R3 of each calling in feature data are extracted.
Claims (6)
1. An anomaly tracking detection system based on micro-services, characterized by: the tracking code implanting module selects an AspectJ implanting mechanism according to requirements, directly embeds tracking codes into a target code module, and the AspectJ implanting mechanism freely inserts the codes into any position of the tracking codes and automatically implants the codes according to configuration information.
2. A microservice-based anomaly tracking detection system according to claim 1, characterised in that: the log collection module comprises: an analysis collector, which adopts an implantable analysis collector and collects data information;
the analysis collector includes: the data extraction unit extracts data from a data source, the data storage unit and the data transmission unit.
3. A microservice-based anomaly tracking detection system according to claim 2, characterised in that: after receiving the log data, the log collection module extracts, combines and converts the attributes of the log data, and finally stores the log data into the log storage module.
4. A microservice-based anomaly tracking detection system according to claim 3, characterised in that: the log storage module acquires tracking data, extracts feature rules and filters through the abnormal feature extraction unit, verifies the tracking record of the abnormal tracking module in the extraction process, if the tracking record is missing or tampered, the module can send an alarm and write corresponding information into the abnormal record database, marks the data as Next and writes the data back into the log storage module.
5. A microservice-based anomaly tracking detection system according to claim 4, characterised in that: the abnormal feature extraction unit extracting step includes:
(1) acquiring abnormal information data in the log storage module through scan;
(2) data normalization processing;
(3) extracting characteristic information;
(4) and judging whether the calling relation, the calling sequence and the called role relation are respectively in the characteristic information sets R1, R2 and R3, if one does not exist, executing an alarm prompt and terminating the service process, otherwise, returning to execute the judging relation and tracking the abnormal dynamic state.
6. A microservice-based anomaly tracking detection system according to claim 5, characterised in that: the extraction method of the abnormal feature extraction unit is as follows: defining each abnormal feature ID, wherein each ID corresponds to an ID generated by each calling request in log tracking, the feature vectors are R1, R2 and R3, and the calling relationship R1, the calling sequence R2 and the calling role R3 of each calling in feature data are extracted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010798815.4A CN111949481A (en) | 2020-08-11 | 2020-08-11 | Anomaly tracking detection system based on micro-service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010798815.4A CN111949481A (en) | 2020-08-11 | 2020-08-11 | Anomaly tracking detection system based on micro-service |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111949481A true CN111949481A (en) | 2020-11-17 |
Family
ID=73331997
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010798815.4A Pending CN111949481A (en) | 2020-08-11 | 2020-08-11 | Anomaly tracking detection system based on micro-service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111949481A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113157478A (en) * | 2021-04-21 | 2021-07-23 | 多点(深圳)数字科技有限公司 | Distributed system configuration data acquisition and service alarm system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150106324A1 (en) * | 2013-10-11 | 2015-04-16 | Accenture Global Services Limited | Contextual graph matching based anomaly detection |
| CN108243238A (en) * | 2016-12-27 | 2018-07-03 | 中国移动通信集团浙江有限公司 | The acquisition method and device of a kind of performance data |
| CN110196790A (en) * | 2018-02-24 | 2019-09-03 | 北京京东尚科信息技术有限公司 | The method and apparatus of abnormal monitoring |
| CN110806959A (en) * | 2019-10-30 | 2020-02-18 | 国云科技股份有限公司 | Method for tracking distributed service chain and aggregating logs |
| CN111427736A (en) * | 2019-01-09 | 2020-07-17 | 北京嘀嘀无限科技发展有限公司 | Log monitoring method, device, equipment and computer readable storage medium |
-
2020
- 2020-08-11 CN CN202010798815.4A patent/CN111949481A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150106324A1 (en) * | 2013-10-11 | 2015-04-16 | Accenture Global Services Limited | Contextual graph matching based anomaly detection |
| CN108243238A (en) * | 2016-12-27 | 2018-07-03 | 中国移动通信集团浙江有限公司 | The acquisition method and device of a kind of performance data |
| CN110196790A (en) * | 2018-02-24 | 2019-09-03 | 北京京东尚科信息技术有限公司 | The method and apparatus of abnormal monitoring |
| CN111427736A (en) * | 2019-01-09 | 2020-07-17 | 北京嘀嘀无限科技发展有限公司 | Log monitoring method, device, equipment and computer readable storage medium |
| CN110806959A (en) * | 2019-10-30 | 2020-02-18 | 国云科技股份有限公司 | Method for tracking distributed service chain and aggregating logs |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113157478A (en) * | 2021-04-21 | 2021-07-23 | 多点(深圳)数字科技有限公司 | Distributed system configuration data acquisition and service alarm system |
| CN113157478B (en) * | 2021-04-21 | 2024-05-10 | 多点(深圳)数字科技有限公司 | Distributed system configured data acquisition and service alarm system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6347374B1 (en) | Event detection | |
| CN105391729A (en) | Web loophole automatic mining method based on fuzzy test | |
| CN108959071B (en) | RASP-based PHP deformation webshell detection method and system | |
| CN104270275A (en) | Auxiliary analysis method for causes of exceptions, server and intelligent equipment | |
| CN102812441A (en) | Automated malware detection and remediation | |
| CN114117311B (en) | Data access risk detection method and device, computer equipment and storage medium | |
| CN105068930A (en) | Intelligent analysis method and device for test generation data of mobile terminal | |
| CN110753038A (en) | Self-adaptive authority control system and method for anomaly detection | |
| CN112612680A (en) | Message warning method, system, computer equipment and storage medium | |
| CN115292163A (en) | Application program detection method and device and computer readable storage medium | |
| CN111949481A (en) | Anomaly tracking detection system based on micro-service | |
| CN111831634A (en) | Cloud platform processing system and method based on big data | |
| CN117235406B (en) | Information content security management and control method and device based on block chain | |
| CN107465652B (en) | Operation behavior detection method, server and system | |
| CN106534162A (en) | Server temperature monitoring system and method based on remote management communication protocol | |
| CN113535823A (en) | Abnormal access behavior detection method and device and electronic equipment | |
| CN115757107A (en) | Embedded point detection method, device, server and storage medium | |
| CN115834188A (en) | Vulnerability scanning monitoring method and system, electronic equipment and storage medium | |
| CN112131110A (en) | Multisource heterogeneous data probe method and device of smart phone system | |
| CN111639249A (en) | Automatic monitoring method, device and equipment for user feedback error reporting | |
| CN114385387B (en) | Access information generation method and device and computer equipment | |
| CN116541448B (en) | Data integration processing method and device based on SaaS | |
| CN109951445A (en) | Network security settlement of insurance claim appraisal procedure and system | |
| CN109450934A (en) | Terminal accesses data exception detection method and system | |
| CN117436073B (en) | Security log alarming method, medium and equipment based on intelligent label |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |