CN109086605A - A kind of distributed data processing method - Google Patents
A kind of distributed data processing method Download PDFInfo
- Publication number
- CN109086605A CN109086605A CN201810831477.2A CN201810831477A CN109086605A CN 109086605 A CN109086605 A CN 109086605A CN 201810831477 A CN201810831477 A CN 201810831477A CN 109086605 A CN109086605 A CN 109086605A
- Authority
- CN
- China
- Prior art keywords
- event
- data
- access interface
- view identifier
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/56—Computer malware detection or handling, e.g. anti-virus arrangements
Abstract
The invention discloses a kind of distributed data processing methods, comprising: receives the machine data for indicating the event occurred on the computer network；Event category belonging to identification events；With based on the event category identified, the primitive event data of annotation event are to include view identifier, view identifier enables downstream entities to pass through specific information or view identifier of the access interface reception about event by being identified by given view identifier, wherein, access interface is based on event data and extracts specific information and/or generation specific information from specific information.
Description
Technical field
The present invention relates to technical field of data processing, in particular to a kind of distributed data processing method.
Background technique
Friendly and malice activity detection is always the matter of priority of computer network management person for a long time.Known
In both privately and publicly owned's computer network, user's use such as desktop computer, laptop computer, tablet computer, smart phone,
The equipment such as browser are interacted with other people by the computer for being coupled to network and server.Usually with Data Packet Forms
Numerical data is transmitted by the network equipment interconnected along network.
However, regrettably, rogue activity may software to network or hardware or its user damage.Rogue activity
May include unwarranted access or then to Internet resources and data do not allow using.Network administrator attempts to detect this
Class activity, for example, pass through the mode of search abnormal behaviour or otherwise change the expection use pattern of special entity, such as
Node etc. in tissue or its subset, personal user, IP address, node or group network
Internet security is provided using safety equipment in known systems.Device, method be related in a network one or
Multiple positions are installed by safety equipment (being usually the server or computer for being configured to provide for safety).After installation, equipment meeting
Monitor the flow of traverses network.The function that equipment provides may include malware detection, intrusion detection, unauthorized access or number
According to unauthorized use.Regrettably, safety equipment can not easy expansion to handle the temporarily or permanently increase of network flow.
Increased network flow usually requires security provisions quotient and executes devices exchange or same time-consuming device upgrade.Equipment often only has
There is limited network visibility, because they are typically configured as the data that monitoring traversal only installs the link of relevant device.This
The equipment of sample will be unaware that the activity occurred on other network segments monitored by other equipment, thus cannot use in other nets
The related additional contextual information of activity that occurs in section detects the malice being likely difficult to from the ingehious design purely detected
Software.Localization information.
The software product of installation rather than safety that secure hardware device is data network yet another method is provided that.This
A little products, such as anti-virus or anti-malware are typically mounted on terminal device (such as desktop computer and laptop, plate
Computer or smart phone) on.Pass through the data of network between terminal devices by the monitoring of the product of installation, with detect inbound or
Malware in outbound data.Regrettably, the software product of installation also shows in terms of scalability or network visibility
It is bad.The product of installation tends to be locally located on the terminal device, therefore also tends to the quite sheet with data on network
The view on ground.They also tend to be mounted on the hardware that can not easily upgrade.
Summary of the invention
The invention proposes a kind of distributed data processing method methods, comprising:
Receive the machine data for indicating the event occurred on the computer network；
Event category belonging to identification events；With
Based on the event category identified, the primitive event data of annotation event are to include view identifier, view identification
Symbol enables downstream entities to receive by the access interface by being identified about the specific of event by given view identifier
Information.View identifier,
Wherein, access interface is based on event data and extracts specific information and/or generation specific information from specific information.
The method, wherein the access interface further includes executing to move for the event data to the expression event
Make to generate the logic of the specific information.
The method, wherein the access interface further include: logic can be used to the thing for indicating the event
Number of packages is acted according to execution to generate the specific information, and wherein, the access interface further includes for controlling the logic
The input of the movement of execution.
The method, which is characterized in that the specific information only includes the complete event data set for indicating the event
Definition subset.
The method, further includes:
The addition step is repeated, for indicating the event data of each of multiple events, to pass through access interface,
To access the unified approach of the information about multiple events.
The method, further includes:
Repetition indicates the addition step for indicating the event data of each of multiple events, to pass through access
Interface realizes unified approach of the access about the information of multiple events, wherein indicating that the event data of multiple events is different number
According to format.
The method, further includes:
The addition step of the event data is repeated, the event data expression belongs to the multiple of classification identical with event
Each of event, with the unified approach by access interface, with access about the information of multiple events.
The method, wherein identifying that the event category includes:
Event category is determined from multiple event categories based on the content of event data, wherein view identifier and access interface
Corresponding to event category.
The method, wherein identifying that the event category includes:
Event category is determined from multiple event categories based on the type of the machine of the event of generation, wherein view identifier and visit
Ask that interface corresponds to event category.
The method, wherein view identifier and access interface correspond to event category belonging to event.
The method, wherein identifying that the event category includes:
The type of machine based on the event of generation, determines event category belonging to event from multiple event categories, wherein regarding
Graph identifier and access interface correspond to event category, and event category characterizes the event generated by event.A plurality of types of machines.
The method, further includes: event data is parsed based on tentation data format.
The method, further includes: the event data is parsed based on tentation data format, the tentation data format refers to
Which data in the fixed event indicate key or value.
The method, wherein the view identifier is added to the field in the event.
The method, further includes:
Multiple attributes based on event data identification events,
Wherein, specific information includes the definition subset of multiple attributes of event.
The method, further includes:
Multiple attributes based on event data identification events,
Wherein specific information includes the definition subset of multiple attributes of event,
Wherein, the attribute includes at least one of the following: key, value or key-value pair.
The method, further includes:
Downstream entities use the information about multiple events to analyze the event data for indicating multiple events.
The method, further includes:
Downstream entities use the information about multiple events to analyze the event data for indicating multiple events,
It wherein, include at least one of the following by the addressable information of access interface: (1) by being included in access interface
In the information that generates of logic, which can operate with to indicating that the event data of multiple events executes movement, or (2) make a reservation for
The subset of a whole set of event data of the multiple events of expression of justice.
The method, further includes:
Determine whether event is related to downstream entities using view identifier.
The method, further includes:
Automatically it routes information to and specifies the downstream entities of view identifier.
The method, further includes:
Automatically it routes information to and specifies the downstream entities of view identifier,
Wherein, downstream entities are configured as identifying the exception towards safety indicated by event.
The method, wherein the event data includes machine data.
The method, wherein the event data includes the machine data with timestamp.
The method, further includes:
The addition step is configured by adjusting configuration file.
The method, wherein the method as in distributed event processing system or abnormality detection system at least
One extraction-transformation-load phase a part executes.
The method, wherein the downstream entities include Complex event processing engine.
Detailed description of the invention
From following description with reference to the accompanying drawings it will be further appreciated that the present invention.Component in figure is not drawn necessarily to scale,
But it focuses on and shows in the principle of embodiment.In the figure in different views, identical appended drawing reference is specified to be corresponded to
Part.
Fig. 1 is the schematic diagram of distributed data processing method of the invention.
Specific embodiment
In order to enable the objectives, technical solutions, and advantages of the present invention are more clearly understood, below in conjunction with embodiment, to this
Invention is further elaborated；It should be appreciated that described herein, the specific embodiments are only for explaining the present invention, and does not have to
It is of the invention in limiting.To those skilled in the art, after access is described in detail below, other systems of the present embodiment
System, method and/or feature will become obvious.All such additional systems, method, feature and advantage are intended to be included in
It in this specification, is included within the scope of the invention, and by the protection of the appended claims.In description described in detail below
The other feature of the disclosed embodiments, and these characteristic roots will be apparent according to described in detail below.
Embodiment one:
As shown in Figure 1, being the schematic diagram of distributed data processing method of the invention, comprising:
Receive the machine data for indicating the event occurred on the computer network；
Event category belonging to identification events；With
Based on the event category identified, the primitive event data of annotation event are to include view identifier, view identification
Symbol enables downstream entities to receive by the access interface by being identified about the specific of event by given view identifier
Information.View identifier,
Wherein, access interface is based on event data and extracts specific information and/or generation specific information from specific information.
The method, wherein the access interface further includes executing to move for the event data to the expression event
Make to generate the logic of the specific information.
The method, wherein the access interface further include: logic can be used to the thing for indicating the event
Number of packages is acted according to execution to generate the specific information, and wherein, the access interface further includes for controlling the logic
The input of the movement of execution.
The method, which is characterized in that the specific information only includes the complete event data set for indicating the event
Definition subset.
The method, further includes:
The addition step is repeated, for indicating the event data of each of multiple events, to pass through access interface,
To access the unified approach of the information about multiple events.
The method, further includes:
Repetition indicates the addition step for indicating the event data of each of multiple events, to pass through access
Interface realizes unified approach of the access about the information of multiple events, wherein indicating that the event data of multiple events is different number
According to format.
The method, further includes:
The addition step of the event data is repeated, the event data expression belongs to the multiple of classification identical with event
Each of event, with the unified approach by access interface, with access about the information of multiple events.
The method, wherein identifying that the event category includes:
Event category is determined from multiple event categories based on the content of event data, wherein view identifier and access interface
Corresponding to event category.
The method, wherein identifying that the event category includes:
Event category is determined from multiple event categories based on the type of the machine of the event of generation, wherein view identifier and visit
Ask that interface corresponds to event category.
The method, wherein view identifier and access interface correspond to event category belonging to event.
The method, wherein identifying that the event category includes:
The type of machine based on the event of generation, determines event category belonging to event from multiple event categories, wherein regarding
Graph identifier and access interface correspond to event category, and event category characterizes the event generated by event.A plurality of types of machines.
The method, further includes: event data is parsed based on tentation data format.
The method, further includes: the event data is parsed based on tentation data format, the tentation data format refers to
Which data in the fixed event indicate key or value.
The method, wherein the view identifier is added to the field in the event.
The method, further includes:
Multiple attributes based on event data identification events,
Wherein, specific information includes the definition subset of multiple attributes of event.
The method, further includes:
Multiple attributes based on event data identification events,
Wherein specific information includes the definition subset of multiple attributes of event,
Wherein, the attribute includes at least one of the following: key, value or key-value pair.
The method, further includes:
Downstream entities use the information about multiple events to analyze the event data for indicating multiple events.
The method, further includes:
Downstream entities use the information about multiple events to analyze the event data for indicating multiple events,
It wherein, include at least one of the following by the addressable information of access interface: (1) by being included in access interface
In the information that generates of logic, which can operate with to indicating that the event data of multiple events executes movement, or (2) make a reservation for
The subset of a whole set of event data of the multiple events of expression of justice.
The method, further includes:
Determine whether event is related to downstream entities using view identifier.
The method, further includes:
Automatically it routes information to and specifies the downstream entities of view identifier.
The method, further includes:
Automatically it routes information to and specifies the downstream entities of view identifier,
Wherein, downstream entities are configured as identifying the exception towards safety indicated by event.
The method, wherein the event data includes machine data.
The method, wherein the event data includes the machine data with timestamp.
The method, further includes:
The addition step is configured by adjusting configuration file.
The method, wherein the method as in distributed event processing system or abnormality detection system at least
One extraction-transformation-load phase a part executes.
The method, wherein the downstream entities include Complex event processing engine.
Embodiment two:
A kind of computer system, comprising:
Communication equipment；With
Processor is configured that
The machine data for indicating the event occurred on the computer network is received by communication equipment；
Event category belonging to identification events；With
Based on the event category identified, the primitive event data of annotation event are to include view identifier, view identification
Symbol enables downstream entities to receive by the access interface identified by the identifier about the thing by given view identifier
The specific information of part.View identifier,
Wherein, access interface is based on event data and extracts specific information and/or generation specific information from specific information.
The computer system, wherein the access interface further includes the thing that can be used to the event is indicated
Number of packages is acted according to execution to generate the logic of the specific information.
The computer system, wherein the processor is also configured to
Repetition is expressed as indicating the addition step of the event data of each of multiple events, to pass through access interface,
To access the unified approach of the information about multiple events.
Embodiment three:
A kind of non-transitory machinable medium for processing system, the non-transitory machine readable storage are situated between
Matter store instruction, execution of the described instruction in the processing system is so that the processing system executes the behaviour including following operation
Make:
Receive the machine data for indicating the event occurred on the computer network；
Event category belonging to identification events；With
Based on the event category identified, the primitive event data of annotation event are to include view identifier, view identification
Symbol enables downstream entities to receive by the access interface by being identified about the specific of event by given view identifier
Information.View identifier,
Wherein, access interface is based on event data and extracts specific information and/or generation specific information from specific information.
Although describing the present invention by reference to various embodiments above, but it is to be understood that of the invention not departing from
In the case where range, many changes and modifications can be carried out.That is methods discussed above, system or equipment etc. show
Example.Various configurations can be omitted suitably, replace or add various processes or component.For example, in alternative configuration, can with
Described order in a different order executes method, and/or can add, and omits and/or combine the various stages.Moreover, about
The feature of certain configuration descriptions can be combined with various other configurations.Can combine in a similar way configuration different aspect and
Element.In addition, many elements are only range of the example without limiting the disclosure or claims with the development of technology.
Give detail in the description to provide to the thorough understanding for including the exemplary configuration realized.However,
Configuration can be practiced without these specific details for example, having been illustrated with well-known circuit, process, calculation
Method, structure and technology are without unnecessary details, to avoid fuzzy configuration.The description only provides example arrangement, and unlimited
The scope of the claims processed, applicability or configuration.It is used on the contrary, front will provide the description of configuration for those skilled in the art
Realize the enabled description of described technology.It, can be to the function of element without departing from the spirit or the scope of the present disclosure
It can and arrange and carry out various changes.
In addition, many operations can be in parallel or concurrently although each operation can describe the operations as sequential process
It executes.Furthermore it is possible to rearrange the sequence of operation.One process may have other steps.Furthermore, it is possible to pass through hardware, soft
Part, firmware, middleware, code, hardware description language or any combination thereof carry out the example of implementation method.When software, firmware, in
Between when realizing in part or code, program code or code segment for executing necessary task can store in such as storage medium
In non-transitory computer-readable medium, and described task is executed by processor.
To sum up, be intended to foregoing detailed description be considered as it is illustrative and not restrictive, and it is to be understood that described
Claim (including all equivalents) is intended to limit the spirit and scope of the present invention.The above embodiment is interpreted as only using
In illustrating the present invention rather than limit the scope of the invention.After the content for having read record of the invention, technology
Personnel can make various changes or modifications the present invention, these equivalence changes and modification equally fall into the claims in the present invention and limited
Fixed range.
Claims (10)
1. a kind of distributed data processing method characterized by comprising
Receive the machine data for indicating the event occurred on the computer network；
Event category belonging to identification events；With
Based on the event category identified, to include view identifier, view identifier is logical for the primitive event data of annotation event
Crossing given view identifier enables downstream entities to receive the specific information about event by the access interface by being identified
Or view identifier,
Wherein, access interface is based on event data and extracts specific information and/or generation specific information from specific information.
2. the method as described in claim 1, which is characterized in that the access interface further includes for the expression event
Event data execution is acted to generate the logic of the specific information.
3. the method as described in claim 1, which is characterized in that the access interface further include: logic can be used to table
Show that the event data execution of the event is acted to generate the specific information, and wherein, the access interface further includes using
In the input for controlling the movement that the logic executes.
4. the method as described in claim 1, which is characterized in that the specific information only includes the complete thing for indicating the event
The definition subset of part data set.
5. the method as described in claim 1, which is characterized in that further include:
The addition step is repeated, for indicating the event data of each of multiple events, to pass through access interface, to visit
Ask the unified approach of the information about multiple events.
6. the method as described in claim 1, which is characterized in that further include:
Repetition indicates the addition step for indicating the event data of each of multiple events, to pass through access interface
Unified approach of the access about the information of multiple events is realized, wherein indicating that the event data of multiple events is different data lattice
Formula.
7. the method as described in claim 1, which is characterized in that further include:
The addition step of the event data is repeated, the event data indicates the multiple events for belonging to classification identical with event
Each of, with the unified approach by access interface, with access about the information of multiple events.
8. the method as described in claim 1, which is characterized in that identify that the event category includes:
Event category is determined from multiple event categories based on the content of event data, and wherein view identifier is corresponding with access interface
In event category.
9. the method as described in claim 1, which is characterized in that identify that the event category includes:
Event category is determined from multiple event categories based on the type of the machine of the event of generation, wherein view identifier and access circle
Face corresponds to event category.
10. the method as described in claim 1, which is characterized in that view identifier and access interface correspond to belonging to event
Event category；Identify that the event category includes:
The type of machine based on the event of generation, determines event category belonging to event from multiple event categories, wherein view mark
Know symbol and access interface corresponds to event category, event category characterizes the event generated by event.A plurality of types of machines；Also wrap
It includes: event data is parsed based on tentation data format；Further include: the event data is parsed based on tentation data format, it is described
Tentation data format specifies which data in the event to indicate key or value；The view identifier is added to the thing
Field in part；
Further include: multiple attributes based on event data identification events, wherein specific information includes determining for multiple attributes of event
Foster son's collection；
Further include: multiple attributes based on event data identification events, wherein specific information includes determining for multiple attributes of event
Foster son's collection, wherein the attribute includes at least one of the following: key, value or key-value pair；
Further include: downstream entities use the information about multiple events to analyze the event data for indicating multiple events；Also
Include:
Downstream entities use the information about multiple events to analyze the event data for indicating multiple events,
It wherein, include at least one of the following by the addressable information of access interface: (1) by including in access interface
The information that logic generates, which can operate with to indicating that the event data of multiple events executes movement, or (2) are predefined
Indicate the subset of a whole set of event data of multiple events；Further include:
Determine whether event is related to downstream entities using view identifier；Further include:
Automatically it routes information to and specifies the downstream entities of view identifier；Further include:
Automatically it routes information to and specifies the downstream entities of view identifier,
Wherein, downstream entities are configured as identifying the exception towards safety indicated by event；The event data includes machine
Data；The event data includes the machine data with timestamp；Further include:
The addition step is configured by adjusting configuration file；The method is as distributed event processing system or abnormal inspection
Extraction-transformation-load phase a part of at least one of examining system executes；The downstream entities include complicated event
Handle engine.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810831477.2A CN109086605A (en) | 2018-07-26 | 2018-07-26 | A kind of distributed data processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810831477.2A CN109086605A (en) | 2018-07-26 | 2018-07-26 | A kind of distributed data processing method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109086605A true CN109086605A (en) | 2018-12-25 |
Family
ID=64838598
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810831477.2A Withdrawn CN109086605A (en) | 2018-07-26 | 2018-07-26 | A kind of distributed data processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109086605A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150039379A1 (en) * | 2013-07-30 | 2015-02-05 | Red Hat, Inc. | Segmented business process engine |
| CN104484180A (en) * | 2014-12-23 | 2015-04-01 | 上海斐讯数据通信技术有限公司 | Event guide method and guide system based on intelligent equipment |
| CN107077476A (en) * | 2014-09-24 | 2017-08-18 | 甲骨文国际公司 | Event is enriched for event handling using the big data of regime type |
| US20180146000A1 (en) * | 2015-08-31 | 2018-05-24 | Splunk Inc. | Event information access interface in data intake stage of a distributed data processing system |
-
2018
- 2018-07-26 CN CN201810831477.2A patent/CN109086605A/en not_active Withdrawn
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150039379A1 (en) * | 2013-07-30 | 2015-02-05 | Red Hat, Inc. | Segmented business process engine |
| CN107077476A (en) * | 2014-09-24 | 2017-08-18 | 甲骨文国际公司 | Event is enriched for event handling using the big data of regime type |
| CN104484180A (en) * | 2014-12-23 | 2015-04-01 | 上海斐讯数据通信技术有限公司 | Event guide method and guide system based on intelligent equipment |
| US20180146000A1 (en) * | 2015-08-31 | 2018-05-24 | Splunk Inc. | Event information access interface in data intake stage of a distributed data processing system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3205072B1 (en) | Differential dependency tracking for attack forensics | |
| US10791133B2 (en) | System and method for detecting and mitigating ransomware threats | |
| US11218510B2 (en) | Advanced cybersecurity threat mitigation using software supply chain analysis | |
| Sharma et al. | CloudPD: Problem determination and diagnosis in shared dynamic clouds | |
| US9071535B2 (en) | Comparing node states to detect anomalies | |
| US10860406B2 (en) | Information processing device and monitoring method | |
| US11061756B2 (en) | Enabling symptom verification | |
| US20180234445A1 (en) | Characterizing Behavior Anomaly Analysis Performance Based On Threat Intelligence | |
| US20160055044A1 (en) | Fault analysis method, fault analysis system, and storage medium | |
| SA515360536B1 (en) | Method, device, and computer program for monitoring an industrial control system | |
| US20220239630A1 (en) | Graphical representation of security threats in a network | |
| AU2019307885B2 (en) | Systems and methods for reporting computer security incidents | |
| JP2021529383A (en) | Automatic threat alert triage through data history | |
| Xiao et al. | From patching delays to infection symptoms: Using risk profiles for an early discovery of vulnerabilities exploited in the wild | |
| US20220210202A1 (en) | Advanced cybersecurity threat mitigation using software supply chain analysis | |
| JP6557774B2 (en) | Graph-based intrusion detection using process trace | |
| JP2007242002A (en) | Network management device and method, and program | |
| WO2006117833A1 (en) | Monitoring simulating device, method, and program | |
| CN111726358A (en) | Attack path analysis method and device, computer equipment and storage medium | |
| JP2019028891A (en) | Information processing device, information processing method and information processing program | |
| KR101281456B1 (en) | Apparatus and method for anomaly detection in SCADA network using self-similarity | |
| US8554908B2 (en) | Device, method, and storage medium for detecting multiplexed relation of applications | |
| KR101444250B1 (en) | System for monitoring access to personal information and method therefor | |
| JP2017211806A (en) | Communication monitoring method, security management system, and program | |
| JP2017199250A (en) | Computer system, analysis method of data, and computer |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20181225 |