CN113259358A

CN113259358A - Data anomaly identification method and device

Info

Publication number: CN113259358A
Application number: CN202110559238.8A
Authority: CN
Inventors: 熊银梅
Original assignee: Hangzhou Hikvision System Technology Co Ltd
Current assignee: Hangzhou Hikvision System Technology Co Ltd
Priority date: 2021-05-21
Filing date: 2021-05-21
Publication date: 2021-08-13

Abstract

The application provides a data anomaly identification method and device, and relates to the technical field of network security. The problems that in the prior art, the popularity rate is low and the abnormal recognition efficiency of network data is extremely low can be effectively solved. The method comprises the following steps: determining a plurality of target components of the event stream and parameters corresponding to each target component; the target component is a component in a preset CEP component library; each component in the CEP component library corresponds to a set of codes. Acquiring the associated information among the target components; the association information is used for indicating the logical connection relation among a plurality of target components. And generating a CEP evaluation rule of the event stream according to the plurality of target components, the parameters corresponding to each target component and the association information among the target components. The embodiment of the application is applied to a computer system.

Description

Data anomaly identification method and device

Technical Field

The present application relates to the field of network security technologies, and in particular, to a method and an apparatus for identifying data anomalies.

Background

The rapid development of network information technology makes people's life and production more convenient, but in the face of data growing in a blowout manner, and the complexity thereof is higher and higher, the security of network data faces a greater threat. In order to ensure the safety of network data, finding out the abnormality of the network data in time becomes an indispensable link.

Complex Event Processing (CEP) is a software technique that has emerged with increasing demands for streaming data processing to find interesting event patterns in event streams where the order of different data sources is mixed. Because the method can well process complex events, the method is a hot technology in the field of network data anomaly identification in the face of the current complex network environment.

Since CEP discovers data anomalies by identifying complex events in the data stream that satisfy a rule, the determination of the rule also becomes the core of implementing the technique. In the prior art, the rules can only be determined by manually writing program codes step by step. Therefore, the user is required to have the programming capability in the aspect, and the popularization rate is low. Meanwhile, a large amount of time resources are consumed in the writing process of the rule program, so that the abnormal identification efficiency of the network data is extremely low.

Disclosure of Invention

The application provides a data anomaly identification method and device, which can effectively solve the problems of low popularization rate and low anomaly identification efficiency of network data in the prior art.

In order to achieve the purpose, the technical scheme is as follows:

in a first aspect, the present application provides a data anomaly identification method, including: determining a plurality of target components of the event stream and parameters corresponding to each target component; the target component is a preset component in a preset CEP component library; each component in the CEP component library corresponds to a set of codes. Acquiring the associated information among the target components; the association information is used for indicating the logical connection relation among a plurality of target components. And generating a CEP evaluation rule of the event stream according to the plurality of target components, the parameters corresponding to each target component and the association information among the target components.

In the above approach, the complex and time-consuming CEP evaluation rule program writing process is replaced with a simple modular combination of components. Therefore, the user can obtain the CEP evaluation rule of the event stream only by knowing the use mode of each component and determining a plurality of target components corresponding to the event stream, the parameters corresponding to each target component and the associated information among the target components. Program code does not need to be written manually, so that the utilization rate of the CEP technology in the field of abnormal recognition of network data is increased. And the efficiency of carrying out exception identification on the data stream corresponding to the event stream through the CEP evaluation rule and the large flush is further improved.

Optionally, each component in the CEP component library is configured with a corresponding control icon. The method for determining a plurality of target components of an event stream comprises the following steps: and determining the designated component as a target component according to the triggering operation of the control icon corresponding to the designated component in the CEP component library.

Optionally, the method for obtaining the association information between the target components includes: acquiring a serial graph; the serial graph includes a plurality of nodes representing target components and at least one directed edge representing an execution order of the plurality of target components. And determining the associated information among the target components according to the directed edges.

Optionally, the method for acquiring the serial map includes: determining a directed edge between the first target component and the second target component in response to a connection operation between the first target component and the second target component; the first target component and the second target component are any two of the plurality of target components. Determining a serial graph under the condition that all target components are determined to have directed edges and no invalid target components exist; an invalid target component is a target component with an in-degree or out-degree greater than one.

Optionally, the method further includes: acquiring an operator packet of the event stream; the computation packets are used to represent the relationships and structures between events. Based on the arithmetic packets, CEP evaluation rules for the event stream are generated. And/or acquiring a program operation statement of the event stream; the program operation statement is used for representing a plurality of target components and the associated information among the target components. And generating a CEP evaluation rule of the event stream according to the program operation statement of the event stream.

Optionally, the method for performing anomaly identification on the data stream corresponding to the event stream based on the CEP evaluation rule and the big data analysis engine flink includes: and acquiring the data stream in real time through the flink, and counting the target data in the data stream. And performing exception identification on the target data based on the CEP evaluation rule.

In a second aspect, there is provided a data anomaly identification apparatus, the apparatus comprising:

the determining unit is used for determining a plurality of target components of the event stream and parameters corresponding to each target component; the target component is a preset component in a preset CEP component library; each component in the CEP component library corresponds to a set of codes. The acquisition unit is used for acquiring the associated information among the target components; the association information is used for indicating the logical connection relation among a plurality of target components. And the processing unit is used for generating a complex event processing CEP evaluation rule of the event stream according to the plurality of target components determined by the determining unit, the parameters corresponding to each target component and the association information among the target components acquired by the acquiring unit. And the processing unit is also used for carrying out exception identification on the data stream corresponding to the event stream based on the CEP evaluation rule and the big data analysis engine flink. Optionally, each component in the CEP component library is configured with a corresponding control icon. And the determining unit is specifically used for determining the specified component as the target component according to the triggering operation of the control icon corresponding to the specified component in the CEP component library.

Optionally, the obtaining unit is specifically configured to: acquiring a serial graph; the serial graph comprises a plurality of nodes and at least one directed edge, wherein the nodes represent target components, and the directed edge represents the execution sequence of the target components; and determining the associated information among the target components according to the directed edges.

Optionally, the obtaining unit is specifically configured to: determining a directed edge between the first target component and the second target component in response to a connection operation between the first target component and the second target component; the first target component and the second target component are any two of the plurality of target components. Determining a serial graph under the condition that all target components are determined to have directed edges and no invalid target components exist; an invalid target component is a target component with an in-degree or out-degree greater than one.

Optionally, the obtaining unit is further configured to obtain an operator packet of the event stream; the computation packets are used to represent the relationships and structures between events. And the processing unit is used for generating the CEP evaluation rule of the event stream based on the operator packet acquired by the acquisition unit.

Optionally, the obtaining unit is further configured to obtain a program operation statement of the event stream; the program operation statement is used for representing a plurality of target components and the associated information among the target components. And generating a CEP evaluation rule of the event stream according to the program operation statement of the event stream.

Optionally, the processing unit is specifically configured to: and acquiring the data stream in real time through the flink, and counting the target data in the data stream. And performing exception identification on the target data based on the CEP evaluation rule.

In a third aspect, an electronic device is provided, which includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the data anomaly identification method as provided in the first aspect above.

In a fourth aspect, there is provided a computer-readable storage medium comprising instructions that, when executed by a processor in an electronic device, cause the electronic device to perform the data anomaly identification method as provided in the first aspect above.

In a fifth aspect, a computer program product is provided, which, when executed by a processor in an electronic device, causes the electronic device to perform the data anomaly identification method as provided in the first aspect above.

It should be noted that all or part of the computer instructions may be stored on the computer readable storage medium. The computer-readable storage medium may be packaged with the processor of the data anomaly recognition device, or may be packaged separately from the processor of the data anomaly recognition device, which is not limited in this application.

For the descriptions of the second, third, fourth and fifth aspects in this application, reference may be made to the detailed description of the first aspect and its various implementations; moreover, for the beneficial effects of the second aspect, the third aspect, the fourth aspect and the fifth aspect, reference may be made to the beneficial effect analysis in the first aspect and various implementation manners thereof, and details are not described here.

In the present application, the names of the data anomaly recognition devices do not limit the devices or the functional modules, and in practical implementation, the devices or the functional modules may appear by other names. Insofar as the functions of the respective devices or functional modules are similar to those of the present application, they fall within the scope of the claims of the present application and their equivalents.

These and other aspects of the present application will be more readily apparent from the following description.

Drawings

Fig. 1 is a schematic flow chart of a data anomaly identification method according to an embodiment of the present application;

fig. 2 is a second schematic flowchart of a data anomaly identification method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a serial diagram provided by an embodiment of the present application;

fig. 4 is a third schematic flowchart of a data anomaly identification method according to an embodiment of the present application;

fig. 5 is a fourth schematic flowchart of a data anomaly identification method according to an embodiment of the present application;

fig. 6A is an interface display diagram of a security analysis development workbench according to an embodiment of the present disclosure;

FIG. 6B is an interface display diagram of a mode setting provided by an embodiment of the present application;

FIG. 6C is an interface display diagram of a merge field according to an embodiment of the present disclosure;

fig. 7 is a schematic structural diagram of a data anomaly recognition apparatus according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of another data anomaly identification device according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a computer program product of a data anomaly identification method according to an embodiment of the present application.

Detailed Description

A data anomaly identification method, a data anomaly identification device, and a storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone.

The terms "first" and "second" and the like in the description and drawings of the present application are used for distinguishing different objects or for distinguishing different processes for the same object, and are not used for describing a specific order of the objects.

Furthermore, the terms "including" and "having," and any variations thereof, as referred to in the description of the present application, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or modules is not limited to the listed steps or modules but may alternatively include other steps or modules not listed or inherent to such process, method, article, or apparatus.

It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to mean serving as examples, illustrations or descriptions. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.

In the description of the present application, the meaning of "a plurality" means two or more unless otherwise specified.

It can be understood that, in practical application, the order of steps of the specific method may be adjusted, and the method provided in the embodiment of the present application is not limited thereto.

Before explaining the embodiments of the present application in detail, some terms related to the embodiments of the present application will be explained.

1. Rules engine

The rule engine is developed by an inference engine, is a component embedded in an application program, and realizes the separation of business decisions from application program codes and the writing of the business decisions by using a predefined semantic module. And receiving data input, interpreting business rules, and making business decisions according to the business rules.

2. Complex event processing CEP

Complex event processing CEP is to output data interested by a user by matching one or more event streams consisting of simple events through a certain rule. Specifically, each piece of data is regarded as different events, and event sequences meeting specified rules in the event stream are identified and responded.

3. Security risk awareness

The safety risk can be timely and comprehensively sensed, all safety events can be warned, safety personnel can be helped to take targeted response treatment measures and trace the source, and the impending safety events can be predicted.

4. Structured query language

Structured Query Language (SQL) is a special-purpose programming Language, a database Query and programming Language, used to access data and Query, update, and manage relational database systems.

Structured query languages are high-level, non-procedural programming languages that allow users to work on high-level data structures. The method does not require a user to specify a data storage method and does not require the user to know a specific data storage mode, so that different database systems with completely different underlying structures can use the same structured query language as an interface for data input and management. The structured query language statements can be nested, which allows for great flexibility and powerful functionality.

5. Event stream

An event stream generally describes an analytic flow event composed of one or more simple events in some logic. The event stream referred to in the embodiments of the present application includes a plurality of target components, and the plurality of target components are arranged in a certain order.

Since CEP discovers data anomalies by identifying complex events in the data stream that satisfy a rule, the determination of the rule also becomes the core of implementing the technique. In the prior art, a user can only determine the rule by writing program codes step by step. Thus, not only is it necessary for the user to have the programming capabilities of this aspect. Meanwhile, a large amount of time resources need to be consumed in the writing process of the rule program, and the efficiency is extremely low.

In view of this, an embodiment of the present disclosure provides a data anomaly identification method, so as to solve the problems in the prior art, and first introduces an application scenario of the data anomaly identification method provided in the embodiment of the present disclosure.

In practical application, the data anomaly identification method provided by the embodiment of the application is applied to electronic equipment. The electronic device may be a terminal, a server, a device for implementing the data abnormality recognition method in the terminal, or a device for implementing the data abnormality recognition method in the server.

For example, when the electronic device in the embodiment of the present application is a terminal, the terminal may be a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a Personal Digital Assistant (PDA), an Augmented Reality (AR) device, and the like, which have a program writing function and operate a related program. The embodiment of the present application does not specifically limit the specific form of the terminal.

The data anomaly identification method provided by the embodiment of the present disclosure is described below with reference to specific embodiments, and the method provided by the embodiment can be applied to the electronic device.

Referring to fig. 1, a data anomaly identification method provided in an embodiment of the present application includes the following steps.

S11, the electronic device determines a plurality of target components of the event stream and corresponding parameters of each target component.

Wherein, the target component is a component in a preset CEP component library; each component in the CEP component library corresponds to a set of codes.

It should be noted that the electronic device determines a plurality of target components of the event stream and parameters corresponding to each target component; code information of the target component can be imported into a CEP component library of the electronic equipment in advance; or respectively storing the code information of the target component in a CEP component library of one or more other electronic devices, and acquiring the information of the relevant target component from one or more electronic devices in real time in the using process.

For example, the data abnormality identification method of the present embodiment may be applied to a server. Assuming that the server 1 stores a CEP component library a and the server 2 stores a CEP component library B, the data anomaly identification method of the present embodiment is deployed in the server 3, where the CEP component library a and the CEP component library B store codes of different target components, respectively. The server 3 may obtain codes of the plurality of target components from the CEP component library a and the CEP component library B according to the service requirement identified by the current data exception and according to the plurality of target components and the parameter corresponding to each target component.

In addition, for each component in the CEP component library, a parameter corresponding to the component may also be set. The parameter values of the parameters are different for different data fusion processes or processing objects, so that when a specific business requirement is faced, the parameter values of the parameters of the corresponding components can be determined according to the specific business requirement.

For example, assume that component A has two parameters: a and b. In the data anomaly identification program corresponding to the business requirement 1, the parameter a is assigned as al, and the parameter b is assigned as b 1. In the data anomaly identification program corresponding to the business requirement 2, the parameter a is assigned as a2, and the parameter b is assigned as b 2.

In one implementation mode, the designated component is determined to be the target component according to the triggering operation of the control icon corresponding to the designated component in the CEP component library.

Specifically, the trigger operation may be a click operation or a drag operation on the control chart of the target component.

For example, a rule device interface can be opened by clicking a specified operation option on a first interface for generating the CEP evaluation rule, a control icon corresponding to each component is displayed in the rule device interface, and when the control icon of a certain component is selected, the control of the component is automatically displayed in a drawing area on the first interface for generating the CEP evaluation rule. Or, the control icon corresponding to the component can be dragged from the rule device interface directly to the drawing area on the first interface of the CEP evaluation rule.

For example, when the triggering operation on the target component is a click operation, the click operation may be any feasible operation such as a single click operation, a double click operation, a sliding operation, and the like, which may be determined according to actual needs, and embodiments of the present disclosure are not limited specifically.

In another example, the component name of the target component may also be added to a specific form, and all components in the specific form are determined as the target components required by the CEP evaluation rule generated this time.

By the triggering operation of the control icon corresponding to the specified component in the CEP component library, the specified component can be determined as the target component. The time for writing codes is saved for the user to obtain the CEP evaluation rule, and the efficiency is improved.

S12, the electronic equipment acquires the associated information among the target components; the association information is used for indicating the logical connection relation among a plurality of target components.

For example, after obtaining the association information between the multiple target components, since the association information indicates a logical connection relationship between the multiple target components, an execution order of codes corresponding to the multiple target components in the program implementing data exception identification may be determined accordingly.

For example, assume that there are 3 target components: component 1, component 2 and component 3. The association information between the 3 components indicates that component 1 is logically connected to component 3, component 2 is logically connected to component 1, and the output of component 2 is the input of component 1 and the output of component 1 is the input of component 3. The execution sequence of the codes corresponding to the component 1, the component 2 and the component 3 in the whole data fusion program is as follows: component 2, component 1, component 3.

S13, the electronic device generates a complex event processing CEP evaluation rule of the event stream according to the target components, the parameters corresponding to each target component and the associated information among the target components.

It should be noted that the CEP evaluation rule is presented in a code manner.

S14, the electronic device conducts exception identification on the data stream corresponding to the event stream based on the CEP evaluation rule and the big data analysis engine flink.

Illustratively, flink-based built-in features can support a variety of data sources, such as CSV, Kafka, Hbase, Text, Socket. Therefore, the source of the data stream can be obtained from any one of the modes of CSV, Kafka, Hbase, Text, Socket, etc., thereby providing convenience for data access.

Specifically, the electronic device obtains the data stream in real time through the flink, and counts the target data in the data stream. And then, performing anomaly identification on the target data based on the CEP evaluation rule. The data stream is obtained in real time through the flink, the target data in the data stream is calculated, and the instantaneity of performing exception identification on the target data is improved.

In one implementation, referring to fig. 2 in conjunction with fig. 1, S12 specifically includes S121 and S122.

S121, the electronic equipment acquires a serial image; the serial graph includes a plurality of nodes representing target components and at least one directed edge representing an execution order of the plurality of target components.

Specifically, the electronic device determines a directed edge between a first target component and a second target component in response to a connection operation between the first target component and the second target component; the first target component and the second target component are any two of the plurality of target components. Determining a serial graph under the condition that all target components are determined to have directed edges and no invalid target components exist; an invalid target component is a target component with an in-degree or out-degree greater than one.

It should be noted that the in-degree of a target component refers to the number of directed edges with the target component as the head; the out-degree of a target component is the number of directed edges that tail the target component.

Illustratively, referring to fig. 3, there is included a serial graph consisting of a target component 1 (corresponding to node 1 in fig. 3), a target component 2 (corresponding to node 2 in fig. 3), a target component 3 (corresponding to node 3 in fig. 3), and directed edges a and b. It can be seen that the in-degree of the target assembly 1 is 1 and the out-degree is 0; the in degree of the target assembly 2 is 1, and the out degree is 1; the in-degree of the target module 3 is 0 and the out-degree is 1.

And S122, the electronic equipment determines the associated information among the target components according to the directed edges.

Illustratively, in one aspect, a directed edge of the serial graph indicates which two target components have an association between them. On the other hand, the directed edges of the serial graph also indicate the execution order of the target components. For example, if there is a directed edge pointing from B to C between the target component B and the target component C in the serial diagram, it indicates that there is an association between the target component B and the target component C, and the code corresponding to the target component B is executed before the code corresponding to the target component C, and the output data of the processing component B is used as the input data of the target component C.

Through the implementation mode, a user can indicate the associated information between the target components by drawing the serial graph on the specified interface. The serial diagram is easy to understand and can clearly show the logical connection relation between target components.

In one implementation, referring to fig. 4 in conjunction with fig. 1, the requirements of different users on the acquisition path of the CEP evaluation rules may be different. Therefore, the embodiment of the present application further provides two ways of obtaining CEP evaluation rules, where the first way includes S31 and S32; the second mode includes S41 and S42.

S31, the electronic equipment acquires an operator packet of the event stream; the computation packets are used to represent the relationships and structures between events.

Illustratively, the software format of the computer packet may be jar or jason description.

Optionally, the user may input the program code of the operator packet related to the event stream in the second interface displayed in the electronic device, or call the program code of the operator packet related to the event stream stored in the database in the second interface to obtain the operator packet.

S32, the electronic device generates a CEP evaluation rule of the event stream based on the arithmetic packets.

Specifically, based on the matching of the operator packet with a preset rule, a CEP evaluation rule of the event stream is determined.

The method provides another implementation mode for determining the CEP evaluation rule for the user through the form of the computer packet, and improves the experience of the user.

S41, the electronic equipment acquires a program running statement of the event stream; the program operation statement is used for representing a plurality of target components and the associated information among the target components.

Optionally, the user inputs a program execution statement of the event stream in a third interface displayed on the electronic device.

And S42, the electronic equipment generates a CEP evaluation rule of the event stream according to the program operation statement of the event stream.

The method provides a further implementation mode for determining the CEP evaluation rule for the user through the form of the program operation statement of the event stream, and improves the experience of the user.

For ease of understanding, referring to fig. 5, 6A-6C, the following is an exemplary description of the CEP evaluation rule obtained through the steps S11 and S12 in the embodiment of the present application.

And S51, the electronic equipment displays the first interface.

Wherein the first interface shows a first area and a second area; wherein the first area presents a target component selected by a user; the second area is used for showing a rule-based control icon.

Illustratively, the first interface refers to the canvas interface in the presentation security analysis development workstation in FIG. 6A. In addition, a second interface and a third interface are also displayed in the display security analysis development workbench. Wherein the second interface corresponds to the algorithm package interface in fig. 6A; the third interface is to the SQL interface in FIG. 6A. The user may display the third interface by clicking the SQL control shown in fig. 6A; the second interface may also be displayed by clicking on the algorithm package control shown in fig. 6A.

And S52, the electronic equipment displays a rule configuration interface according to the clicking operation of the user on the rule-based control icon. The rule configuration interface includes a control icon for at least one component.

Illustratively, the user presents a control icon 02 for at least one component by presenting a rule-based control icon 01 in FIG. 6A. The control icons 02 showing at least one component in fig. 6A may include, but are not limited to, any of field-ordered control icons 021, mode-set control icons 022, and field-merged control icons 023. Besides the illustration, the control icons 02 of the component may also include a control icon of a protocol, a control icon of a frequency of occurrence, a control icon of an action, a control icon of an alarm, and the like, which is not limited in this embodiment of the present application.

And S53, the electronic equipment determines the designated component as a target component according to the triggering operation of the control icon corresponding to the designated component in the CEP component library.

For example, when the user triggers the field-sorted control icon 021, the mode-set control icon 022, and the merged field control icon 023 in fig. 6A to operate, it is determined that the field-sorted, mode-set, and merged fields are all target components, and the field-sorted, mode-set, and merged field icons are displayed in the first area.

S54, the electronic device determines a directed edge between the first target component and the second target component in response to the connection operation between the first target component and the second target component.

For example, for the join operation of the field ordering and mode setting illustrated in the first region of FIG. 6A, a directed edge A1 between the field ordering and the mode setting is determined. Wherein the directional edge arrow between the field ordering and the mode setting points to the mode setting. Similarly, the join operation on the mode set and merge fields determines a directed edge A2 between the mode set and merge fields. Wherein the arrow with a directed edge between the mode setting and the merge field points to the merge field.

And S55, the electronic equipment determines the associated information among the target components according to the directed edges.

Specifically, for the join operation of the field ordering and the mode setting illustrated in the first region of fig. 6A, a directed edge a1 between the field ordering and the mode setting is determined. Wherein the directional edge arrow between the field ordering and the mode setting points to the mode setting. The execution sequence of the field sorting and the mode setting is to execute the mode setting after the field sorting is executed. Similarly, the join operation on the mode set and merge fields determines a directed edge A2 between the mode set and merge fields. Wherein the arrow with a directed edge between the mode setting and the merge field points to the merge field. The execution sequence of the mode setting and merging field is the execution sequence of the mode setting and merging field, and then the merging field is executed.

And S56, the electronic device determines the serial graph under the condition that all the target components have directed edges and no invalid target components exist.

For example, the determined field ordering, mode setting, and merging fields of fig. 6A are sequentially connected by the directed edge in turn, and the arrows of the directed edge are oriented in the same direction, thereby determining the serial graph.

S57, the electronic equipment responds to the triggering operation of the user on the icon of the target component in the first interface, and the target component interface is displayed; the target component interface is used for setting parameters of each target component.

For example, after the user triggers the icon of the mode setting in fig. 6A, the mode setting interface as shown in fig. 6B is displayed, and the mode name and its corresponding input box B1, the number of occurrences and corresponding input box B5, and the condition setting area are displayed in the mode setting interface. At least one sub-region is shown in the condition setting region, and a condition field and its corresponding input box b2, a condition operator and its corresponding input box b3 and a condition field value and its corresponding input box b4 are shown in each sub-region. When there are multiple sub-regions, the logical operators and their corresponding input boxes b6 are also exposed between the two sub-regions in the mode device interface.

For another example, after the user triggers the icon of the merge field in fig. 6A, the merge field interface shown in fig. 6C is shown, and the merge field and the corresponding input box C are shown in the interface. The fields to be merged are determined by entering the field names to be merged in the input box c. The field names may be event names, event major types, event minor types, and event source IPs as shown in fig. 6C. Of course, the field name may also be other event information, which is not limited in this embodiment of the present application.

S58, the electronic device generates a complex event processing CEP evaluation rule of the event stream according to the target components, the parameters corresponding to each target component and the associated information among the target components.

After the CEP evaluation rule of the event stream is generated, the evaluation rule can be sent to a data development workbench in an SQL form to form an analysis task for the event stream, then the data development workbench schedules a big data analysis engine flink in a job mode based on the analysis task, the flink acquires a data stream (such as a log) and target data in a statistical data stream in real time, performs exception identification on the target data based on the CEP evaluation rule, and stores an identification result in a security service analysis result base, so that the timeliness is high, security risks can be timely and comprehensively sensed, and a security event is alarmed. The data development workbench and the security analysis development workbench may be different modules integrated on one hardware device, may also be independent and separate hardware devices, and of course, may also be in other product forms, which is not limited in this application embodiment.

The scheme provided by the embodiment of the application is mainly introduced from the perspective of a method. To implement the above functions, it includes hardware structures and/or software modules for performing the respective functions. Those of skill in the art would readily appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

Fig. 7 is a schematic structural diagram of a data anomaly recognition apparatus 70 according to an embodiment of the present application. The data abnormality recognition device 70 is used to execute the data abnormality recognition method shown in fig. 1. The data abnormality recognition device 70 includes: an acquisition unit 701, a processing unit 702, and a determination unit 703.

Specifically, the determining unit 703 is configured to determine a plurality of target components of the event stream and a parameter corresponding to each target component; the target component is a preset component in a preset CEP component library; each component in the CEP component library corresponds to a set of codes. For example, the determining unit 703 may be used to implement S11 shown in fig. 1.

An obtaining unit 701, configured to obtain association information between target components; the association information is used for indicating the logical connection relation among a plurality of target components. For example, the obtaining unit 701 may be used to implement S12 shown in fig. 1.

The processing unit 702 is configured to generate a complex event processing CEP evaluation rule of the event stream according to the multiple target components determined by the determining unit 703, the parameter corresponding to each target component, and the association information between the target components acquired by the acquiring unit 701. For example, the processing unit 702 may be configured to implement S13 as shown in fig. 1.

The processing unit 702 is further configured to perform exception identification on the data stream corresponding to the event stream based on the CEP evaluation rule and the big data analysis engine flink. For example, the processing unit 702 may be configured to implement S14 as shown in fig. 1.

Optionally, each component in the CEP component library is configured with a corresponding control icon.

The determining unit 703 is specifically configured to determine, according to a trigger operation on a control icon corresponding to a specified component in the CEP component library, that the specified component is a target component.

Optionally, the obtaining unit 701 is specifically configured to: acquiring a serial graph; the serial graph includes a plurality of nodes representing target components and at least one directed edge representing an execution order of the plurality of target components. And determining the associated information among the target components according to the directed edges.

The optional obtaining unit 701 is specifically configured to: determining a directed edge between the first target component and the second target component in response to a connection operation between the first target component and the second target component; the first target component and the second target component are any two of the plurality of target components. Determining a serial graph under the condition that all target components are determined to have directed edges and no invalid target components exist; an invalid target component is a target component with an in-degree or out-degree greater than one.

Optionally, the obtaining unit 701 is further configured to obtain an operator packet of the event stream; the computation packets are used to represent the relationships and structures between events.

And the processing unit 702 is configured to generate a CEP evaluation rule of the event stream based on the operator packet acquired by the acquiring unit 701.

Optionally, the obtaining unit 701 is further configured to obtain a program running statement of the event stream; the program operation statement is used for representing a plurality of target components and the associated information among the target components.

The processing unit 702 is configured to generate a CEP evaluation rule of the event stream according to the program execution statement of the event stream acquired by the acquiring unit 701.

Optionally, the processing unit 702 is specifically configured to: and acquiring the data stream in real time through the flink, and counting the target data in the data stream. And performing exception identification on the target data based on the CEP evaluation rule.

Of course, the data anomaly identification device 70 provided in the embodiment of the present application includes, but is not limited to, the above modules, for example, the data anomaly identification device 70 may further include a sending unit 703 and a storage unit 704. The sending unit 703 may be configured to send the relevant data in the data anomaly recognition apparatus 70 to other devices, so as to implement data interaction with other devices. The storage unit 704 may be used for storing the program code of the data exception identifying apparatus 70, and may also be used for storing data generated by the data exception identifying apparatus 70 during operation, such as data in a write request.

Here, the system architecture and the service scenario described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not constitute a limitation to the technical solution provided in the embodiment of the present application, and it is known by a person of ordinary skill in the art that the technical solution provided in the embodiment of the present application is also applicable to similar technical problems along with the evolution of the network architecture and the appearance of a new service scenario.

Alternatively, the data anomaly identification device 70 mentioned in the embodiment of the present application may be implemented by the communication device shown in fig. 8.

The communication device includes a processor 81, a communication bus 84, and at least one transceiver (illustrated in fig. 8 as including transceiver 83 for exemplary purposes only).

Processor 81 may include one or more processing units, such as: the processor 81 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a Video Processing Unit (VPU) controller, a memory, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors.

The controller can be a neural center and a command center of the communication device. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.

A memory may also be provided in processor 81 for storing instructions and data. In some embodiments, the memory in processor 81 is a cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 81. If the processor 81 needs to use the instruction or data again, it can be called directly from the memory. Avoiding repeated accesses reduces the latency of the processor 81 and thus increases the efficiency of the system.

In some embodiments, processor 81 may include one or more interfaces. The interface may include an integrated circuit (I8C) interface, a universal asynchronous receiver/transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a general-purpose input/output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and/or a Universal Serial Bus (USB) interface, a Serial Peripheral Interface (SPI) interface, and/or the like.

The communication bus 84 may include a path to transfer information between the aforementioned components.

The transceiver 83 may be any device, such as a transceiver, for communicating with other devices or communication networks, such as an ethernet, a Radio Access Network (RAN), a Wireless Local Area Network (WLAN), etc.

Optionally, the communication device may also include a memory 82.

The memory 82 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that may store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that may store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, optical disk storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory may be self-contained and coupled to the processor via a communication bus 84. The memory may also be integral to the processor.

The memory 82 is used for storing computer-executable instructions for executing the present application, and is controlled by the processor 81 to execute the instructions. The processor 81 is configured to execute computer-executable instructions stored in the memory 82, so as to implement the data anomaly identification method provided by the following embodiments of the present application.

Optionally, the computer-executable instructions in the embodiment of the present invention may also be referred to as application program codes, which is not specifically limited in this embodiment of the present invention.

In particular implementations, processor 81 may include one or more CPUs such as CPU0 and CPU1 in fig. 8 as an example.

In particular implementations, the communication device may include multiple processors, such as processor 81 and processor 85 in fig. 8, for example, as an example. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).

In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a computer-readable storage medium in a machine-readable format or encoded on other non-transitory media or articles of manufacture.

Fig. 9 schematically illustrates a conceptual partial view of a computer program product comprising a computer program for executing a computer process on a computing device provided by an embodiment of the application.

In one embodiment, the computer program product is provided using a signal bearing medium 410. The signal bearing medium 410 may include one or more program instructions that, when executed by one or more processors, may provide the functions or portions of the functions described above with respect to fig. 3. Thus, for example, referring to the embodiment shown in FIG. 3, one or more features of S11-S14 may be undertaken by one or more instructions associated with the signal bearing medium 410. Further, the program instructions in FIG. 9 also describe example instructions.

In some examples, signal bearing medium 410 may include a computer readable medium 411, such as, but not limited to, a hard disk drive, a Compact Disc (CD), a Digital Video Disc (DVD), a digital tape, a memory, a read-only memory (ROM), a Random Access Memory (RAM), or the like.

In some implementations, the signal bearing medium 410 may comprise a computer recordable medium 412 such as, but not limited to, a memory, a read/write (R/W) CD, a R/W DVD, and the like.

In some implementations, the signal bearing medium 410 may include a communication medium 413, such as, but not limited to, a digital and/or analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.).

The signal bearing medium 410 may be conveyed by a wireless form of communication medium 413, such as a wireless communication medium compliant with the IEEE802.41 standard or other transport protocol. The one or more program instructions may be, for example, computer-executable instructions or logic-implementing instructions.

In some examples, a data writing apparatus, such as that described with respect to fig. 3, may be configured to provide various operations, functions, or actions in response to one or more program instructions via the computer-readable medium 411, the computer-recordable medium 412, and/or the communication medium 413.

In addition, the embodiment of the application also provides a chip system, and the chip system is applied to the data anomaly identification device; the chip system includes one or more interface circuits, and one or more processors. The interface circuit and the processor are interconnected through a line; the interface circuit is configured to receive signals from a memory of the data anomaly identification device and send signals to the processor, the signals including computer instructions stored in the memory. When the processor executes the computer instructions, the data anomaly identification device executes the data anomaly identification method as provided by the first aspect or any one of the possible design manners of the first aspect.

Through the above description of the embodiments, it is clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the modules or units is only one logical functional division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another device, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may be one physical unit or a plurality of physical units, that is, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially or partially contributed to by the prior art, or all or part of the technical solutions may be embodied in the form of a software product, where the software product is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

The above description is only an embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A data anomaly identification method is characterized by comprising the following steps:

determining a plurality of target components of the event stream and parameters corresponding to each target component; the target component is a component in a preset CEP component library; each component in the CEP component library corresponds to a set of codes;

acquiring the associated information among the target components; the association information is used for indicating the logical connection relation among the target components;

generating a Complex Event Processing (CEP) evaluation rule of the event stream according to the target components, the parameters corresponding to each target component and the correlation information among the target components;

and performing exception identification on the data stream corresponding to the event stream based on the CEP evaluation rule and a big data analysis engine flink.

2. The data anomaly identification method according to claim 1, wherein each component in said CEP component library is configured with a corresponding control icon;

the determining a plurality of target components of an event stream comprises:

and determining the designated component as the target component according to the triggering operation of the control icon corresponding to the designated component in the CEP component library.

3. The method for identifying data anomalies according to claim 1, wherein the acquiring of the association information between the target components includes:

acquiring a serial graph; the serial graph comprises a plurality of nodes and at least one directed edge, wherein the nodes represent the target components, and the directed edge represents the execution sequence of the target components;

and determining the associated information among the target components according to the directed edges.

4. The data anomaly identification method according to claim 3, wherein said obtaining a serial map comprises:

determining a directed edge between the first target component and the second target component in response to a connection operation between the first target component and the second target component; the first target component and the second target component are any two of the plurality of target components;

determining a serial graph under the condition that all target components are determined to have directed edges and no invalid target components exist; an invalid target component is a target component with an in-degree or out-degree greater than one.

5. The data anomaly identification method according to any one of claims 1-4, further comprising:

acquiring an operator packet of the event stream; the computer sub-package is used for representing the relationship and the structure between the events;

generating a CEP evaluation rule for the event stream based on the computation packet; and/or the presence of a gas in the gas,

acquiring a program running statement of the event stream; the program operation statement is used for representing the association information between the target components and the plurality of target components;

and generating a CEP evaluation rule of the event stream according to the program operation statement of the event stream.

6. The data anomaly identification method according to claim 1, wherein the performing anomaly identification on the data stream corresponding to the event stream based on the CEP evaluation rule and a big data analysis engine flink comprises:

acquiring a data stream in real time through the flink, and counting target data in the data stream;

and performing exception identification on the target data based on the CEP evaluation rule.

7. A data abnormality recognition apparatus, characterized by comprising:

the determining unit is used for determining a plurality of target components of the event stream and parameters corresponding to each target component; the target component is a component in a preset CEP component library; each component in the CEP component library corresponds to a set of codes;

the acquisition unit is used for acquiring the associated information among the target components; the association information is used for indicating the logical connection relation among the target components;

the processing unit is used for generating a complex event processing CEP evaluation rule of the event stream according to the plurality of target components determined by the determining unit, the parameters corresponding to each target component and the association information among the target components acquired by the acquiring unit;

the processing unit is further configured to perform exception identification on the data stream corresponding to the event stream based on the CEP evaluation rule and a big data analysis engine flink.

8. The data anomaly identification device according to claim 7,

the obtaining unit is further configured to obtain an operator packet of the event stream; the computer sub-package is used for representing the relationship and the structure between the events;

the processing unit is configured to generate a CEP evaluation rule of the event stream based on the computation packet acquired by the acquisition unit; and/or the presence of a gas in the gas,

the obtaining unit is further configured to obtain a program running statement of the event stream; the program operation statement is used for representing the association information between the target components and the plurality of target components;

the processing unit is configured to generate a CEP evaluation rule of the event stream according to the program running statement of the event stream acquired by the acquiring unit.

9. An electronic device, comprising: a memory for storing a computer program and a processor for executing the computer program to perform the data anomaly identification method of any one of claims 1-6.

10. A computer-readable storage medium, having stored thereon a computer program which, when run on a computer, causes the computer to execute the data anomaly identification method according to any one of claims 1-6.