CN117033318B - Method and device for generating data to be tested, storage medium and electronic equipment - Google Patents

Method and device for generating data to be tested, storage medium and electronic equipment Download PDF

Info

Publication number
CN117033318B
CN117033318B CN202311290884.4A CN202311290884A CN117033318B CN 117033318 B CN117033318 B CN 117033318B CN 202311290884 A CN202311290884 A CN 202311290884A CN 117033318 B CN117033318 B CN 117033318B
Authority
CN
China
Prior art keywords
judgment
value
rule
array
hit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311290884.4A
Other languages
Chinese (zh)
Other versions
CN117033318A (en
Inventor
陈伟胜
孙洪伟
肖新光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Antan Network Security Technology Co ltd
Original Assignee
Shenzhen Antan Network Security Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Antan Network Security Technology Co ltd filed Critical Shenzhen Antan Network Security Technology Co ltd
Priority to CN202311290884.4A priority Critical patent/CN117033318B/en
Publication of CN117033318A publication Critical patent/CN117033318A/en
Application granted granted Critical
Publication of CN117033318B publication Critical patent/CN117033318B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
    • G06F16/1815—Journaling file systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The present invention relates to the field of data processing, and in particular, to a method and apparatus for generating data to be tested, a storage medium, and an electronic device. The method comprises the following steps: acquiring a judging rule set corresponding to a target event; and carrying out first data generation processing on the judgment value corresponding to each judgment rule in the log of the target event, and generating a judgment value set corresponding to the target event. In the invention, through the first data processing, a plurality of judgment values corresponding to the same hit field can exist in the form of a standard three-dimensional array. Therefore, the data structure forms of the acquired target data can be unified, and the plurality of judgment values can be conveniently processed by using the same data analysis method later. The applicability of the acquired data to be detected is improved, and the convenience of later data processing is improved.

Description

Method and device for generating data to be tested, storage medium and electronic equipment
Technical Field
The present invention relates to the field of data processing, and in particular, to a method and apparatus for generating data to be tested, a storage medium, and an electronic device.
Background
As data analysis has received increasing attention in recent years, so has the ability to do more and more with information in event logs (e.g., json logs). For example, according to the data recorded in the log, certain characteristics of the event generating the log can be obtained through corresponding analysis and calculation, and more accurate and effective information can be obtained.
Before data analysis is performed, the data to be tested needs to be extracted, but the complexity of the data structure in the Json log corresponding to different events is different due to different event types. And with the increase of the complexity of the data structure, the judgment value obtained from the corresponding Json log finally exists in different forms, such as a single value form or a one-dimensional array or a two-dimensional array. And further, the structure forms corresponding to the data of different events are different. Meanwhile, after the data extraction is completed, the subsequent data analysis processing is required, and when the data processing is performed, the data structure required to be input by the same data analysis method is consistent, otherwise, the calculation processing cannot be performed. The data structure forms of the target data obtained by the value method in the prior art have larger difference, so that the applicability of obtaining the data to be measured is reduced, and the convenience of later data processing is reduced.
Disclosure of Invention
Aiming at the technical problems that the difference of the data structure forms of the target data obtained by the value method in the prior art is large, the applicability of obtaining the data to be measured is further reduced, and the convenience of the later data processing is reduced, the invention adopts the following technical scheme:
according to an aspect of the present invention, there is provided a method of generating data to be measured, the method comprising the steps of:
acquiring a judging rule set corresponding to a target event; the set of decision rules includes a plurality of decision rules; each judgment rule comprises a unique corresponding hit field name and at least one corresponding rule hit path;
performing first data generation processing on the corresponding judgment value of each judgment rule in the log of the target event to generate a judgment value set corresponding to the target event; the decision value is a parameter value representing at least part of the characteristics of the target event;
a first data generation process comprising:
generating at least one value index corresponding to the judgment rule according to the hit field name and at least one rule hit path in the judgment rule;
if the judging rule corresponds to a plurality of value indexes, generating a plurality of group values of each value index corresponding to the judging rule according to the data format of the data content corresponding to each sub-index included in each value index in the log of the target event;
generating an initial array of the judgment rule according to the maximum value of the array values corresponding to all the value indexes, wherein the dimension of the initial array is the same as the maximum value of the array values of all the value indexes corresponding to the judgment rule;
according to each value index, at least one initial judgment value corresponding to each value index is obtained from a log of the target event;
respectively placing initial judgment values corresponding to each value index into corresponding initial arrays to generate initial data structures corresponding to each value index;
the initial data structure body corresponding to each value index is placed in the same first expansion array, and a three-dimensional judgment array corresponding to the judgment rule is generated; dimension W of first extended array K The following conditions are satisfied: w (W) K =3-W C ；W C Is the dimension of the initial array.
Further, after generating at least one value index corresponding to the decision rule, the first data generating process further includes:
if the judging rule only corresponds to one value index, generating a second expansion array corresponding to the judging rule, wherein the second expansion array is a three-dimensional array;
according to the value index, at least one initial judgment value corresponding to the value index is obtained from the log of the target event;
and all the initial judgment values are put into the second expansion array, and a three-dimensional judgment array corresponding to the judgment rule is generated.
Further, generating an array value of each value index corresponding to the decision rule according to a data format of data content corresponding to each sub-index included in each value index in the log of the target event, including:
configuring the same initial array value for each value index;
acquiring the data content of each value index in the log of the target event;
and when the index content corresponding to any sub-index in the value indexes is provided with the array identifier, carrying out accumulated counting on the initial array value corresponding to the value index so as to generate the array value of each value index corresponding to the judging rule.
Further, after generating the set of determination values corresponding to the target event, the method further includes:
determining each of the set of determination values using the determination information in each of the determination rules;
if the judging value accords with the judging condition in the corresponding judging information, determining that the judging value hits the judging rule corresponding to the judging information;
taking the corresponding judgment rule when the judgment value hits the judgment information as the hit judgment rule corresponding to the judgment value;
taking preset output information corresponding to the hit judgment rule as marking information of the target event; the preset output information comprises event portrait tags and mark judgment information.
Further, determining each of the set of determination values using the determination information in each of the determination rules includes:
if the hit field name corresponding to the three-dimensional judgment array is the same as the hit field name corresponding to the judgment rule, determining the three-dimensional judgment array as a target three-dimensional judgment array of the judgment rule;
and performing traversal judgment on the judgment values in the target three-dimensional judgment array by using the judgment information corresponding to each judgment rule.
Further, the judging rule set further comprises at least one mark hit information, wherein the mark hit information comprises a mark hit path, an event portrait tag and mark judging information; the marked hit path is one or more regular hit paths marked in advance;
the preset output information corresponding to the hit judgment rule is used as the marking information of the target event, and the method comprises the following steps:
if the rule hit path in the hit judgment rule and any mark hit path have an intersection, and a path node in the intersection is larger than or equal to a first node threshold value, determining that mark hit information corresponding to the mark hit path is preset output information corresponding to the hit judgment rule;
and taking event portrait labels and mark judgment information in preset output information corresponding to the hit judgment rule as mark information of the target event.
Further, the log of the target event is a json log of the target event.
According to a second aspect of the present invention, there is provided an apparatus for generating data to be measured, the apparatus comprising:
the rule acquisition module is used for acquiring a judging rule set corresponding to the target event; the set of decision rules includes a plurality of decision rules; each judgment rule comprises a unique corresponding hit field name and at least one corresponding rule hit path;
the data generation module is used for carrying out first data generation processing on the judgment value corresponding to each judgment rule in the log of the target event to generate a judgment value set corresponding to the target event; the decision value is a parameter value representing at least part of the characteristics of the target event;
a first data generation process comprising:
generating at least one value index corresponding to the judgment rule according to the hit field name and at least one rule hit path in the judgment rule;
if the judging rule corresponds to a plurality of value indexes, generating a plurality of group values of each value index corresponding to the judging rule according to the data format of the data content corresponding to each sub-index included in each value index in the log of the target event;
generating an initial array of the judgment rule according to the maximum value of the array values corresponding to all the value indexes, wherein the dimension of the initial array is the same as the maximum value of the array values of all the value indexes corresponding to the judgment rule;
according to each value index, at least one initial judgment value corresponding to each value index is obtained from a log of the target event;
respectively placing initial judgment values corresponding to each value index into corresponding initial arrays to generate initial data structures corresponding to each value index;
the initial data structure body corresponding to each value index is placed in the same first expansion array, and a three-dimensional judgment array corresponding to the judgment rule is generated; dimension W of first extended array K The following conditions are satisfied: w (W) K =3-W C ；W C Is the dimension of the initial array.
According to a third aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing a computer program which, when executed by a processor, implements a method of generating data to be measured as described above.
According to a fourth aspect of the present invention, there is provided an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing a method of generating data to be measured as described above when executing the computer program.
The invention has at least the following beneficial effects:
in the invention, a plurality of value indexes can be generated by hit field names and at least one corresponding rule hit path, and each value index is used for the value of the hit field in an event log of a corresponding type. Meanwhile, since different types of event logs may come from different service scenarios, the decision value is finally obtained from the corresponding Json log and also exists in different forms. The invention can make a plurality of judgment values corresponding to the same hit field exist in the form of a standard three-dimensional array through the first data processing. Therefore, the data structure forms of the acquired target data can be unified, and the plurality of judgment values can be conveniently processed by using the same data analysis method later. The applicability of the acquired data to be detected is improved, and the convenience of later data processing is improved.
In addition, because of the complexity of two-dimensional array form in the log data of the existing business scene, the design of the data structure can already satisfy almost all data relations. Therefore, the method and the device select to set the finally generated data structure in the three-dimensional array form, so that almost all log data in the existing service scene can be compatible, further, dynamic value taking of any data structure within the three-dimensional array structure required by most of the existing service scenes can be realized, and applicability of acquiring the judgment value is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a method for generating data to be tested according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a corresponding tree structure in a Json log of a certain target event according to an embodiment of the present invention;
fig. 3 is a block diagram of a device for generating data to be tested according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
As a possible embodiment of the present invention, as shown in fig. 1, there is provided a method for generating data to be measured, the method including the steps of:
s100: and acquiring a judging rule set corresponding to the target event. The set of decision rules includes a plurality of decision rules, each decision rule including a unique corresponding hit field name and at least one corresponding rule hit path.
Because the invention is applicable to the scenario of taking the value of the log data in various business scenarios. Because different types of event logs may come from different service scenarios, the value paths corresponding to the same hit field in different event logs are different, that is, the corresponding rule hit paths are different. Therefore, each decision rule includes a unique corresponding hit field name and at least one corresponding rule hit path. And by combining the hit field name with at least one corresponding regular hit path, a plurality of value indexes may be generated, each value index being used to value a hit field in an event log of a corresponding type.
S200: and carrying out first data generation processing on the judgment value corresponding to each judgment rule in the log of the target event, and generating a judgment value set corresponding to the target event. The decision value is a parameter value representing at least part of the characteristics of the target event.
A first data generation process comprising:
s201: and generating at least one value index corresponding to the judgment rule according to the hit field name and at least one rule hit path in the judgment rule.
As illustrated in the following example, the hit field name is md5, and its corresponding regular hit paths may be process_info_parent.
Thus, two value indexes, process_info_parent, file_info.md5 and process_info_self_info.md5, respectively, are generated.
S202: if the judging rule corresponds to a plurality of value indexes, generating a plurality of group values of each value index corresponding to the judging rule according to the data format of the data content corresponding to each sub-index included in each value index in the log of the target event.
Specifically, S202 includes:
s222: the same initial array value is configured for each value index. The initial array value is 0.
S223: and acquiring the data content of each value index in the log of the target event. Preferably, the log of the target event is a json log of the target event.
S224: and when the index content corresponding to any sub-index in the value indexes is provided with the array identifier, carrying out accumulated counting on the initial array value corresponding to the value index so as to generate the array value of each value index corresponding to the judging rule.
The following examples are specific:
the sub-indices in process_info_parent.file_info.md5 are process_info_parent and file_info, respectively, and when the corresponding data content of the sub-index in the Json log exists in the form of an array, a partial array identifier [ -follows the sub-index. The data content corresponding to the sub-index in the Json log is: "Process_info_parent: and [ ("), thus, traversing the data content of the command line where each sub-index in the Json log is located, and adding 1 to the initial array value whenever" [ ") is found, thus finally generating the array value of each value index. Typically the array value is less than or equal to 2.
S203: and generating an initial array of the judgment rule according to the maximum value of the array values corresponding to all the value indexes, wherein the dimension of the initial array is the same as the maximum value of the array values of all the value indexes corresponding to the judgment rule.
S204: and according to each value index, acquiring at least one initial judgment value corresponding to each value index from the log of the target event.
S205: and respectively placing the initial judgment values corresponding to each value index into corresponding initial arrays to generate initial data structures corresponding to each value index.
S206: and placing the initial data structure body corresponding to each value index into the same first expansion array to generate a three-dimensional judgment array corresponding to the judgment rule. Dimension W of first extended array K The following conditions are satisfied: w (W) K =3-W C 。W C Is the dimension of the initial array.
Specifically, S204 to S206 will be described as examples below, where the array value corresponding to process_info_parent. The array value corresponding to process_info_self.file_info.md5 is 2. Both initial arrays finally generated are two-dimensional arrays [ [ ] ].
If the initial determination value corresponding to the process_info_parent.file_info.md5 is 995, 55664. The initial determination value corresponding to process_info_self.file_info.md5 is 325, 55354,6665. The initial data structure that is ultimately generated is [ [995, 55664] ] and [ [325, 55354,6665] ].
According to the initial data structure, the first expansion array is a one-dimensional array [ ]. Finally, a three-dimensional judgment array corresponding to the judgment rule is generated as [ [995, 55664] ], and [ (325, 55354,6665] ].
In addition, after S201, the first data generation process further includes:
s211: if the judging rule only corresponds to one value index, generating a second expansion array corresponding to the judging rule, wherein the second expansion array is a three-dimensional array.
S212: and according to the value index, at least one initial judgment value corresponding to the value index is obtained from the log of the target event.
S213: and all the initial judgment values are put into the second expansion array, and a three-dimensional judgment array corresponding to the judgment rule is generated.
Specifically, S211 to S213 will be described as the following examples, and the value indexes are only: process_info_parent. Its corresponding initial decision value is 995, 55664. The second expanded array is generated as a three-dimensional array [ [ ] ].
All the initial judgment values are put into a second expansion array, and finally, the three-dimensional judgment array corresponding to the judgment rule is generated as [ [995, 55664] ].
Therefore, after the first data processing in the embodiment, the data to be extracted in different forms in different service scenes can be finally generated into the standard three-dimensional array format, so that the purpose of unifying the data forms is achieved. Specifically, the generated standard three-dimensional array format has the following forms: [ [ xxx, xxx, xx ] ], [ [ [ xxx, xxx ], [ [ xx ], [ ] ] ] and [ [ [ xx, xx ] ], [ [ xxx, xx ] ] ].
Therefore, the data structure forms of the acquired target data can be unified, and the plurality of judgment values can be conveniently processed by using the same data analysis method later. The applicability of the acquired data to be detected is improved, and the convenience of later data processing is improved.
As another embodiment of the present invention, after generating the set of decision values corresponding to the target event, the method further includes:
s300: each of the set of decision values is decided using the decision information in each decision rule.
Specifically, S300 includes:
s301: if the hit field name corresponding to the three-dimensional judgment array is the same as the hit field name corresponding to the judgment rule, determining the three-dimensional judgment array as a target three-dimensional judgment array of the judgment rule.
S302: and performing traversal judgment on the judgment values in the target three-dimensional judgment array by using the judgment information corresponding to each judgment rule.
For example, the three-dimensional judgment array is [ [995, 55664] ], [ (325, 55354,6665] ], and the judgment rule is to judge whether each judgment value is larger than 1000. The method can directly designate the corresponding data traversing range as [ [995, 55664] ], [ [325, 55354,6665] ] ] when the for statement is used for cyclic traversal, and then write a set of judging conditions, namely, the cyclic judgment can be carried out on 995, 55664, 325, 55354 and 6665 in the data traversing range, and corresponding judging results are respectively generated. Because the data structures corresponding to the [ [995, 55664] ] and the [ [325, 55354,6665] ] are two-dimensional arrays, the consistency of the data structures is higher, and a set of value and judgment rules are more convenient to use for judging different values in the data structures.
At the same time, there is also a possibility that a plurality of different rule hit paths are included in the same decision rule. And each rule hit path may be respectively in one-to-one correspondence with each subarray in the three-dimensional array. The first rule hit path is corresponding to [ [995, 55664] ] in [ [995, 55664] ], [ (325, 55354,6665] ]; [ [325, 55354,6665] ] corresponds to the second rule hit path. Therefore, the correspondence between the judging result and the rule hit path can be determined according to the correspondence between the rule hit path and the subarray, and the matching accuracy between the judging results is improved.
Through the first data generation processing, the plurality of initial determination values corresponding to the same determination rule may be represented in the same data format, and the determination rules of the plurality of initial determination values corresponding to the same determination rule may be the same. The decision operations performed on each initial decision value are the same, so that the for loop statement can be used to traverse the corresponding three-dimensional array to perform anomaly detection on multiple initial decision values corresponding to the same decision rule. According to the method, the developer does not need to rewrite the same judgment rules for each data structure in different forms, and the development workload of the developer can be saved.
S400: if the judging value accords with the judging condition in the corresponding judging information, determining that the judging value hits the judging rule corresponding to the judging information.
S500: and taking the judgment rule corresponding to the judgment value when the judgment value hits the judgment information as the hit judgment rule corresponding to the judgment value.
S600: and taking preset output information corresponding to the hit judgment rule as the marking information of the target event. The preset output information comprises event portrait tags and mark judgment information.
Specifically, the decision rule set further includes at least one tag hit information, where the tag hit information includes a tag hit path, an event portrayal tag, and tag decision information. The tag hit path is one or more regular hit paths that were previously tagged. The tag determination information is a determination value corresponding to the tag hit path.
S600 includes:
s601: if the rule hit path in the hit judgment rule and any mark hit path have an intersection, and a path node in the intersection is greater than or equal to a first node threshold value, determining that mark hit information corresponding to the mark hit path is preset output information corresponding to the hit judgment rule.
S602: and taking event portrait labels and mark judgment information in preset output information corresponding to the hit judgment rule as mark information of the target event.
As shown in fig. 2, the corresponding tree structure in the Json log of a certain target event. Each leaf node may generate a plurality of corresponding leaf nodes, with the leaf node of the upper layer being the parent of the adjacent leaf node of the next layer. Wherein each circle represents a leaf node and the letters in the circles represent the node name for that node.
If the tag hit information is a decision rule corresponding to a rule hit path a.b.d.h.j, that is, the tag hit path a.b.d.h.j. The tag judgment information corresponding to the tag hit information is the md5 value in the a.b.d.h.j path.
If the hit determination rule corresponding to the determination value is a.b.c, the intersection between the hit determination rule and the tag hit path a.b.d.h.j is a.b, the number of path nodes in the intersection is a and b, and the number is 2, which meets the condition. Therefore, the mark hit information corresponding to the mark hit path a.b.d.h.j is outputted, and the image information corresponding to a.b.c itself is not outputted.
Similarly, if the tag hit information with the regular hit path a.b.c is preset, the tag hit information with the tag hit path a.b.c is hit by the above determination value, and the tag hit information with the tag hit path a.b.c is output at the same time.
If the determination value also hits the determination rule corresponding to the rule hit path a.p.r.s and/or a.p.r.t is also the tag hit path, the tag hit information corresponding to the tag hit path a.p.r.s and/or a.p.r.t is also output.
Often different sub-features may represent the same feature of the target event, whereas the decision rule in the present invention is transformed according to each sub-feature. Therefore, when different decision rules are hit, the same event portrait tag may be output. However, since the tag judgment information finally generated in the present invention is for use by a downstream person, only one piece of the most appropriate tag judgment information is output from the same event portrait tag.
Meanwhile, as each judging rule is a judging rule for determining the same characteristic of the target event from different angles, judging values of the different angles can appear in each stage of the running process of the target event, and meanwhile, based on the data storage rule of the Json log, the judging values of the different angles can be stored in each father-son node of the Json log corresponding to the target event according to the corresponding appearance sequence. This causes overlapping portions of the storage paths (value indexes) of the respective determination values. In the invention, the regular hit path is the storage path corresponding to the judgment value. So that there will be intersections of the regular hit paths corresponding to the same event portrait tag. According to the characteristics, the rule hit path and each mark hit path are compared, so that when the judgment rule corresponding to any sub-feature is hit, the same mark judgment information which is convenient for downstream personnel to use can be output.
According to a second aspect of the present invention, as shown in fig. 3, there is provided a generating apparatus of data to be measured, comprising:
and the rule acquisition module is used for acquiring a judging rule set corresponding to the target event. The set of decision rules includes a plurality of decision rules. Each decision rule includes a unique corresponding hit field name and at least one corresponding rule hit path.
And the data generation module is used for carrying out first data generation processing on the judgment value corresponding to each judgment rule in the log of the target event to generate a judgment value set corresponding to the target event. The decision value is a parameter value representing at least part of the characteristics of the target event.
A first data generation process comprising:
and generating at least one value index corresponding to the judgment rule according to the hit field name and at least one rule hit path in the judgment rule.
If the judging rule corresponds to a plurality of value indexes, generating a plurality of group values of each value index corresponding to the judging rule according to the data format of the data content corresponding to each sub-index included in each value index in the log of the target event.
And generating an initial array of the judgment rule according to the maximum value of the array values corresponding to all the value indexes, wherein the dimension of the initial array is the same as the maximum value of the array values of all the value indexes corresponding to the judgment rule.
And according to each value index, acquiring at least one initial judgment value corresponding to each value index from the log of the target event.
And respectively placing the initial judgment values corresponding to each value index into corresponding initial arrays to generate initial data structures corresponding to each value index.
And placing the initial data structure body corresponding to each value index into the same first expansion array to generate a three-dimensional judgment array corresponding to the judgment rule. Dimension W of first extended array K Satisfy the following requirementsThe following conditions were: w (W) K =3-W C 。W C Is the dimension of the initial array.
Furthermore, although the steps of the methods in the present disclosure are depicted in a particular order in the drawings, this does not require or imply that the steps must be performed in that particular order or that all illustrated steps be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (may be a personal computer, a server, a mobile terminal, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
Those skilled in the art will appreciate that the various aspects of the invention may be implemented as a system, method, or program product. Accordingly, aspects of the invention may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.
An electronic device according to this embodiment of the invention. The electronic device is merely an example, and should not impose any limitations on the functionality and scope of use of embodiments of the present invention.
The electronic device is in the form of a general purpose computing device. Components of an electronic device may include, but are not limited to: the at least one processor, the at least one memory, and a bus connecting the various system components, including the memory and the processor.
Wherein the memory stores program code that is executable by the processor to cause the processor to perform steps according to various exemplary embodiments of the present invention described in the above section of the exemplary method of this specification.
The storage may include readable media in the form of volatile storage, such as Random Access Memory (RAM) and/or cache memory, and may further include Read Only Memory (ROM).
The storage may also include a program/utility having a set (at least one) of program modules including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
The bus may be one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.
The electronic device may also communicate with one or more external devices (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device, and/or with any device (e.g., router, modem, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication may be through an input/output (m/O) interface. And, the electronic device may also communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through a network adapter. The network adapter communicates with other modules of the electronic device via a bus. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with an electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAmD systems, tape drives, data backup storage systems, and the like.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (may be a personal computer, a server, a terminal device, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, a computer-readable storage medium having stored thereon a program product capable of implementing the method described above in the present specification is also provided. In some possible embodiments, the aspects of the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the invention as described in the "exemplary method" section of this specification, when the program product is run on the terminal device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
Furthermore, the above-described drawings are only schematic illustrations of processes included in the method according to the exemplary embodiment of the present invention, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
The present invention is not limited to the above embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.

Claims (10)

1. A method of generating data to be measured, the method comprising the steps of:
acquiring a judging rule set corresponding to a target event; the set of decision rules includes a plurality of decision rules; each judgment rule comprises a unique corresponding hit field name and at least one corresponding rule hit path;
performing first data generation processing on the judgment value corresponding to each judgment rule in the log of the target event to generate a judgment value set corresponding to the target event; the decision value is a parameter value representing at least part of the characteristics of the target event;
the first data generation process includes:
generating at least one value index corresponding to the judgment rule according to the hit field name and at least one rule hit path in the judgment rule;
if the judging rule corresponds to a plurality of value indexes, generating a plurality of groups of values of each value index corresponding to the judging rule according to the data format of the data content corresponding to each sub-index included in each value index in the log of the target event;
generating an initial array of the judgment rule according to the maximum value of array values corresponding to all the value indexes, wherein the dimension of the initial array is the same as the maximum value of the array values of all the value indexes corresponding to the judgment rule;
according to each value index, at least one initial judgment value corresponding to each value index is obtained from the log of the target event;
respectively placing initial judgment values corresponding to each value index into corresponding initial arrays to generate initial data structures corresponding to each value index;
placing the initial data structure body corresponding to each value index into the same first expansion array to generate a three-dimensional judgment array corresponding to the judgment rule; dimension W of the first expansion array K The following conditions are satisfied: w (W) K =3-W C ；W C Is the dimension of the initial array.
2. The method of claim 1, wherein after generating at least one value index corresponding to the decision rule, the first data generation process further comprises:
if the judging rule only corresponds to one value index, generating a second expansion array corresponding to the judging rule, wherein the second expansion array is a three-dimensional array;
according to the value index, at least one initial judgment value corresponding to the value index is obtained from the log of the target event;
and all the initial judgment values are placed into a second expansion array, and a three-dimensional judgment array corresponding to the judgment rule is generated.
3. The method of claim 1, wherein generating the array value for each value index corresponding to the decision rule according to the data format of the data content corresponding to each sub-index included in each value index in the log of the target event, comprises:
configuring the same initial array value for each value index;
acquiring the data content of each value index in the log of the target event;
and when the index content corresponding to any sub-index in the value indexes is provided with the array identifier, carrying out accumulated counting on the initial array value corresponding to the value index so as to generate the array value of each value index corresponding to the judging rule.
4. The method of claim 1, wherein after generating the set of decision values for the target event, the method further comprises:
determining each of the set of determination values using the determination information in each determination rule;
if the judging value accords with the judging condition in the corresponding judging information, determining that the judging value hits the judging rule corresponding to the judging information;
taking the corresponding judgment rule when the judgment value hits the judgment information as the hit judgment rule corresponding to the judgment value;
taking preset output information corresponding to the hit judgment rule as the marking information of the target event; the preset output information comprises event portrait labels and mark judging information.
5. The method of claim 4, wherein determining each of the set of decision values using the decision information in each decision rule comprises:
if the hit field name corresponding to the three-dimensional judgment array is the same as the hit field name corresponding to the judgment rule, determining the three-dimensional judgment array as a target three-dimensional judgment array of the judgment rule;
and performing traversal judgment on the judgment values in the target three-dimensional judgment array by using the judgment information corresponding to each judgment rule.
6. The method of claim 4, wherein the set of decision rules further comprises at least one tag hit information, the tag hit information comprising a tag hit path, an event portrayal tag, and tag decision information; the marked hit path is one or more regular hit paths marked in advance;
and taking preset output information corresponding to the hit judgment rule as marking information of the target event, wherein the preset output information comprises the following components:
if the rule hit path in the hit judgment rule and any mark hit path have an intersection, and a path node in the intersection is larger than or equal to a first node threshold value, determining mark hit information corresponding to the mark hit path as preset output information corresponding to the hit judgment rule;
and taking event portrait labels and mark judgment information in preset output information corresponding to the hit judgment rule as mark information of the target event.
7. The method of claim 1, wherein the log of the target event is a json log of the target event.
8. A device for generating data to be measured, comprising:
the rule acquisition module is used for acquiring a judging rule set corresponding to the target event; the set of decision rules includes a plurality of decision rules; each judgment rule comprises a unique corresponding hit field name and at least one corresponding rule hit path;
the data generation module is used for carrying out first data generation processing on the judgment value corresponding to each judgment rule in the log of the target event to generate a judgment value set corresponding to the target event; the decision value is a parameter value representing at least part of the characteristics of the target event;
the first data generation process includes:
generating at least one value index corresponding to the judgment rule according to the hit field name and at least one rule hit path in the judgment rule;
if the judging rule corresponds to a plurality of value indexes, generating a plurality of groups of values of each value index corresponding to the judging rule according to the data format of the data content corresponding to each sub-index included in each value index in the log of the target event;
generating an initial array of the judgment rule according to the maximum value of array values corresponding to all the value indexes, wherein the dimension of the initial array is the same as the maximum value of the array values of all the value indexes corresponding to the judgment rule;
according to each value index, at least one initial judgment value corresponding to each value index is obtained from the log of the target event;
respectively placing initial judgment values corresponding to each value index into corresponding initial arrays to generate initial data structures corresponding to each value index;
placing the initial data structure body corresponding to each value index into the same first expansion array to generate a three-dimensional judgment array corresponding to the judgment rule; dimension W of the first expansion array K The following conditions are satisfied: w (W) K =3-W C ；W C Is the dimension of the initial array.
9. A non-transitory computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements a method of generating data to be measured according to any one of claims 1 to 7.
10. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements a method of generating data to be measured according to any of claims 1 to 7 when executing the computer program.
CN202311290884.4A 2023-10-08 2023-10-08 Method and device for generating data to be tested, storage medium and electronic equipment Active CN117033318B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311290884.4A CN117033318B (en) 2023-10-08 2023-10-08 Method and device for generating data to be tested, storage medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311290884.4A CN117033318B (en) 2023-10-08 2023-10-08 Method and device for generating data to be tested, storage medium and electronic equipment

Publications (2)

Publication Number Publication Date
CN117033318A CN117033318A (en) 2023-11-10
CN117033318B true CN117033318B (en) 2023-12-08

Family

ID=88632221

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311290884.4A Active CN117033318B (en) 2023-10-08 2023-10-08 Method and device for generating data to be tested, storage medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN117033318B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111144697A (en) * 2019-11-29 2020-05-12 泰康保险集团股份有限公司 Data processing method, data processing device, storage medium and electronic equipment
WO2023056946A1 (en) * 2021-10-09 2023-04-13 上海淇馥信息技术有限公司 Data caching method and apparatus, and electronic device
CN116170300A (en) * 2023-02-24 2023-05-26 山东云天安全技术有限公司 Data processing method, electronic equipment and medium for determining abnormal log information

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111144697A (en) * 2019-11-29 2020-05-12 泰康保险集团股份有限公司 Data processing method, data processing device, storage medium and electronic equipment
WO2023056946A1 (en) * 2021-10-09 2023-04-13 上海淇馥信息技术有限公司 Data caching method and apparatus, and electronic device
CN116170300A (en) * 2023-02-24 2023-05-26 山东云天安全技术有限公司 Data processing method, electronic equipment and medium for determining abnormal log information

Also Published As

Publication number Publication date
CN117033318A (en) 2023-11-10

Similar Documents

Publication Publication Date Title
CN110457277B (en) Service processing performance analysis method, device, equipment and storage medium
CN107506300B (en) User interface testing method, device, server and storage medium
CN104346153A (en) Method and system for translating text information of application programs
CN109298855B (en) Network target range management system, implementation method and device thereof, and storage medium
CN114328208A (en) Code detection method and device, electronic equipment and storage medium
CN112015467A (en) Point burying method, medium, device and computing equipment
CN113114680A (en) Detection method and detection device for file uploading vulnerability
CN113094625B (en) Page element positioning method and device, electronic equipment and storage medium
CN117009911B (en) Abnormality determination method and device for target event, medium and electronic equipment
CN110990346A (en) File data processing method, device, equipment and storage medium based on block chain
CN110555352A (en) interest point identification method, device, server and storage medium
CN117033318B (en) Method and device for generating data to be tested, storage medium and electronic equipment
CN112235262A (en) Message analysis method and device, electronic equipment and computer readable storage medium
CN110515758A (en) A kind of Fault Locating Method, device, computer equipment and storage medium
CN115964701A (en) Application security detection method and device, storage medium and electronic equipment
CN117034210B (en) Event image generation method and device, storage medium and electronic equipment
CN117034260B (en) Event judgment information generation method and device, medium and electronic equipment
CN113535660B (en) Android log storage method and device
CN115600216B (en) Detection method, detection device, detection equipment and storage medium
US11449408B2 (en) Method, device, and computer program product for obtaining diagnostic information
CN114611816B (en) Potential event prediction method, device, equipment and storage medium
CN112000573B (en) Code quality monitoring method and device, computer equipment and medium
CN116305131B (en) Static confusion removing method and system for script
CN115757145A (en) Method, device, equipment and storage medium for developing client interface
EP4235407A1 (en) Method and system for mapping intermediate representation objects for facilitating incremental analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant