CN114491198A

CN114491198A - Data labeling, packaging and acquiring method and system

Info

Publication number: CN114491198A
Application number: CN202111509356.4A
Authority: CN
Inventors: 司红星; 鲍晨阳
Original assignee: Siwei Chuangzhi Beijing Technology Development Co ltd
Current assignee: Siwei Chuangzhi Beijing Technology Development Co ltd
Priority date: 2021-12-10
Filing date: 2021-12-10
Publication date: 2022-05-13

Abstract

The invention discloses a method and a system for marking, packaging and acquiring data, which relate to the field of data processing, wherein the attribute of the data is determined according to the type of the data, the label of the data is determined according to the characteristics of the data, and the data is marked according to the attribute and the label of the data; the method comprises the steps of generating second dictionary type data according to the data requirement condition of a script, the data attribute and the label of the data, associating the script with the second dictionary type data, obtaining data required by executing the script from a database according to the second dictionary type data, obtaining the data required by the new script through keywords when the new script is added, reducing the complexity, improving the efficiency and ensuring that the script is executed correctly.

Description

Data labeling, packaging and acquiring method and system

Technical Field

The invention relates to the field of data processing, in particular to a method and a system for labeling, packaging and acquiring data.

Background

In the process of automatically permeating data by using a permeation tool, data is required to be transferred among scripts with different functions, wherein the data comprises data input by a user and data obtained by executing the last script. In a typical infiltration tool, data required for script execution is confirmed in advance. After the last script is executed, the obtained data is automatically sent to the next determined script and the script is executed. In this mode, the script is highly coupled to the data. In the mode, a large amount of operations are required to add or delete the script, so that the complexity is high, the efficiency is low, and the script execution error is easily caused.

Disclosure of Invention

The embodiment of the invention provides a data marking, packaging and acquiring method and system, which are used for solving the defects of high complexity, low efficiency and easy script execution error in the prior art.

In order to achieve the above object, the method and system for labeling, encapsulating and acquiring data provided by the embodiment of the present invention include the following technical solutions:

in a first aspect, a data annotation method provided in an embodiment of the present invention includes the following steps:

determining the attribute of the data according to the type of the data;

determining a label of the data according to the characteristics of the data;

and marking the data according to the attribute and the label of the data.

As a preferred implementation manner of the first aspect, the determining the label of the data according to the characteristic of the data includes:

and determining the label of the data according to the surface characteristics of the data.

and determining the label of the data according to the operating characteristics of the data.

In a second aspect, a data encapsulation method provided in an embodiment of the present invention includes the following steps:

according to the attributes and the labels of the data, a first dictionary structure is constructed for the data, and first dictionary type data are generated;

the format of the first dictionary-type data is converted into a json format.

As a preferred embodiment of the second aspect, the first dictionary structure is: { tag: { param: [ data ] }, wherein tag is a label and is a key of the first-level dictionary; param is attribute information and is a key of the second-level dictionary.

In a third aspect, a data acquisition method provided in an embodiment of the present invention includes the following steps:

generating second dictionary type data according to the requirement condition of the script on the data, the attribute of the data and the label;

associating the script with the second dictionary type data;

and acquiring data required by executing the script from a database according to the second dictionary type data.

As a preferred implementation of the third aspect, generating the second dictionary type data according to the requirement condition of the script on the data, the attribute of the data, and the tag includes:

the data is divided into necessary data, dynamic data and optional data, wherein the necessary data is the data necessary for executing the script, the dynamic data is at least one of the data necessary for executing the script, and the optional data is the data selectable for executing the script.

As a preferred implementation manner of the third aspect, the second dictionary type data has a structure that:

{ "required" { "tag1" [ "param1" ] … }, "choice" { "tag2" [ "param2" ], "tag3" [ "param3" ] … }, "options" "{" tag 4"] [" param4"] } … }, wherein required is an essential data identifier, choice is a dynamic data identifier, options are optional data identifiers, tag1, tag2, tag3, and tag4 are tags of the respective data, and param1, param2, am3, and param4 are attributes of the respective data.

In a fourth aspect, a data annotation system provided in an embodiment of the present invention includes:

a first determination module configured to determine an attribute of data according to a type of the data;

a second determination module configured to determine a tag of the data according to a usage of the data;

and the standard module is configured to label the data according to the attributes and the labels of the data.

In a fifth aspect, a data acquisition system provided in an embodiment of the present invention includes:

the generating module is configured to generate second dictionary type data according to the requirement condition of the script on the data, the attribute of the data and the label;

an association module configured to associate the script with the second dictionary type data;

and the acquisition module is configured to acquire data required for executing the script from a database according to the second dictionary type data.

In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium storing a computer program for executing the method according to any one of the first, second, and third aspects.

In a seventh aspect, an electronic device provided in an embodiment of the present invention includes:

a processor;

a memory for storing the processor-executable instructions;

the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any of the first, second, and third aspects described above.

The data marking, packaging and acquiring method and system provided by the embodiment of the invention have the following beneficial effects:

the data are labeled by using the labels and the attributes, the type and the purpose of the data can be described at the same time, the data are filtered in the data acquisition process, the data required by the new script can be acquired by the keywords when the new script is added, the complexity is reduced, the efficiency is improved, and the script can be correctly executed.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.

Fig. 1 is a schematic flow chart of a data annotation method according to an embodiment of the present invention.

Fig. 2 is a schematic flow chart of a data encapsulation method according to an embodiment of the present invention.

Fig. 3 is a schematic flow chart of a data acquisition method according to an embodiment of the present invention.

Fig. 4 is a diagram of a dictionary type data structure framework according to an embodiment of the present invention.

Fig. 5 is a schematic diagram of another dictionary type data structure framework according to an embodiment of the present invention.

Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.

Example 1

As shown in fig. 1, the data annotation method provided in the embodiment of the present invention includes the following steps:

s101, determining the attribute of the data according to the type of the data.

Specifically, the attribute of the data is a type of the data, for example: data http:// www.xxx.com is used as a uniform resource locator (URL for short) in the script, and the attribute of the data is URL; for another example, a user name and a password are admin/admin, the attribute of the first data admin is username, and the attribute of the second data admin is password.

S102, according to the characteristics of the data, the label of the data is determined.

Specifically, the label of the data is represented as a feature of the data, and simply, the label is a description of the use of the data in the permeation process. For example: the data http:// www.xxx.com/login is a url, and the content displayed in the page by the url is found to be a landing page after the url is accessed, so that the label of the data is defined as login _ url.

As an alternative embodiment of the present invention, the process comprises:

Specifically, data http:// www.xxx.com/asd. jspid 123 is a url, which is found to be at risk of injection by observing the surface features of the url, so the label of the url is defined as sql _ inj _ pre _ test.

As an optional embodiment of the present invention, the process further comprises:

Specifically, acquiring the running characteristic of the data, and if finding that the data http:// www.xxx.com/asd. jspid ═ 123 can really complete the injection, creating a label for the url as sql _ inj _ success.

When describing data using tags and attributes, the data "what" and "what to do" can be described simultaneously. The two dimensions can also sufficiently describe the characteristics of the penetration data, and play a role in filtering data in the data searching process. When a new script is added, the method is used for acquiring the data required by the new script from the database only through the keywords, so that the complexity is reduced, and the efficiency is improved.

And S103, marking the data according to the attribute and the label of the data.

Example 2

As shown in fig. 2, the data encapsulation method provided by the embodiment of the present invention includes the following steps:

s201, constructing a first dictionary structure for the data according to the attributes and the labels of the data, and generating first dictionary type data;

s202, converting the format of the first dictionary type data into a json format.

As shown in fig. 4, as an alternative embodiment of the present invention, the first dictionary structure is: { tag: { param: [ data ] }, wherein tag is a label and is a key of the first-level dictionary; param is attribute information and is a key of the second-level dictionary.

Specifically, when data is packaged, the data is assumed to be data1 and data2 …, wherein data1 and data2 … … are specific contents in the data generated by the script, a two-layer dictionary storage structure { tag: { param: [ data ] } is constructed, wherein tag is label information and is a key of a first-layer dictionary, and param is attribute information and is a key of a second-layer dictionary. The data is penetration data and is the value of the second layer dictionary. And initializing the double-layer dictionary storage structure to generate empty dictionary type data. When each piece of data1, data2 … … stored in the database is operated, assuming that the corresponding tags and attributes of the data1 are data1_ tag and data1_ param, then the double-layer dictionary type data is newly created according to the data, and the target storage dictionary is updated by using the piece of dictionary data. If the label and the attribute do not exist in the target storage dictionary, directly updating the target storage dictionary by using the dictionary data; and if the data list corresponding to the label and the attribute exists in the target storage dictionary, merging the corresponding two list data. And in the same way, the creation of the storage dictionary data corresponding to the penetration data is completed. And converting the dictionary data into json-format data and storing the json-format data into a database, finishing the storage process of the penetration data, and performing basic function operations such as transmission, storage and the like on the character string data.

Example 3

As shown in fig. 3, the data acquisition method provided in the embodiment of the present invention includes the following steps:

s301, generating second dictionary type data according to the data requirement condition of the script, the attribute of the data and the label.

As an alternative embodiment of the present invention, the process comprises:

dividing data into necessary data, dynamic data and optional data, wherein the necessary data is necessary data for executing the script; the dynamic data is required to have at least one of the data for executing the script, and if only one piece of data exists under the dynamic data, the data is not substantially different from the necessary data; the optional data is data which can be selected by the execution script and does not influence the execution of the script if not.

S302, the script is associated with second dictionary type data.

And S303, acquiring data required for executing the script from the database according to the second dictionary type data.

As shown in fig. 5, as an alternative embodiment of the present invention, the structure of the second dictionary type data is:

Specifically, when defining data required by a script, firstly, constructing dictionary type data { "required": { }, "choice": and "options": and defining the labels and the attributes of the required data under the keys of three dictionaries of required, choice and options respectively by a user according to the requirements of the script. When defining the label and attribute of the required data, creating a new dictionary type data, using the required label as the key of the dictionary type data, and storing the attribute corresponding to the label into a list as the value of the dictionary type data. And finally, storing the obtained dictionary under required, choice and options keys corresponding to the data required by the script. For example, when writing data required for detecting sql injection scripts, the required data is labeled sql _ inj _ pre _ test, and then the definition of the data required by the scripts is:

{ "required" { "sql _ inj _ pre _ test": [ "url" ] }, "choice": { }, "options": { } }; for another example, when writing data required by a script for landing blasting, the penetration data required by the script is user name and password labeled as logic _ branch _ message, and furthermore, url labeled as logic _ url or logic _ verify _ code _ url, that is, a landing url not including a verification code and a landing url including a verification code, are required. The data required by the script is defined as { "required" { "logic _ branch _ message": [ "user name", "password" ] }, "choice": "{" logic _ vertical _ code _ url ": [" url "], [" url "] }," options ": { }. When the system executes the script, the corresponding data is searched from the database through the label and the attribute in the definition of the required data, and the data is packaged and then sent to the script to be used as the parameter of the script to participate in operation.

Example 4

The data annotation system provided by the embodiment of the invention comprises:

Example 5

The data acquisition system provided by the embodiment of the invention comprises:

Example 6

Fig. 6 is a structure of an electronic device provided by an exemplary embodiment of the present invention. The electronic device may be either or both of the first device and the second device, or a stand-alone device separate from them, which stand-alone device may communicate with the first device and the second device to receive the acquired input signals therefrom. FIG. 6 illustrates a block diagram of an electronic device in accordance with an embodiment of the disclosure. As shown in fig. 6, the electronic device includes one or more processors 401 and memory 402.

The processor 401 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

Memory 402 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by the processor 401 to implement the method of information mining of historical change records of the software program of the disclosed embodiments described above and/or other desired functions. In one example, the electronic device may further include: an input device 403 and an output device 404, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

The input device 403 may also include, for example, a keyboard, a mouse, and the like.

The output device 404 can output various information to the outside. The output devices 404 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.

Of course, for simplicity, only some of the components of the electronic device relevant to the present disclosure are shown in fig. 6, omitting components such as buses, input/output interfaces, and the like. In addition, the electronic device may include any other suitable components, depending on the particular application.

Example 7

In addition to the methods and apparatus described above, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the data labeling, encapsulation, and retrieval methods according to various embodiments of the present disclosure described in the "exemplary methods" section above of this specification.

The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps in the data labeling, packaging, and acquisition methods according to various embodiments of the present disclosure described in the "exemplary methods" section above in this specification.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.

In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts in the embodiments are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by one skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be considered equivalents of the present disclosure. The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

It will be appreciated that the relevant features of the method and apparatus described above are referred to one another. In addition, "first", "second", and the like in the above embodiments are used to distinguish the embodiments, and do not represent merits of the embodiments.

The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

It should be noted that the above-mentioned embodiments do not limit the present invention in any way, and all technical solutions obtained by using equivalent alternatives or equivalent variations fall within the protection scope of the present invention.

Claims

1. A method for annotating data, comprising:

determining the attribute of the data according to the type of the data;

determining a label of the data according to the characteristics of the data;

and marking the data according to the attribute and the label of the data.

2. The data annotation method of claim 1, wherein the determining the tag of the data according to the characteristic of the data comprises:

3. The data annotation method of claim 2, wherein the determining the tag of the data according to the characteristic of the data comprises:

4. A method for encapsulating data, comprising:

the format of the first dictionary-type data is converted into a json format.

5. The data packaging method of claim 4, wherein the first dictionary structure is: { tag: { param: [ data ] }, wherein tag is a label and is a key of the first-level dictionary; param is attribute information and is a key of the second-level dictionary.

6. A method of data acquisition, comprising:

associating the script with the second dictionary type data;

7. The data acquisition method according to claim 6, wherein the generating of the second dictionary type data according to the requirement of the script on the data, the attribute of the data and the tag comprises:

8. The data acquisition method according to claim 6, wherein the second dictionary type data is structured as:

9. A data annotation system, comprising:

a second determination module configured to determine a tag of the data according to a characteristic of the data;

10. A data acquisition system, comprising:

11. A computer-readable storage medium, characterized in that the storage medium stores a computer program for performing the method of any of the preceding claims 1-3 or 4 or 6-7.

12. An electronic device, characterized in that the electronic device comprises:

a processor;

a memory for storing the processor-executable instructions;

the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1-3 or 4 or 6-7.