CN116384352B - Data set generation method, device, equipment and medium - Google Patents
Data set generation method, device, equipment and medium Download PDFInfo
- Publication number
- CN116384352B CN116384352B CN202310665922.3A CN202310665922A CN116384352B CN 116384352 B CN116384352 B CN 116384352B CN 202310665922 A CN202310665922 A CN 202310665922A CN 116384352 B CN116384352 B CN 116384352B
- Authority
- CN
- China
- Prior art keywords
- parameter
- api interface
- api
- data set
- data packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000004458 analytical method Methods 0.000 claims abstract description 14
- 230000007246 mechanism Effects 0.000 claims abstract description 12
- 238000004590 computer program Methods 0.000 claims description 12
- 230000015654 memory Effects 0.000 claims description 10
- 238000012795 verification Methods 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 4
- 239000008186 active pharmaceutical agent Substances 0.000 description 84
- 239000000243 solution Substances 0.000 description 8
- 238000012545 processing Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/174—Form filling; Merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- User Interface Of Digital Computer (AREA)
- Communication Control (AREA)
- Computer And Data Communications (AREA)
Abstract
The application provides a data set generation method, a device, equipment and a medium, wherein the method comprises the following steps: after sending a data acquisition request, receiving an API interface corresponding to the data acquisition request; the API interface carries parameter attributes of parameters for generating a data set and request parameters matched with a data acquisition request; carrying out validity check on the parameter attribute of the API through a pre-checking mechanism; if the parameter attribute of the API interface is legal, verifying the request parameter through an API signature rule, and judging the safety of the API interface; if the API interface is safe, acquiring a parameter data packet which is output by the API interface and used for generating a data set; setting a pass token for the received parameter data packet conforming to the preset line rule according to the preset current limiting rule; and carrying out field analysis on the parameter data packet provided with the pass token to generate a data set.
Description
Technical Field
The present application relates to the field of data processing, and in particular, to a method, an apparatus, a device, and a medium for generating a data set.
Background
Along with the progress of society, office software used by people is more and more intelligent, wherein a report is automatically generated and the field of vision of people is gradually entered, before the report is automatically generated, a data set for filling the report is manually arranged, data in the data set is manually imported into the report, each data in the data set is manually checked in the process of importing the data, so that the accuracy of data filling is ensured, but the data volume in the data set is too large, the condition of missed detection is easy to occur, and the data filling accuracy in the table is low.
Disclosure of Invention
Accordingly, an object of the present application is to provide a method, apparatus, device and medium for generating a data set, which are used for solving the problem of how to reduce the complexity of offline deployment in the prior art.
In a first aspect, an embodiment of the present application provides a method for generating a data set, including:
after sending a data acquisition request, receiving an API interface corresponding to the data acquisition request; the API interface carries parameter attributes of parameters for generating a data set and request parameters matched with a data acquisition request;
carrying out validity check on the parameter attribute of the API through a pre-checking mechanism;
if the parameter attribute of the API interface is legal, verifying the request parameter through an API signature rule, and judging the safety of the API interface;
if the API interface is safe, acquiring a parameter data packet which is output by the API interface and used for generating a data set;
setting a pass token for the received parameter data packet conforming to the preset line rule according to the preset current limiting rule;
and carrying out field analysis on the parameter data packet provided with the pass token to generate a data set.
Optionally, field parsing is performed on the parameter data packet provided with the pass token, so as to generate a data set, including:
performing field type analysis on the parameter data packet provided with the pass token, and determining the field type of each field; the field types include any one or more of the following types: character string type, numerical value type, picture type, boolean value type;
a data set is generated based on a field type of each field in the parameter data packet provided with the pass token.
Optionally, the method further comprises:
if the parameter data packet provided with the pass token contains a character string type field, converting an English field of the character string type in the parameter data packet provided with the pass token into a Chinese field by utilizing a character recognition model.
Optionally, the method further comprises:
if the parameter data packet provided with the pass token contains a field of the picture type, marking an icon at the field of the picture type in the parameter data packet provided with the pass token.
Optionally, the validity check is performed on the parameter attribute of the API through a pre-checking mechanism, including:
judging whether the parameter type in the parameter attribute of the API is matched with the parameter type of the data set or not to obtain a first matching result;
judging whether each parameter value in the parameter attribute of the API meets the standard requirement or not to obtain a second matching result;
judging whether each parameter value in the parameter attribute of the API meets the business rule of the data set or not to obtain a third matching result;
and if the first matching result, the second matching result and the third matching result all meet the preset requirements, the parameter attribute of the API interface is legal.
Optionally, the standard requirements include either or both of the following:
the parameter value cannot be null and the string length of the parameter value cannot exceed the preset length.
Optionally, the business rule includes any one or two of the following rules:
the parameter opening authority accords with the authority of the current business rule, and the parameter encryption mode accords with the encryption requirement of the current business rule.
In a second aspect, an embodiment of the present application provides a data set generating apparatus, including:
the receiving module is used for receiving an API interface corresponding to the data acquisition request after the data acquisition request is sent; the API interface carries parameter attributes of parameters for generating a data set and request parameters matched with a data acquisition request;
the first verification module is used for verifying the validity of the parameter attribute of the API through a pre-detection mechanism;
the second checking module is used for checking the request parameters through an API signature rule if the parameter attribute of the API interface is legal, and judging the safety of the API interface;
the acquisition module is used for acquiring a parameter data packet which is output by the API interface and used for generating a data set if the API interface is safe;
the setting module is used for setting a pass token for the received parameter data packet conforming to the preset line rule according to the preset current limiting rule;
and the generation module is used for carrying out field analysis on the parameter data packet provided with the pass token to generate a data set.
In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any one of the methods described above when the computer program is executed.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of any of the above.
The data set generation method provided by the embodiment of the application comprises the steps of firstly, after a data acquisition request is sent, receiving an API interface corresponding to the data acquisition request; the API interface carries parameter attributes of parameters for generating a data set and request parameters matched with a data acquisition request; secondly, carrying out validity check on parameter attributes of the API through a pre-checking mechanism; thirdly, if the parameter attribute of the API interface is legal, verifying the request parameter through an API signature rule, and judging the safety of the API interface; if the API interface is safe, acquiring a parameter data packet which is output by the API interface and used for generating a data set; setting a pass token for the received parameter data packet conforming to the preset line rule according to the preset current limiting rule; and carrying out field analysis on the parameter data packet provided with the pass token to generate a data set.
In some embodiments, after the data acquisition API is acquired, validity verification is performed on parameters in the API, and security of the API is verified only if the parameter verification is successful, so that security of acquired data is ensured by the two verification modes, and under the condition of data security, the acquired parameter data packet is limited, so that accuracy of data analysis is further ensured.
In order to make the above objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of a method for generating a data set according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a data set generating device according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present application.
At present, the data set for automatically generating the report is filled and arranged manually, and the data in the data set is filled and arranged in the report manually.
In order to solve the above-mentioned drawbacks, an embodiment of the present application provides a method for generating a data set, as shown in fig. 1, including the following steps:
s101, after a data acquisition request is sent, an API interface corresponding to the data acquisition request is received; the API interface carries parameter attributes of parameters for generating a data set and request parameters matched with a data acquisition request;
s102, verifying the validity of the parameter attribute of the API through a pre-detection mechanism;
s103, if the parameter attribute of the API interface is legal, verifying the request parameter through an API signature rule, and judging the safety of the API interface;
s104, if the API interface is safe, acquiring a parameter data packet which is output by the API interface and used for generating a data set;
s105, setting a pass token for the received parameter data packet conforming to the preset line rule according to the preset current limit rule;
s106, carrying out field analysis on the parameter data packet provided with the pass token to generate a data set.
In the step S101, the method for generating a data set provided by the present application is applied to a data set analysis engine, the data set analysis engine sends a data acquisition request to an API server, and the server returns a corresponding data acquisition API interface to the data set analysis engine after receiving the data acquisition request. The parameter attribute of the parameter used for generating the data set and the request parameter matched with the data acquisition request are carried in the data acquisition API interface.
In step S102, the pre-checking mechanism is configured to check whether the parameters in the API interface are capable of generating the data set, and if the parameters in the API interface are illegal, the data of the generated data set is also erroneous, so that the data acquisition from the API interface is directly stopped, or the data acquisition request is sent to the server again.
In step S103, the verification of the API signature is continued only if the parameter attribute of the API interface is legal, two parameters, namely, a random number and a time stamp, are added to the request parameter before the data acquisition request is sent, and an md5 character string is added on the basis of the two parameters, namely, the random number and the time stamp, and the md5 character string, so as to generate the data signature. When the server feeds back the data to acquire the API, the corresponding data signature is carried in the API. And verifying the fed-back data signature according to the generation rule of the data signature, and if only the difference between the time stamp and the current time in the fed-back data signature is smaller than the preset difference and the request parameter, the random number and the md5 character string are completely the same, passing the security verification of the API interface.
In step S104, the parameter data packet for generating the data set output by the API interface is acquired only when the API interface is secure, and the acquisition of data from the API interface is stopped or the data acquisition request is resent to the server when the API interface is not secure.
In step S105, in order to prevent network congestion, there may be data errors in the generated data set due to too much data, and the acquired parameter data packets need to be limited. The application adopts a token bucket mode to carry out current limiting, and specifically comprises the steps of setting a virtual token bucket, setting a preset number of tokens in the token bucket at intervals, and setting pass tokens for each parameter data packet acquired in one time period. And stopping setting the pass tokens to the parameter data packets when the number of the parameter data packets in the preset time period is larger than the preset number, wherein the parameter data packets can wait for the next time period to be configured with the pass tokens.
In the step S106, only the parameter data packet of the pass token can be field-parsed, and the final parsing result can be used to generate the data set. The parameter data without the pass token is not required to be analyzed, and the data analysis error condition caused by too much data is avoided.
In the embodiment of the application, after the data acquisition API interface is acquired through the six steps, the validity of the parameters in the API interface is verified, the safety of the API interface is verified only under the condition that the parameter verification is successful, the safety of the acquired data is ensured through the two verification modes, and the acquired parameter data packet is limited under the condition of the data safety, so that the accuracy of data analysis is further ensured.
The data in the data set in the present application is used for filling the table, so the data in the data set includes a plurality of types, and the filling contents of the different types of data are different when filling the table, that is, step S106 includes:
step 1061, performing field type analysis on the parameter data packet provided with the pass token, and determining a field type of each field; the field types include any one or more of the following types: character string type, numerical value type, picture type, boolean value type;
step 1062, a data set is generated based on the field type of each field in the parameter data packet provided with the pass token.
In steps 1061 through 1062 described above, each field in the parameter data packet is traversed, and a field type of each field is determined, where the field type includes any one or more of the following types: string type, value type, picture type, boolean value type. The fields of the value type are operated according to the functional expressions in the table. The field of the picture type is used to display the corresponding chart in the table. After the field types of the different fields are determined, the fields of the different types are subjected to response processing, and then a data set is obtained.
For the character string type field and the picture type field, the present application provides a detailed processing manner, and step 1062 includes:
in step 10621, if the parameter data packet with the pass token includes a field of a character string type, the text recognition model is used to convert the english field of the character string type in the parameter data packet with the pass token into the chinese field.
In step 10621, since the parameter data packet is returned to the english character string by default, the application generates the chinese data table, so if the character string type field exists in the parameter data packet, it is necessary to identify the english field by using the text recognition model, and translate the english field into chinese. And the Chinese fields are conveniently utilized to form a data set, and when the data set is further used for rendering a table, the Chinese data table is directly generated.
In step 10622, if the parameter data packet with the pass token includes a field of the picture type, the icon is marked at the field of the picture type in the parameter data packet with the pass token.
In step 10622, when the parameter data packet includes a field of a picture type, a corresponding icon is set for the field, and only field identification is not needed, so as to prevent the situation of confusion in identification between the picture field and the character string field.
In a data table, the data amount is relatively large, and the data table needs to enable the user to clearly understand the meaning to be represented by the data table, so the data requirement for filling the data table is relatively high, and the following verification needs to be performed on the data in the parameter data packet, and step S102 includes:
step 1021, judging whether the parameter type in the parameter attribute of the API is matched with the parameter type of the data set, and obtaining a first matching result;
step 1022, judging whether each parameter value in the parameter attribute of the API meets the standard requirement, and obtaining a second matching result;
step 1023, judging whether each parameter value in the parameter attribute of the API meets the business rule of the data set, and obtaining a third matching result;
step 1024, if the first matching result, the second matching result and the third matching result all meet the preset requirements, the parameter attribute of the API interface is legal.
In step 1021, the data set specifies the parameter type of the required parameter, if the parameter type of the parameter provided by the API interface does not match the parameter type of the parameter specified in the data set, the parameter provided by the API interface cannot be accurately added into the data set, so that it is required to determine whether the parameter type of the parameter of the API interface matches the parameter type of the data set, to obtain a first matching result, where the first matching result includes complete matching or incomplete matching.
That is, the numeric parameter can be matched only with the numeric parameter, and the numeric parameter cannot be matched with the string type parameter.
In the step 1022, parameters in the dataset are preset with a certain standard, which specifically includes: the parameter value cannot be null and the string length of the parameter value cannot exceed the preset length.
In the implementation, parameter values of all parameters of the API interface are traversed, and each parameter value is subjected to standard requirement auditing to obtain a second matching result which is successful or unsuccessful in auditing.
In step 1023 above, business rules are determined from the business scenario generated from the data in the dataset, including either or both of the following rules: the parameter opening authority accords with the authority of the current business rule, and the parameter encryption mode accords with the encryption requirement of the current business rule. For example, in some business scenes, the authority of the acquired data is set, some people can only see 50% of the data, and other people can see 80% of the data; or in some business scenarios, the data may be encrypted, such as an identification card number, a mobile phone number, a bank card number, etc. The business rules can be customized according to actual conditions.
In specific implementation, after judging whether the data provided by the API interface meets the requirements according to the business rules, obtaining a third matching result which meets the rules or does not meet the rules.
For example, the current business scenario is that the identification card number and the bank card number need to be encrypted, but the data sent by the API interface does not encrypt the identification card number and the bank card number, and the data sent by the API interface does not conform to the rule.
Besides the above-mentioned model for verifying the data of the API interface, the application can also adopt modes of fuzzy test, injection attack, invalidation/out-of-range and the like to continuously verify the safety of the API interface, thereby further ensuring the safety of the acquired data.
The application also provides a device for generating the data set, as shown in fig. 2, comprising:
a receiving module 201, configured to receive an API interface corresponding to a data acquisition request after sending the data acquisition request; the API interface carries parameter attributes of parameters for generating a data set and request parameters matched with a data acquisition request;
a first checking module 202, configured to perform validity checking on the parameter attribute of the API through a pre-checking mechanism;
the second checking module 203 is configured to verify the request parameter through an API signature rule if the parameter attribute of the API interface is legal, and determine the security of the API interface;
the obtaining module 204 is configured to obtain a parameter data packet output by the API interface and used for generating a data set if the API interface is secure;
the setting module 205 is configured to set a pass token for the received parameter data packet according to a preset line rule according to a preset current limit rule;
the generating module 206 is configured to perform field parsing on the parameter data packet provided with the pass token, and generate a data set.
Optionally, field parsing is performed on the parameter data packet provided with the pass token, so as to generate a data set, including:
performing field type analysis on the parameter data packet provided with the pass token, and determining the field type of each field; the field types include any one or more of the following types: character string type, numerical value type, picture type, boolean value type;
a data set is generated based on a field type of each field in the parameter data packet provided with the pass token.
Optionally, the apparatus further includes:
if the parameter data packet provided with the pass token contains a character string type field, converting an English field of the character string type in the parameter data packet provided with the pass token into a Chinese field by utilizing a character recognition model.
Optionally, the apparatus further includes:
if the parameter data packet provided with the pass token contains a field of the picture type, marking an icon at the field of the picture type in the parameter data packet provided with the pass token.
Optionally, the validity check is performed on the parameter attribute of the API through a pre-checking mechanism, including:
judging whether the parameter type in the parameter attribute of the API is matched with the parameter type of the data set or not to obtain a first matching result;
judging whether each parameter value in the parameter attribute of the API meets the standard requirement or not to obtain a second matching result;
judging whether each parameter value in the parameter attribute of the API meets the business rule of the data set or not to obtain a third matching result;
and if the first matching result, the second matching result and the third matching result all meet the preset requirements, the parameter attribute of the API interface is legal.
Optionally, the standard requirements include either or both of the following:
the parameter value cannot be null and the string length of the parameter value cannot exceed the preset length.
Optionally, the business rule includes any one or two of the following rules:
the parameter opening authority accords with the authority of the current business rule, and the parameter encryption mode accords with the encryption requirement of the current business rule.
Corresponding to the method for generating a data set in fig. 1, the embodiment of the present application further provides a computer device 300, as shown in fig. 3, where the device includes a memory 301, a processor 302, and a computer program stored in the memory 301 and capable of running on the processor 302, where the processor 302 implements the method for generating a data set when executing the computer program.
Specifically, the above memory 301 and the processor 302 can be general-purpose memories and processors, which are not limited herein, and when the processor 302 runs a computer program stored in the memory 301, the above method for generating a data set can be executed, which solves the problem of how to reduce the complexity of offline deployment in the prior art.
Corresponding to the method of generating a data set in fig. 1, an embodiment of the present application further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of generating a data set described above.
Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk and the like, when a computer program on the storage medium is run, the method for generating the data set can be executed, so that the problem of how to reduce complexity of offline deployment in the prior art is solved.
In the embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments provided in the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
It should be noted that: like reference numerals and letters in the following figures denote like items, and thus once an item is defined in one figure, no further definition or explanation of it is required in the following figures, and furthermore, the terms "first," "second," "third," etc. are used merely to distinguish one description from another and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above examples are only specific embodiments of the present application, and are not intended to limit the scope of the present application, but it should be understood by those skilled in the art that the present application is not limited thereto, and that the present application is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the corresponding technical solutions. Are intended to be encompassed within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method of generating a data set, comprising:
after sending a data acquisition request, receiving an API interface corresponding to the data acquisition request; the API interface carries parameter attributes of parameters for generating a data set and request parameters matched with a data acquisition request;
carrying out validity check on the parameter attribute of the API through a pre-checking mechanism;
if the parameter attribute of the API interface is legal, verifying the request parameter through an API signature rule, and judging the safety of the API interface;
if the API interface is safe, acquiring a parameter data packet which is output by the API interface and used for generating a data set;
setting a pass token for the received parameter data packet conforming to the preset line rule according to the preset current limiting rule;
performing field analysis on the parameter data packet provided with the pass token to generate a data set;
setting a pass token for the received parameter data packet conforming to the preset line rule according to the preset current limiting rule, including:
setting a virtual token bucket, setting a preset number of tokens in the token bucket at intervals, and setting pass tokens for each parameter data packet obtained in one time period;
and stopping setting the pass tokens to the parameter data packets when the number of the parameter data packets in the preset time period is larger than the preset number, wherein the parameter data packets can be configured with the pass tokens after waiting for the next time period.
2. The method of claim 1, wherein field parsing the parameter data packet with the pass token set to generate the data set comprises:
performing field type analysis on the parameter data packet provided with the pass token, and determining the field type of each field; the field types include any one or more of the following types: character string type, numerical value type, picture type, boolean value type;
a data set is generated based on a field type of each field in the parameter data packet provided with the pass token.
3. The method according to claim 2, wherein the method further comprises:
if the parameter data packet provided with the pass token contains a character string type field, converting an English field of the character string type in the parameter data packet provided with the pass token into a Chinese field by utilizing a character recognition model.
4. The method according to claim 2, wherein the method further comprises:
if the parameter data packet provided with the pass token contains a field of the picture type, marking an icon at the field of the picture type in the parameter data packet provided with the pass token.
5. The method of claim 1, wherein verifying the validity of the parameter attribute of the API interface by the pre-check mechanism comprises:
judging whether the parameter type in the parameter attribute of the API is matched with the parameter type of the data set or not to obtain a first matching result;
judging whether each parameter value in the parameter attribute of the API meets the standard requirement or not to obtain a second matching result;
judging whether each parameter value in the parameter attribute of the API meets the business rule of the data set or not to obtain a third matching result;
and if the first matching result, the second matching result and the third matching result all meet the preset requirements, the parameter attribute of the API interface is legal.
6. The method of claim 5, wherein the standard requirements include either or both of the following:
the parameter value cannot be null and the string length of the parameter value cannot exceed the preset length.
7. The method of claim 5, wherein the business rules comprise either or both of the following rules:
the parameter opening authority accords with the authority of the current business rule, and the parameter encryption mode accords with the encryption requirement of the current business rule.
8. A data set generating apparatus, comprising:
the receiving module is used for receiving an API interface corresponding to the data acquisition request after the data acquisition request is sent; the API interface carries parameter attributes of parameters for generating a data set and request parameters matched with a data acquisition request;
the first verification module is used for verifying the validity of the parameter attribute of the API through a pre-detection mechanism;
the second checking module is used for checking the request parameters through an API signature rule if the parameter attribute of the API interface is legal, and judging the safety of the API interface;
the acquisition module is used for acquiring a parameter data packet which is output by the API interface and used for generating a data set if the API interface is safe;
the setting module is used for setting a pass token for the received parameter data packet conforming to the preset line rule according to the preset current limiting rule;
the generation module is used for carrying out field analysis on the parameter data packet provided with the pass token to generate a data set;
the setting module is specifically used for setting a virtual token bucket, setting a preset number of tokens in the token bucket at intervals, and setting pass tokens for each parameter data packet obtained in one time period;
and stopping setting the pass tokens to the parameter data packets when the number of the parameter data packets in the preset time period is larger than the preset number, wherein the parameter data packets can be configured with the pass tokens after waiting for the next time period.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of the preceding claims 1-7 when the computer program is executed.
10. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor performs the steps of the method of any of the preceding claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310665922.3A CN116384352B (en) | 2023-06-07 | 2023-06-07 | Data set generation method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310665922.3A CN116384352B (en) | 2023-06-07 | 2023-06-07 | Data set generation method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116384352A CN116384352A (en) | 2023-07-04 |
| CN116384352B true CN116384352B (en) | 2023-08-25 |
Family
ID=86966058
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310665922.3A Active CN116384352B (en) | 2023-06-07 | 2023-06-07 | Data set generation method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116384352B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106452737A (en) * | 2010-08-11 | 2017-02-22 | 安全第公司 | Systems and methods for secure multi-tenant data storage |
| EP3477527A1 (en) * | 2017-10-31 | 2019-05-01 | Twinpeek | Privacy management |
| CN114614994A (en) * | 2022-03-25 | 2022-06-10 | 芜湖雄狮汽车科技有限公司 | API interface data communication method, device, client and storage medium |
-
2023
- 2023-06-07 CN CN202310665922.3A patent/CN116384352B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106452737A (en) * | 2010-08-11 | 2017-02-22 | 安全第公司 | Systems and methods for secure multi-tenant data storage |
| EP3477527A1 (en) * | 2017-10-31 | 2019-05-01 | Twinpeek | Privacy management |
| CN114614994A (en) * | 2022-03-25 | 2022-06-10 | 芜湖雄狮汽车科技有限公司 | API interface data communication method, device, client and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 软件开发开放API接口的安全处理;刘俊;;信息与电脑(理论版)(第08期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116384352A (en) | 2023-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111177005B (en) | Service application testing method, device, server and storage medium | |
| CN110688643A (en) | Processing method for platform identity identification and authority authentication | |
| CN108075888B (en) | Dynamic URL generation method and device, storage medium and electronic equipment | |
| CN106850231A (en) | A kind of method of protection interface safety, service end and system, a kind of client | |
| CN115580494B (en) | Method, device and equipment for detecting weak password | |
| CN111679975A (en) | Document generation method and device, electronic equipment and medium | |
| CN107358763A (en) | A kind of method, apparatus and system of ATM checking identity | |
| CN111277418B (en) | Method for realizing Api interface security | |
| CN110113329A (en) | A kind of verification method and device of identifying code | |
| CN111124421B (en) | Abnormal contract data detection method and device for blockchain intelligent contract | |
| CN109246113B (en) | REST API SQL injection vulnerability detection method and device | |
| CN105631669A (en) | Method and device which verifies payment data | |
| CN116384352B (en) | Data set generation method, device, equipment and medium | |
| CN112613893A (en) | Method, system, equipment and medium for identifying malicious user registration | |
| CN110097258B (en) | User relationship network establishment method, device and computer readable storage medium | |
| CN114422586B (en) | Event notification method, event notification device, computer equipment and storage medium | |
| CN107995167B (en) | Equipment identification method and server | |
| US10652276B1 (en) | System and method for distinguishing authentic and malicious electronic messages | |
| CN108052842B (en) | Signature data storage and verification method and device | |
| CN115357894A (en) | Application program bug detection method and system with custom verification function | |
| CN111901299A (en) | Application authentication method and device, electronic equipment and storage medium | |
| CN111382050B (en) | Network service interface testing method and device | |
| CN113674083A (en) | Internet financial platform credit risk monitoring method, device and computer system | |
| CN110020239B (en) | Malicious resource transfer webpage identification method and device | |
| CN115225341B (en) | Website access method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |