CN111026737A - Task processing method and device - Google Patents
Task processing method and device Download PDFInfo
- Publication number
- CN111026737A CN111026737A CN201911072882.1A CN201911072882A CN111026737A CN 111026737 A CN111026737 A CN 111026737A CN 201911072882 A CN201911072882 A CN 201911072882A CN 111026737 A CN111026737 A CN 111026737A
- Authority
- CN
- China
- Prior art keywords
- file
- task
- verified
- verification
- checked
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1012—Server selection for load balancing based on compliance of requirements or conditions with available server resources
Abstract
The embodiment of the invention relates to the technical field of financial technology (Finech), and discloses a task processing method and a task processing device. In the embodiment of the invention, the data quality problem in the intermediate result data can be detected by checking the intermediate result data of the task, so that the wrong intermediate result data can be removed from the task, and thus, the subsequent task is executed based on the accurate intermediate result data, the task execution error caused by the quality problem of the intermediate result data can be avoided, and the task execution accuracy is improved.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a task processing method and apparatus.
Background
With the development of computer technology, more and more technologies are applied in the financial field, and the traditional financial industry is gradually changing to financial technology (Fintech), however, the financial technology also puts higher demands on the technology due to the requirements of security and real-time performance of the financial industry. Financial technology generally relates to a large number of transaction tasks, such as transfer transaction tasks, card purchase transaction tasks, deposit transaction tasks, insurance transaction tasks and the like, each task corresponds to more data, and in order to effectively manage task data, the financial technology can store the task data in a form of a table. However, in order to ensure the processing quality of the task, before the operation is performed on the data in the table, the data quality in the table can be checked, so that the data quality problem caused by errors when the table is manually input is avoided.
In the prior art, some mature data quality verification methods exist, but the methods usually only perform data quality verification on actually existing tables, but do not perform data quality verification on intermediate result data. However, in many scenarios, there is still a need to perform data quality verification on intermediate result data, for example, in a certain scenario, after a user aggregates each actually existing table based on an aggregation algorithm to obtain a report result, a transaction is directly executed based on the report result; in this scenario, if the compiling of the aggregation algorithm is wrong or there is a user input error in a certain actually existing table, the data quality of the report result is problematic, and thus, if the data quality check is not performed on the report result, the accuracy of the transaction result of the subsequent transaction may be affected. In such a scenario, since the report result belongs to intermediate result data of the task, and is not an actual table, the data quality verification method in the prior art cannot be used for verification.
In summary, there is a need for a task processing method for verifying intermediate results (e.g., report results) of a task to improve the accuracy of data processing.
Disclosure of Invention
The embodiment of the invention provides a task processing method and a task processing device, which are used for verifying an intermediate result (such as a report result) of a task so as to improve the accuracy of data processing.
In a first aspect, a task processing method provided in an embodiment of the present invention includes:
acquiring intermediate result data of a task, and converting the intermediate result data of the task into a file to be verified if the intermediate result data of the task is determined to be verified according to task configuration information of the task; correspondingly, according to the task configuration information of the task, a field to be verified is obtained from the file to be verified, a verification template corresponding to the file to be verified is determined, the field to be verified in the file to be verified is verified by using the verification template corresponding to the file to be verified, and a verification result is obtained.
In the design, the data quality problem in the intermediate result data can be detected by checking the intermediate result data of the task, so that the wrong intermediate result data can be removed from the task, and thus, the subsequent task is executed based on the accurate intermediate result data, the task execution error caused by the quality problem of the intermediate result data can be avoided, and the task execution accuracy is improved.
In one possible design, the converting the intermediate result data of the task into a file to be verified includes: extracting each content element and the value of each object at each content element from the intermediate result data of the task, setting a corresponding row index for any object, constructing row data of the object based on the value of the object at each content element, and constructing and obtaining the file to be verified according to the row index of each object, the row data of each object and each content element.
In the design, the intermediate result data can be converted into files with the same format by extracting the line data of each content element and each object in the intermediate result data and combining the line data of each content element and each object according to the set sequence, and compared with the intermediate result data with disordered data, the verification process is executed based on the files with the same format, so that the verification is more ordered, and the verification efficiency can be improved.
In a possible design, the determining a verification template corresponding to the file to be verified includes: determining the positions of an initial verification template and the field to be verified according to the task configuration information of the task; the initial checking template comprises a first variable and a second variable, wherein the first variable is used for identifying a path of a file to be checked, and the second variable is used for identifying the position of a field to be checked in the file to be checked; further, assigning the path of the file to be checked to the first variable, and assigning the position of the field to be checked to the second variable, so as to obtain a check template corresponding to the file to be checked.
In the design, multiple initial check templates are preset, so that the corresponding initial check templates can be directly called to execute the check process in the check process, the data check efficiency can be improved, the first variable and the second variable are set in the initial check templates, and the check templates corresponding to the files to be checked can be obtained in a variable assignment mode, so that the check process is more convenient; and the initial verification template can be set according to the requirements of the user, and the types of the templates are richer, so that the actual requirements are met better, and the user experience is better.
In a possible design, the verification template corresponding to the file to be verified is a verification template in an sql format; in a specific implementation, the verifying the field to be verified in the file to be verified by using the verification template corresponding to the file to be verified includes: acquiring each content element and line data corresponding to each object from the file to be verified, converting each content element into a first feature class, converting the line data of each object into a feature vector corresponding to each object, constructing a second feature class according to the feature vector corresponding to each object, and aggregating the first feature class and the second feature class so as to convert the file to be verified into a feature class file supporting a verification template in an sql format for verification; and thus, the characteristic class file corresponding to the file to be verified is verified by using the verification template corresponding to the file to be verified, and a verification result is obtained.
In the design, when the verification template belongs to the sql format, the file to be verified is converted into the feature file supporting the verification of the verification template in the sql format, so that the feature file corresponding to the file to be verified can be matched with the format of the verification template, and thus, the verification template can accurately verify the feature file corresponding to the file to be verified, and a better verification effect can be obtained; in addition, the verification process is executed in a class file mode, so that the identification efficiency of the verification nodes can be improved, and the data verification efficiency is improved.
In one possible design, the task configuration information of the task further includes a user list and a notification mode of the notification when the verification fails, and the task configuration information further includes a notification condition; the method further comprises the following steps: and if the verification result meets the notification condition, notifying the user in the user list of the verification result according to the notification mode.
In the design, after the verification result is obtained, whether the verification result meets the requirements of the user or not can be determined by automatically comparing the verification result with the notification conditions, and when the verification result meets the requirements set by the user, the verification result is automatically sent to the user in the user list, so that the verification and alarm process is automatically linked without user operation, the maintenance cost is low, and the user experience is good.
In a second aspect, an embodiment of the present invention provides a message processing apparatus, where the apparatus includes:
the acquisition module is used for acquiring intermediate result data of the task;
the conversion module is used for converting the intermediate result data of the task into a file to be verified if the intermediate result data of the task is determined to be verified according to the task configuration information of the task;
the determining module is used for acquiring a field to be verified from the file to be verified according to the task configuration information of the task and determining a verification template corresponding to the file to be verified;
and the verification module is used for verifying the field to be verified in the file to be verified by using the verification template corresponding to the file to be verified to obtain a verification result.
In one possible design, the conversion module is specifically configured to: extracting each content element and the value of each object at each content element from the intermediate result data of the task, setting a corresponding row index for any object, constructing row data of the object based on the value of the object at each content element, and constructing and obtaining the file to be verified according to the row index of each object, the row data of each object and each content element.
In one possible design, the determining module is specifically configured to: determining the positions of an initial verification template and the field to be verified according to the task configuration information of the task; the initial checking template comprises a first variable and a second variable, wherein the first variable is used for identifying a path of a file to be checked, and the second variable is used for identifying the position of a field to be checked in the file to be checked; further, assigning the path of the file to be checked to the first variable, and assigning the position of the field to be checked to the second variable, so as to obtain a check template corresponding to the file to be checked.
In a possible design, the verification template corresponding to the file to be verified is a verification template in an sql format written based on a Spark platform; in a specific implementation, the verification module is specifically configured to: acquiring each content element and line data corresponding to each object from the file to be verified, converting each content element into a first feature class, converting the line data of each object into a feature vector corresponding to each object, constructing a second feature class according to the feature vector corresponding to each object, and aggregating the first feature class and the second feature class so as to convert the file to be verified into a feature class file supporting a verification template in an sql format for verification; and thus, the characteristic class file corresponding to the file to be verified is verified by using the verification template corresponding to the file to be verified, and a verification result is obtained.
In one possible design, the task configuration information of the task further includes a user list and a notification mode of the notification when the verification fails, and the task configuration information further includes a notification condition; in a specific implementation, the verification module is further configured to: and if the verification result meets the notification condition, notifying the user in the user list of the verification result according to the notification mode.
In a third aspect, an embodiment of the present invention provides a computing device, including at least one processor and at least one memory, where the memory stores a computer program, and when the program is executed by the processor, the processor is caused to execute the task processing method according to any of the first aspect.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program executable by a computing device, and when the program runs on the computing device, the computer program causes the computing device to execute the task processing method according to any of the first aspects.
These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic diagram of an architecture of a task processing system according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating a task processing method according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a task processing device according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computing device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic architecture diagram of a possible task processing system according to an embodiment of the present invention, as shown in fig. 1, an execution layer may be disposed in the task processing system, a load balancing node 101 and a plurality of execution nodes, such as an execution node 102, an execution node 103, and an execution node 104, may be disposed in the execution layer, and the plurality of execution nodes may be arranged in a cluster; the load balancing node 101 may be connected to each execution node, and count the service status and resource usage of each execution node. The connection mode may be a wired mode or a wireless mode, and is not particularly limited.
As shown in fig. 1, the load balancing node 101 may also be connected with at least one client, such as a client 121, a client 122, and a client 123. In a specific implementation, any client may send a task processing instruction to the execution layer, where the task processing instruction is first obtained by the load balancing node 101 in the execution layer, and thus, after receiving the task processing instruction sent by the client, the load balancing node 101 may first determine, from each execution node, an execution node in a serviceable state, then select, based on the resource usage of each execution node, the execution node that is the most idle execution node from the execution nodes in the serviceable state as a target execution node, and send the task processing instruction to the target execution node, so that the target execution node processes a task.
As shown in fig. 1, a check layer may be further disposed in the task processing system, a check node 110 may be disposed in the check layer, and the check node 110 may be connected to each execution node, so that intermediate result data generated by each execution node executing a task may be transmitted to the check node 110 via the check layer, so that the check node 110 checks the intermediate result data. The number of check nodes in the check layer may be set by a person skilled in the art according to experience, for example, a plurality of check nodes arranged in a cluster may also be set in the check layer, so as to improve the processing efficiency of the task processing system.
In the embodiment of the present invention, a node may refer to a server or a process in the server, for example, the execution node 102, the execution node 103, and the execution node 104 may be three different execution servers or three different processes in the same execution server, and are not limited specifically.
It should be noted that fig. 1 is only an exemplary and simple illustration, and does not constitute a limitation to the scheme, and in a specific implementation, the check node may also be a process in the execution node, so that the execution node may have functions of task processing, intermediate result data checking, and the like at the same time, and is not limited specifically.
Based on the system architecture illustrated in fig. 1, fig. 2 is a schematic flowchart corresponding to a task processing method provided in an embodiment of the present invention, where the method includes:
In the embodiment of the present invention, the intermediate result data refers to intermediate data generated during a process from the start of a processing task to the end of the processing task, specifically, a task may be composed of a plurality of parallel or dependent subtasks, and each subtask is processed to obtain result data corresponding to the subtask, where the result data corresponding to the subtask belongs to the intermediate result data of the task as long as the result data is not a final task result. For example, if a task is to check the shopping list of user 1 in the first quarter of 2019, the execution flow of the task may be: aggregating a shopping list table of a user 1 in the month 1 of 2019, a shopping list table of the user 1 in the month 2 of 2019 and a shopping list table of the user 1 in the month 3 of 2019 to obtain a shopping list table of the user 1 in the first quarter of 2019, and checking each shopping transaction in the shopping list table of the user 1 in the first quarter of 2019; therefore, the task can correspond to a first subtask, a second subtask and a third subtask, the first subtask is to acquire a shopping list table of the user 1 in month 1 of 2019, a shopping list table of the user 1 in month 2 of 2019 and a shopping list table of the user 1 in month 3 of 2019, the second subtask is to aggregate the shopping list table of the user 1 in the first quarter of 2019, and the third subtask is to check the shopping list table of the user 1 in the first quarter of 2019. Taking the second subtask as an example, since the final task result is the check result and the task result of the second subtask is the shopping list table of the first quarter of the year 2019 of the user 1, the shopping list table of the first quarter of the year 2019 of the user 1 belongs to the intermediate result data of the task.
In a possible implementation manner, the task processing instruction may include task configuration information, the task configuration information is configured on the client in advance by a user, the task configuration information may include a plurality of subtasks corresponding to the task, a dependency relationship of the plurality of subtasks, each workflow node through which each subtask flows, a dependency relationship of each workflow node, and configuration information of each workflow node, and the configuration information of each workflow node is used to execute the workflow node.
In specific implementation, after the execution node receives the task processing instruction, the task processing instruction may be analyzed to obtain task configuration information, then the current subtask to be executed is determined according to the dependency relationship of the plurality of subtasks in the task configuration information, then the current workflow node to be processed is determined based on the dependency relationship of the workflow node through which the current subtask to be executed needs to flow, and the current workflow node to be processed is processed based on the configuration information of the current workflow node to be processed. For example, if a user configures a task including a first subtask, a second subtask and a third subtask that depend in sequence, where the first subtask flows through an execution stream node, the second subtask flows through the execution stream node and a check stream node, and the third subtask flows through the execution node, the task processing system may first invoke the execution node to execute the first subtask to obtain intermediate result data of the first subtask, then invoke the execution node to execute the second subtask based on the intermediate result data of the first subtask to obtain intermediate result data of the second subtask, since the second subtask flows through the check stream node, the task processing system may invoke the check node 110 to check the intermediate result data of the second subtask, if the check is passed, may invoke the execution node to execute the third subtask based on the intermediate result data of the second subtask, and if the check is not passed, the third subtask may not be executed.
In one example, after receiving the task processing instruction, the executing node may place the parsed task configuration information in a set location, where the data placed in the set location can be acquired by any node in the task processing system. In specific implementation, for any subtask included in a task, after an execution node executes the subtask to obtain result data of the subtask (i.e., intermediate result data of the task), all workflow nodes through which the subtask flows may be obtained from task configuration information placed at a set position, and if all workflow nodes include check workflow nodes, it may be determined that the result data of the subtask is checked; correspondingly, if the check stream node is not included in all the workflow nodes, it may be determined that the result data of the subtask is not checked.
Accordingly, since the task configuration information is placed at a common location that can be acquired by each node, after the execution node obtains the result data of a certain subtask of the task, if it is determined that another execution node is adopted to execute the next subtask of the task, the result data of the subtask can be directly sent to another execution node without sending the task configuration information.
In this example, the task configuration information is placed by using the set position, so that each node in the task processing system can obtain the task configuration information by accessing the set position without transmitting the task configuration information, thereby reducing the data transmission amount, reducing the communication loss, and improving the processing efficiency of the task.
In another example, after the execution node obtains the execution result of a certain subtask, if it is determined that another execution node is used to execute a next subtask of the task, the execution result of the subtask and the task configuration information may be simultaneously sent to another execution node. In this example, each node in the task processing system transmits the task configuration information, so that a fixed storage space can be prevented from being arranged in the task processing system, the occupation amount of data is reduced, and the performance loss of the system is reduced.
In this embodiment of the present invention, the execution node and the check node 110 may communicate through a file transfer protocol, and if it is determined that the intermediate result data of the task (i.e., the result data of the sub-task) is checked, the execution node may store the intermediate result data of the task according to a file position set by the check node 110, so that the check node 110 obtains the intermediate result data of the task from the set file position through the file transfer protocol, or if it is determined that the intermediate result data of the task (i.e., the result data of the sub-task) is checked, the execution node may also directly send the intermediate result data of the task to the check node 110, which is not limited specifically.
Correspondingly, after receiving the intermediate result data of the task, the check node 110 may convert the intermediate result data of the task according to a preset file format, so as to obtain a file to be checked corresponding to the intermediate result data of the task. The preset file format may be set by a person skilled in the art according to experience, and may be any one of a TXT format, a DOC format, an XLS format, a PPT format, a DOCX format, an XLSX format, and a PPTX format, which is not limited specifically.
Table 1 is a structural schematic table of a new file format provided in the embodiment of the present invention, and for convenience of description, the following embodiment of the present invention refers to the file format as a dopin file format, and correspondingly refers to a file in the file format as a dopin file.
Table 1: structural schematic of Dophin file format
| Line data |
| Row length index |
| Line data |
| Row length index |
| Basic information |
| Basic information length |
| Dophin |
As shown in table 1, a dopin file includes a first part, a second part, a third part and a fourth part, the first part is located at a tail end position of the dopin file, and the first part is used for storing a file identifier of the dopin file, that is, dopin, so that if a node detects that the tail end position of a certain file is dopin, the file can be determined to be a dopin file; the second part is adjacent to the first part and is positioned in the middle of the Dophin file, and the second part is used for storing the length of the basic information; the third part is adjacent to the second part and is positioned in the middle of the Dophin file, the third part is used for storing basic information, the basic information can comprise the data type of the middle result data, content meta information in the middle result data, the encoding format of the middle result data and statistical information, and the statistical information comprises the line number, the maximum line length, the minimum line length and the like of data information in the middle result data; and the fourth part is adjacent to the third part and is positioned at the top layer of the Dophin file, and the fourth part is used for storing data information in the intermediate result data and comprises a plurality of sub-parts, and each sub-part stores a row index and data corresponding to the row index.
In a possible implementation manner, the type of the intermediate result data may be any one of a JSON type, a Table type, an Txt type, and a CSV type; correspondingly, the embodiment of the invention sets the extraction mode of JSON type matching, the extraction mode of Table type matching, the extraction mode of Txt type matching and the extraction mode of CSV type matching. In specific implementation, after the check node 110 acquires the intermediate result data of the task, a matching extraction mode can be selected according to the type of the intermediate result data, and then content meta-information and data information are extracted from the intermediate result data based on the matching extraction mode; the content meta-information is tag data of each object in the intermediate result data, and the data information is data of each object divided based on the content meta-information.
After determining the matching extraction manner, the check node 110 may determine, based on the extraction manner, a symbolic bytecode of each content element corresponding to the type of intermediate result data, and then determine, based on the symbolic bytecode, a location where each content element is located from the intermediate result data, so that the check node 110 may extract each content element from the intermediate result data based on the location where each content element is located; further, the check node 110 may distinguish data of each object in the intermediate result data using the respective content elements, thereby extracting line data of each object from the intermediate result data, the line data of each object including a value of the object at the respective content elements.
Further, the check node 110 may set a corresponding row index for the row data of each object, then sequentially fill the row indexes and the row data of the objects corresponding to the row indexes into the fourth part of the Dophin file, and count the total row number, the maximum row length and the minimum row length in all the rows in the filling process; thus, after the fourth part is filled, the check node 110 may use the counted total line number, maximum line length, and minimum line length as statistical information, and may generate basic information according to the statistical information, each content element, the encoding mode of the intermediate result data, and the type of the intermediate result data, then fill the basic information into the third part of the dopin file, and count the total length of the third part in the filling process to obtain the length of the basic information; further, after the third portion is filled, the check node 110 may fill the total length of the basic information into the second portion of the dopin file, and after the second portion is filled, fill the dopin identifier into the first portion of the dopin file.
So far, the intermediate result data of the task is converted into a Dophin file.
In one example, if the intermediate result data of the task is a report result, the check node 110 may respectively extract header information and table content information from the report result, and since the header information includes a field identifier of each column in the report result, the check node 110 may directly use the header information as each content element, and since the table content information includes a plurality of rows of data arranged according to the field identifiers of the columns (i.e., each content element), the check node 110 may use the plurality of rows of data as row data of a plurality of objects. In this way, the check node 110 may fill the dopin file corresponding to the intermediate result data according to the above-described manner.
For example, if the intermediate result data of the task is a CSV type report result composed of the age, the gender, the identification number, and the residence information of each user who purchased the product within 10/20/2019, the header information extracted from the report result by the verification node 110 may include the age, the gender, the identification number, and the residence, the table content information extracted may include the line data corresponding to each user, and the line data corresponding to each user includes the age, the gender, the identification number, and the residence of the user. Assuming that there are user 1, user 2, and user 3, when filling the dopin file, the check node 110 may set corresponding row index 1, row index 2, and row index 3 for user 1, user 2, and user 3, then filling a fourth part of the Dophin file according to the sequence of the row index 1, the row data of the user 1, the row index 2, the row data of the user 2, the row index 3 and the row data of the user 3, and after the fourth part is filled, the third part of the Dophin file may be filled in according to the order of the identification fields, the coding mode, and the report type (i.e., CSV type) of each content element, such as the total line number (i.e., 3), the maximum line length, the minimum line length, the age, the gender, the identification number, and the place of residence, the second part of the Dophin file may be filled in using the total length of the third part, and finally the first part of the Dophin file may be filled with the Dophin identifier, so as to obtain the Dophin file corresponding to the intermediate result data.
In the embodiment of the invention, the intermediate result data with disordered data can be converted into the file to be verified with a fixed format by extracting each content element in the intermediate result data and the value of each object at each content element and combining the contents according to the set sequence, so that the information of the file to be verified is clearer and more ordered, the verification process is more convenient and faster, and the verification efficiency is higher.
In a possible implementation manner, a user may set a plurality of verification rules in the verification node 110 in advance, where each verification rule may include an initial verification template, and may further include a user list that needs to be notified when verification fails, such as a mailbox, a nailed account, a micro signal, a mobile phone number, and the like of the user. In the verification rule, the first variable in the initial verification template is assigned as the initial file path by default, and the second variable is assigned as the initial position by default.
In the embodiment of the invention, various initial verification templates are preset, so that the corresponding initial verification template can be directly called to execute the verification process in the verification process, the efficiency of data verification can be improved, and the verification template corresponding to the file to be verified can be obtained in a variable assignment mode by setting the first variable and the second variable in the initial verification template, so that the verification process is more convenient and faster; and the initial verification template can be set according to the requirements of the user, so that the actual requirements are met, and the user experience is better.
In specific implementation, different users have different verification permissions, when the verification node 110 receives a request message for setting a verification rule by a user, the verification node 110 may perform authentication on the user first, and if the authentication fails, the identity of the user is not legal, so that the verification node 110 may refuse to set the verification rule and may send a response message indicating that the authentication fails to the user; if the identity of the user is determined to be legal, the user can be subjected to authority verification, if the authority verification fails, the user does not have the authority for setting the verification rule, therefore, the verification node 110 can refuse to set the verification rule and can send a response message of failure of the authority verification to the user, if the authority verification passes, the user is indicated to have the authority for setting the verification rule, and therefore, the verification node 110 can set the verification rule according to the request message of setting the verification rule by the user.
In an example, a plurality of check codes are set in the check node 110, and the plurality of check codes may be check codes written by a user and stored in the check node 110, or may be check codes in an open source architecture, which is not limited. For example, some check codes may check the position of each null value in the field to be checked, some check codes may check the position of each value that does not meet the set threshold range in the field to be checked, and other check codes may check the position of each field of the abnormal field type in the field to be checked, and so on.
In this example, after the user passes both the authentication and the authorization authentication, the check node 110 may provide a check rule setting interface to the user, so that the user sets the check rule configuration information in the check rule setting interface; the check rule configuration information may include a check code, a first variable, a second variable, and may further include a user list and a notification mode that need to be notified when the check fails. Therefore, a user can check the check code in the check rule setting interface, assign the first variable as an initial file path and assign the second variable as an initial position, and can input a user list and a notification mode which need to be notified when the check fails in a text mode or a voice mode; correspondingly, after the check node 110 obtains the check rule configuration information, an initial check template may be generated according to the check code, the initial file path and the initial position, and may be packaged into a check rule according to the initial check template, the user list that needs to be notified when the check fails, and the notification mode, and may also set an identifier, such as a number, for the check rule.
In the embodiment of the present invention, the task configuration information may further include an identifier of the initial verification template and a position of the field to be verified, where the position of the field to be verified refers to row information of the field to be verified in the file to be verified; several possible ways of determining the verification template corresponding to the file to be verified are described below:
implementation mode one
In a specific implementation, after obtaining the file to be checked corresponding to the intermediate result data, the check node 110 may randomly store the file to be checked at a certain position, and record a file path where the file to be checked is located. The location may be a certain location in the internal memory of the check node 110, or may also be a location in the task processing system, which is not limited.
In this way, after the check node 110 obtains the file to be checked corresponding to the intermediate result data, the field to be checked can be obtained from the file to be checked according to the position of the field to be checked set in the task configuration information, and the initial check template can be obtained according to the identifier of the initial check template.
In the embodiment of the present invention, the check node 110 may further generate the check rule corresponding to the file to be checked according to the check template corresponding to the file to be checked, the user list that needs to be notified when the check fails, and the notification mode.
Implementation mode two
In a specific implementation, the task configuration information may further include a file path of the file to be checked set by the user, and after the check node 110 obtains the file to be checked corresponding to the intermediate result data, the file to be checked may be stored according to the file path of the file to be checked in the task configuration information. The file path of the file to be checked set by the user may be a certain path in the internal memory of the check node 110, or may also be a path in the task processing system, which is not limited.
In this way, after obtaining the file to be checked corresponding to the intermediate result data, the check node 110 may obtain the file path of the file to be checked, the position of the field to be checked, and the initial check template according to the task configuration information, then obtain the field to be checked from the file to be checked according to the position of the field to be checked and the file path of the file to be checked, and may obtain the initial check template according to the identifier of the initial check template, further replace the initial file path of the first variable assignment with the file path of the file to be checked corresponding to the intermediate result data, and replace the initial position of the second variable assignment with the position of the field to be checked set in the task configuration information, thereby obtaining the check template corresponding to the file to be checked.
Implementation mode three
In specific implementation, the task configuration information may further include an identifier of a check rule, after obtaining the file to be checked corresponding to the intermediate result data, the check node 110 may obtain the positions of the initial check template and the field to be checked according to the identifier of the check rule in the task configuration information, replace the path of the initial file of the file to be checked in the check template with the path of the file to be checked corresponding to the intermediate result data, and replace the initial position of the second variable assignment with the position of the field to be checked set in the task configuration information by the user, thereby obtaining the check template corresponding to the file to be checked.
In an example, the second variable in the initial check template may also be assigned as a placeholder by default, and therefore, after the check node 110 obtains the position of the field to be checked by parsing from the task configuration information, the placeholder assigned by the second variable may be replaced by the position of the field to be checked, and thus the check template corresponding to the file to be checked is obtained.
And 204, verifying the field to be verified in the file to be verified by using a verification template corresponding to the file to be verified to obtain a verification result.
In specific implementation, a check code can be obtained from a check template corresponding to a file to be checked, the field to be checked can be obtained from the file to be checked based on the position of the field to be checked, if the compiling language of the check code is the same as that of the field to be checked, the field to be checked can be checked directly by using the check code, if the compiling language of the check code is different from that of the field to be checked, the field to be checked can be firstly converted into a class to be checked which can be understood by the check code, and then the class to be checked is checked by using the check code, so that a check result is obtained.
In a possible implementation manner, when the verification template corresponding to the file to be verified is a sql-format verification template, the verification node 110 may first obtain each content element and line data of each object from the file to be verified, then convert each content element into a first feature class, convert the line data of each object into a feature vector corresponding to each object, construct a second feature class according to the feature vector corresponding to each object, and aggregate the first feature class and the second feature class, thereby converting the file to be verified into a feature class file supporting the sql-format verification template for verification, so that the verification template corresponding to the file to be verified can realize verification of the feature class file corresponding to the file to be verified.
As a specific example, if the check code is a sql-format check code written based on a Spark platform, the check node 110 may first obtain each content element from the file to be checked, then convert each content element into a StructType class based on the Spark platform, then obtain row data of each object from the file to be checked, convert the row data of each object into a row vector in an ArrayList [ ] class based on the Spark platform, and then convert the row data of each object into an ArrayList [ ] class based on the Spark platform; thus, the check node 110 may aggregate the structype class and the ArrayList [ ] class by using a Spark.
Further, since the Spark DateFrame type file and the check code are both obtained based on the Spark platform, the Spark DateFrame type file corresponding to the file to be checked can be checked by the check code in the sql format, and thus, the check node 110 can directly call the check code to check the Spark DateFrame type file corresponding to the file to be checked, and obtain a field which does not meet the check condition in the field to be checked. The check condition may be a condition set by the user in the check code, for example, when the check template checks a null value in the field, the check condition may be that the field does not belong to the null value, so that, for any field in the field to be checked, if the field belongs to the null value, it is determined that the field does not conform to the check condition, and therefore, the field may be checked by the check code.
For example, if the file to be checked is a dopin file, the check node 110 may first obtain each content element from the second portion of the dopin file, then convert each content element into a structype class based on a Spark platform, then obtain each line data from the fourth portion of the dopin file, and then convert each line data into an ArrayList [ ] class based on the Spark platform, so as to check the Spark DateFrame class file aggregated based on the structype class and the ArrayList [ ] class.
In the implementation manner, when the verification template belongs to the sql format, the spare dataframe type file corresponding to the file to be verified can be matched with the verification template in format by converting the file to be verified into the spare dataframe type file, so that the verification template can verify the spare dataframe type file corresponding to the file to be verified, and the verification effect can be improved; in addition, the verification process is executed in a class file mode, so that the identification efficiency of the verification nodes can be improved, and the data verification efficiency is improved.
In an example, the task configuration information may further include a notification condition, and after the verification result is obtained, the verification node 110 may further determine whether the verification result satisfies the notification condition, if not, may execute a next subtask based on the intermediate result data, and if so, may generate a verification report according to the position of each field that does not satisfy the verification condition, and may notify the verification report to the user list in the verification template according to a set notification manner. For example, if the notification condition is that the number of null values is greater than the preset number, after each field that does not meet the verification condition in the field to be verified is verified, the verification node 110 may count the number of each field that does not meet the verification condition, and if the number is greater than the preset number, it may be determined that the intermediate result data verification meets the notification condition, so the verification node 110 may generate a verification report according to the position of each field that does not meet the verification condition, and may notify the verification report to the user list in the verification template according to the set notification manner; accordingly, if the number is less than or equal to the preset number, it may be determined that the intermediate result data does not satisfy the notification condition, and thus, the check node 110 may execute the next subtask according to the intermediate result data.
In the above example, after the verification result is obtained, whether the verification result meets the requirements of the user can be determined by automatically comparing the verification result with the notification conditions, and when the verification result meets the requirements set by the user, the verification result is automatically sent to the user in the user list, so that the verification and alarm process can be automatically linked without user operation, thereby achieving low maintenance cost and good user experience.
In the embodiment of the invention, the intermediate result data of the task is acquired, and according to the task configuration information of the task, if the intermediate result data of the task is determined to be verified, the intermediate result data of the task is converted into a file to be verified; correspondingly, according to the task configuration information of the task, a field to be verified is obtained from the file to be verified, a verification template corresponding to the file to be verified is determined, the field to be verified in the file to be verified is verified by using the verification template corresponding to the file to be verified, and a verification result is obtained. In the embodiment of the invention, the data quality problem in the intermediate result data can be detected by checking the intermediate result data of the task, so that the wrong intermediate result data can be removed from the task, and thus, the subsequent task is executed based on the accurate intermediate result data, the task execution error caused by the quality problem of the intermediate result data can be avoided, and the task execution accuracy is improved.
In view of the above method flow, an embodiment of the present invention further provides a task processing device, and specific contents of the device may be implemented with reference to the above method.
Fig. 3 is a schematic structural diagram of a task processing device according to an embodiment of the present invention, including:
an obtaining module 301, configured to obtain intermediate result data of a task;
a conversion module 302, configured to, according to the task configuration information of the task, if it is determined that the intermediate result data of the task is verified, convert the intermediate result data of the task into a file to be verified;
a determining module 303, configured to obtain a field to be verified from the file to be verified according to the task configuration information of the task, and determine a verification template corresponding to the file to be verified;
the verification module 304 is configured to verify the field to be verified in the file to be verified by using a verification template corresponding to the file to be verified, so as to obtain a verification result.
Optionally, the conversion module 302 is specifically configured to:
extracting various content elements and the value of each object at the various content elements from the intermediate result data of the task;
setting a corresponding row index for any object, and constructing row data of the object based on the values of the object at the content elements;
and constructing and obtaining the file to be verified according to the row index of each object, the row data of each object and each content element.
Optionally, the determining module 303 is specifically configured to:
determining the positions of an initial verification template and the field to be verified according to the task configuration information of the task; the initial checking template comprises a first variable and a second variable, wherein the first variable is used for identifying a path of a file to be checked, and the second variable is used for identifying the position of a field to be checked in the file to be checked;
and assigning the path of the file to be checked to the first variable, and assigning the position of the field to be checked to the second variable to obtain a check template corresponding to the file to be checked.
Optionally, the verification template corresponding to the file to be verified is a verification template in an sql format;
the verification module 304 is specifically configured to:
acquiring each content element and row data corresponding to each object from the file to be verified;
converting each content element into a first feature class, converting the line data of each object into a feature vector corresponding to each object, and constructing a second feature class according to the feature vector corresponding to each object;
aggregating the first feature class and the second feature class to convert the file to be verified into a feature class file supporting verification of a verification template in an sql format;
and verifying the feature class file corresponding to the file to be verified by using the verification template corresponding to the file to be verified to obtain a verification result.
Optionally, the task configuration information of the task further includes a user list and a notification mode of the notification when the verification fails, and the task configuration information further includes a notification condition;
the verification module 304 is further configured to:
and if the verification result meets the notification condition, notifying the user in the user list of the verification result according to the notification mode.
From the above, it can be seen that: in the embodiment of the invention, the intermediate result data of the task is acquired, and according to the task configuration information of the task, if the intermediate result data of the task is determined to be verified, the intermediate result data of the task is converted into a file to be verified; correspondingly, according to the task configuration information of the task, a field to be verified is obtained from the file to be verified, a verification template corresponding to the file to be verified is determined, the field to be verified in the file to be verified is verified by using the verification template corresponding to the file to be verified, and a verification result is obtained. In the embodiment of the invention, the data quality problem in the intermediate result data can be detected by checking the intermediate result data of the task, so that the wrong intermediate result data can be removed from the task, and thus, the subsequent task is executed based on the accurate intermediate result data, the task execution error caused by the quality problem of the intermediate result data can be avoided, and the task execution accuracy is improved.
Based on the same inventive concept, an embodiment of the present invention further provides a computing device, as shown in fig. 4, including at least one processor 401 and a memory 402 connected to the at least one processor, where a specific connection medium between the processor 401 and the memory 402 is not limited in the embodiment of the present invention, and the processor 401 and the memory 402 are connected through a bus in fig. 4 as an example. The bus may be divided into an address bus, a data bus, a control bus, etc.
In the embodiment of the present invention, the memory 402 stores instructions executable by the at least one processor 401, and the at least one processor 401 may execute the steps included in the foregoing task processing method by executing the instructions stored in the memory 402.
The processor 401 is a control center of the computing device, and may be connected to various parts of the computing device through various interfaces and lines, and may perform data processing by executing or executing instructions stored in the memory 402 and calling data stored in the memory 402. Optionally, the processor 401 may include one or more processing units, and the processor 401 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, an application program, and the like, and the modem processor mainly processes an instruction issued by an operation and maintenance worker. It will be appreciated that the modem processor described above may not be integrated into the processor 401. In some embodiments, processor 401 and memory 402 may be implemented on the same chip, or in some embodiments, they may be implemented separately on separate chips.
The processor 401 may be a general-purpose processor, such as a Central Processing Unit (CPU), a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, configured to implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present invention. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the task processing embodiments may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.
Based on the same inventive concept, the embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program executable by a computing device, and when the program runs on the computing device, the computer program causes the computing device to execute the task processing method as described in any of fig. 2.
It should be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (12)
1. A method for processing a task, the method comprising:
acquiring intermediate result data of the task;
according to the task configuration information of the task, if the intermediate result data of the task is determined to be verified, converting the intermediate result data of the task into a file to be verified;
according to the task configuration information of the task, acquiring a field to be verified from the file to be verified, and determining a verification template corresponding to the file to be verified;
and checking the field to be checked in the file to be checked by using a checking template corresponding to the file to be checked to obtain a checking result.
2. The method of claim 1, wherein the converting the intermediate result data of the task into a file to be verified comprises:
extracting various content elements and the value of each object at the various content elements from the intermediate result data of the task;
setting a corresponding row index for any object, and constructing row data of the object based on the values of the object at the content elements;
and constructing and obtaining the file to be verified according to the row index of each object, the row data of each object and each content element.
3. The method according to claim 1, wherein the determining the verification template corresponding to the file to be verified comprises:
determining the positions of an initial verification template and the field to be verified according to the task configuration information of the task; the initial checking template comprises a first variable and a second variable, wherein the first variable is used for identifying a path of a file to be checked, and the second variable is used for identifying the position of a field to be checked in the file to be checked;
and assigning the path of the file to be checked to the first variable, and assigning the position of the field to be checked to the second variable to obtain a check template corresponding to the file to be checked.
4. The method according to claim 1, wherein the verification template corresponding to the file to be verified is a verification template in an sql format;
the checking the field to be checked in the file to be checked by using the checking template corresponding to the file to be checked includes:
acquiring each content element and row data corresponding to each object from the file to be verified;
converting each content element into a first feature class, converting the line data of each object into a feature vector corresponding to each object, and constructing a second feature class according to the feature vector corresponding to each object;
aggregating the first feature class and the second feature class to convert the file to be verified into a feature class file supporting verification of a verification template in an sql format;
and verifying the feature class file corresponding to the file to be verified by using the verification template corresponding to the file to be verified to obtain a verification result.
5. The method according to any one of claims 1 to 4, wherein the task configuration information of the task further includes a list of users who are notified when the verification fails and a notification manner, and the task configuration information further includes a notification condition;
the method further comprises the following steps:
and if the verification result meets the notification condition, notifying the user in the user list of the verification result according to the notification mode.
6. A task processing apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring intermediate result data of the task;
the conversion module is used for converting the intermediate result data of the task into a file to be verified if the intermediate result data of the task is determined to be verified according to the task configuration information of the task;
the determining module is used for acquiring a field to be verified from the file to be verified according to the task configuration information of the task and determining a verification template corresponding to the file to be verified;
and the verification module is used for verifying the field to be verified in the file to be verified by using the verification template corresponding to the file to be verified to obtain a verification result.
7. The apparatus of claim 6, wherein the conversion module is specifically configured to:
extracting various content elements and the value of each object at the various content elements from the intermediate result data of the task;
setting a corresponding row index for any object, and constructing row data of the object based on the values of the object at the content elements;
and constructing and obtaining the file to be verified according to the row index of each object, the row data of each object and each content element.
8. The apparatus of claim 6, wherein the determining module is specifically configured to:
determining the positions of an initial verification template and the field to be verified according to the task configuration information of the task; the initial checking template comprises a first variable and a second variable, wherein the first variable is used for identifying a path of a file to be checked, and the second variable is used for identifying the position of a field to be checked in the file to be checked;
and assigning the path of the file to be checked to the first variable, and assigning the position of the field to be checked to the second variable to obtain a check template corresponding to the file to be checked.
9. The apparatus according to claim 6, wherein the verification template corresponding to the file to be verified is a verification template in an sql format;
the verification module is specifically configured to:
acquiring each content element and row data corresponding to each object from the file to be verified;
converting each content element into a first feature class, converting the line data of each object into a feature vector corresponding to each object, and constructing a second feature class according to the feature vector corresponding to each object;
aggregating the first feature class and the second feature class to convert the file to be verified into a feature class file supporting verification of a verification template in an sql format;
and verifying the feature class file corresponding to the file to be verified by using the verification template corresponding to the file to be verified to obtain a verification result.
10. The apparatus according to any one of claims 6 to 9, wherein the task configuration information of the task further includes a list of users who notify when the check fails and a notification manner, and the task configuration information further includes a notification condition;
the verification module is further configured to:
and if the verification result meets the notification condition, notifying the user in the user list of the verification result according to the notification mode.
11. A computing device comprising at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method of any of claims 1 to 5.
12. A computer-readable storage medium storing a computer program executable by a computing device, the program, when run on the computing device, causing the computing device to perform the method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911072882.1A CN111026737A (en) | 2019-11-05 | 2019-11-05 | Task processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911072882.1A CN111026737A (en) | 2019-11-05 | 2019-11-05 | Task processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111026737A true CN111026737A (en) | 2020-04-17 |
Family
ID=70200853
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911072882.1A Pending CN111026737A (en) | 2019-11-05 | 2019-11-05 | Task processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111026737A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015763A (en) * | 2020-07-14 | 2020-12-01 | 北京旷视科技有限公司 | JSON data verification method and device, electronic equipment and computer readable medium |
| CN113050949A (en) * | 2021-03-23 | 2021-06-29 | 上海万向区块链股份公司 | JAVA-based method and system for preventing data confusion after execution of parent-child dependence JOB |
| CN114661704A (en) * | 2022-03-23 | 2022-06-24 | 杭州半云科技有限公司 | Data resource full life cycle management method, system, terminal and medium |
-
2019
- 2019-11-05 CN CN201911072882.1A patent/CN111026737A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015763A (en) * | 2020-07-14 | 2020-12-01 | 北京旷视科技有限公司 | JSON data verification method and device, electronic equipment and computer readable medium |
| CN113050949A (en) * | 2021-03-23 | 2021-06-29 | 上海万向区块链股份公司 | JAVA-based method and system for preventing data confusion after execution of parent-child dependence JOB |
| CN114661704A (en) * | 2022-03-23 | 2022-06-24 | 杭州半云科技有限公司 | Data resource full life cycle management method, system, terminal and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109474578B (en) | Message checking method, device, computer equipment and storage medium | |
| CN108595157B (en) | Block chain data processing method, device, equipment and storage medium | |
| CN111026737A (en) | Task processing method and device | |
| US10496843B2 (en) | Systems and method for automatically securing sensitive data in public cloud using a serverless architecture | |
| CN108304554B (en) | File splitting method and device, computer equipment and storage medium | |
| CN106327140B (en) | Method and device for monitoring data modification | |
| CN110780870A (en) | Service execution method, device, equipment and storage medium | |
| CN111273891A (en) | Business decision method and device based on rule engine and terminal equipment | |
| CN110222323A (en) | A kind of voucher generation method, device, readable storage medium storing program for executing and terminal device | |
| CN111932257A (en) | Block chain parallelization processing method and device | |
| CN114091099A (en) | Authority hierarchical control method, equipment and storage medium for business system | |
| CN113360269A (en) | Task allocation method, device, server and storage medium | |
| CN111475494A (en) | Mass data processing method, system, terminal and storage medium | |
| CN110659943A (en) | Method and device for adjusting delivery channel based on user structure characteristics and electronic equipment | |
| CN113806816A (en) | Electronic file management method and device based on block chain and electronic equipment | |
| CN109284268A (en) | A kind of method, system and the electronic equipment of fast resolving log | |
| CN111737351A (en) | Transaction management method and device for distributed management system | |
| CN114625723A (en) | Data migration method and device, computer equipment and storage medium | |
| CN113342647A (en) | Test data generation method and device | |
| CN105989021A (en) | Document processing method and device | |
| CN117076546B (en) | Data processing method, terminal device and computer readable storage medium | |
| US20230342226A1 (en) | Event handling based on tracing execution paths | |
| CN114693449A (en) | Block chain based transaction request processing method and device and storage medium | |
| CN114710333A (en) | Data transmission and verification method, system, computer equipment and storage medium | |
| CN105912605A (en) | Statistical method and system for BI report |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |