Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
FIG. 1 is a diagram of an application environment of a database data processing method. Referring to fig. 1, the database data processing method is applied to a database data processing system. The database data processing system includes a terminal 110 and a server 120, wherein the terminal 110 is connected to the server 120 through a network. The terminal 110 may be a fixed terminal, which may specifically be at least one of a printer, a scanner, and a monitor, or a mobile terminal, which may specifically be at least one of a tablet, a smartphone, a personal data assistant, and a digital camera.
FIG. 2 is a diagram illustrating an internal configuration of server 120 in the database data processing system of FIG. 1, in accordance with one embodiment. As shown in fig. 2, the server 120 includes a processor, a nonvolatile storage medium, an internal memory, and a network interface, which are connected through a system bus. The processor of the server 120 is used to provide computing and control capabilities, to support the operation of the entire server 120, the memory is used to store data, code instructions, etc., and the network interface is used to communicate with the terminal 110 over a network. The memory has stored thereon at least one computer-executable instruction executable by the processor to implement the database data processing method provided in the embodiments of the present application and applied to the server 120. The Memory may include a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM). For example, in one embodiment, the memory includes a non-volatile storage medium and an internal memory; the nonvolatile storage medium stores a processing system, a database and computer-executable instructions, wherein the database comprises a plurality of databases corresponding to a plurality of database types respectively, target data are stored in the databases, and the computer-executable instructions can be executed by a processor to realize the database data processing method; the internal memory provides a cached operating environment for the processing system and computer-executable instructions in the non-volatile storage medium.
As shown in fig. 3, in an embodiment, a database data processing method is provided, which is exemplified by applying the method to the server 120 in fig. 1, and the method specifically includes the following steps:
s302, a data processing task execution instruction is obtained.
Specifically, the terminal 110 acquires a user account input by a user, generates a page data acquisition request according to the acquired user account, and sends the page data acquisition request to the server 120. After receiving the page data acquisition request sent by the terminal 110, the server 120 parses the page data acquisition request, and extracts the user account in the page data acquisition request by parsing. The server 120 performs authority verification on the extracted user account, and returns data processing page data to the terminal 110 according to the page data acquisition request when the extracted user account is verified to have the access authority to access the data processing page. After the terminal 110 processes the received data processing page data, it displays the data processing page according to the data processing page data. The terminal 110 obtains the task identifier selected by the user in the displayed data processing page, and generates a data processing task execution instruction according to the selected task identifier. The terminal 110 transmits a data processing task execution instruction to the server 120.
S304, extracting the task identification in the data processing task execution instruction.
Specifically, the server 120, upon receiving the data processing task execution instruction sent by the terminal 110, parses the data processing task execution instruction, and extracts the task identifier in the task execution instruction through parsing. The task identifier is a unique identifier of the task.
S306, acquiring a task script corresponding to the task identifier.
Specifically, the terminal 110 obtains a task identifier and a task script corresponding to the task identifier, which are input by a user in a displayed data processing page, and sends the obtained task identifier and task script to the server 120. The server 120 correspondingly stores the received task identification and the task script. The task script is a script executed when executing a data processing task. After extracting the task identifier, the server 120 queries a task script corresponding to the extracted task identifier from the stored task script, and obtains the queried task script.
S308, extracting the data processing instruction and the data acquisition condition in the task script.
Specifically, after extracting the task script, the server 120 parses the task script, and extracts the data processing instruction and the data acquisition condition from the task script by parsing. The data processing instruction is an instruction for processing data.
In one embodiment, the server 120 parses the task script, and configures the operation resources for the data processing instructions according to the operation resource configuration parameters by parsing and identifying the data processing instructions, the data acquisition conditions, and the operation resource configuration parameters in the task script.
And S310, acquiring target data from the databases corresponding to the database types according to the data acquisition conditions.
In particular, server 120 may access databases corresponding to a plurality of database types. The databases corresponding to the multiple database types may be set on the server 120, or may be set on other servers. The server 120 obtains the access data interface of the database corresponding to each database type, and obtains the target data meeting the data obtaining condition from the corresponding database through the access data interface.
In one embodiment, the server 120 extracts the database identifier in the data obtaining condition, identifies the database type corresponding to the database identifier, obtains the database access interface corresponding to the database type, and obtains the target data meeting the data obtaining condition from the database corresponding to the database identifier according to the database access interface.
In one embodiment, the server 120 extracts the database type in the data obtaining condition, obtains a database access interface of the database type and a database corresponding to the query database type, and obtains the target data meeting the data obtaining condition from the queried database according to the database access interface.
And S312, respectively creating a temporary data table according to the target data corresponding to each database type.
Specifically, after the server 120 obtains the target data corresponding to each database type, a temporary data table in a preset format is respectively created according to the target data corresponding to each database type, so as to obtain a temporary data table corresponding to each database type, where the target data is stored in the temporary data table.
In one embodiment, S312 specifically includes the following: acquiring temporary table establishing instructions corresponding to the database types; and respectively establishing a temporary data table for the target data corresponding to the corresponding database type according to the temporary table establishing instruction.
Specifically, the server 120 obtains target data corresponding to each database type, obtains temporary table creation instructions corresponding to each database type, creates a temporary data table corresponding to each database type according to the temporary table creation instructions, converts the data format of the target data corresponding to each database type into a data format matched with the temporary data table, and stores the target data after data format conversion in the corresponding temporary data table.
And S314, processing the target data in the temporary data table according to the data processing instruction.
Specifically, after creating the temporary data tables corresponding to the database types, the server 120 performs data processing on the target data in the temporary data tables corresponding to the database types according to the data processing, and obtains the processing result by processing the target data in the temporary data tables.
In this embodiment, the edited task script is acquired according to the data processing task execution instruction, the task script is analyzed to generate a data processing task, the data processing instruction and the data acquisition condition in the data processing task are extracted, and the target data is acquired from the database corresponding to each database type according to the data acquisition condition. And respectively creating temporary data tables according to the target data corresponding to each database type, and processing the target data corresponding to each database type in the temporary data tables according to the data processing instruction by establishing the temporary data tables corresponding to each database type. The target data corresponding to each database type does not need to be independently operated, and the processing efficiency of the data corresponding to each database type is improved.
As shown in fig. 4, in an embodiment, the step S306 further includes a step of reading the task identifier according to the task dependency relationship, where the step includes the following steps:
s402, reading task identifiers according to task dependency in the data processing task execution instruction.
Specifically, the data processing task execution instruction includes a plurality of task identifiers and task dependency relationships corresponding to the plurality of task identifiers. The server 120 analyzes the data processing task execution instruction, extracts the task dependency relationship and the task identifier in the data processing task execution instruction through analysis, and reads the task identifier according to the task dependency relationship.
In one embodiment, the task identification in the data processing task execution instruction is a task flow identification arranged in terms of task dependencies. The server 120 reads the task identification from the task stream identification according to the task dependency relationship.
For example, the task flow identifier in the data processing task execution instruction is "1, 2|3|4, 5", and the task flow identifier includes task identifiers of task 1, task 2, task 3, task 4, and task 5. Wherein, the ' represents that the task has dependence, the ' is executed firstly ', the ' task corresponding to the previous task identifier, the ' is executed later ', and the ' task corresponding to the next task identifier; the "|" indicates that the tasks before and after the "|" can be executed in parallel to identify the corresponding tasks. The task execution sequence of "1, 2|3|4, 5" is to execute task 1 first, then execute task 2, task 3, and task 4 in parallel after task 1 is executed, and then execute task 5 after task 2, task 3, and task 4 are executed in parallel.
S404, acquiring the task script corresponding to the read task identifier.
Specifically, after reading the task identifier from the task execution instruction according to the task dependency relationship, the server 120 queries the task script corresponding to the task identifier from the stored task scripts, and obtains the queried task script.
In this embodiment, the data processing task execution instruction includes a plurality of task identifiers and task dependencies corresponding to the plurality of task identifiers, reads the task identifiers according to the task dependencies, and obtains the task script according to the read task identifiers. The task corresponding to each task identifier is guaranteed to be executed according to the task dependency relationship, the operation steps for executing the task are simplified, the task execution efficiency is improved, a plurality of data processing tasks can be executed according to one data processing task execution instruction, and the data processing efficiency is improved.
As shown in fig. 5, in an embodiment, the step S314 specifically includes a step of obtaining a custom function, where the step specifically includes the following steps:
and S502, extracting the self-defined function identification in the data processing instruction.
Specifically, the server 120 parses the data processing instruction, and extracts the custom function identifier in the data processing instruction through parsing. The user-defined function identifier is the unique identifier of the user-defined function.
S504, acquiring a data processing function from the custom function library according to the custom function identifier.
Specifically, the terminal 110 acquires a data processing function input by the user in the data processing page, and sends the acquired data processing function to the server 120. After receiving the data processing function, the server 120 extracts the custom function identifier in the data processing function, and correspondingly stores the received data processing function and the extracted custom function identifier in the custom function library. After extracting the custom function identifier from the data processing instruction, the server 120 queries the data processing function in the custom function library according to the extracted custom function identifier, and obtains the queried data processing function.
And S506, processing the target data in the temporary data table according to the acquired data processing function.
Specifically, the server 120 processes the target data in the temporary data table according to the obtained data processing function, and processes the target data according to the data processing function to obtain a processing result. The server 120 may transmit the obtained processing result to the terminal 110, so that the terminal 110 is displayed in the data processing page.
In the embodiment, the target data in the temporary data table of each database type can be processed according to the user-defined data processing function, so that the target data can be processed more flexibly, and the accuracy and the efficiency of data processing are improved.
As shown in fig. 6, in an embodiment, after S314, a step of storing the processing result is specifically included, and the step specifically includes the following steps:
s602, a processing result storage instruction is obtained.
Specifically, the terminal 110 displays the processing result obtained by the server 120 in the data processing page, and when detecting that the result saving button in the data processing page is clicked, triggers the processing result storage instruction, and sends the triggered processing result storage instruction to the server 120. The server 120 receives the processing result storage instruction transmitted by the terminal 110.
In one embodiment, the server 120 may detect whether the data processing process is finished, and upon detecting that the data processing process is finished, trigger a processing result storing instruction.
S604, extracting the database type in the processing result storage instruction.
Specifically, the server 120 parses the processing result storage instruction, and extracts the database type in the processing result storage instruction by parsing. And the extracted database type is the database type corresponding to the database stored in the processing result.
And S606, converting the processing result obtained by processing the target data into a data format matched with the extracted database type.
Specifically, the server 120 obtains a processing result obtained by processing the target data, determines a data format matching the extracted database type, performs data format conversion on the processing result, and converts the processing result into a data format matching the extracted database type.
And S608, storing the processing result after format conversion into a database corresponding to the extracted database type.
Specifically, the server 120 queries a database corresponding to the extracted database type, extracts a data identifier of the processing result, and stores the extracted data identifier and the processing result in the queried database in a corresponding manner.
In one embodiment, the server 120 extracts the database identifier in the processing result storage instruction, and identifies the database type to which the database corresponding to the extracted database identifier belongs. The server 120 determines the data format corresponding to the identified database type, converts the data format of the processing result into the determined data format, and stores the processing result after the format conversion into the database corresponding to the extracted database identifier.
In this embodiment, the database type in the processing result storage instruction is extracted according to the processing result storage instruction, the processing result is subjected to data format conversion, it is ensured that the data format of the processing result is matched with the database type, the processing result after format conversion is stored in the database corresponding to the database type in the processing result storage instruction, and the correctness of the data format of the processing result is ensured.
In one embodiment, a user enters a user account and a user password through a browser page and a data processing application page. The terminal 110 acquires the input user account and user password, generates a page data request according to the acquired user account and user password, and the terminal 110 sends the page data request to the server 120.
After receiving the page data request, the server 120 extracts the user account and the user password in the page data request, and verifies whether the user corresponding to the user account has the access right according to the user account and the user password. After verifying that the user corresponding to the user account has the access right, the server 120 returns data processing page data to the terminal 110.
The terminal 110 displays a data processing page according to the data processing page data. The user selects a task identifier in the data processing page and clicks a data task execution button through an input device of the terminal 110. When detecting that the data task execution button in the data processing page is clicked, the terminal 110 generates a data processing task execution instruction according to the selected task identifier and the task dependency relationship corresponding to the selected task identifier, and sends the data processing task execution instruction to the server 120.
The server 120 extracts the task identifier in the data processing task execution instruction and the task dependency corresponding to the task identifier, and splits the plurality of data processing tasks according to the task dependency and the task identifier. The server 120 extracts the task script according to the task identifier in each data processing task, and extracts the data processing instruction and the data acquisition condition in the task script.
The server 120 queries the target data from the databases corresponding to the accessible database types according to the data acquisition conditions, and acquires the target data corresponding to the database types. The server 120 creates a temporary data table corresponding to the database type according to the temporary table creation instruction respectively obtained corresponding to each database type, and stores corresponding target data to the temporary data table corresponding to the database type. Temporary data tables corresponding to each database type may be previously interacted with. And the server 120 processes the target data in the temporary data table corresponding to each database type according to the data processing instruction to obtain a processing result.
The server 120 acquires at least one of a user account, data processing time, a data processing type, a database type, and data acquisition conditions logged in by the terminal 110, and generates a data processing log according to the acquired at least one of the user account, the data processing time, the data processing type, the database type, and the data acquisition conditions.
As shown in fig. 7, in one embodiment, a database data processing apparatus 700 is provided, which specifically includes the following: instruction obtaining module 702, identification extracting module 704, script obtaining module 706, script parsing module 708, data obtaining module 710, data table creating module 712, and data processing module 714.
An instruction obtaining module 702 is configured to obtain a data processing task execution instruction.
And the identifier extracting module 704 is used for extracting the task identifier in the data processing task execution instruction.
And the script obtaining module 706 is configured to obtain a task script corresponding to the task identifier.
And the script parsing module 708 is used for extracting the data processing instruction and the data acquisition condition in the task script.
And the data acquisition module 710 is configured to acquire target data from the databases corresponding to the database types according to the data acquisition conditions.
And a data table creating module 712, configured to create temporary data tables according to the target data corresponding to each database type.
And the data processing module 714 is used for processing the target data in the temporary data table according to the data processing instruction.
In one embodiment, the data table creating module 712 is further configured to obtain a temporary table creating instruction corresponding to each database type; and respectively establishing a temporary data table for the target data corresponding to the corresponding database type according to the temporary table establishing instruction.
In this embodiment, the edited task script is acquired according to the data processing task execution instruction, the task script is analyzed to generate a data processing task, the data processing instruction and the data acquisition condition in the data processing task are extracted, and the target data is acquired from the database corresponding to each database type according to the data acquisition condition. And respectively creating temporary data tables according to the target data corresponding to each database type, and processing the target data corresponding to each database type in the temporary data tables according to the data processing instruction by establishing the temporary data tables corresponding to each database type. The target data corresponding to each database type does not need to be independently operated, and the processing efficiency of the data corresponding to each database type is improved.
In one embodiment, the script obtaining module 706 is further configured to read task identifiers according to task dependencies in the data processing task execution instruction; and acquiring a task script corresponding to the read task identifier.
In this embodiment, the data processing task execution instruction includes a plurality of task identifiers and task dependencies corresponding to the plurality of task identifiers, reads the task identifiers according to the task dependencies, and obtains the task script according to the read task identifiers. The task corresponding to each task identifier is guaranteed to be executed according to the task dependency relationship, the operation steps for executing the task are simplified, the task execution efficiency is improved, a plurality of data processing tasks can be executed according to one data processing task execution instruction, and the data processing efficiency is improved.
As shown in fig. 8, in an embodiment, the data processing module 714 specifically includes the following: a function identifier extracting module 714a, a processing function obtaining module 714b and a processing result obtaining module 714 c.
And the function identifier extracting module 714a is used for extracting the custom function identifier in the data processing instruction.
And the processing function obtaining module 714b is configured to obtain the data processing function from the custom function library according to the custom function identifier.
And a processing result obtaining module 714c, configured to process the target data in the temporary data table according to the obtained data processing function.
In the embodiment, the target data in the temporary data table of each database type can be processed according to the user-defined data processing function, so that the target data can be processed more flexibly, and the accuracy and the efficiency of data processing are improved.
As shown in fig. 9, in an embodiment, the database data processing apparatus 700 further includes the following contents: an instruction fetch module 716, a type extraction module 718, a format conversion module 720, and a data storage module 722.
And an instruction obtaining module 716, configured to obtain a processing result storage instruction.
The type extraction module 718 is configured to extract a database type in the processing result storage instruction.
And a format conversion module 720, configured to convert a processing result obtained by processing the target data into a data format matched with the extracted database type.
And the data storage module 722 is configured to store the processing result after the format conversion to the database corresponding to the extracted database type.
In this embodiment, the database type in the processing result storage instruction is extracted according to the processing result storage instruction, the processing result is subjected to data format conversion, it is ensured that the data format of the processing result is matched with the database type, the processing result after format conversion is stored in the database corresponding to the database type in the processing result storage instruction, and the correctness of the data format of the processing result is ensured.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program: acquiring a data processing task execution instruction; extracting a task identifier in a data processing task execution instruction; acquiring a task script corresponding to the task identifier; extracting a data processing instruction and a data acquisition condition in the task script; acquiring target data from databases corresponding to various database types according to data acquisition conditions; respectively creating a temporary data table according to the target data corresponding to each database type; and processing the target data in the temporary data table according to the data processing instruction.
In one embodiment, obtaining a task script corresponding to a task identifier includes: reading task identifiers according to task dependency in a data processing task execution instruction; and acquiring a task script corresponding to the read task identifier.
In one embodiment, creating the temporary data table according to the target data corresponding to each database type respectively includes: acquiring temporary table establishing instructions corresponding to the database types; and respectively establishing a temporary data table for the target data corresponding to the corresponding database type according to the temporary table establishing instruction.
In one embodiment, processing the target data in the temporary data table according to the data processing instruction includes: extracting a custom function identifier in the data processing instruction; acquiring a data processing function from a custom function library according to the custom function identifier; and processing the target data in the temporary data table according to the acquired data processing function.
In one embodiment, after processing the target data in the temporary data table according to the data processing instruction, the processor executes the computer program to further implement the following steps: acquiring a processing result storage instruction; extracting the database type in the processing result storage instruction; converting a processing result obtained by processing the target data into a data format matched with the extracted database type; and storing the processing result after the format conversion to a database corresponding to the extracted database type.
In this embodiment, the edited task script is acquired according to the data processing task execution instruction, the task script is analyzed to generate a data processing task, the data processing instruction and the data acquisition condition in the data processing task are extracted, and the target data is acquired from the database corresponding to each database type according to the data acquisition condition. And respectively creating temporary data tables according to the target data corresponding to each database type, and processing the target data corresponding to each database type in the temporary data tables according to the data processing instruction by establishing the temporary data tables corresponding to each database type. The target data corresponding to each database type does not need to be independently operated, and the processing efficiency of the data corresponding to each database type is improved.
One or more storage media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: acquiring a data processing task execution instruction; extracting a task identifier in a data processing task execution instruction; acquiring a task script corresponding to the task identifier; extracting a data processing instruction and a data acquisition condition in the task script; acquiring target data from databases corresponding to various database types according to data acquisition conditions; respectively creating a temporary data table according to the target data corresponding to each database type; and processing the target data in the temporary data table according to the data processing instruction.
In one embodiment, obtaining a task script corresponding to a task identifier includes: reading task identifiers according to task dependency in a data processing task execution instruction; and acquiring a task script corresponding to the read task identifier.
In one embodiment, creating the temporary data table according to the target data corresponding to each database type respectively includes: acquiring temporary table establishing instructions corresponding to the database types; and respectively establishing a temporary data table for the target data corresponding to the corresponding database type according to the temporary table establishing instruction.
In one embodiment, processing the target data in the temporary data table according to the data processing instruction includes: extracting a custom function identifier in the data processing instruction; acquiring a data processing function from a custom function library according to the custom function identifier; and processing the target data in the temporary data table according to the acquired data processing function.
In one embodiment, after processing the target data in the temporary data table according to the data processing instruction, the processor further performs the following steps: acquiring a processing result storage instruction; extracting the database type in the processing result storage instruction; converting a processing result obtained by processing the target data into a data format matched with the extracted database type; and storing the processing result after the format conversion to a database corresponding to the extracted database type.
In this embodiment, the edited task script is acquired according to the data processing task execution instruction, the task script is analyzed to generate a data processing task, the data processing instruction and the data acquisition condition in the data processing task are extracted, and the target data is acquired from the database corresponding to each database type according to the data acquisition condition. And respectively creating temporary data tables according to the target data corresponding to each database type, and processing the target data corresponding to each database type in the temporary data tables according to the data processing instruction by establishing the temporary data tables corresponding to each database type. The target data corresponding to each database type does not need to be independently operated, and the processing efficiency of the data corresponding to each database type is improved.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the computer program is executed. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM).
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.