CN107665233A - Database data processing method, device, computer equipment and storage medium - Google Patents
Database data processing method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN107665233A CN107665233A CN201710608523.8A CN201710608523A CN107665233A CN 107665233 A CN107665233 A CN 107665233A CN 201710608523 A CN201710608523 A CN 201710608523A CN 107665233 A CN107665233 A CN 107665233A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- data processing
- type
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24561—Intermediate data storage techniques for performance improvement
Abstract
The present invention relates to a kind of database data processing method, device, computer equipment and storage medium, methods described includes：Obtain data processing task execute instruction；Extract the task identification in the data processing task execute instruction；Obtain mission script corresponding to the task identification；Extract the data processing instructions and data acquisition conditions in the mission script；Target data is obtained according in database corresponding to the data acquisition conditions from each type of database；Temporary data table is respectively created according to target data corresponding to each type of database；The target data in the temporary data table is handled according to the data processing instructions.Database data processing method, device, computer equipment and storage medium provided by the invention, without individually to corresponding to each type of database target data carry out it is individually operated, improve the treatment effeciency of data corresponding to each type of database.
Description
Technical field
The present invention relates to technical field of data processing, more particularly to a kind of database data processing method, device, calculating
Machine equipment and storage medium.
Background technology
The development of computer technology at any time, substantial amounts of data are required for being handled with computer.With data processing needs
Continuous growth, the data-handling capacity of computer also constantly lifting.
In the storing process of data, the database of disparate databases type can be stored data according to business demand
In.Traditional data processing technique, the data in the database of single database type can only be handled.It is more when being related to
During the data of individual type of database so that data-handling efficiency is relatively low.
The content of the invention
Based on this, it is necessary to for database data treatment effeciency it is low the problem of, there is provided a kind of processing of database data
Method, apparatus, computer equipment and storage medium.
A kind of database data processing method, methods described include：
Obtain data processing task execute instruction；
Extract the task identification in the data processing task execute instruction；
Obtain mission script corresponding to the task identification；
Extract the data processing instructions and data acquisition conditions in the mission script；
Target data is obtained according in database corresponding to the data acquisition conditions from each type of database；
Temporary data table is respectively created according to target data corresponding to each type of database；
The target data in the temporary data table is handled according to the data processing instructions.
A kind of database data processing unit, described device include：
Instruction acquisition module, for obtaining data processing task execute instruction；
Marker extraction module, for extracting the task identification in the data processing task execute instruction；
Script acquisition module, for obtaining mission script corresponding to the task identification；
Script parsing module, for extracting data processing instructions and data acquisition conditions in the mission script；
Data acquisition module, for being obtained in the database according to corresponding to the data acquisition conditions from each type of database
Target data；
Tables of data creation module, ephemeral data is respectively created for the target data according to corresponding to each type of database
Table；
Data processing module, for being carried out according to the data processing instructions to the target data in the temporary data table
Processing.
A kind of computer equipment, the computer equipment include memory, processor and are stored on the memory simultaneously
The computer program that can be run on the processor, following steps are realized during computer program described in the computing device：
Obtain data processing task execute instruction；
Extract the task identification in the data processing task execute instruction；
Obtain mission script corresponding to the task identification；
Extract the data processing instructions and data acquisition conditions in the mission script；
Target data is obtained according in database corresponding to the data acquisition conditions from each type of database；
Temporary data table is respectively created according to target data corresponding to each type of database；
The target data in the temporary data table is handled according to the data processing instructions.
One or more is stored with the storage medium of computer executable instructions, and the computer executable instructions are by one
Or during multiple computing devices so that one or more of computing device following steps：
Obtain data processing task execute instruction；
Extract the task identification in the data processing task execute instruction；
Obtain mission script corresponding to the task identification；
Extract the data processing instructions and data acquisition conditions in the mission script；
Target data is obtained according in database corresponding to the data acquisition conditions from each type of database；
Temporary data table is respectively created according to target data corresponding to each type of database；
The target data in the temporary data table is handled according to the data processing instructions.
Above-mentioned database data processing method, device, computer equipment and storage medium, are performed according to data processing task
Instruction obtains the mission script of editor, and generation data processing task is parsed to mission script, extracts the number in data processing task
According to process instruction and data acquisition conditions, target is obtained according in database corresponding to data acquisition conditions from each type of database
Data.Temporary data table is respectively created according to target data corresponding to each type of database, by establishing each type of database pair
The temporary data table answered, according to data processing instructions to passing through the target data in temporary data table corresponding to each type of database
Handled.Need not be individually individually operated to target data progress corresponding to each type of database, improve each class database
The treatment effeciency of data corresponding to type.
Brief description of the drawings
Fig. 1 is the applied environment figure of database data processing method in one embodiment；
Fig. 2 is the structured flowchart of the server in database data processing system in one embodiment；
Fig. 3 is the schematic flow sheet of database data processing method in one embodiment；
Fig. 4 is schematic flow sheet the step of reading task identification according to task dependence in one embodiment；
Fig. 5 is schematic flow sheet the step of SQL is obtained in one embodiment；
Fig. 6 is schematic flow sheet the step of result is stored in one embodiment；
Fig. 7 is the structured flowchart of database data processing unit in one embodiment；
Fig. 8 is the structured flowchart of data processing module in one embodiment；
Fig. 9 is the structured flowchart of database data processing unit in another embodiment.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, it is right below in conjunction with drawings and Examples
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
Fig. 1 is the applied environment figure of a database data processing method.Reference picture 1, the database data processing method
Applied to database data processing system.Database data processing system includes terminal 110 and server 120, wherein terminal 110
It is connected by network with server 120.Terminal 110 can be fixed terminal or mobile terminal, and fixed terminal can be beaten
At least one of print machine, scanner and monitor, mobile terminal can be specifically tablet personal computer, smart mobile phone, personal data
At least one of assistant and digital camera.
Fig. 2 is the internal structure schematic diagram of the server 120 in Fig. 1 database data processing systems in one embodiment.
As shown in Fig. 2 the server 120 include by system bus connection processor, non-volatile memory medium, built-in storage and
Network interface.The processor of server 120 is used to provide calculating and control ability, supports the operation of whole server 120, stores
Device is used for data storage, code command etc., and network interface is used to carry out network service with terminal 110.Be stored with memory to
Few computer executable instructions, the computer executable instructions can be executed by processor, to realize in the embodiment of the present application
The database data processing method suitable for server 120 provided.Memory may include magnetic disc, CD, read-only store-memory
The non-volatile memory mediums such as body (Read-Only Memory, ROM), or random access memory (Random Access
Memory, RAM) etc..For example, in one embodiment, memory includes non-volatile memory medium and built-in storage；It is non-volatile
Property storage medium be stored with processing system, database and computer executable instructions, database includes multiple type of database point
Not corresponding multiple databases, store target data in database, the computer executable instructions can be executed by processor with
Realize above-mentioned database data processing method；Built-in storage can for the processing system in non-volatile memory medium and computer
Execute instruction provides the running environment of cache.
As shown in figure 3, in one embodiment, there is provided a kind of database data processing method, the present embodiment is in this way
Come applied to the server 120 in Fig. 1 for example, this method specifically includes herein below：
S302, obtain data processing task execute instruction.
Specifically, terminal 110 obtains the user account of user's input, and page data is generated according to the user account got
Request is obtained, page data is obtained into request sends to server 120.Server 120 receives the page number of the transmission of terminal 110
After request is obtained, request is obtained to page data and parsed, the user in asking is obtained by parsing extraction page data
Account.Server 120 carries out Authority Verification to the user account extracted, has in the user account that checking is extracted and accesses number
During according to the access rights for handling the page, request is obtained according to page data and handles page data to the returned data of terminal 110.Terminal
110 after the data manipulation page data received, and the page is handled according to data manipulation page data display data.Terminal 110
Obtain user and choose task identification in the data manipulation page of display, data processing task is generated according to the task identification of selection
Execute instruction.Terminal 110 sends data processing task execute instruction to server 120.
S304, extract the task identification in data processing task execute instruction.
Specifically, server 120 is receiving the data processing task execute instruction of the transmission of terminal 110, to data processing
Task-performance instructions are parsed, by parsing the task identification extracted in task-performance instructions.Task identification be task only
One mark.
S306, obtain mission script corresponding to task identification.
Specifically, terminal 110 obtains the task identification and task identification that user inputs in the data manipulation page of display
Corresponding mission script, the task identification got and mission script are sent to server 120.Server 120 will receive
Task identification and mission script is corresponding stores.Execution script when mission script is performs data processing task.Server 120
After task identification is extracted, mission script corresponding with the task identification extracted is inquired about in the mission script of storage, is obtained
Take the mission script inquired.
S308, extract data processing instructions and data acquisition conditions in mission script.
Specifically, server 120 parses after mission script is extracted to mission script, by parsing from task
Data processing instructions and data acquisition conditions are extracted in script.Wherein, data processing instructions are the instruction handled data.
In one embodiment, server 120 parses to mission script, by parsing the number in identification mission script
It is data processing instructions according to calculation resources configuration parameter according to process instruction, data acquisition conditions and calculation resources configuration parameter
Configure calculation resources.
S310, target data is obtained according in database corresponding to data acquisition conditions from each type of database.
Specifically, server 120 can conduct interviews to database corresponding to various database types.Wherein, a variety of numbers
It can be arranged on server 120, can also be arranged on other servers according to database corresponding to the type of storehouse.Server 120
The access data-interface of database corresponding to various type of database is obtained, by accessing data access interface from corresponding data
The target data for meeting data acquisition conditions is obtained in storehouse.
In one embodiment, server 120 extracts the Database Identification in data acquisition conditions, identification database mark
Corresponding type of database, the database access interface answered of type of database is obtained, according to database access interface from database
The target data for meeting data acquisition conditions is obtained in database corresponding to mark.
In one embodiment, server 120 extracts the type of database in data acquisition conditions, obtains type of database
Database access interface and inquiry type of database corresponding to database, according to database access interface from the data inquired
The target data for meeting data acquisition conditions is obtained in storehouse.
S312, temporary data table is respectively created according to target data corresponding to each type of database.
Specifically, after server 120 gets target data corresponding to each type of database, according to each type of database pair
The temporary data table of preset format is respectively created in the target data answered, obtain each type of database corresponding to temporary data table, face
When tables of data in store target data.
In one embodiment, S312 specifically includes herein below：Interim table corresponding to each type of database is obtained to create
Instruction；Instruction is created according to interim table temporary data table is created to target data corresponding to corresponding type of database respectively.
Specifically, server 120 gets target data corresponding to each type of database, obtains each type of database respectively
Corresponding interim table creates instruction, the interim table of data according to corresponding to the establishment instruction of interim table creates each type of database, will be each
The Data Format Transform of target data corresponding to type of database is the data format matched with temporary data table, by data format
Target data after conversion is stored in corresponding temporary data table.
S314, the target data in temporary data table is handled according to data processing instructions.
Specifically, after server 120 creates temporary data table corresponding to each type of database, performed according to data processing
Data processing is carried out to the target data in temporary data table corresponding to each type of database, by each temporary data table
The processing of target data obtains result.
In the present embodiment, the mission script of editor is obtained according to data processing task execute instruction, mission script is parsed
Data processing task is generated, data processing instructions and data acquisition conditions in data processing task are extracted, according to data acquisition
Target data is obtained in database corresponding to condition from each type of database.According to target data corresponding to each type of database point
Temporary data table is not created, by establishing temporary data table corresponding to each type of database, according to data processing instructions to each number
Handled according to corresponding to the type of storehouse by the target data in temporary data table.Without individually corresponding to each type of database
Target data carry out it is individually operated, improve the treatment effeciency of data corresponding to each type of database.
As shown in figure 4, in one embodiment, S306 specifically also includes reading task identification according to task dependence
Step, the step specifically include herein below：
S402, the task dependence in data processing task execute instruction read each task identification.
Specifically, data processing task execute instruction includes task corresponding to multiple tasks mark and multiple tasks mark
Dependence.Server 120 parses to data processing task-performance instructions, is performed by parsing extraction data processing task
Task dependence and task identification in instruction, task identification is read according to task dependence.
In one embodiment, task identification is arranged according to task dependence in data processing task execute instruction
Task traffic identifier.Server 120 reads task identification according to task dependence from task traffic identifier.
For example, the task flow in data processing task execute instruction be identified as " 1,2 | 3 | 4,5 ", in task traffic identifier
Include the task identification of task 1, task 2, task 3, task 4 and task 5.Wherein ", " expression task has dependence, first carries out ", "
Task corresponding to preceding task identification, task corresponding to the task identification after ", " is performed afterwards；" | " is represented before can performing " | " parallel
Task corresponding to task identification afterwards." 1,2 | 3 | 4,5 " tasks carrying order is finished to first carry out task 1 in task 1
Afterwards, then executing tasks parallelly 2, task 3 and task 4, task 2, task 3 and task 4 are parallel be finished after, then perform task
5。
S404, obtain mission script corresponding with the task identification read.
Specifically, server 120 is from task-performance instructions according to task dependence after task identification is read,
Mission script corresponding with task identification is inquired about in the mission script of storage, obtains the mission script inquired.
In the present embodiment, data processing task execute instruction includes appointing corresponding to multiple tasks mark and multiple tasks mark
Business dependence, task identification is read according to task dependence, and mission script is obtained according to the task identification read.Ensure
The task according to corresponding to task dependence performs each task identification, simplifies the operating procedure of execution task, improves task
Execution efficiency, multiple data processing tasks are can perform according to a data processing task-performance instructions, improve data processing
Efficiency.
As shown in figure 5, in one embodiment, S314 specifically includes the step of obtaining SQL, and the step is specific
Including herein below：
S502, extract the SQL mark in data processing instructions.
Specifically, server 120 parses to data processing instructions, self-defined in extraction data processing instructions by parsing
Function identification.Wherein, SQL is identified as the unique mark of user-defined function.
S504, data processing function is obtained from SQL storehouse according to SQL mark.
Specifically, terminal 110 obtains user's data processing function that user inputs in data manipulation page, will get
Data processing function send to server 120.Server 120 extracts data processing letter after data processing function is received
SQL mark in number, is stored in the data processing function received is corresponding with the SQL mark extracted
In SQL storehouse.After server 120 extracts from data processing instructions SQL mark, according to extract from
Defined function mark inquires about data processing function in SQL storehouse, obtains the data processing function inquired.
S506, the target data in temporary data table is handled according to the data processing function got.
Specifically, server 120 is carried out according to the data processing function got to the target data in temporary data table
Processing, is handled target data to obtain result according to data processing function.The processing that server 120 will can obtain
As a result send to terminal 110, terminal 110 is shown in data manipulation page.
In the present embodiment, in the temporary data table that each type of database of function pair can be handled according to custom data
Target data handled so that the processing to target data is more flexible, improve data processing accuracy and efficiently
Property.
As shown in fig. 6, in one embodiment, specific the step of also including storage result, the step after S314
Specifically also include herein below：
S602, obtain result store instruction.
Specifically, the result that terminal 110 obtains server 120 is included in data manipulation page, detects number
When being clicked according to the result save button in the processing page, result store instruction is triggered, the result of triggering is stored
Instruction is sent to server 120.The result store instruction that the receiving terminal 110 of server 120 is sent.
In one embodiment, server 120 can detect whether data handling procedure terminates, and detect data processing
At the end of process, result store instruction is triggered.
S604, the type of database in extraction process result store instruction.
Specifically, server 120 parses to result store instruction, is referred to by parsing extraction process result storage
Type of database in order.Wherein, the type of database extracted is the database corresponding to the database of result storage
Type.
S606, the result that will be handled to obtain to target data, the type of database for being converted to and extracting
The data format matched somebody with somebody.
Specifically, server 120, which obtains, is handled target data obtained result, it is determined that the data extracted
The data format of storehouse type matching, result is subjected to Data Format Transform, result is converted to the data extracted
The data format of storehouse type matching.
S608, the result after format transformation is stored to database corresponding to the type of database extracted.
Specifically, server 120 inquires about database corresponding to the type of database extracted, the data of extraction process result
Mark, the Data Identification extracted and result are correspondingly stored into the database inquired.
In one embodiment, the Database Identification in the extraction process result store instruction of server 120, identification are extracted
Database Identification corresponding to type of database belonging to database.The type of database that server 120 determines to recognize corresponds to
Data format, by the Data Format Transform of result for determine data format, the result after format transformation is deposited
Storage is into database corresponding to the Database Identification extracted.
In the present embodiment, according to result store instruction, the type of database in extraction process result store instruction will
Result carries out Data Format Transform, ensures that the data format of result matches with type of database, after format transformation
Result store the number that result in database corresponding to type of database, is ensure that into result store instruction
According to the correctness of form.
In one embodiment, user is by inputting user account and use in browser page and data processing five application page
Family password.Terminal 110 obtains the user account and user cipher of input, is generated according to the user account and user cipher got
Page data request, terminal 110 send page data request to server 120.
After server 120 receives page data request, user account and user cipher in page data request are extracted,
Whether access rights are had according to user corresponding to user account and user cipher checking user account.Server 120 is being verified
After user has access rights corresponding to user account, page data is handled to the returned data of terminal 110.
Terminal 110 handles the page according to data manipulation page data display data.User chooses in data manipulation page
Task identification, pass through the input equipment click data tasks carrying button of terminal 110.Terminal 110 is detecting data processing page
When data task executive button in face is clicked, according to the task identification of selection and choose task identification corresponding to task according to
The relation of relying generation data processing task execute instruction, data processing task execute instruction is sent to server 120.
Task corresponding to task identification and task identification in the extraction data processing task execute instruction of server 120 relies on
Relation, the multiple data processing tasks in city are split according to task dependence and task identification.Server 120 is respectively according to each data
Task identification extraction mission script in processing task, extracts the data processing instructions and data acquisition conditions in mission script.
Server 120 inquires about mesh in the database according to corresponding to data acquisition conditions from each type of database that can be accessed
Data are marked, obtain target data corresponding to each type of database.Server 120 according to corresponding to obtaining each type of database respectively
Interim table creates instruction, and creating instruction according to interim table creates temporary data table corresponding with type of database, by corresponding mesh
Mark temporary data table corresponding to data storage to type of database.It can be handed over before temporary data table corresponding to each type of database
Mutually.Server 120 according to data processing instructions to the target data in temporary data table corresponding to each type of database at
Reason obtains result.
Server 120 obtains user account, data processing time, data processing type, the class database that terminal 110 logs in
At least one of type and data acquisition conditions, according to get user account, data processing time, data processing type,
At least one of type of database and data acquisition conditions generate data processing daily record.
As shown in fig. 7, in one embodiment, there is provided a kind of database data processing unit 700, the device specifically include
Herein below：Instruction acquisition module 702, marker extraction module 704, script acquisition module 706, script parsing module 708, data
Acquisition module 710, tables of data creation module 712 and data processing module 714.
Instruction acquisition module 702, for obtaining data processing task execute instruction.
Marker extraction module 704, for extracting the task identification in data processing task execute instruction.
Script acquisition module 706, for obtaining mission script corresponding to task identification.
Script parsing module 708, for extracting data processing instructions and data acquisition conditions in mission script.
Data acquisition module 710, for being obtained in the database according to corresponding to data acquisition conditions from each type of database
Target data.
Tables of data creation module 712, ephemeral data is respectively created for the target data according to corresponding to each type of database
Table.
Data processing module 714, for being handled according to data processing instructions the target data in temporary data table.
In one embodiment, tables of data creation module 712 is additionally operable to obtain interim table wound corresponding to each type of database
Build instruction；Instruction is created according to interim table temporary data table is created to target data corresponding to corresponding type of database respectively.
In the present embodiment, the mission script of editor is obtained according to data processing task execute instruction, mission script is parsed
Data processing task is generated, data processing instructions and data acquisition conditions in data processing task are extracted, according to data acquisition
Target data is obtained in database corresponding to condition from each type of database.According to target data corresponding to each type of database point
Temporary data table is not created, by establishing temporary data table corresponding to each type of database, according to data processing instructions to each number
Handled according to corresponding to the type of storehouse by the target data in temporary data table.Without individually corresponding to each type of database
Target data carry out it is individually operated, improve the treatment effeciency of data corresponding to each type of database.
In one embodiment, script acquisition module 706 is additionally operable to the task in data processing task execute instruction
Dependence reads each task identification；Obtain mission script corresponding with the task identification read.
In the present embodiment, data processing task execute instruction includes appointing corresponding to multiple tasks mark and multiple tasks mark
Business dependence, task identification is read according to task dependence, and mission script is obtained according to the task identification read.Ensure
The task according to corresponding to task dependence performs each task identification, simplifies the operating procedure of execution task, improves task
Execution efficiency, multiple data processing tasks are can perform according to a data processing task-performance instructions, improve data processing
Efficiency.
As shown in figure 8, in one embodiment, data processing module 714 specifically includes herein below：Function identification is extracted
Module 714a, processing function acquisition module 714b and result obtain module 714c.
Function identification extraction module 714a, for extracting the mark of the SQL in data processing instructions.
Function acquisition module 714b is handled, for being obtained according to SQL mark from SQL storehouse at data
Manage function.
Result obtains module 714c, for according to the data processing function that gets to the target in temporary data table
Data are handled.
In the present embodiment, in the temporary data table that each type of database of function pair can be handled according to custom data
Target data handled so that the processing to target data is more flexible, improve data processing accuracy and efficiently
Property.
As shown in figure 9, in one embodiment, database data processing unit 700 specifically also includes herein below：Instruction
Acquisition module 716, type-collection module 718, format converting module 720 and data memory module 722.
Instruction acquisition module 716, for obtaining result store instruction.
Type-collection module 718, for the type of database in extraction process result store instruction.
Format converting module 720, for the result that will be handled to obtain to target data, be converted to and extract
Type of database matching data format.
Data memory module 722, for the result after format transformation to be stored to the type of database pair extracted
The database answered.
In the present embodiment, according to result store instruction, the type of database in extraction process result store instruction will
Result carries out Data Format Transform, ensures that the data format of result matches with type of database, after format transformation
Result store the number that result in database corresponding to type of database, is ensure that into result store instruction
According to the correctness of form.
A kind of computer equipment, computer equipment include memory, processor and storage on a memory and can handled
The computer program run on device, following steps are realized during computing device computer program：Data processing task is obtained to perform
Instruction；Extract the task identification in data processing task execute instruction；Obtain mission script corresponding to task identification；Extraction task
Data processing instructions and data acquisition conditions in script；According to database corresponding to data acquisition conditions from each type of database
Middle acquisition target data；Temporary data table is respectively created according to target data corresponding to each type of database；According to data processing
Instruction is handled the target data in temporary data table.
In one embodiment, mission script corresponding to task identification is obtained, including：Performed and referred to according to data processing task
Task dependence in order reads each task identification；Obtain mission script corresponding with the task identification read.
In one embodiment, temporary data table is respectively created in the target data according to corresponding to each type of database, including：
Obtain interim table corresponding to each type of database and create instruction；Instruction is created respectively to corresponding type of database according to interim table
Corresponding target data creates temporary data table.
In one embodiment, the target data in temporary data table is handled according to data processing instructions, including：
Extract the SQL mark in data processing instructions；Data are obtained from SQL storehouse according to SQL mark
Handle function；The target data in temporary data table is handled according to the data processing function got.
In one embodiment, after being handled according to data processing instructions the target data in temporary data table,
Following steps are also realized during computing device computer program：Obtain result store instruction；Extraction process result storage refers to
Type of database in order；The result that will be handled to obtain to target data, the class database for being converted to and extracting
The data format of type matching；Result after format transformation is stored to database corresponding to the type of database extracted.
In the present embodiment, the mission script of editor is obtained according to data processing task execute instruction, mission script is parsed
Data processing task is generated, data processing instructions and data acquisition conditions in data processing task are extracted, according to data acquisition
Target data is obtained in database corresponding to condition from each type of database.According to target data corresponding to each type of database point
Temporary data table is not created, by establishing temporary data table corresponding to each type of database, according to data processing instructions to each number
Handled according to corresponding to the type of storehouse by the target data in temporary data table.Without individually corresponding to each type of database
Target data carry out it is individually operated, improve the treatment effeciency of data corresponding to each type of database.
One or more is stored with the storage medium of computer executable instructions, and computer executable instructions are by one or more
During individual computing device so that one or more processors perform following steps：Obtain data processing task execute instruction；Extraction
Task identification in data processing task execute instruction；Obtain mission script corresponding to task identification；Extract in mission script
Data processing instructions and data acquisition conditions；Mesh is obtained according in database corresponding to data acquisition conditions from each type of database
Mark data；Temporary data table is respectively created according to target data corresponding to each type of database；According to data processing instructions to facing
When tables of data in target data handled.
In one embodiment, mission script corresponding to task identification is obtained, including：Performed and referred to according to data processing task
Task dependence in order reads each task identification；Obtain mission script corresponding with the task identification read.
In one embodiment, temporary data table is respectively created in the target data according to corresponding to each type of database, including：
Obtain interim table corresponding to each type of database and create instruction；Instruction is created respectively to corresponding type of database according to interim table
Corresponding target data creates temporary data table.
In one embodiment, the target data in temporary data table is handled according to data processing instructions, including：
Extract the SQL mark in data processing instructions；Data are obtained from SQL storehouse according to SQL mark
Handle function；The target data in temporary data table is handled according to the data processing function got.
In one embodiment, after being handled according to data processing instructions the target data in temporary data table,
Processor also performs following steps：Obtain result store instruction；Type of database in extraction process result store instruction；
The result that will be handled to obtain to target data, be converted to the data format matched with the type of database extracted；
Result after format transformation is stored to database corresponding to the type of database extracted.
In the present embodiment, the mission script of editor is obtained according to data processing task execute instruction, mission script is parsed
Data processing task is generated, data processing instructions and data acquisition conditions in data processing task are extracted, according to data acquisition
Target data is obtained in database corresponding to condition from each type of database.According to target data corresponding to each type of database point
Temporary data table is not created, by establishing temporary data table corresponding to each type of database, according to data processing instructions to each number
Handled according to corresponding to the type of storehouse by the target data in temporary data table.Without individually corresponding to each type of database
Target data carry out it is individually operated, improve the treatment effeciency of data corresponding to each type of database.
One of ordinary skill in the art will appreciate that realize all or part of flow in above-described embodiment method, being can be with
The hardware of correlation is instructed to complete by computer program, the computer program can be stored in a computer-readable storage and be situated between
In matter, the program is upon execution, it may include such as the flow of the embodiment of above-mentioned each method.Wherein, foregoing storage medium can be
The non-volatile memory mediums such as magnetic disc, CD, read-only memory (Read-Only Memory, ROM), or random storage note
Recall body (Random Access Memory, RAM) etc..
Each technical characteristic of embodiment described above can be combined arbitrarily, to make description succinct, not to above-mentioned reality
Apply all possible combination of each technical characteristic in example to be all described, as long as however, the combination of these technical characteristics is not deposited
In contradiction, the scope that this specification is recorded all is considered to be.
Embodiment described above only expresses the several embodiments of the present invention, and its description is more specific and detailed, but simultaneously
Can not therefore it be construed as limiting the scope of the patent.It should be pointed out that come for one of ordinary skill in the art
Say, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to the protection of the present invention
Scope.Therefore, the protection domain of patent of the present invention should be determined by the appended claims.
Claims (10)
1. a kind of database data processing method, methods described include：
Obtain data processing task execute instruction；
Extract the task identification in the data processing task execute instruction；
Obtain mission script corresponding to the task identification；
Extract the data processing instructions and data acquisition conditions in the mission script；
Target data is obtained according in database corresponding to the data acquisition conditions from each type of database；
Temporary data table is respectively created according to target data corresponding to each type of database；
The target data in the temporary data table is handled according to the data processing instructions.
2. according to the method for claim 1, it is characterised in that it is described to obtain mission script corresponding to the task identification,
Including：
Task dependence in the data processing task execute instruction reads each task identification；
Obtain mission script corresponding with the task identification read.
3. according to the method for claim 2, it is characterised in that the number of targets according to corresponding to each type of database
According to temporary data table is respectively created, including：
Obtain interim table corresponding to each type of database and create instruction；
Instruction is created according to the interim table temporary data table is created to target data corresponding to corresponding type of database respectively.
4. according to the method for claim 3, it is characterised in that it is described according to the data processing instructions to the nonce
Handled according to the target data in table, including：
Extract the SQL mark in data processing instructions；
Data processing function is obtained from SQL storehouse according to SQL mark；
The target data in the temporary data table is handled according to the data processing function got.
5. according to the method any one of claim 1-4, it is characterised in that described according to the data processing instructions pair
After target data in the temporary data table is handled, in addition to：
Obtain result store instruction；
Extract the type of database in the result store instruction；
The result that will be handled to obtain to the target data, be converted to the number matched with the type of database extracted
According to form；
Result after format transformation is stored to database corresponding to the type of database extracted.
6. a kind of database data processing unit, it is characterised in that described device includes：
Instruction acquisition module, for obtaining data processing task execute instruction；
Marker extraction module, for extracting the task identification in the data processing task execute instruction；
Script acquisition module, for obtaining mission script corresponding to the task identification；
Script parsing module, for extracting data processing instructions and data acquisition conditions in the mission script；
Data acquisition module, for obtaining target in the database according to corresponding to the data acquisition conditions from each type of database
Data；
Tables of data creation module, temporary data table is respectively created for the target data according to corresponding to each type of database；
Data processing module, at according to the data processing instructions to the target data in the temporary data table
Reason.
7. device according to claim 6, it is characterised in that the data processing module includes：
Function identification extraction module, for extracting the mark of the SQL in data processing instructions；
Function acquisition module is handled, for obtaining data processing letter from SQL storehouse according to SQL mark
Number；
Result obtains module, for according to the data processing function that gets to the target data in the temporary data table
Handled.
8. the device according to claim 6 or 7, it is characterised in that described device also includes：
Instruction acquisition module, for obtaining result store instruction；
Type-collection module, for extracting the type of database in the result store instruction；
Format converting module, for the result that will be handled to obtain to the target data, be converted to and extract
The data format of type of database matching；
Data memory module, for the result after format transformation to be stored to corresponding to the type of database extracted
Database.
9. a kind of computer equipment, the computer equipment includes memory, processor and is stored on the memory and can
The computer program run on the processor, following steps are realized during computer program described in the computing device：
Obtain data processing task execute instruction；
Extract the task identification in the data processing task execute instruction；
Obtain mission script corresponding to the task identification；
Extract the data processing instructions and data acquisition conditions in the mission script；
Target data is obtained according in database corresponding to the data acquisition conditions from each type of database；
Temporary data table is respectively created according to target data corresponding to each type of database；
The target data in the temporary data table is handled according to the data processing instructions.
10. one or more is stored with the storage medium of computer executable instructions, the computer executable instructions are by one
Or during multiple computing devices so that one or more of computing device following steps：
Obtain data processing task execute instruction；
Extract the task identification in the data processing task execute instruction；
Obtain mission script corresponding to the task identification；
Extract the data processing instructions and data acquisition conditions in the mission script；
Target data is obtained according in database corresponding to the data acquisition conditions from each type of database；
Temporary data table is respectively created according to target data corresponding to each type of database；
The target data in the temporary data table is handled according to the data processing instructions.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710608523.8A CN107665233B (en) | 2017-07-24 | 2017-07-24 | Database data processing method and device, computer equipment and storage medium |
| PCT/CN2017/104253 WO2019019361A1 (en) | 2017-07-24 | 2017-09-29 | Method and apparatus for processing data of database, computer device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710608523.8A CN107665233B (en) | 2017-07-24 | 2017-07-24 | Database data processing method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107665233A true CN107665233A (en) | 2018-02-06 |
| CN107665233B CN107665233B (en) | 2020-07-31 |
Family
ID=61122580
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710608523.8A Active CN107665233B (en) | 2017-07-24 | 2017-07-24 | Database data processing method and device, computer equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107665233B (en) |
| WO (1) | WO2019019361A1 (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108446175A (en) * | 2018-03-12 | 2018-08-24 | 平安普惠企业管理有限公司 | Task processing method, device, computer equipment and storage medium |
| CN108446371A (en) * | 2018-03-15 | 2018-08-24 | 平安科技（深圳）有限公司 | Data return guiding method, device, computer equipment and storage medium |
| CN109284097A (en) * | 2018-09-07 | 2019-01-29 | 武汉轻工大学 | Realize method, equipment, system and the storage medium of complex data analysis |
| CN110020773A (en) * | 2019-01-30 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of data processing method, equipment and computer-readable medium |
| CN110019434A (en) * | 2018-08-23 | 2019-07-16 | 深圳市信锐网科技术有限公司 | Internet of things data processing method, device, platform and storage medium |
| CN110309214A (en) * | 2018-04-10 | 2019-10-08 | 腾讯科技（深圳）有限公司 | A kind of instruction executing method and its equipment, storage medium, server |
| CN110489213A (en) * | 2018-05-15 | 2019-11-22 | 华为技术有限公司 | A kind of task processing method and processing unit, computer system |
| CN110750308A (en) * | 2019-09-11 | 2020-02-04 | 东软集团股份有限公司 | Task processing method and device, storage medium and electronic equipment |
| CN110750597A (en) * | 2018-07-23 | 2020-02-04 | 北京国双科技有限公司 | Data processing method and device, storage medium and processor |
| CN111078733A (en) * | 2019-11-26 | 2020-04-28 | 金蝶软件（中国）有限公司 | Batch task processing method and device, computer equipment and storage medium |
| CN111339170A (en) * | 2018-12-19 | 2020-06-26 | 深圳市优必选科技有限公司 | Data processing method and device, computer equipment and storage medium |
| CN111813783A (en) * | 2020-07-27 | 2020-10-23 | 南方电网数字电网研究院有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN112000644A (en) * | 2020-08-21 | 2020-11-27 | 北京浪潮数据技术有限公司 | Data processing method and system, electronic equipment and storage medium |
| CN112988806A (en) * | 2019-12-17 | 2021-06-18 | 北京沃东天骏信息技术有限公司 | Data processing method and device |
| CN113269547A (en) * | 2021-05-31 | 2021-08-17 | 中国农业银行股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113778999A (en) * | 2021-09-29 | 2021-12-10 | 平安资产管理有限责任公司 | Data modularization processing method and device, computer equipment and storage medium |
| CN114238286A (en) * | 2022-02-28 | 2022-03-25 | 连连（杭州）信息技术有限公司 | Data warehouse data processing method and device, electronic equipment and storage medium |
| CN115952172A (en) * | 2023-03-08 | 2023-04-11 | 畅捷通信息技术股份有限公司 | Data matching method and device based on temporary table of database |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101944128A (en) * | 2010-09-25 | 2011-01-12 | 中兴通讯股份有限公司 | Data export and import method and device |
| CN102346775A (en) * | 2011-09-26 | 2012-02-08 | 苏州博远容天信息科技有限公司 | Method for synchronizing multiple heterogeneous source databases based on log |
| US20160246824A1 (en) * | 2011-11-10 | 2016-08-25 | Treasure Data, Inc. | System and method for operating a big-data platform |
| CN105930389A (en) * | 2016-04-14 | 2016-09-07 | 北京京东尚科信息技术有限公司 | Method and system for transferring data |
| CN105989142A (en) * | 2015-02-28 | 2016-10-05 | 华为技术有限公司 | Data query method and device |
| CN106649355A (en) * | 2015-10-30 | 2017-05-10 | 北京国双科技有限公司 | Method and device for processing data |
| CN106815353A (en) * | 2017-01-20 | 2017-06-09 | 星环信息科技（上海）有限公司 | A kind of method and apparatus of data query |
-
2017
- 2017-07-24 CN CN201710608523.8A patent/CN107665233B/en active Active
- 2017-09-29 WO PCT/CN2017/104253 patent/WO2019019361A1/en active Application Filing
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101944128A (en) * | 2010-09-25 | 2011-01-12 | 中兴通讯股份有限公司 | Data export and import method and device |
| CN102346775A (en) * | 2011-09-26 | 2012-02-08 | 苏州博远容天信息科技有限公司 | Method for synchronizing multiple heterogeneous source databases based on log |
| US20160246824A1 (en) * | 2011-11-10 | 2016-08-25 | Treasure Data, Inc. | System and method for operating a big-data platform |
| CN105989142A (en) * | 2015-02-28 | 2016-10-05 | 华为技术有限公司 | Data query method and device |
| CN106649355A (en) * | 2015-10-30 | 2017-05-10 | 北京国双科技有限公司 | Method and device for processing data |
| CN105930389A (en) * | 2016-04-14 | 2016-09-07 | 北京京东尚科信息技术有限公司 | Method and system for transferring data |
| CN106815353A (en) * | 2017-01-20 | 2017-06-09 | 星环信息科技（上海）有限公司 | A kind of method and apparatus of data query |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108446175A (en) * | 2018-03-12 | 2018-08-24 | 平安普惠企业管理有限公司 | Task processing method, device, computer equipment and storage medium |
| CN108446371A (en) * | 2018-03-15 | 2018-08-24 | 平安科技（深圳）有限公司 | Data return guiding method, device, computer equipment and storage medium |
| CN108446371B (en) * | 2018-03-15 | 2020-10-27 | 平安科技（深圳）有限公司 | Data back-leading method and device, computer equipment and storage medium |
| CN110309214A (en) * | 2018-04-10 | 2019-10-08 | 腾讯科技（深圳）有限公司 | A kind of instruction executing method and its equipment, storage medium, server |
| CN110309214B (en) * | 2018-04-10 | 2023-06-23 | 腾讯科技（深圳）有限公司 | Instruction execution method and equipment, storage medium and server thereof |
| CN110489213A (en) * | 2018-05-15 | 2019-11-22 | 华为技术有限公司 | A kind of task processing method and processing unit, computer system |
| CN110489213B (en) * | 2018-05-15 | 2022-04-05 | 华为技术有限公司 | Task processing method and processing device and computer system |
| US11941434B2 (en) | 2018-05-15 | 2024-03-26 | Huawei Technologies Co., Ltd. | Task processing method, processing apparatus, and computer system |
| CN110750597A (en) * | 2018-07-23 | 2020-02-04 | 北京国双科技有限公司 | Data processing method and device, storage medium and processor |
| CN110019434A (en) * | 2018-08-23 | 2019-07-16 | 深圳市信锐网科技术有限公司 | Internet of things data processing method, device, platform and storage medium |
| CN109284097A (en) * | 2018-09-07 | 2019-01-29 | 武汉轻工大学 | Realize method, equipment, system and the storage medium of complex data analysis |
| CN111339170A (en) * | 2018-12-19 | 2020-06-26 | 深圳市优必选科技有限公司 | Data processing method and device, computer equipment and storage medium |
| CN110020773A (en) * | 2019-01-30 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of data processing method, equipment and computer-readable medium |
| CN110750308B (en) * | 2019-09-11 | 2024-03-01 | 东软集团股份有限公司 | Task processing method and device, storage medium and electronic equipment |
| CN110750308A (en) * | 2019-09-11 | 2020-02-04 | 东软集团股份有限公司 | Task processing method and device, storage medium and electronic equipment |
| CN111078733B (en) * | 2019-11-26 | 2024-02-09 | 金蝶软件（中国）有限公司 | Batch task processing method, device, computer equipment and storage medium |
| CN111078733A (en) * | 2019-11-26 | 2020-04-28 | 金蝶软件（中国）有限公司 | Batch task processing method and device, computer equipment and storage medium |
| CN112988806A (en) * | 2019-12-17 | 2021-06-18 | 北京沃东天骏信息技术有限公司 | Data processing method and device |
| CN111813783A (en) * | 2020-07-27 | 2020-10-23 | 南方电网数字电网研究院有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN111813783B (en) * | 2020-07-27 | 2024-03-26 | 南方电网数字电网研究院有限公司 | Data processing method, device, computer equipment and storage medium |
| CN112000644A (en) * | 2020-08-21 | 2020-11-27 | 北京浪潮数据技术有限公司 | Data processing method and system, electronic equipment and storage medium |
| CN112000644B (en) * | 2020-08-21 | 2022-08-02 | 北京浪潮数据技术有限公司 | Data processing method and system, electronic equipment and storage medium |
| CN113269547A (en) * | 2021-05-31 | 2021-08-17 | 中国农业银行股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113778999A (en) * | 2021-09-29 | 2021-12-10 | 平安资产管理有限责任公司 | Data modularization processing method and device, computer equipment and storage medium |
| CN114238286B (en) * | 2022-02-28 | 2022-08-05 | 连连（杭州）信息技术有限公司 | Data warehouse data processing method and device, electronic equipment and storage medium |
| CN114238286A (en) * | 2022-02-28 | 2022-03-25 | 连连（杭州）信息技术有限公司 | Data warehouse data processing method and device, electronic equipment and storage medium |
| CN115952172A (en) * | 2023-03-08 | 2023-04-11 | 畅捷通信息技术股份有限公司 | Data matching method and device based on temporary table of database |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107665233B (en) | 2020-07-31 |
| WO2019019361A1 (en) | 2019-01-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107665233A (en) | Database data processing method, device, computer equipment and storage medium | |
| CA3051393C (en) | Method for providing recording and verification service for data received and transmitted by messenger service, and server using method | |
| US8813214B1 (en) | Method and system for providing secure peer-to-peer file transfers | |
| CN104796257B (en) | Flexible data authentication | |
| CN104866985B (en) | The recognition methods of express delivery odd numbers, apparatus and system | |
| CN104468531A (en) | Authorization method, device and system for sensitive data | |
| CN102833235A (en) | Identity authentication and management device and identity authentication and management method | |
| CN110377762A (en) | Information query method, device and computer equipment based on electronics folder | |
| CN112036125B (en) | Document management method and device and computer equipment | |
| CN110738323B (en) | Method and device for establishing machine learning model based on data sharing | |
| CN112286896B (en) | Log file processing method and device, computer equipment and storage medium | |
| CN111460394A (en) | Copyright file verification method and device and computer readable storage medium | |
| CN110798445A (en) | Public gateway interface testing method and device, computer equipment and storage medium | |
| CN105554136B (en) | Backup Data restoring method, apparatus and system | |
| CN104092653A (en) | Data processing method and system | |
| CN110019640A (en) | Confidential document inspection method and device | |
| CN111506551B (en) | Conference file extraction method, system and computer equipment | |
| CN106899564A (en) | A kind of login method and device | |
| CN110955758A (en) | Code detection method, code detection server and index server | |
| CN105184559B (en) | A kind of payment system and method | |
| CN113038192B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN108133026B (en) | Multi-data processing method, system and storage medium | |
| CN104991941A (en) | Information processing apparatus, information processing method, and computer program | |
| CN106209584A (en) | User profile processing method based on the Internet, client and server | |
| CN107862035A (en) | Network read method, device, Intelligent flat and the storage medium of minutes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20180530 Address after: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.) Applicant after: Shenzhen one ledger Intelligent Technology Co., Ltd. Address before: 200000 Xuhui District, Shanghai Kai Bin Road 166, 9, 10 level. Applicant before: Shanghai Financial Technologies Ltd |
|
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 518000 Room 201, building A, 1 front Bay Road, Shenzhen Qianhai cooperation zone, Shenzhen, Guangdong Applicant after: Shenzhen one ledger Intelligent Technology Co., Ltd. Address before: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.) Applicant before: Shenzhen one ledger Intelligent Technology Co., Ltd. |
|
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1250801 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |