CN104123288A - Method and device for inquiring data - Google Patents
Method and device for inquiring data Download PDFInfo
- Publication number
- CN104123288A CN104123288A CN201310146187.1A CN201310146187A CN104123288A CN 104123288 A CN104123288 A CN 104123288A CN 201310146187 A CN201310146187 A CN 201310146187A CN 104123288 A CN104123288 A CN 104123288A
- Authority
- CN
- China
- Prior art keywords
- inquiry
- inquiry clause
- clause
- data
- metadata
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
Abstract
The invention discloses a method and device for inquiring data. The method comprises the steps that an inquiry statement is decomposed into one or more inquiry clauses; traversal is carried out on the inquiry clauses, the inquiry clauses are matched with metadata, and the inquiry clauses are replaced with execution results stored in the metadata; otherwise, the inquiry clauses are executed, and the metadata are updated; the final execution result is used for main inquiry to obtain an inquiry result. The executed inquiry clauses and the execution results of the executed inquiry clauses are used as the metadata to be managed, the identical inquiry clauses in the current session or different sessions are directly replaced with the execution results stored in the metadata, the inquiry clauses are prevented from being executed repeatedly, and therefore the inquiry performance of the whole system is improved to a greater extent. Meanwhile, compared with optimization of the existing inquiry clause level, the inquiry optimization granularity is thinner, the method and device are more suitable for inquiring large data in data warehouse application, and therefore the inquiry performance of the whole system is further improved.
Description
Technical field
The application relates to data storage and analytical technology, espespecially a kind of in data warehouse applications, for magnanimity, read to write few data more and carry out data enquire method and device.
Background technology
In many applications such as scientific research, Computer Simulation, internet, applications, ecommerce, data volume increases at a terrific speed.The explosive increase of information explosion, the especially unstructured data of large data age, just whole data storage and analysis field in profound influence.
In order to meet emerging business demand, start the data processing method of abandoning tradition gradually, then attempt new pattern, the various types of data including unstructured data is conducted interviews, processed and analyzes.At data storage and analysis field, MapReduce is undoubtedly technology of new generation of greatest concern, utilize MapReduce programming framework, developer can develop across program processor distributed type assemblies or stand-alone computer, can parallel processing magnanimity unstructured data.Wherein, MapReduce is a kind of programming model, and for the concurrent operation of large-scale data, Map represents mapping, and Reduce represents abbreviation.
At large data age, traditional Structured Query Language (SQL) (SQL, Structure Query Language) can not meet all business demands, although and the application of MapReduc is very extensive, but, the DLL (dynamic link library) that MapReduc framework comes out is still more rudimentary, develop more consuming time, and code be difficult to multiplexing.Then, turn to the SQL that user can be write to be converted into corresponding MapReduce program, utilize MapReduce framework to carry out the program of these tasks so that the mass data in Hadoop distributed file system (HDFS) is processed.At present most widely used is the Hive of Facebook contribution.Hive based on MapReduce is having powerful advantage aspect extendability and fault-tolerance.Wherein, Hive is a Tool for Data Warehouse based on Hadoop, structurized data file can be mapped as to a database table, and complete SQL query function is provided, and SQL statement can be converted to MapReduce task and move.
In business data mining process, conventionally the mining task of carrying out every day all can be hundreds and thousands of, even reach up to ten thousand, so also just there is unavoidably the problem of double counting in different mining tasks, such as: slip-stick artist A need to filter out the data of a collection of condition W from table S1 and table S2, then carries out with table S3 the data that combination regeneration becomes slip-stick artist A to want.Meanwhile, slip-stick artist B also need to filter out identical data with the same terms W from table S1 and table S2, then with Table X combination, the data that regeneration slip-stick artist B wants.Between these two slip-stick artists, conventionally and do not know the demand that has each other part identical, so in production task, identical calculating can carry out twice, identical common query information is repeated accessing.Undoubtedly, reduced like this query performance of total system.
Summary of the invention
In order to solve the problems of the technologies described above, the application provides a kind of data enquire method and device, can avoid reruning of common query information, thereby improves the query performance of total system.
In order to reach the application's object, the application provides a kind of data enquire method, it is characterized in that, comprising: query statement is decomposed into one or more inquiry clauses;
Traversal queries clause, inquiry clause and meta data match, replaced with the execution result of preserving in metadata, otherwise, execution inquiry clause more new metadata;
The execution result finally obtaining is obtained to Query Result for main inquiry.
Before the method, also comprise: obtain query statement, the validity of revene lookup statement and correctness.
The method also comprises: described query statement is standardized and query optimization.
Described traversal queries clause specifically comprises:
The inquiry clause operator tree that described inquiry clause is formed is carried out postorder traversal, utilizes the triplet information of inquiry clause to mate with metadata,
If the triplet information of described inquiry clause exists in metadatabase, the triplet information of this inquiry clause and meta data match, replaces with corresponding execution result by described inquiry clause; The out-of-service time of this metadata of resetting;
If the triplet information of described inquiry clause does not exist in metadatabase, the triplet information of this inquiry clause is not mated with metadata, carries out described inquiry clause, persistence execution result, and in metadata, add corresponding triplet information; The out-of-service time of this metadata is set, and this inquiry clause is in the level of inquiry clause operator tree.
When the triplet information of described inquiry clause and meta data match, also comprise: record corresponding described execution result is added to one by access times.
Described, when query statement is decomposed into multiple queries clause, the method also comprises: optimize described inquiry clause.
The method also comprises: set in advance inquiry clause optimisation strategy;
Described Optimizing Queries clause comprises: be optimized decomposing the inquiry clause obtaining according to described inquiry clause optimisation strategy, reject unnecessary inquiry clause.
Described inquiry clause optimisation strategy comprises:
In the time that the input set of inquiry clause collects consistent with output, delete described inquiry clause; And/or,
Difference when between inquiry clause is output row, and the output of the first inquiry clause row are while being the subset of output row of the second inquiry clause, carry out the second inquiry clause, and the Output rusults collection of the second inquiry clause is incorporated to the first inquiry clause; And/or,
Difference when between inquiry clause is output row, and the output of the first inquiry clause row and the output of the second inquiry clause are while being listed as each other supplementary set, merge the first inquiry clause and the second inquiry clause; And/or,
When the input set of inquiry clause identical, when filtercondition is different, extract described input set identical, the public part of the different each inquiry clause of filtercondition forms new inquiry clause, carry out this new inquiry clause, and it is identical that its Output rusults collection is incorporated to respectively to described input set, each inquiry clause that filtercondition is different; And/or,
The inquiry clause of described existence or relation is split as to two or more inquiry clauses.
Described metadata store is in metadatabase; The method also comprises described metadatabase is managed:
The out-of-service time corresponding when described metadata expires, or the User-Defined Functions UDF that generates described inquiry clause result data is expired, or generate that in the input data set of described inquiry clause result data, data change occurs for one of them, or the output collection of described inquiry clause is while being reclaimed by force by user, deletes the inquiry clause losing efficacy;
And/or,
According to scanning metadata the interval time of original setting, delete the inquiry clause information losing efficacy.
The inquiry clause that described deletion was lost efficacy comprises: the inquiry clause of deleting all execution results that depend on described inquiry clause;
If the execution result that described inquiry clause is corresponding is intermediate result, from distributed file storage system, delete this intermediate result simultaneously.
Described metadata comprises: out-of-service time, the inquiry clause of inquiry clause corresponding to triplet information, the triplet information corresponding with each inquiry clause are identified at level, the Query Result of query node tree by access times, inquiry clause.
Described tlv triple comprises input set, output collection, and completes the operational order collection that is input to output conversion.
The application provides a kind of data query device, at least comprises metadatabase, decomposing module, processing module and output module, wherein,
Metadatabase, for storing metadata, comprises the triplet information corresponding with each inquiry clause, and out-of-service time of inquiry clause corresponding to triplet information, inquiry clause be level, the Query Result identification information in query node tree by access times, inquiry clause;
Decomposing module, for being decomposed into query statement one or more inquiry clauses;
Processing module, for traversal queries clause, inquiry clause and meta data match, replaced with the execution result of preserving in metadata; Otherwise, execution inquiry clause more new metadata;
Output module, for obtaining Query Result by the execution result finally obtaining for main inquiry.
This device also comprises: acquisition module, and for obtaining query statement, the validity of revene lookup statement and correctness.
Described acquisition module, also for standardizing and query optimization to described query statement.
Described processing module, mate with metadata specifically for the triplet information of utilizing inquiry clause:
In the time that the triplet information of inquiry clause exists in metadatabase, inquiry clause is replaced with to execution result; The out-of-service time of this metadata of resetting;
In the time that the triplet information of inquiry clause does not exist in metadatabase, carry out current inquiry clause, persistence execution result, and in metadata, add corresponding triplet information; The out-of-service time of this metadata is set, and this inquiry clause is in the level of inquiry clause operator tree.
Described processing module also for, in the time of the triplet information of described inquiry clause and meta data match, record corresponding described execution result is added to one by access times.
Described output module, specifically for carrying out the execution result of subquery in described inquiry clause iteration and for main inquiry, obtain Query Result.
In described decomposing module, set in advance inquiry clause optimisation strategy;
Described decomposing module, also for being optimized decomposing the inquiry clause obtaining according to optimisation strategy.
Described processing module, also for metadatabase is managed, remove the inquiry clause losing efficacy:
The out-of-service time corresponding when described metadata expires, or the UDF that generates described inquiry clause result data is expired, or generate that in the input data set of described inquiry clause result data, data change occurs for one of them, or the output collection of described inquiry clause is while being reclaimed by force by user, deletes the inquiry clause losing efficacy;
And/or,
According to scanning metadata the interval time of original setting, delete the inquiry clause losing efficacy;
Wherein, the inquiry clause that described deletion was lost efficacy is the inquiry clause of deleting all execution results that depend on described inquiry clause; If the execution result that described inquiry clause is corresponding is intermediate result, from distributed file storage system, delete this intermediate result simultaneously.
Compared with prior art, the application comprises query statement is decomposed into one or more inquiry clauses; Traversal queries clause, inquiry clause and meta data match, replaced with the execution result of preserving in metadata, otherwise, execution inquiry clause more new metadata; The execution result finally obtaining is obtained to Query Result for main inquiry.The application manages the inquiry clause of having carried out and execution result thereof as metadata, for same queries clause in current sessions or different sessions, directly inquiry clause is replaced with to the execution result of preserving in metadata, avoid repeating of inquiry clause, thereby improved to a greater extent the query performance of total system.Simultaneously, the application's query statement is decomposed into multiple queries clause, compared with the optimal way of existing query statement aspect, the more refinement of query optimization granularity of the application's data enquire method, be more suitable for the large data query in data warehouse applications, thereby further improved the query performance of total system.
The application's further feature and advantage will be set forth in the following description, and, partly from instructions, become apparent, or understand by implementing the application.The application's object and other advantages can be realized and be obtained by specifically noted structure in instructions, claims and accompanying drawing.
Brief description of the drawings
Accompanying drawing is used to provide the further understanding to present techniques scheme, and forms a part for instructions, is used from the application's embodiment mono-technical scheme of explaining the application, does not form the restriction to present techniques scheme.
Fig. 1 is the process flow diagram of the application's data enquire method;
Fig. 2 is the composition structural representation of the application's data query device.
Embodiment
For making the application's object, technical scheme and advantage clearer, hereinafter in connection with accompanying drawing, the application's embodiment is elaborated.It should be noted that, in the situation that not conflicting, the combination in any mutually of the feature in embodiment and embodiment in the application.
In typical configuration of the application, computing equipment comprises one or more processors (CPU), input/output interface, network interface and internal memory.
Internal memory may comprise the volatile memory in computer-readable medium, and the forms such as random access memory (RAM) and/or Nonvolatile memory, as ROM (read-only memory) (ROM) or flash memory (flashRAM).Internal memory is the example of computer-readable medium.
Computer-readable medium comprises that permanent and impermanency, removable and non-removable media can realize information storage by any method or technology.Information can be module or other data of computer-readable instruction, data structure, program.The example of the storage medium of computing machine comprises, but be not limited to phase transition internal memory (PRAM), static RAM (SRAM), dynamic RAM (DRAM), the random access memory (RAM) of other types, ROM (read-only memory) (ROM), Electrically Erasable Read Only Memory (EEPROM), fast flash memory bank or other memory techniques, read-only optical disc ROM (read-only memory) (CD-ROM), digital versatile disc (DVD) or other optical memory, magnetic magnetic tape cassette, the storage of tape magnetic rigid disk or other magnetic storage apparatus or any other non-transmission medium, can be used for the information that storage can be accessed by computing equipment.According to defining herein, computer-readable medium does not comprise non-temporary computer readable media (transitory media), as data-signal and the carrier wave of modulation.
Can in the computer system such as one group of computer executable instructions, carry out in the step shown in the process flow diagram of accompanying drawing.And, although there is shown logical order in flow process, in some cases, can carry out shown or described step with the order being different from herein.
Inquiry clause herein refers to, in the time that an inquiry is the condition of another inquiry, claims that this inquiry is subquery, and wherein, outer query is referred to as main inquiry.Subquery is better than main inquiry first to be carried out, and the execution result of subquery is for main inquiry.Subquery and main inquiry are referred to as to inquiry clause herein.An inquiry clause has definite input set Pin and output collection Pout, and completes the operational order collection Φ that is input to output conversion
operatordeng three part compositions, and referred to as inquiry clause tlv triple, be designated as τ={ Pin, Pout, Φ
operator, for unique identification inquiry clause.The input source of an inquiry clause, or be another one inquiry clause, or be exactly from table Table or table subregion Partition, be designated as Pin={dom| τ, Table, Partition}.
Fig. 1 is the process flow diagram of the application's data enquire method, as shown in Figure 1, mainly comprises:
Step 100: query statement is decomposed into one or more inquiry clauses.
In this step, before inquiry, first query statement is decomposed into one or more inquiry clauses, and the inquiry clause that utilizes decomposition to obtain is constructed definite inquiry clause operator tree.Each inquiry clause is made up of with operation operator tree the input set of determining, output collection, and wherein, operation operator is set by one group of operational order collection Φ
operatorform; Common operation Operator instruction is as shown in table 1:
| Operation Operator instruction | Implication |
| TableScanOperator | Scan table data |
| SelectOperator | Select output row |
| FilterOperator | Filter input data |
| FileSinkOperator | Set up result data, export file to |
| ReduceSinkOperator | Create right to<Key, the Value>that send to Reducer end |
| JoinOperator | Multi-source data JOIN |
| GroupByOperator | GroupBy statement |
| LimitOperator | Limit statement |
| UnionOperator | Union statement |
| … | ? |
Table 1
In this step, the specific implementation of the method that decomposition query statement is inquiry clause belongs to those skilled in the art's conventional techniques means, and its specific implementation is also not used in the protection domain that limits the application.This step is emphasized, before inquiry, query statement is decomposed into one or more inquiry clauses.Like this, compared with the existing inquiry mode that is positioned at query statement layer, more refinement of the granularity of query of the application's data enquire method, is more suitable for the large data query in data warehouse applications.
Before step 100, also comprise: obtain query statement, the validity of revene lookup statement and correctness, also further comprise that the query statement to obtaining standardizes and query optimization.Wherein, standardization is exactly normalization with the object of query optimization, according to format rule and the optimisation strategy of known set, the query statement of realizing identical function is expressed by a kind of mode of common the best.Conventional optimisation strategy has that row are reduced, subregion is reduced, push away under predicate etc.Specific implementation belongs to those skilled in the art's conventional techniques means, repeats no more here, is also not used in the protection domain that limits the application.
Step 101: traversal queries clause, inquiry clause and meta data match, replaced with the execution result of preserving in metadata; Otherwise, execution inquiry clause more new metadata.
Wherein, metadata store is in the metadatabase setting in advance, metadata comprises the triplet information corresponding with each inquiry clause, also comprises the out-of-service time of inquiry clause corresponding to triplet information, the information such as level, Query Result mark that inquiry clause is set at query node by access times, inquiry clause.Wherein, the out-of-service time, for the out-of-service time then, remove metadata, reclaim the shared resource of long-term no inquiry clause; By access times, for the execution priority of the angle Optimizing Queries clause from entire system, be can be used as by access times and query node number (level) weight that priority is calculated, such as inquiry times is more, priority is more high, and the higher inquiry clause of priority is enjoyed more computational resource and temporal right of priority; The level of inquiry clause is that the maximal value of each inquiry clause level in input set adds 1, if input set is table or table subregion, inquiring about level is 0; Query Result mark can be understood as table name and claims, if inquiry clause root node and user have specified table name to claim, uses this table name to claim; Otherwise the table name of getting a temporary table claims, user is not needed transparent, whether the existence of not perception of user temporary table to be.According to this table name claim can locating query clause result set data.
This step specifically comprises: inquiry clause operator tree definite in step 100 traveled through as postorder traversal, utilizes the triplet information of inquiry clause to mate with metadata,
If the triplet information of current inquiry clause exists in metadatabase, the triplet information of this inquiry clause and meta data match, now, show to carry out before this inquiry clause, and the data of execution result are effective, so, directly inquiry clause is replaced with to the table relevant information that the execution result of preserving in metadata is corresponding record; The out-of-service time (if session parameter setting does not exist, can Use Defaults as 3 days) of resetting this metadata is set according to already-existing session parameter, further, this record can be added to one by access times;
If the triplet information of current inquiry clause does not exist in metadatabase, the triplet information of this inquiry clause is not mated with metadata, now, show not carry out before this inquiry clause, so, carry out current inquiry clause, persistence execution result is kept in distributed file storage system by execution result, and in metadata, adds corresponding triplet information; The out-of-service time (if session parameter setting does not exist, can Use Defaults as 3 days) of this metadata is set according to already-existing session parameter, and this inquiry clause is in the degree of depth (level) of inquiry clause operator tree.It should be noted that, if inquiry clause is subquery, output data set title identifies taking TMP_ as prefix, and concrete title can generate at random, ensures that title is that the overall situation is unique; If inquiry clause is main inquiry, if there is definition output table name to claim, use this title, the juxtaposition out-of-service time is never lost efficacy (table that respective user creates, only just invalid in the time that user deletes); Otherwise method to set up is consistent with subquery.
Step 102: the execution result finally obtaining is obtained to Query Result for main inquiry.
Traveling through after all inquiry clauses, the execution result of subquery in inquiry clause is carried out iteration by this step, is used for main inquiry by the execution result of inquiry clause operator tree root node inquiry clause, finally obtains Query Result.
To sum up, the application specifically comprises: query statement is decomposed into one or more inquiry clauses, output inquiry clause operator tree; Traversal queries clause operator tree, inquiry clause and meta data match, replace with by the inquiry clause of coupling the execution result of preserving in metadata; Carry out unmatched inquiry clause and execution result information is updated to metadata; The execution result of inquiry clause operator tree root node inquiry clause is obtained to Query Result for main inquiry.
Can see from the matching process of the application's step 101, the application manages the inquiry clause of having carried out and execution result thereof as metadata, for inquiry clause identical in current sessions or different sessions, directly inquiry clause is replaced with to the execution result of preserving in metadatabase, avoid repeating of inquiry clause, thereby improved to a greater extent the query performance of total system.Simultaneously, the application's method was decomposed into multiple queries clause by query statement before inquiry, compared with the optimization of existing query statement aspect, the more refinement of query optimization granularity of the application's data enquire method, be more suitable for the large data query in data warehouse applications, thereby further improved the query performance of total system.
Further, in the application's step 100, in the time that query statement is decomposed into multiple queries clause, can also comprise: Optimizing Queries clause.By the further optimization to inquiry clause, weed out unnecessary inquiry clause, reduce unnecessary inquiry, thereby further improved the query performance of total system.
Optimizing Queries clause specifically comprise: set in advance inquiry clause optimisation strategy, be optimized decomposing the inquiry clause obtaining according to optimisation strategy, reject unnecessary inquiry clause.Wherein, optimisation strategy can include, but are not limited to following several:
1) input set of inquiry clause and output collection consistent (Pin=Pout), it is the situation that inquiry clause is directly equal to input set, the result data collection of this inquiry clause itself does not have any help to optimizing, do not need synchronously to add in the metadatabase of inquiry clause, delete this inquiry clause;
2) difference between inquiry clause is only output row, and the output of inquiry clause row are subsets of another one inquiry clause output row, such as:
Inquiry clause is a): SELECT name FROM employee WHERE title=' TeamLeader ';
Inquiry clause is b): SELECT*FROM employee WHERE title=' TeamLeader ';
Wherein, inquiry clause a completes the inquiry to employee table name field, takes out the value that meets title field and equal the record of TeamLeader; Inquiry clause b completes the inquiry of employee being shown to all fields, takes out the value that meets title field and equal the record of TeamLeader.
That is to say, inquiry clause data result collection a) is inquiry clause subset b); Now, the method for optimization is: first carry out inquiry clause b), suppose that Output rusults collection is: TMP_20130318_012345, so, inquiry clause can be optimized for a): SELECT name FROM TMP_20130318_012345.This situation is equivalent to inquiry clause Output rusults collection b) to be incorporated to inquiry clause a), thereby has also deleted inquiry clause b) in a) having simplified inquiry clause.
3) difference between inquiry clause is only output row, and the output row of an inquiry clause are listed as and have each other supplementary set with another one output, such as:
Inquiry clause is a): SELECT name FROM employee WHERE title=' TeamLeader ';
Query statement is b): SELECT addr FROM employee WHERE title=' TeamLeader ';
Wherein, inquiry clause a) completes the inquiry to employee table name field, and the value of taking out all title of meeting fields equals the record of TeamLeader; Inquiry clause b) completes the inquiry to employee table addr field, and the value of taking out all title of meeting fields equals the record of TeamLeader.
Now, the method for optimization is: these two inquiry clauses are merged, the inquiry clause after merging be inquiry clause c): SELECT name, addr FROM employee WHERE title=' 11 '.
4) two inquiry clause input sets are identical, filtercondition difference, such as:
Inquiry clause is a): SELECT name FROM employee WHERE title=' TeamLeader ';
Inquiry clause is b): SELECT addr FROM employee WHERE title=' TeamLeader ' and name=' Smich ';
Wherein, inquiry clause a) completes the inquiry to employee table name field, and the value of taking out all title of meeting fields equals the record of TeamLeader; Inquiry clause b) completes the inquiry to employee table addr field, and the value that the value of taking out all title of meeting fields equals TeamLeader and name field equals the record of Smich;
Now, the method for optimization is: extract the public part composition inquiry clause of inquiry clause c), such as:
Inquiry clause is c): SELECT name, addr FROM employee WHERE title=' TeamLeader ';
Wherein, inquiry clause c) completes employee table name, the inquiry of addr field, and the value of taking out all title of meeting fields equals the record of TeamLeader.
Suppose that inquiry clause Output rusults collection c) is: TMP_20130318_222222, so, can simplify inquiry clause according to the Output rusults collection obtaining, particularly:
Inquiry clause a) can be optimized for: SELECT name FROM TMP_20130318_222222;
Inquiry clause b) is optimized for: SELECT addr FROM TMP_20130318_222222 WHERE name=' Smich ';
5) inquiry clause of existence or relation is split as to more than two or two inquiry clause, can improves like this common query clause's hit rate.Such as for
Inquiry clause is a): SELECT*FROM employee WHERE title=' TeamLeader ' or title=' Manager ';
Wherein, inquiry clause a) completes the inquiry of employee being shown to all fields, and the value of taking out all title of meeting fields equals the record of TeamLeader or Manager.
Can be split as:
Inquiry clause is b): SELECT*FROM employee WHERE title=' TeamLeader ';
Inquiry clause is c): SELECT*FROM employee WHERE title=' Manager '
Wherein, inquiry clause b) completes the inquiry of employee being shown to all fields, and the value of taking out all title of meeting fields equals the record of TeamLeader; Inquiry clause c) completes the inquiry of employee being shown to all fields, and the value of taking out all title of meeting fields equals the record of Manager.
The application's method also comprises: metadatabase is managed, i.e. certain interval of time scanning metadata, removes expired inquiry clause.Wherein, can be set in advance as interval time day or hour;
Except normal condition is as expire cause expired and removing corresponding inquiry clause of out-of-service time, also have several situations also can cause the result data of inquiry clause expired, such as: the User-Defined Functions (UDF) that generates certain inquiry clause result data is expired; And for example: generate that in the input data set of certain inquiry clause result data, data change has occurred for one of them; For another example: the output collection of certain inquiry clause is reclaimed by force by user.When inquiry clause lost efficacy, mean that the inquiry clause of all execution results that depend on this inquiry clause lost efficacy simultaneously; If the execution result that inquiry clause is corresponding is intermediate result, from distributed file storage system, delete this intermediate result simultaneously.
The application also provides a kind of data query device, as shown in Figure 2, at least comprises metadatabase, decomposing module, processing module and output module, wherein,
Metadatabase, for storing metadata, comprise the triplet information corresponding with each inquiry clause, also comprise the out-of-service time of inquiry clause corresponding to triplet information, the information such as level, Query Result mark that inquiry clause is set at query node by access times, inquiry clause.
Decomposing module, for being decomposed into query statement one or more inquiry clauses;
Processing module, for traversal queries clause, inquiry clause and meta data match, replaced with the execution result of preserving in metadata; Otherwise, execution inquiry clause more new metadata;
Output module, for obtaining Query Result by the execution result finally obtaining for main inquiry.
The application's data query device also comprises: acquisition module, and for obtaining query statement, the validity of revene lookup statement and correctness; Be further used for, the query statement obtaining is standardized and query optimization.
Wherein,
Processing module, mate with metadata specifically for the triplet information of utilizing inquiry clause:
In the time that the triplet information of inquiry clause exists in metadatabase, inquiry clause is replaced with to the table relevant information that execution result is corresponding record; The out-of-service time of this metadata of resetting, further, this record is added to one by access times;
In the time that the triplet information of inquiry clause does not exist in metadatabase, carry out current inquiry clause, persistence execution result, and in metadata, add corresponding triplet information; The out-of-service time of this metadata is set, and this inquiry clause is in the degree of depth (level) of inquiry clause operator tree.
Output module, specifically for carrying out the execution result of subquery in inquiry clause iteration and for main inquiry, finally obtaining Query Result.
Outside, in decomposing module, also set in advance inquiry clause optimisation strategy, decomposing module also for, be optimized decomposing the inquiry clause that obtains according to optimisation strategy.
Processing module, also for metadatabase is managed, remove the inquiry clause losing efficacy:
The out-of-service time corresponding when described metadata expires, or the UDF that generates described inquiry clause result data is expired, or generate that in the input data set of described inquiry clause result data, data change occurs for one of them, or the output collection of described inquiry clause is while being reclaimed by force by user, deletes the inquiry clause losing efficacy;
And/or,
According to scanning metadata the interval time of original setting, delete the inquiry clause losing efficacy;
Wherein, the inquiry clause that described deletion was lost efficacy is the inquiry clause of deleting all execution results that depend on described inquiry clause; If the execution result that described inquiry clause is corresponding is intermediate result, from distributed file storage system, delete this intermediate result simultaneously.
From the application's device, the application manages the inquiry clause of having carried out and execution result thereof as metadata, for inquiry clause identical in current sessions or different sessions, directly inquiry clause is replaced with to the execution result of preserving in metadatabase, avoid repeating of inquiry clause, thereby improved to a greater extent the query performance of total system.Simultaneously, the application's method was decomposed into multiple queries clause by query statement before inquiry, compared with the optimization of existing query statement aspect, the more refinement of query optimization granularity of the application's data enquire method, be more suitable for the large data query in data warehouse applications, thereby further improved the query performance of total system.
In addition, the application, by the further optimization to inquiry clause, weeds out unnecessary inquiry clause, has reduced unnecessary inquiry, thereby has further improved the query performance of total system.
It is apparent to those skilled in the art that each ingredient of the device that above-mentioned the embodiment of the present application provides, and each step in method, they can concentrate on single calculation element, or are distributed on the network that multiple calculation elements form.Alternatively, they can be realized with the executable program code of calculation element.Thereby, they can be stored in memory storage and be carried out by calculation element, or they are made into respectively to each integrated circuit modules, or the multiple modules in them or step are made into single integrated circuit module realize.Like this, the application is not restricted to any specific hardware and software combination.
Although the disclosed embodiment of the application as above, the embodiment that described content only adopts for ease of understanding the application, not in order to limit the application.Those of skill in the art under any the application; do not departing under the prerequisite of the disclosed spirit and scope of the application; can in the form of implementing and details, carry out any amendment and variation; but the application's scope of patent protection, still must be as the criterion with the scope that appending claims was defined.
Claims (20)
1. a data enquire method, is characterized in that, comprising:
Query statement is decomposed into one or more inquiry clauses;
Traversal queries clause, inquiry clause and meta data match, replaced with the execution result of preserving in metadata, otherwise, execution inquiry clause more new metadata;
The execution result finally obtaining is obtained to Query Result for main inquiry.
2. data enquire method according to claim 1, is characterized in that, also comprises: obtain query statement, the validity of revene lookup statement and correctness before the method.
3. data enquire method according to claim 2, is characterized in that, the method also comprises: described query statement is standardized and query optimization.
4. according to the data enquire method described in claim 1～3 any one, it is characterized in that, described traversal queries clause specifically comprises:
The inquiry clause operator tree that described inquiry clause is formed is carried out postorder traversal, utilizes the triplet information of inquiry clause to mate with metadata,
If the triplet information of described inquiry clause exists in metadatabase, the triplet information of this inquiry clause and meta data match, replaces with corresponding execution result by described inquiry clause; The out-of-service time of this metadata of resetting;
If the triplet information of described inquiry clause does not exist in metadatabase, the triplet information of this inquiry clause is not mated with metadata, carries out described inquiry clause, persistence execution result, and in metadata, add corresponding triplet information; The out-of-service time of this metadata is set, and this inquiry clause is in the level of inquiry clause operator tree.
5. data enquire method according to claim 4, is characterized in that, when the triplet information of described inquiry clause and meta data match, also comprises: record corresponding described execution result is added to one by access times.
6. according to the data enquire method described in claim 1～3 any one, it is characterized in that, described, when query statement is decomposed into multiple queries clause, the method also comprises: optimize described inquiry clause.
7. data enquire method according to claim 6, is characterized in that, the method also comprises: set in advance inquiry clause optimisation strategy;
Described Optimizing Queries clause comprises: be optimized decomposing the inquiry clause obtaining according to described inquiry clause optimisation strategy, reject unnecessary inquiry clause.
8. data enquire method according to claim 7, is characterized in that, described inquiry clause optimisation strategy comprises:
In the time that the input set of inquiry clause collects consistent with output, delete described inquiry clause; And/or,
Difference when between inquiry clause is output row, and the output of the first inquiry clause row are while being the subset of output row of the second inquiry clause, carry out the second inquiry clause, and the Output rusults collection of the second inquiry clause is incorporated to the first inquiry clause; And/or,
Difference when between inquiry clause is output row, and the output of the first inquiry clause row and the output of the second inquiry clause are while being listed as each other supplementary set, merge the first inquiry clause and the second inquiry clause; And/or,
When the input set of inquiry clause identical, when filtercondition is different, extract described input set identical, the public part of the different each inquiry clause of filtercondition forms new inquiry clause, carry out this new inquiry clause, and it is identical that its Output rusults collection is incorporated to respectively to described input set, each inquiry clause that filtercondition is different; And/or,
The inquiry clause of described existence or relation is split as to two or more inquiry clauses.
9. according to the data enquire method described in claim 1～3 any one, it is characterized in that, described metadata store is in metadatabase; The method also comprises described metadatabase is managed:
The out-of-service time corresponding when described metadata expires, or the User-Defined Functions UDF that generates described inquiry clause result data is expired, or generate that in the input data set of described inquiry clause result data, data change occurs for one of them, or the output collection of described inquiry clause is while being reclaimed by force by user, deletes the inquiry clause losing efficacy;
And/or,
According to scanning metadata the interval time of original setting, delete the inquiry clause information losing efficacy.
10. data enquire method according to claim 9, is characterized in that, the inquiry clause that described deletion was lost efficacy comprises: the inquiry clause of deleting all execution results that depend on described inquiry clause;
If the execution result that described inquiry clause is corresponding is intermediate result, from distributed file storage system, delete this intermediate result simultaneously.
11. require the data enquire method described in 10 according to claim, it is characterized in that, described metadata comprises: out-of-service time, the inquiry clause of inquiry clause corresponding to triplet information, the triplet information corresponding with each inquiry clause are identified at level, the Query Result of query node tree by access times, inquiry clause.
12. data enquire methods according to claim 11, is characterized in that, described tlv triple comprises input set, output collection, and completes the operational order collection that is input to output conversion.
13. 1 kinds of data query devices, is characterized in that, at least comprise metadatabase, decomposing module, processing module and output module, wherein,
Metadatabase, for storing metadata, comprises the triplet information corresponding with each inquiry clause, and out-of-service time of inquiry clause corresponding to triplet information, inquiry clause be level, the Query Result identification information in query node tree by access times, inquiry clause;
Decomposing module, for being decomposed into query statement one or more inquiry clauses;
Processing module, for traversal queries clause, inquiry clause and meta data match, replaced with the execution result of preserving in metadata; Otherwise, execution inquiry clause more new metadata;
Output module, for obtaining Query Result by the execution result finally obtaining for main inquiry.
14. data query devices according to claim 13, is characterized in that, this device also comprises: acquisition module, and for obtaining query statement, the validity of revene lookup statement and correctness.
15. data query devices according to claim 14, is characterized in that, described acquisition module, also for standardizing and query optimization to described query statement.
16. according to the data query device described in claim 13～15 any one, it is characterized in that described processing module is mated with metadata specifically for the triplet information of utilizing inquiry clause:
In the time that the triplet information of inquiry clause exists in metadatabase, inquiry clause is replaced with to execution result; The out-of-service time of this metadata of resetting;
In the time that the triplet information of inquiry clause does not exist in metadatabase, carry out current inquiry clause, persistence execution result, and in metadata, add corresponding triplet information; The out-of-service time of this metadata is set, and this inquiry clause is in the level of inquiry clause operator tree.
17. data query devices according to claim 16, is characterized in that, described processing module also for, in the time of the triplet information of described inquiry clause and meta data match, record corresponding described execution result is added to one by access times.
18. according to the data query device described in claim 13～15 any one, it is characterized in that, described output module, specifically for carrying out the execution result of subquery in described inquiry clause iteration and for main inquiry, obtain Query Result.
19. according to the data query device described in claim 13～15 any one, it is characterized in that, has set in advance inquiry clause optimisation strategy in described decomposing module;
Described decomposing module, also for being optimized decomposing the inquiry clause obtaining according to optimisation strategy.
20. according to the data query device described in claim 13～15 any one, it is characterized in that described processing module, also for metadatabase is managed, is removed the inquiry clause losing efficacy:
The out-of-service time corresponding when described metadata expires, or the UDF that generates described inquiry clause result data is expired, or generate that in the input data set of described inquiry clause result data, data change occurs for one of them, or the output collection of described inquiry clause is while being reclaimed by force by user, deletes the inquiry clause losing efficacy;
And/or,
According to scanning metadata the interval time of original setting, delete the inquiry clause losing efficacy;
Wherein, the inquiry clause that described deletion was lost efficacy is the inquiry clause of deleting all execution results that depend on described inquiry clause; If the execution result that described inquiry clause is corresponding is intermediate result, from distributed file storage system, delete this intermediate result simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310146187.1A CN104123288B (en) | 2013-04-24 | 2013-04-24 | A kind of data query method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310146187.1A CN104123288B (en) | 2013-04-24 | 2013-04-24 | A kind of data query method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104123288A true CN104123288A (en) | 2014-10-29 |
| CN104123288B CN104123288B (en) | 2018-06-01 |
Family
ID=51768701
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310146187.1A Active CN104123288B (en) | 2013-04-24 | 2013-04-24 | A kind of data query method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104123288B (en) |
Cited By (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104462317A (en) * | 2014-12-01 | 2015-03-25 | 苏州朗米尔照明科技有限公司 | Data screening method based on tag |
| CN104866608A (en) * | 2015-06-05 | 2015-08-26 | 中国人民大学 | Query optimization method based on join index in data warehouse |
| CN105930407A (en) * | 2016-04-18 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Cross-database associated query method and system for distributed database |
| CN106325832A (en) * | 2015-06-18 | 2017-01-11 | 阿里巴巴集团控股有限公司 | Data generation method and device |
| CN106411826A (en) * | 2015-08-03 | 2017-02-15 | 阿里巴巴集团控股有限公司 | Data access method and equipment thereof |
| CN106598963A (en) * | 2015-10-14 | 2017-04-26 | 五八同城信息技术有限公司 | Method and device for optimizing query statement |
| CN106776639A (en) * | 2015-11-24 | 2017-05-31 | 腾讯科技（深圳）有限公司 | Data processing method and data processing equipment based on SQL |
| CN106802927A (en) * | 2016-12-22 | 2017-06-06 | 上海大汉三通数据通信有限公司 | A kind of date storage method and querying method |
| CN107239710A (en) * | 2016-03-29 | 2017-10-10 | 北京明略软件系统有限公司 | A kind of data base authority method and system |
| CN107315790A (en) * | 2017-06-14 | 2017-11-03 | 腾讯科技（深圳）有限公司 | A kind of optimization method and device of irrelevant subquery |
| WO2018045610A1 (en) * | 2016-09-06 | 2018-03-15 | 北京百度网讯科技有限公司 | Method and device for executing distributed computing task |
| CN108089915A (en) * | 2016-11-22 | 2018-05-29 | 北京京东尚科信息技术有限公司 | The method and system of business controlization processing based on message queue |
| CN108268589A (en) * | 2017-12-05 | 2018-07-10 | 北京百度网讯科技有限公司 | Aggregate query method, apparatus, computer equipment and the readable medium of time series data |
| CN108388606A (en) * | 2018-02-06 | 2018-08-10 | 福建星瑞格软件有限公司 | A kind of method and computer equipment verifying library literary name section name in Sql sentences |
| CN108431805A (en) * | 2015-10-23 | 2018-08-21 | 甲骨文国际公司 | It is parallel to execute the inquiry with recursion clause |
| WO2018218504A1 (en) * | 2017-05-31 | 2018-12-06 | 华为技术有限公司 | Method and device for data query |
| CN109308300A (en) * | 2018-09-27 | 2019-02-05 | 上海达梦数据库有限公司 | A kind of processing method of logical operation, device, conversion plug-in unit and storage medium |
| CN109508339A (en) * | 2018-11-12 | 2019-03-22 | 上海达梦数据库有限公司 | A kind of data query method, apparatus, terminal device and storage medium |
| CN109947804A (en) * | 2019-03-20 | 2019-06-28 | 上海达梦数据库有限公司 | Optimization method, device, server and the storage medium of data acquisition system inquiry |
| CN110008244A (en) * | 2019-03-29 | 2019-07-12 | 国家计算机网络与信息安全管理中心 | A kind of data query method and data query device |
| CN110019394A (en) * | 2017-11-28 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Data query method and apparatus |
| CN110096489A (en) * | 2019-04-30 | 2019-08-06 | 阿里巴巴集团控股有限公司 | A kind of data query method, system, device and electronic equipment |
| CN110109946A (en) * | 2019-04-23 | 2019-08-09 | 北京小米移动软件有限公司 | Problem querying method and device |
| CN110297710A (en) * | 2018-03-23 | 2019-10-01 | 畅想科技有限公司 | Public precedence information for more resource arbitrations |
| CN110727672A (en) * | 2018-06-28 | 2020-01-24 | 北京京东尚科信息技术有限公司 | Data mapping relation query method and device, electronic equipment and readable medium |
| CN110968615A (en) * | 2018-09-30 | 2020-04-07 | 北京国双科技有限公司 | Data query method and device |
| CN111125155A (en) * | 2018-10-31 | 2020-05-08 | 北京国双科技有限公司 | Data query method and device based on access path, storage medium and processor |
| CN111159218A (en) * | 2019-12-31 | 2020-05-15 | 中科曙光国际信息产业有限公司 | Data processing method and device and readable storage medium |
| CN111488326A (en) * | 2019-01-25 | 2020-08-04 | 阿里巴巴集团控股有限公司 | MapReduce job execution method, node device and storage medium |
| CN111597214A (en) * | 2019-02-21 | 2020-08-28 | 北京京东尚科信息技术有限公司 | Method, apparatus, computer system and medium for data query |
| CN111694943A (en) * | 2020-05-29 | 2020-09-22 | 刘韩 | Accurate information query method and system |
| CN111949684A (en) * | 2020-08-06 | 2020-11-17 | 上海达梦数据库有限公司 | SQL statement execution method, device, equipment and storage medium |
| CN113010539A (en) * | 2019-12-20 | 2021-06-22 | 北京奇艺世纪科技有限公司 | Data processing method and device |
| WO2022127417A1 (en) * | 2020-12-16 | 2022-06-23 | 中兴通讯股份有限公司 | Data query method, and electronic device and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030046073A1 (en) * | 2001-08-24 | 2003-03-06 | International Business Machines Corporation | Word predicting method, voice recognition method, and voice recognition apparatus and program using the same methods |
| KR100413784B1 (en) * | 1997-04-29 | 2004-02-14 | 삼성전자주식회사 | Device for translating english-korean having clause unit separator and method for the same |
| US20060122993A1 (en) * | 2004-12-06 | 2006-06-08 | International Business Machines Corporation | Abstract query plan |
| CN101075255A (en) * | 2007-06-08 | 2007-11-21 | 北京神舟航天软件技术有限公司 | Method for removing SQL query statement constant condition |
| CN102045652A (en) * | 2009-10-21 | 2011-05-04 | 深圳市彩讯科技有限公司 | Garbage short message interception method based on characteristic similarity |
| CN102073719A (en) * | 2011-01-10 | 2011-05-25 | 复旦大学 | Interval coding-based geography markup language (GML) document index method |
| CN102682047A (en) * | 2011-10-18 | 2012-09-19 | 国网电力科学研究院 | Mixed structured query language (SQL) injection protection method |
| CN102789475A (en) * | 2012-05-08 | 2012-11-21 | 华为技术有限公司 | Data synchronization method and device for result set cache at server end of database |
| CN102799622A (en) * | 2012-06-19 | 2012-11-28 | 北京大学 | Distributed structured query language (SQL) query method based on MapReduce expansion framework |
-
2013
- 2013-04-24 CN CN201310146187.1A patent/CN104123288B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100413784B1 (en) * | 1997-04-29 | 2004-02-14 | 삼성전자주식회사 | Device for translating english-korean having clause unit separator and method for the same |
| US20030046073A1 (en) * | 2001-08-24 | 2003-03-06 | International Business Machines Corporation | Word predicting method, voice recognition method, and voice recognition apparatus and program using the same methods |
| US20060122993A1 (en) * | 2004-12-06 | 2006-06-08 | International Business Machines Corporation | Abstract query plan |
| CN101075255A (en) * | 2007-06-08 | 2007-11-21 | 北京神舟航天软件技术有限公司 | Method for removing SQL query statement constant condition |
| CN102045652A (en) * | 2009-10-21 | 2011-05-04 | 深圳市彩讯科技有限公司 | Garbage short message interception method based on characteristic similarity |
| CN102073719A (en) * | 2011-01-10 | 2011-05-25 | 复旦大学 | Interval coding-based geography markup language (GML) document index method |
| CN102682047A (en) * | 2011-10-18 | 2012-09-19 | 国网电力科学研究院 | Mixed structured query language (SQL) injection protection method |
| CN102789475A (en) * | 2012-05-08 | 2012-11-21 | 华为技术有限公司 | Data synchronization method and device for result set cache at server end of database |
| CN102799622A (en) * | 2012-06-19 | 2012-11-28 | 北京大学 | Distributed structured query language (SQL) query method based on MapReduce expansion framework |
Cited By (55)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104462317A (en) * | 2014-12-01 | 2015-03-25 | 苏州朗米尔照明科技有限公司 | Data screening method based on tag |
| CN104866608A (en) * | 2015-06-05 | 2015-08-26 | 中国人民大学 | Query optimization method based on join index in data warehouse |
| CN104866608B (en) * | 2015-06-05 | 2018-01-09 | 中国人民大学 | Enquiring and optimizing method based on join index in a kind of data warehouse |
| CN106325832A (en) * | 2015-06-18 | 2017-01-11 | 阿里巴巴集团控股有限公司 | Data generation method and device |
| CN106411826A (en) * | 2015-08-03 | 2017-02-15 | 阿里巴巴集团控股有限公司 | Data access method and equipment thereof |
| CN106411826B (en) * | 2015-08-03 | 2019-06-18 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of data access |
| CN106598963A (en) * | 2015-10-14 | 2017-04-26 | 五八同城信息技术有限公司 | Method and device for optimizing query statement |
| CN108431805B (en) * | 2015-10-23 | 2021-08-17 | 甲骨文国际公司 | Parallel execution of queries with recursive clauses |
| CN108431805A (en) * | 2015-10-23 | 2018-08-21 | 甲骨文国际公司 | It is parallel to execute the inquiry with recursion clause |
| CN106776639A (en) * | 2015-11-24 | 2017-05-31 | 腾讯科技（深圳）有限公司 | Data processing method and data processing equipment based on SQL |
| CN106776639B (en) * | 2015-11-24 | 2020-04-10 | 腾讯科技（深圳）有限公司 | Data processing method and data processing device based on structured query language |
| CN107239710B (en) * | 2016-03-29 | 2020-06-16 | 北京明略软件系统有限公司 | Database permission implementation method and system |
| CN107239710A (en) * | 2016-03-29 | 2017-10-10 | 北京明略软件系统有限公司 | A kind of data base authority method and system |
| CN105930407A (en) * | 2016-04-18 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Cross-database associated query method and system for distributed database |
| CN105930407B (en) * | 2016-04-18 | 2019-05-17 | 北京思特奇信息技术股份有限公司 | A kind of inter-library relation query method of distributed data base and system |
| JP2019531539A (en) * | 2016-09-06 | 2019-10-31 | ベイジン バイドゥ ネットコム サイエンス テクノロジー カンパニー リミテッド | Method and apparatus for performing distributed computing tasks |
| US11379499B2 (en) | 2016-09-06 | 2022-07-05 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for executing distributed computing task |
| WO2018045610A1 (en) * | 2016-09-06 | 2018-03-15 | 北京百度网讯科技有限公司 | Method and device for executing distributed computing task |
| CN108089915A (en) * | 2016-11-22 | 2018-05-29 | 北京京东尚科信息技术有限公司 | The method and system of business controlization processing based on message queue |
| CN106802927A (en) * | 2016-12-22 | 2017-06-06 | 上海大汉三通数据通信有限公司 | A kind of date storage method and querying method |
| WO2018218504A1 (en) * | 2017-05-31 | 2018-12-06 | 华为技术有限公司 | Method and device for data query |
| US11269878B2 (en) | 2017-06-14 | 2022-03-08 | Tencent Technology (Shenzhen) Company Limited | Uncorrelated subquery optimization method and apparatus, and storage medium |
| CN107315790A (en) * | 2017-06-14 | 2017-11-03 | 腾讯科技（深圳）有限公司 | A kind of optimization method and device of irrelevant subquery |
| CN107315790B (en) * | 2017-06-14 | 2021-07-06 | 腾讯科技（深圳）有限公司 | Optimization method and device for non-relevant sub-queries |
| CN110019394A (en) * | 2017-11-28 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Data query method and apparatus |
| CN108268589A (en) * | 2017-12-05 | 2018-07-10 | 北京百度网讯科技有限公司 | Aggregate query method, apparatus, computer equipment and the readable medium of time series data |
| CN108388606A (en) * | 2018-02-06 | 2018-08-10 | 福建星瑞格软件有限公司 | A kind of method and computer equipment verifying library literary name section name in Sql sentences |
| CN108388606B (en) * | 2018-02-06 | 2022-01-25 | 福建星瑞格软件有限公司 | Method for checking base table field names in Sql sentences and computer equipment |
| CN110297710A (en) * | 2018-03-23 | 2019-10-01 | 畅想科技有限公司 | Public precedence information for more resource arbitrations |
| CN110297710B (en) * | 2018-03-23 | 2022-05-10 | 畅想科技有限公司 | Common priority information for multiple resource arbitration |
| CN110727672A (en) * | 2018-06-28 | 2020-01-24 | 北京京东尚科信息技术有限公司 | Data mapping relation query method and device, electronic equipment and readable medium |
| CN109308300A (en) * | 2018-09-27 | 2019-02-05 | 上海达梦数据库有限公司 | A kind of processing method of logical operation, device, conversion plug-in unit and storage medium |
| CN110968615B (en) * | 2018-09-30 | 2023-05-23 | 北京国双科技有限公司 | Data query method and device |
| CN110968615A (en) * | 2018-09-30 | 2020-04-07 | 北京国双科技有限公司 | Data query method and device |
| CN111125155A (en) * | 2018-10-31 | 2020-05-08 | 北京国双科技有限公司 | Data query method and device based on access path, storage medium and processor |
| CN111125155B (en) * | 2018-10-31 | 2023-10-27 | 北京国双科技有限公司 | Access path-based data query method, device, storage medium and processor |
| CN109508339A (en) * | 2018-11-12 | 2019-03-22 | 上海达梦数据库有限公司 | A kind of data query method, apparatus, terminal device and storage medium |
| CN109508339B (en) * | 2018-11-12 | 2020-10-09 | 上海达梦数据库有限公司 | Data query method and device, terminal equipment and storage medium |
| CN111488326A (en) * | 2019-01-25 | 2020-08-04 | 阿里巴巴集团控股有限公司 | MapReduce job execution method, node device and storage medium |
| CN111488326B (en) * | 2019-01-25 | 2023-04-07 | 阿里巴巴集团控股有限公司 | MapReduce job execution method, node device and storage medium |
| CN111597214A (en) * | 2019-02-21 | 2020-08-28 | 北京京东尚科信息技术有限公司 | Method, apparatus, computer system and medium for data query |
| CN109947804A (en) * | 2019-03-20 | 2019-06-28 | 上海达梦数据库有限公司 | Optimization method, device, server and the storage medium of data acquisition system inquiry |
| CN109947804B (en) * | 2019-03-20 | 2021-04-06 | 上海达梦数据库有限公司 | Data set query optimization method and device, server and storage medium |
| CN110008244A (en) * | 2019-03-29 | 2019-07-12 | 国家计算机网络与信息安全管理中心 | A kind of data query method and data query device |
| CN110109946A (en) * | 2019-04-23 | 2019-08-09 | 北京小米移动软件有限公司 | Problem querying method and device |
| CN110109946B (en) * | 2019-04-23 | 2021-09-14 | 北京小米移动软件有限公司 | Question query method and device |
| CN110096489A (en) * | 2019-04-30 | 2019-08-06 | 阿里巴巴集团控股有限公司 | A kind of data query method, system, device and electronic equipment |
| CN113010539A (en) * | 2019-12-20 | 2021-06-22 | 北京奇艺世纪科技有限公司 | Data processing method and device |
| CN111159218B (en) * | 2019-12-31 | 2023-10-31 | 中科曙光国际信息产业有限公司 | Data processing method, device and readable storage medium |
| CN111159218A (en) * | 2019-12-31 | 2020-05-15 | 中科曙光国际信息产业有限公司 | Data processing method and device and readable storage medium |
| CN111694943B (en) * | 2020-05-29 | 2021-03-23 | 刘韩 | Accurate information query method and system |
| CN111694943A (en) * | 2020-05-29 | 2020-09-22 | 刘韩 | Accurate information query method and system |
| CN111949684B (en) * | 2020-08-06 | 2023-08-11 | 上海达梦数据库有限公司 | SQL sentence execution method, device, equipment and storage medium |
| CN111949684A (en) * | 2020-08-06 | 2020-11-17 | 上海达梦数据库有限公司 | SQL statement execution method, device, equipment and storage medium |
| WO2022127417A1 (en) * | 2020-12-16 | 2022-06-23 | 中兴通讯股份有限公司 | Data query method, and electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104123288B (en) | 2018-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104123288A (en) | Method and device for inquiring data | |
| JP6617117B2 (en) | Scalable analysis platform for semi-structured data | |
| CN105122243B (en) | Expansible analysis platform for semi-structured data | |
| US8700605B1 (en) | Estimating rows returned by recursive queries using fanout | |
| CN111026874A (en) | Data processing method and server of knowledge graph | |
| CN111382226A (en) | Database query retrieval method and device and electronic equipment | |
| CN106294695A (en) | A kind of implementation method towards the biggest data search engine | |
| CN111221791A (en) | Method for importing multi-source heterogeneous data into data lake | |
| DE112011101200T5 (en) | Column-oriented memory representations of data records | |
| WO2019161679A1 (en) | Data processing method and device for use in online analytical processing | |
| Alarabi et al. | TAREEG: A MapReduce-based system for extracting spatial data from OpenStreetMap | |
| US20140229429A1 (en) | Database management delete efficiency | |
| US20200226130A1 (en) | Vertical union of feature-based datasets | |
| CN112434027A (en) | Indexing method and device for multi-dimensional data, computer equipment and storage medium | |
| US10678789B2 (en) | Batch data query method and apparatus | |
| Kricke et al. | Graph data transformations in Gradoop | |
| Mehmood et al. | Distributed real-time ETL architecture for unstructured big data | |
| US11567969B2 (en) | Unbalanced partitioning of database for application data | |
| Mittal et al. | Efficient random data accessing in MapReduce | |
| CN110928963B (en) | Column-level authority knowledge graph construction method for operation and maintenance service data table | |
| CN116894035A (en) | Multi-source heterogeneous data blood-edge relation construction method, system, equipment and medium | |
| CN115658680A (en) | Data storage method, data query method and related device | |
| US20220215021A1 (en) | Data Query Method and Apparatus, Computing Device, and Storage Medium | |
| Albahli et al. | TripleFCA: FCA-based approach to enhance semantic web data management | |
| CN113297252A (en) | Data query service method with mode being unaware |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20211109 Address after: Room 507, floor 5, building 3, No. 969, Wenyi West Road, Wuchang Street, Yuhang District, Hangzhou City, Zhejiang Province Patentee after: ZHEJIANG TMALL TECHNOLOGY Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Patentee before: ALIBABA GROUP HOLDING Ltd. |