CN107633094A - The method and apparatus of data retrieval in a kind of cluster environment - Google Patents
The method and apparatus of data retrieval in a kind of cluster environment Download PDFInfo
- Publication number
- CN107633094A CN107633094A CN201710939998.5A CN201710939998A CN107633094A CN 107633094 A CN107633094 A CN 107633094A CN 201710939998 A CN201710939998 A CN 201710939998A CN 107633094 A CN107633094 A CN 107633094A
- Authority
- CN
- China
- Prior art keywords
- keywords
- character string
- cluster
- retrieval
- cluster environment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The method that the present invention provides data retrieval in a kind of cluster environment, this method using rule, using resolver come configuration information, the set of definition of keywords and predefined keywords based on cluster environment to inquiry relational expression including being parsed, the result according to parsing, and the thin consolidation after parsing is sent into cluster to make requests on, obtain response and final retrieval result is obtained from response to be inquired about, by the result of integration.The present invention has the advantages of inquiry is convenient, fast, and cost is low.
Description
Technical field
The present invention relates to information control technology field, and more particularly, to data retrieval in a kind of cluster environment
Method and apparatus.
Background technology
At present, the retrieval to company-data is mainly using the API (application programming interfaces) of the opening based on cluster itself
The mode of interface.But bigger limitation be present in this mode, the inquiry mode and mesh that are mainly reflected in distributed type assemblies
Preceding ripe relevant database difference is huge, and the T-SQL modes that current industry can not be used general carry out quick search；
Somewhat complicated inquiry needs to write code realization, and cost is larger；Operation maintenance personnel has one when quickly and efficiently using cluster
Fixed difficulty.
Therefore, a kind of method of data retrieval how is designed, industry can be easily used in distributed type assemblies environment
Ripe T-SQL standards turn into technical problem urgently to be resolved hurrily quickly and efficiently to inquire about, retrieve data.
The content of the invention
Therefore, it is an object of the invention to propose a kind of method of data retrieval in cluster environment.
Another object of the present invention is to propose a kind of device of data retrieval in cluster environment.
To achieve these goals, technical scheme according to an aspect of the present invention, it is proposed that in a kind of cluster environment
The method of data retrieval, this method are included based on cluster environment come configuration information；The set of definition of keywords and predefined pass
Key word uses rule；Inquiry relational expression is parsed using resolver；According to the result of parsing, by the portion after parsing
Divide and integrate to be inquired about；The result of integration is sent to cluster to make requests on；And obtain and respond and obtained from response
Take final retrieval result.
According to one embodiment of present invention, packet IP containing cluster server, cluster name, port numbers.
According to one embodiment of present invention, set includes the keyword used in T-SQL inquiries.
According to one embodiment of present invention, parsing is carried out to inquiry relational expression using resolver to specifically include：
Character string is obtained from inquiry relational expression, character string and SELECT keywords are subjected to matching verification, match verification
Rule is：
If character string is equal with SELECT keywords, character string is retrieval class,
If character string and SELECT keywords are unequal, character string is non-retrieval class.
According to one embodiment of present invention, for retrieving class, inquiry relational expression is closed according to SELECT keywords, FROM
Key word and WHERE keywords are divided, and the part between SELECT keywords and FROM keywords is divided into M sections, FROM
The part that part between keyword and WHERE keywords is divided into behind N sections, and WHERE keywords is divided into Q
Section.
Technical scheme according to another aspect of the present invention, there is provided the device of data retrieval in a kind of cluster environment, should
Device is included based on cluster environment come the module of configuration information；The set of definition of keywords and the use rule of predefined keywords
Module then；The module parsed using resolver to inquiry relational expression；According to the result of parsing, by the portion after parsing
Divide the module integrated to be inquired about；The module that the result of the integration is sent to cluster to make requests on；And obtain and ring
The module of final retrieval result and should be obtained from the response.
According to one embodiment of present invention, packet IP containing cluster server, cluster name, port numbers.
According to one embodiment of present invention, set includes the keyword used in T-SQL inquiries.
According to one embodiment of present invention, the module parsed using resolver to inquiry relational expression is further wrapped
Include：
Character string is obtained from inquiry relational expression, character string with SELECT keywords match to the submodule of verification, its
In, the rule of the matching verification is：
If character string is equal with the SELECT keywords, character string is retrieval class,
If character string and the SELECT keywords are unequal, character string is non-retrieval class.
Technical scheme according to another aspect of the invention, a kind of computer-readable recording medium is additionally provided, the calculating
Computer program (instruction) is stored with machine readable storage medium storing program for executing, for realizing data retrieval in cluster environment, described program (refers to
Make) method described in any of the above-described technical scheme is realized when being executed by processor.
The additional aspect and advantage of the present invention will become obvious in the following description, or the practice by the present invention
Solve.
Brief description of the drawings
Fig. 1 shows the flow chart of the method for data retrieval in cluster environment according to an embodiment of the invention；
Fig. 2 shows the workflow diagram of resolver according to another embodiment of the present invention.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, below in conjunction with the accompanying drawings, the present invention is entered
Row is further described.It should be appreciated that specific embodiment described herein is not used to only to explain the present invention
Limit the present invention.
The method that Fig. 1 shows data retrieval in cluster environment according to one embodiment of present invention, this method is in step
S01 starts.In step S01, relevant information is configured based on cluster environment, including cluster server IP address, the title of cluster,
The key messages such as port numbers, then method proceed to step S02.In the collection for the keyword that step S02, definition resolver are supported
Close, the set of wherein keyword includes the keyword used in T-SQL inquiries, and method then proceeds to step S03.In step
S03, predefined keywords use rule, for example, keyword etc. is applied in combination, method then proceeds to step S04.In step
S04, query statement is parsed using resolver, i.e., using resolver come resolver in the query statement that uses user
The keyword supported is parsed, specific as follows：
For each action statement, part before intercepting first space simultaneously filters out the space in intercepted part
Deng idle character, then the final character string got is judged, i.e., by character string and SELECT (SQL query statement)
Matching verification is carried out, if character string is equal with SELECT keywords (i.e. the two is just the same), the character string is retrieval class
Character string, if character string and SELECT keywords are unequal (incomplete both i.e.), the character string is non-retrieval class
Character string.
For retrieving class character string, T-SQL query statements are divided into three parts according to SELECT, FROM and WHERE：
The N sections between M sections, FROM and WHERE, the Q sections after WHERE between SELECT and FROM.
For M sections, cut to obtain array, and each single item in array is removed front and rear by using space
Space.Then check in array whether include other T-SQL keywords, if comprising the T-SQL keywords are parsed simultaneously
Its query type is judged according to the keyword, if do not included, each single item in array is resolved into inquiry needs what is obtained
Field.
For N sections, similar to M sections, cut by using space to obtain array, and to each in array
Item removes front and rear space.Then, obtained array is resolved to data source to be checked.
For Q sections, judge whether it includes any one keyword in ORDER BY, GROUP BY, LIMIT etc., such as
Fruit includes, then by positioned at WHERE and ORDER BY GROUPBY part analysis between LIMIT be querying condition；If do not wrap
Contain, be then querying condition by all part analysis after WHERE.Wherein, LIMIT situation is included in Q sections, is obtained
After LIMIT numeral and resolve to：Filter out that LIMIT defines in the data for meeting inquiry from which bar to which bar
Data.Then, method proceeds to step S05.
In step S05, according to the analysis result in step S04, by the various pieces obtained after parsing by using corresponding
Api interface be integrated into final calling form, method then proceeds to step S06.In step S06, by the API after integration
Calling form is sent to cluster to make requests on, and method then proceeds to step S07.In step S07, the sound from cluster is obtained
Answer and according to the search field parsed, response results are filtered to obtain final retrieval result, method terminates.
Fig. 2 shows the workflow diagram of resolver according to another embodiment of the present invention.First, user is according to parsing
The keyword that device is supported determines corresponding query statement, and query statement is parsed to determine inquiry mode, if looked into
Inquiry mode is data class inquiry mode, then the field for needing to inquire about is parsed from query statement, is then obtained according to analysis result
The data source with inquiry is taken, if inquiry mode is statistics class inquiry, directly query statement is parsed to obtain inquiry
Data source.Then, querying condition is parsed from query statement, and parsing packet, sort criteria are carried out according to querying condition
Deng being parsed to obtain the data slot for meeting that querying condition need to return, will be returned to the data source of inquiry according to querying condition
The data slot returned is combined into inquiry API to be parsed.
According to still another embodiment of the invention, in a kind of cluster environment data retrieval device include based on cluster environment come
The module of configuration information, the information include cluster server IP address, the title of cluster, port numbers etc.；The collection of definition of keywords
Merging and the module using regular (for example, keyword is applied in combination) of predefined keywords, wherein, the set of keyword includes
The keyword used in T-SQL inquiries；The module parsed using resolver to inquiry relational expression, specifically, the module
Parsed using resolver come the keyword that the resolver in the query statement that uses user is supported；According to the knot of parsing
Fruit, the various pieces obtained after parsing are integrated into final calling form to enter by using corresponding api interface
The module of row inquiry；The module that API Calls form after integration is sent to cluster to make requests on；And obtain and come from cluster
Response and according to the search field parsed, the response results from cluster are filtered to obtain the mould of final result
Block.
On process described here, system, method etc., it should be understood that although the step of such process etc. is described as
Arrangement occurs in a certain order, but such process can use what is completed with the order outside order described herein
The step of description, implements operation.Further it is appreciated that some steps can perform simultaneously, other steps can be added,
Or some steps described here can be omitted.In other words, the description of process here is provided for illustrating some embodiments
Purpose, and should not be construed in any way for limit claimed invention.
Correspondingly, it should be understood that the purpose of above description illustrates rather than limitation.When reading above description,
Many embodiments and application will be apparent from addition to the example of offer.The scope of the present invention should refer to appended claims
And the four corner equivalent with the right required by claim and determine, rather than determined with reference to explanation above.Can
To be contemplated that field discussed herein will appear from further developing, and disclosed system and method can combine
Into such following embodiment.In a word, it should be understood that the present invention can be modified and change.
It is to be further understood that any described process or it is described during the step of can with other disclosed processes or
Step is combined to form the structure in the range of the disclosure.Example arrangement and process disclosed herein be it is for illustrative purposes,
And it is not necessarily to be construed as limiting.
Claims (10)
1. a kind of method of data retrieval in cluster environment, it is characterised in that the described method comprises the following steps：
Based on the cluster environment come configuration information；
The use rule gathered and predefine the keyword of definition of keywords；
Inquiry relational expression is parsed using resolver；
According to the result of the parsing, by the thin consolidation after the parsing to be inquired about；
The result of the integration is sent to the cluster to make requests on；With
Obtain and respond and obtained from the response final retrieval result.
2. the method for data retrieval in cluster environment according to claim 1, it is characterised in that described information includes cluster
Server ip, cluster name, port numbers.
3. the method for data retrieval in cluster environment according to claim 1, it is characterised in that the set includes T-
The keyword used in SQL query.
4. the method for data retrieval in cluster environment according to claim 1, it is characterised in that described to use resolver pair
Inquiry relational expression carries out parsing and specifically included：
Character string is obtained from the inquiry relational expression, the character string and SELECT keywords are subjected to matching verification, described
Rule with verification is：
If the character string is equal with the SELECT keywords, the character string is retrieval class,
If the character string and the SELECT keywords are unequal, the character string is non-retrieval class.
5. the method for data retrieval in cluster environment according to claim 4, it is characterised in that for the retrieval class,
The inquiry relational expression is divided according to SELECT keywords, FROM keywords and WHERE keywords, the SELECT is closed
Part between key word and the FROM keywords is divided into M sections, between the FROM keywords and the WHERE keywords
The part that is divided into behind N sections, and the WHERE keywords of part be divided into Q sections.
6. the device of data retrieval in a kind of cluster environment, it is characterised in that described device is included with lower module：
Based on the cluster environment come the module of configuration information；
The module using rule gathered and predefine the keyword of definition of keywords；
The module parsed using resolver to inquiry relational expression；
According to the result of the parsing, by module of the thin consolidation after the parsing to be inquired about；
The module that the result of the integration is sent to the cluster to make requests on；With
Obtain and respond and obtained from the response module of final retrieval result.
7. the device of data retrieval in cluster environment according to claim 6, it is characterised in that described information includes cluster
Server ip, cluster name, port numbers.
8. the device of data retrieval in cluster environment according to claim 6, it is characterised in that the set includes T-
The keyword used in SQL query.
9. the device of data retrieval in cluster environment according to claim 6, it is characterised in that described to use resolver pair
The module that inquiry relational expression is parsed further comprises：
Character string is obtained from the inquiry relational expression, the character string with SELECT keywords match to the submodule of verification
Block, wherein, the rule of the matching verification is：
If the character string is equal with the SELECT keywords, the character string is retrieval class,
If the character string and the SELECT keywords are unequal, the character string is non-retrieval class.
10. a kind of computer-readable recording medium, computer program (instruction) is stored thereon with, for realizing number in cluster environment
According to retrieval, it is characterised in that described program (instruction) realizes the side described in claim any one of 1-5 when being executed by processor
Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710939998.5A CN107633094B (en) | 2017-10-11 | 2017-10-11 | Method and device for data retrieval in cluster environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710939998.5A CN107633094B (en) | 2017-10-11 | 2017-10-11 | Method and device for data retrieval in cluster environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107633094A true CN107633094A (en) | 2018-01-26 |
| CN107633094B CN107633094B (en) | 2020-12-29 |
Family
ID=61104284
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710939998.5A Active CN107633094B (en) | 2017-10-11 | 2017-10-11 | Method and device for data retrieval in cluster environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107633094B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109299101A (en) * | 2018-10-15 | 2019-02-01 | 上海达梦数据库有限公司 | Data retrieval method, device, server and storage medium |
| CN111782766A (en) * | 2020-06-30 | 2020-10-16 | 福建健康之路信息技术有限公司 | Method and system for retrieving all resources in Kubernetes cluster through keywords |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130185408A1 (en) * | 2012-01-18 | 2013-07-18 | Dh2I Company | Systems and Methods for Server Cluster Application Virtualization |
| CN103412897A (en) * | 2013-07-25 | 2013-11-27 | 中国科学院软件研究所 | Parallel data processing method based on distributed structure |
| CN104657439A (en) * | 2015-01-30 | 2015-05-27 | 欧阳江 | Generation system and method for structured query sentence used for precise retrieval of natural language |
| CN106649455A (en) * | 2016-09-24 | 2017-05-10 | 孙燕群 | Big data development standardized systematic classification and command set system |
| CN106844380A (en) * | 2015-12-04 | 2017-06-13 | 阿里巴巴集团控股有限公司 | A kind of database operation method, information processing method and related device |
| CN106991183A (en) * | 2017-03-27 | 2017-07-28 | 福建数林信息科技有限公司 | A kind of business intelligence ETL method for packing and system |
| CN107180113A (en) * | 2017-06-16 | 2017-09-19 | 成都亿橙科技有限公司 | A kind of big data searching platform |
-
2017
- 2017-10-11 CN CN201710939998.5A patent/CN107633094B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130185408A1 (en) * | 2012-01-18 | 2013-07-18 | Dh2I Company | Systems and Methods for Server Cluster Application Virtualization |
| CN103412897A (en) * | 2013-07-25 | 2013-11-27 | 中国科学院软件研究所 | Parallel data processing method based on distributed structure |
| CN104657439A (en) * | 2015-01-30 | 2015-05-27 | 欧阳江 | Generation system and method for structured query sentence used for precise retrieval of natural language |
| CN106844380A (en) * | 2015-12-04 | 2017-06-13 | 阿里巴巴集团控股有限公司 | A kind of database operation method, information processing method and related device |
| CN106649455A (en) * | 2016-09-24 | 2017-05-10 | 孙燕群 | Big data development standardized systematic classification and command set system |
| CN106991183A (en) * | 2017-03-27 | 2017-07-28 | 福建数林信息科技有限公司 | A kind of business intelligence ETL method for packing and system |
| CN107180113A (en) * | 2017-06-16 | 2017-09-19 | 成都亿橙科技有限公司 | A kind of big data searching platform |
Non-Patent Citations (1)
| Title |
|---|
| 张建中 等: ""基于ElasticSearch的数字图书馆检索系统"", 《计算机与现代化》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109299101A (en) * | 2018-10-15 | 2019-02-01 | 上海达梦数据库有限公司 | Data retrieval method, device, server and storage medium |
| CN109299101B (en) * | 2018-10-15 | 2020-12-01 | 上海达梦数据库有限公司 | Data retrieval method, device, server and storage medium |
| CN111782766A (en) * | 2020-06-30 | 2020-10-16 | 福建健康之路信息技术有限公司 | Method and system for retrieving all resources in Kubernetes cluster through keywords |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107633094B (en) | 2020-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110908997B (en) | Data blood relationship construction method and device, server and readable storage medium | |
| CA2562281C (en) | Partial query caching | |
| US7111025B2 (en) | Information retrieval system and method using index ANDing for improving performance | |
| US11687546B2 (en) | Executing conditions with negation operators in analytical databases | |
| US7680821B2 (en) | Method and system for index sampled tablescan | |
| US20070239673A1 (en) | Removing nodes from a query tree based on a result set | |
| IL218803A (en) | System and method for data masking | |
| US8229940B2 (en) | Query predicate generator to construct a database query predicate from received query conditions | |
| US11775767B1 (en) | Systems and methods for automated iterative population of responses using artificial intelligence | |
| US9454561B2 (en) | Method and a consistency checker for finding data inconsistencies in a data repository | |
| CN107577787B (en) | Method and system for storing associated data information | |
| JP6763830B2 (en) | Platform-based data isolation | |
| US20090132607A1 (en) | Techniques for log file processing | |
| US10380115B2 (en) | Cross column searching a relational database table | |
| CN107633094A (en) | The method and apparatus of data retrieval in a kind of cluster environment | |
| WO2018107942A1 (en) | System and method of adaptively partitioning data to speed up join queries on distributed and parallel database systems | |
| US8880503B2 (en) | Value-based positioning for outer join queries | |
| US20230153455A1 (en) | Query-based database redaction | |
| CN105589969A (en) | Data processing method and device | |
| KR20190129474A (en) | Apparatus and method for retrieving data | |
| CN115687392A (en) | SQL statement optimized execution method and device, electronic equipment and medium | |
| CN115658680A (en) | Data storage method, data query method and related device | |
| US20200342139A1 (en) | High-dimensional data anonymization for in- memory applications | |
| US9002827B2 (en) | Database query table substitution | |
| US11023469B2 (en) | Value list compression (VLC) aware qualification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 2298, Yingying building, 99 Tuanjie Road, yanchuangyuan, Jiangbei new district, Nanjing, Jiangsu Province, 211800 Applicant after: Beixinyuan system integration Co., Ltd Address before: No.3 Ruiyun Road, Jiangpu street, Pukou District, Nanjing, Jiangsu Province, 211899 Applicant before: JIANGSU SHENZHOU XINYUAN SYSTEM ENGINEERING Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |