CN109189800A - A kind of data Layer paging query model and querying method - Google Patents
A kind of data Layer paging query model and querying method Download PDFInfo
- Publication number
- CN109189800A CN109189800A CN201810933428.XA CN201810933428A CN109189800A CN 109189800 A CN109189800 A CN 109189800A CN 201810933428 A CN201810933428 A CN 201810933428A CN 109189800 A CN109189800 A CN 109189800A
- Authority
- CN
- China
- Prior art keywords
- module
- data
- unified
- condition
- retrieval module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of data Layer paging query model and querying methods, model includes near line WEB application end, cleaning module, adaptation module, unified retrieval module and data source, the present invention passes through unified paging query condition, the data of different data sources are subjected to unified retrieval, by encapsulating front end paging logic, unitized paging activity, encapsulation of data layer query logic, enhancing system safety, realize the accurate lookup of paged data small data quantity, model of the invention, which has, to be easy to extend, it is easy to the feature of distributing, the unified data Layer querying condition of querying method, simplify interface to call, use value with higher.
Description
Technical field
The invention belongs to data query technique fields, and in particular to a kind of data Layer paging query model and issuer
Method.
Background technique
In software development, the inquiry of big data quantity is a common problem, is frequently encountered and carries out to mass data
The scene of inquiry, the common solution to big data quantity inquiry have following two: 1, total data first being inquired memory
In, paging is then carried out in memory, and this mode is larger to EMS memory occupation, it is necessary to limit the data volume of one query；2, it adopts
Paging is carried out in the database with storing process, and this mode is larger to the dependence of database, different database realizing mechanism
It is obstructed, and search efficiency is not ideal enough.Both the above mode is all not friendly enough for users.And existing exploitation software,
Such as JAVA only supports MySQL, Oracle relational data library inquiry, only supports Solr, the inspection of the search engines data such as Lucene
Rope needs to encapsulate different query objects for different data sources, due to paging condition disunity, it is difficult to specification query criteria,
For upper-layer service, when needing to do corresponding adjustment according to data source, then it is related to that face is wider, and data volume is big, can not accurately accomplishes
Data routing, load balancing, business separation, final query results are more, can not accomplish flexible conversion.
Summary of the invention
According to above-mentioned elaboration, the purpose of the present invention is to provide a kind of data Layer paging query model and querying method,
By packaging bottom layer logic of standardizing, uniform data layer querying condition simplifies interface and calls.
Technical solution provided by the invention:
A kind of data Layer paging query model, comprising: near line WEB application end, cleaning module, adaptation module, unified retrieval
Module and data source, near line WEB application end, cleaning module, adaptation module data carry out sequential delivery, unified retrieval module
Api interface is set, and the unified retrieval module includes parallel retrieval module, load module, the retrieval module, load module
It is separately connected data source, data source includes Solr, Elasticsearch, Mysql and Oracle.
A kind of data Layer paging query method, steps are as follows:
Q1, querying condition is obtained by line WEB application end；
Q2, cleaning module carry out classification client query condition and movement, are unified for standard paging querying condition；
Standard queries condition is converted to rear end querying condition by Q3, adaptation module；
Q4, the retrieval module that standard queries condition is transferred to unified retrieval module, retrieval module are inquired in data source, and
The query result of return is matched back to load module with rear end querying condition, final result is exported by api interface.
The present invention passes through unified paging query condition, the data of different data sources is carried out unified retrieval, before encapsulation
Paging logic is held, unitize paging activity, encapsulation of data layer query logic, and enhancing system safety realizes paged data small data
The accurate lookup of amount, model of the invention, which has, to be easy to extend, and is easy to the feature of distributing, and the unified data Layer of querying method inquires item
Part simplifies interface and calls, use value with higher.
Detailed description of the invention
Fig. 1 the structural representation of present invention；
Specific embodiment
Technical solution of the present invention is clearly and completely described with reference to the accompanying drawings and detailed description, it is clear that
Described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Based on the implementation in the present invention
Example, every other embodiment obtained by those of ordinary skill in the art without making creative efforts belong to
The scope of protection of the invention.
According to Fig. 1, a kind of data Layer paging query model, comprising: near line WEB application end, cleaning module, adaptation mould
Block, unified retrieval module and data source, near line WEB application end, cleaning module, adaptation module data carry out sequential delivery,
Api interface is arranged in unified retrieval module, and the unified retrieval module includes parallel retrieval module, load module, the retrieval
Module, load module are separately connected data source, and data source includes Solr, Elasticsearch, Mysql and Oracle.
A kind of data Layer paging query method, steps are as follows:
Q1, querying condition is obtained by line WEB application end；
Q2, cleaning module carry out classification client query condition and movement, are unified for standard paging querying condition；
Standard queries condition is converted to rear end querying condition by Q3, adaptation module；
Q4, the retrieval module that standard queries condition is transferred to unified retrieval module, retrieval module are inquired in data source, and
The query result of return is matched back to load module with rear end querying condition, final result is exported by api interface.
The foregoing describe basic principle of the invention and advantages.It should be understood by those skilled in the art that the present invention not by
The limitation of above-mentioned case study on implementation is merely illustrated the principles of the invention described in above-mentioned case study on implementation and specification, is not being departed from
Under the premise of spirit and scope of the invention, various changes and improvements may be made to the invention, these changes and improvements both fall within requirement
In the scope of the invention of protection.Claimed range of the invention is defined by the appending claims and its equivalent thereof.
Claims (2)
1. a kind of data Layer paging query model, it is characterised in that: including near line WEB application end, cleaning module, adaptation module,
Unified retrieval module and data source, near line WEB application end, cleaning module, adaptation module data carry out sequential delivery, unified
Api interface is arranged in retrieval module, and the unified retrieval module includes parallel retrieval module, load module, the retrieval module,
Load module is separately connected data source, and data source includes Solr, Elasticsearch, Mysql and Oracle.
2. a kind of a kind of data Layer paging query method based on claim 1 data Layer paging query model, it is characterised in that:
Steps are as follows:
Q1, querying condition is obtained by line WEB application end；
Q2, cleaning module carry out classification client query condition and movement, are unified for standard paging querying condition；
Standard queries condition is converted to rear end querying condition by Q3, adaptation module；
Q4, the retrieval module that standard queries condition is transferred to unified retrieval module, retrieval module is inquired in data source, and will be returned
The query result returned is matched back to load module with rear end querying condition, and final result is exported by api interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810933428.XA CN109189800A (en) | 2018-08-16 | 2018-08-16 | A kind of data Layer paging query model and querying method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810933428.XA CN109189800A (en) | 2018-08-16 | 2018-08-16 | A kind of data Layer paging query model and querying method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109189800A true CN109189800A (en) | 2019-01-11 |
Family
ID=64918153
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810933428.XA Pending CN109189800A (en) | 2018-08-16 | 2018-08-16 | A kind of data Layer paging query model and querying method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109189800A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110134696A (en) * | 2019-05-21 | 2019-08-16 | 国网山东省电力公司 | A kind of comprehensive budget editing method, apparatus and system based on big data |
| CN111026574A (en) * | 2019-11-25 | 2020-04-17 | 中盈优创资讯科技有限公司 | Method and device for diagnosing Elasticissearch cluster problems |
| CN111767309A (en) * | 2020-07-07 | 2020-10-13 | 中电万维信息技术有限责任公司 | Method for optimizing retrieval based on switch design mode |
| CN112487275A (en) * | 2020-12-11 | 2021-03-12 | 杭州安恒信息技术股份有限公司 | Data retrieval method, system, equipment and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1920817A (en) * | 2006-09-14 | 2007-02-28 | 浙江大学 | Method for multiple resources pools integral parallel search in open websites |
| CN101739460A (en) * | 2009-12-16 | 2010-06-16 | 中国科学院对地观测与数字地球科学中心 | Grid-based spatial data source unification service system and method |
| CN104636502A (en) * | 2015-03-10 | 2015-05-20 | 浪潮集团有限公司 | Accelerated data query method of query system |
| CN105450680A (en) * | 2014-08-09 | 2016-03-30 | 星际空间（天津）科技发展有限公司 | Service method for data sharing |
| CN105912624A (en) * | 2016-04-07 | 2016-08-31 | 北京中安智达科技有限公司 | Query method for distributed deployed heterogeneous database |
| CN108009236A (en) * | 2017-11-29 | 2018-05-08 | 北京锐安科技有限公司 | A kind of big data querying method, system, computer and storage medium |
-
2018
- 2018-08-16 CN CN201810933428.XA patent/CN109189800A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1920817A (en) * | 2006-09-14 | 2007-02-28 | 浙江大学 | Method for multiple resources pools integral parallel search in open websites |
| CN101739460A (en) * | 2009-12-16 | 2010-06-16 | 中国科学院对地观测与数字地球科学中心 | Grid-based spatial data source unification service system and method |
| CN105450680A (en) * | 2014-08-09 | 2016-03-30 | 星际空间（天津）科技发展有限公司 | Service method for data sharing |
| CN104636502A (en) * | 2015-03-10 | 2015-05-20 | 浪潮集团有限公司 | Accelerated data query method of query system |
| CN105912624A (en) * | 2016-04-07 | 2016-08-31 | 北京中安智达科技有限公司 | Query method for distributed deployed heterogeneous database |
| CN108009236A (en) * | 2017-11-29 | 2018-05-08 | 北京锐安科技有限公司 | A kind of big data querying method, system, computer and storage medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110134696A (en) * | 2019-05-21 | 2019-08-16 | 国网山东省电力公司 | A kind of comprehensive budget editing method, apparatus and system based on big data |
| CN111026574A (en) * | 2019-11-25 | 2020-04-17 | 中盈优创资讯科技有限公司 | Method and device for diagnosing Elasticissearch cluster problems |
| CN111026574B (en) * | 2019-11-25 | 2024-03-12 | 中盈优创资讯科技有限公司 | Method and device for diagnosing elastiscearch cluster problem |
| CN111767309A (en) * | 2020-07-07 | 2020-10-13 | 中电万维信息技术有限责任公司 | Method for optimizing retrieval based on switch design mode |
| CN111767309B (en) * | 2020-07-07 | 2022-06-24 | 中电万维信息技术有限责任公司 | Method for optimizing retrieval based on switch design mode |
| CN112487275A (en) * | 2020-12-11 | 2021-03-12 | 杭州安恒信息技术股份有限公司 | Data retrieval method, system, equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109189800A (en) | A kind of data Layer paging query model and querying method | |
| Yang et al. | Design and implementation of cloud platform for intelligent logistics in the trend of intellectualization | |
| US20130191523A1 (en) | Real-time analytics for large data sets | |
| CN107895046B (en) | Heterogeneous data integration platform | |
| CN101901242A (en) | Federated configuration data management | |
| CN104200402A (en) | Publishing method and system of source data of multiple data sources in power grid | |
| CN105683956A (en) | Densely grouping dimensional data | |
| CN104239572A (en) | System and method for achieving metadata analysis based on distributed cache | |
| CN105224613A (en) | Based on integrating heterogeneous data source system and the integration method thereof of the federal technology of data | |
| CN101739460A (en) | Grid-based spatial data source unification service system and method | |
| CN102609464A (en) | Method and device for associative table query of MONGODB shards | |
| CN104598587A (en) | Invoice authenticity inquiry method and system | |
| US10140347B2 (en) | Exposing data to query generating applications using usage profiles | |
| CN104636329A (en) | Unified management method for large-scale cross-platform heterogeneous data | |
| CN105045897A (en) | Business processing system and method capable of supporting capacity expansion of database | |
| CN108241539A (en) | Interactive big data querying method, device, storage medium and terminal device based on distributed system | |
| CN104008199A (en) | Data inquiring method | |
| CN104537091A (en) | Networked relational data query method based on hierarchical identification routing | |
| CN115687478A (en) | Standardized service data sharing system and method | |
| CN106970943A (en) | Fusion query method based on heterogeneous data source and distributed file system | |
| CN1744638A (en) | Advertising issueing system and method for issueing advertising | |
| CN105718592A (en) | Data calling method and system based on Redis | |
| CN109739872A (en) | A kind of implementation method, system and the operating method of SQL statement processing | |
| CN110245043B (en) | Tracking system for call relation between distributed systems | |
| CN107656943B (en) | Task query method and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190111 |