CN111611328A - Large-scale two-dimensional and three-dimensional spatial data search engine based on network - Google Patents
Large-scale two-dimensional and three-dimensional spatial data search engine based on network Download PDFInfo
- Publication number
- CN111611328A CN111611328A CN201910139222.4A CN201910139222A CN111611328A CN 111611328 A CN111611328 A CN 111611328A CN 201910139222 A CN201910139222 A CN 201910139222A CN 111611328 A CN111611328 A CN 111611328A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- module
- application program
- interface
- space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Remote Sensing (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a large-scale two-dimensional and three-dimensional space data search engine based on a network, which structurally comprises: the system comprises a space indexer module, a space server module and an application program interface module; the spatial indexer module is used for establishing a spatial index record database; the space server module comprises a queue response processor and a solr search server module; the application program interface module comprises a query interface and a feedback interface; the output end of the spatial index record database and the query interface of the application program interface module are respectively connected with the solr search server module, and the output end of the solr search server module is connected with the feedback interface of the application program interface module and is output to the outside through the feedback interface. The invention adopts the modular design, is convenient to search for data of different scales, is easy to expand, greatly improves the query speed of large-scale two-dimensional and three-dimensional space information, and obviously improves the quality of a data source and the retrieval accuracy.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to a large-scale two-three dimensional space data searching technology based on a network.
Background
With the rapid development of computer technology and the rapid increase of social demand, the application of spatial data is more and more extensive, wherein the map platforms for civil use including high-grade and Baidu maps and the large Geographic Information Systems (GIS) for commercial use including ArcGIS and hypergraph are all established on the basis of the spatial data; at present, for the application of spatial data, the search of a large amount of even massive spatial data has become a key problem faced in the above system, especially the high combination of two-dimensional (such as road information) and three-dimensional (such as building information) is difficult for spatial search. Therefore, how to solve the search problem in the two-dimensional and three-dimensional spatial information field by using the big data processing technology of the traditional search engine is urgent to establish a fast and efficient search engine, such as google, hundredth and the like.
At present, the traditional spatial search engine still depends on the retrieval technology of a relational database, and the query efficiency is low; moreover, two-dimensional data and three-dimensional data cannot be unified in the prior art, and the solution for the heterogeneity without integration is difficult to meet the requirement of social development.
Disclosure of Invention
In view of the above circumstances, the present invention aims to: the method provides a large-scale two-dimensional and three-dimensional spatial data search engine based on a network, solves the search problem in the field of two-dimensional and three-dimensional spatial information by using a novel search engine and a big data processing technology, and establishes a rapid and efficient multi-dimensional spatial search method.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows: providing a large-scale two-dimensional and three-dimensional spatial data search engine based on a network, wherein the engine structure comprises: the system comprises a space indexer module, a space server module and an application program interface module; the spatial indexer module is used for establishing a spatial index record database; the space server module comprises a queue response processor and a solr search server module; the application program interface module comprises a query interface and a feedback interface; the output end of the spatial index record database and the query interface of the application program interface module are respectively connected with the solr search server module, the output end of the solr search server module is connected with the feedback interface of the application program interface module, and the output end of the solr search server module is output to the outside through the feedback interface and aims at index results of different data types.
The index record database automatically analyzes different types of databases, the solr search server analyzes and classifies established two-dimensional and three-dimensional spatial data indexes, analyzes the timeliness and the regionality of the two-dimensional and three-dimensional spatial data, adds time and region information into the index record database or establishes a connection with the index record database, the lucene search application server adopts an optimal priority search strategy to extract data, selects one or more data with the best evaluation according to an analysis algorithm to extract, and processes the data to obtain an optimal data index result.
Compared with the prior art, the technical scheme has the beneficial effects that: the engine adopts a modular design, so that the data of different scales can be conveniently searched, and the expansion is easy; the engine establishes a unified index mode for the two-dimensional data and the three-dimensional data by using solr, so that the query speed of large-scale two-dimensional and three-dimensional spatial information can be greatly improved, and the quality of a data source and the retrieval accuracy can be obviously improved.
Drawings
Fig. 1 is a schematic structural diagram of a spatial data search engine.
Detailed Description
The present application is described below by way of example, and as shown in fig. 1, the present invention provides a large-scale two-three dimensional space data search engine based on a network, which comprises: the system comprises a space indexer module, a space server module and an application program interface module. The spatial indexer module is used for establishing an index record database; the space server module comprises a queue response processor and a solr search server; the application program interface module comprises a query interface and a feedback interface. The output end and the query interface of the spatial index record database are respectively connected with the solr search server module, the output end of the solr search server module is connected with the feedback interface, and index results aiming at different data types (two-dimensional or three-dimensional) are output to the outside through the feedback interface.
The spatial data search engine provided by the invention realizes indexing through the following process.
1. Data acquisition preprocessing
The data acquisition mainly comprises the steps of automatically extracting two-dimensional and three-dimensional space data from a space database, preprocessing the data mainly comprises the steps of analyzing and classifying the captured two-dimensional and three-dimensional space data, analyzing the timeliness and the regionality of the two-dimensional and three-dimensional space data, adding time and region information into an index database or establishing a connection with the index database, establishing a professional knowledge information base, and solving the problem of accurate searching from a data source.
2. Core search service:
in the technical scheme provided by the invention, the core search service adopts a distributed search structure, and the realization mechanism mainly comprises the following steps:
1) distributed search
Dividing the original large index into a plurality of small indexes (the small indexes are not of a certain scale and are called as small indexes which are relative to the whole index in the prior art), integrating the results returned from the small indexes by the system, and feeding back the results to the client; if a single search is fast enough, but only to increase the overall number of requests/second, then an attempt to duplicate with the index may be made.
2) Multi-core search service
In order to achieve the greatest degree of customization, a configuration scheme of a multi-core search mode is provided. Multiple cores allow independent multiple configurations and indexes in a single search service instance, with each different application having its own configuration and schema files, but with a unified administration. Each index is independent, managing them within an instance. A new core may be built to add a new index and even one core may replace another without closing the Servlet container.
Claims (2)
1. A large-scale two-dimensional and three-dimensional space data search engine based on network, the structure comprises: the system comprises a space indexer module, a space server module and an application program interface module; the method is characterized in that:
the spatial indexer module is used for establishing a spatial index record database;
the space server module comprises a queue response processor and a solr search server module;
the application program interface module comprises a query interface and a feedback interface;
the output end of the spatial index record database and the query interface of the application program interface module are respectively connected with the solr search server module, and the output end of the solr search server module is connected with the feedback interface of the application program interface module and is output to the outside through the feedback interface.
2. The web-based massive two-dimensional and three-dimensional spatial data search engine of claim 1, wherein: and the solr search server analyzes the timeliness and the regionality of the two-dimensional and three-dimensional spatial data, and adds the time and region information into the index record database or establishes a connection with the index record database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910139222.4A CN111611328A (en) | 2019-02-26 | 2019-02-26 | Large-scale two-dimensional and three-dimensional spatial data search engine based on network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910139222.4A CN111611328A (en) | 2019-02-26 | 2019-02-26 | Large-scale two-dimensional and three-dimensional spatial data search engine based on network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111611328A true CN111611328A (en) | 2020-09-01 |
Family
ID=72199735
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910139222.4A Pending CN111611328A (en) | 2019-02-26 | 2019-02-26 | Large-scale two-dimensional and three-dimensional spatial data search engine based on network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111611328A (en) |
-
2019
- 2019-02-26 CN CN201910139222.4A patent/CN111611328A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109284338B (en) | Satellite remote sensing big data optimization query method based on mixed index | |
| US9798772B2 (en) | Using persistent data samples and query-time statistics for query optimization | |
| CN106709067B (en) | Multisource heterogeneous space data circulation method based on Oracle database | |
| CN103491187B (en) | A kind of big data united analysis processing method based on cloud computing | |
| CN106933833B (en) | Method for quickly querying position information based on spatial index technology | |
| CN105912666B (en) | A kind of mixed structure data high-performance storage of facing cloud platform, querying method | |
| Zou et al. | Pareto-based dominant graph: An efficient indexing structure to answer top-k queries | |
| CN112000773B (en) | Search engine technology-based data association relation mining method and application | |
| CN107506464A (en) | A kind of method that HBase secondary indexs are realized based on ES | |
| CN102200979A (en) | Distributed parallel information retrieval system and distributed parallel information retrieval method | |
| CN108009265B (en) | Spatial data indexing method in cloud computing environment | |
| CN106599052A (en) | Data query system based on ApacheKylin, and method thereof | |
| CN103226608A (en) | Parallel file searching method based on folder-level telescopic Bloom Filter bit diagram | |
| CN117112691A (en) | Storage method of big data-oriented multi-storage engine database | |
| Alsubaiee et al. | Asterix: scalable warehouse-style web data integration | |
| He et al. | JUST-Traj: A distributed and holistic trajectory data management system | |
| CN112540987A (en) | Big data management system of distribution and utilization electricity based on data mart | |
| Cassavia et al. | Data preparation for tourist data big data warehousing | |
| CN111611328A (en) | Large-scale two-dimensional and three-dimensional spatial data search engine based on network | |
| Alkathiri et al. | Geo-spatial big data mining techniques | |
| CN113806376A (en) | Index construction method and device | |
| Li et al. | SP-phoenix: a massive spatial point data management system based on phoenix | |
| CN107480220B (en) | Rapid text query method based on online aggregation | |
| CN110990430A (en) | Large-scale data parallel processing system | |
| Ming et al. | Research on multidimensional analysis method of drilling information based on Hadoop |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200901 |