CN108182291A - It is a kind of based on NoSql databases with the data interaction mechanism of relevant database - Google Patents
It is a kind of based on NoSql databases with the data interaction mechanism of relevant database Download PDFInfo
- Publication number
- CN108182291A CN108182291A CN201810089032.1A CN201810089032A CN108182291A CN 108182291 A CN108182291 A CN 108182291A CN 201810089032 A CN201810089032 A CN 201810089032A CN 108182291 A CN108182291 A CN 108182291A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- interaction mechanism
- relevant database
- interaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 17
- 230000007246 mechanism Effects 0.000 title claims abstract description 12
- 230000008859 change Effects 0.000 claims description 7
- 238000000034 method Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 2
- 230000005012 migration Effects 0.000 claims description 2
- 238000013508 migration Methods 0.000 claims description 2
- 239000000284 extract Substances 0.000 claims 1
- 230000008676 import Effects 0.000 claims 1
- 238000010937 topological data analysis Methods 0.000 abstract description 2
- 238000007405 data analysis Methods 0.000 abstract 1
- 230000000694 effects Effects 0.000 description 1
- 230000003362 replicative effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Multi Processors (AREA)
Abstract
The invention discloses it is a kind of based on NoSql databases with the data interaction mechanism of relevant database, be real-time, validity and the integrity issue for solving interaction between cross-platform data.A kind of NoSql and the middleware of relevant database synchrodata are developed, by active request GIS service, GIS service receives to ask and return request data.After middleware obtains data, and topological analysis and data check are carried out to data, related data information write-in MongoDB databases are extracted after verifying no problem.
Description
Technical field
Present invention relates particularly to it is a kind of based on NoSql databases with the data interaction mechanism of relevant database, belong to letter
Cease technical field.
Background technology
Relevant database needs to create index to accelerate inquiry velocity, just has to change to increase necessary field
Become table structure, in order to carry out these processing, need to carry out table shared locking, data change, update, are inserted into, delete during this
Deng can not all carry out.Relevant database is to carry out digital independent using special sql like language, it is needed to SQL languages
Speech is parsed, and is bad to simply inquiring quick return structure.Also have simultaneously to additional as the locking and unlock of table
Expense.NoSql databases are only applied in specific area, substantially without complicated processing, but it exactly compensate for before institute
The shortcoming for the relevant database enumerated.
Invention content
After middleware obtains data, carry out data conversion into data object, analyze data object topological relation and
Data check.It verifies in the map file table that data information out of question is written in NoSql databases, problematic data are stored in
In wrong data table.Interaction mechanism is divided into two ways：
1st, full dose data interaction：Full dose is extracted to be replicated similar to Data Migration or data, it is by the table in data source or regards
The data of figure are all extracted from database, then carry out subsequent conversion and loading operation.Full dose extraction can use number
It is completed according to the mode for replicating, importing or backing up, realization mechanism is fairly simple.
2nd, incremental data interacts：Change catching method by analyzing more existing database, in addition a kind of task based access control
Control table method.This method increases the relevant informations such as task, group and type on the basis of control table method, while with net change
Extraction assembly is measured as auxiliary, can more effectively be suitable for the data exchange of heterogeneous database between multinode.Incremented data
Piecemeal handles model, thought and the Data Stream Processing thought of this Model Fusion batch processing, periodically to increasing in a period of time
Data carry out batch processing.
Advantageous effect
The present invention program solves real-time, validity and the integrity issue of interaction between cross-platform data.
Description of the drawings
Fig. 1 for it is a kind of based on NoSql databases with the data interaction mechanism of relevant database.
Specific embodiment
Invention provides it is a kind of based on NoSql databases with the data interaction mechanism of relevant database, solve across
Real-time, validity and the integrity issue of interaction between platform data.Develop a kind of NoSql and relevant database synchrodata
Middleware, by active request GIS service, GIS service receives to ask simultaneously to return to request data.Middleware obtains data
Afterwards, and to data topological analysis and data check are carried out, related data information write-in MongoDB numbers is extracted after verifying no problem
According to library.Specific implementation is as follows：
Step 1) obtains agent data service by calling the components of PMS plateform systems.
Step 2) request data agency services obtain all devices data in data source, will obtain data and pass through centre
Part is verified, is converted, is assembled, and data information is being written to MongoDB databases, is carrying out data full dose interaction.
The daily timed increase data updates of step 3) call agent data service, and comparing acquisition by data increases newly, repaiies
Change data, verified, converted and assembled by middleware, in data information update to MongoDB databases.
Step 4) according to acquisition delete data information delete MongoDB databases in data, daily right place into
Row data update realizes real-time, validity and the integrality of interaction between platform data.
Claims (2)
1. it is a kind of based on NoSql databases with the data interaction mechanism of relevant database, which is characterized in that middleware obtains
Access carries out data conversion into data object, analyzes the topological relation and data check of data object after；It verifies out of question
Data information is written in the map file table in NoSql databases, and problematic data are stored in wrong data table；Interaction mechanism
It is divided into two ways：
1), full dose data interaction：Full dose is extracted to be replicated similar to Data Migration or data, by the table or the number of view in data source
According to all being extracted from database, then carry out subsequent conversion and loading operation；Full dose, which extracts, can use data to replicate,
It imports or the mode of backup is completed, realization mechanism is fairly simple；
2), incremental data interacts：Change catching method by analyzing more existing database, in addition a kind of control of task based access control
Tabulating method is increased task, group and type of relevant information on the basis of control table method, while is made with net change amount extraction assembly
For auxiliary, it can more effectively be suitable for the data exchange of heterogeneous database between multinode.
2. according to the data interaction mechanism described in claims 1, which is characterized in that it is described 2) using incremented data piecemeal at
Manage model, thought and the Data Stream Processing thought of this Model Fusion batch processing, periodically to data increased in a period of time
Carry out batch processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810089032.1A CN108182291A (en) | 2018-01-30 | 2018-01-30 | It is a kind of based on NoSql databases with the data interaction mechanism of relevant database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810089032.1A CN108182291A (en) | 2018-01-30 | 2018-01-30 | It is a kind of based on NoSql databases with the data interaction mechanism of relevant database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108182291A true CN108182291A (en) | 2018-06-19 |
Family
ID=62551764
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810089032.1A Pending CN108182291A (en) | 2018-01-30 | 2018-01-30 | It is a kind of based on NoSql databases with the data interaction mechanism of relevant database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108182291A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109271452A (en) * | 2018-10-19 | 2019-01-25 | 武汉达梦数据库有限公司 | DB2 database data synchronization updating method and apparatus |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573107A (en) * | 2015-02-02 | 2015-04-29 | 国家计算机网络与信息安全管理中心江苏分中心 | Network security application NoSQL database and relational database fusion interface method |
| CN105512939A (en) * | 2015-12-04 | 2016-04-20 | 中国建设银行股份有限公司 | Foreign exchange transaction-related data storage and query method, declaration method and system |
| CN106372176A (en) * | 2016-08-30 | 2017-02-01 | 东华大学 | Method for supporting uniform SQL (Structured Query Language) query of embedded document |
-
2018
- 2018-01-30 CN CN201810089032.1A patent/CN108182291A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573107A (en) * | 2015-02-02 | 2015-04-29 | 国家计算机网络与信息安全管理中心江苏分中心 | Network security application NoSQL database and relational database fusion interface method |
| CN105512939A (en) * | 2015-12-04 | 2016-04-20 | 中国建设银行股份有限公司 | Foreign exchange transaction-related data storage and query method, declaration method and system |
| CN106372176A (en) * | 2016-08-30 | 2017-02-01 | 东华大学 | Method for supporting uniform SQL (Structured Query Language) query of embedded document |
Non-Patent Citations (1)
| Title |
|---|
| 王培建: "云计算环境下大规模数据存储技术研究"" * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109271452A (en) * | 2018-10-19 | 2019-01-25 | 武汉达梦数据库有限公司 | DB2 database data synchronization updating method and apparatus |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103914564B (en) | Method and system for updating and warehousing urban pipelines on basis of ArcGIS (geographic information system) | |
| CN103699693B (en) | A kind of data quality management method based on metadata and system | |
| CN103914526A (en) | Interface method and interface device for SAP ERP (enterprise resource planning) system and ORACLE ERP system | |
| CN103473342A (en) | Report data generating method and system | |
| CN104991955A (en) | Method and system for automatically constructing template library | |
| CN111897863A (en) | Multi-source heterogeneous data fusion and convergence method | |
| CN106557470A (en) | data extraction method and device | |
| Matthes et al. | Testing & quality assurance in data migration projects | |
| CN111598382A (en) | Service chain data checking method and device based on block chain, storage medium and electronic equipment | |
| CN103559025A (en) | Software refactoring method through clustering | |
| CN104767795A (en) | LTE MRO data statistical method and system based on HADOOP | |
| CN111881105B (en) | Labeling model of business data and model training method thereof | |
| CN109039710A (en) | Route data auditing method, device, server and storage medium | |
| Fernando et al. | Hybrid framework for master data management | |
| CN115086277A (en) | Identification management method, system, terminal equipment and storage medium | |
| CN105069138A (en) | Association analysis system and method | |
| CN108182291A (en) | It is a kind of based on NoSql databases with the data interaction mechanism of relevant database | |
| CN112948473A (en) | Data processing method, device and system of data warehouse and storage medium | |
| CN114611859A (en) | Method and system for intelligent recruitment of software platform | |
| CN112835877A (en) | Epidemic situation big data cleaning method for public burst transactions | |
| CN107766033A (en) | A kind of quick demand method for splitting based on problem framework | |
| US20170270111A1 (en) | System migration using selective envelope management | |
| TW201327433A (en) | Design method of flexible signature process | |
| Jeong et al. | A study on effective requirement traceability management method in implementation project of information system | |
| KR102562186B1 (en) | System for providing rental property management based official letter sending service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 225009 Weiyang Road, Jiangsu, China, No. 179, No. Applicant after: YANGZHOU POWER SUPPLY BRANCH OF STATE GRID JIANGSU ELECTRIC POWER Co.,Ltd. Applicant after: XIAMEN GREAT POWER GEO INFORMATION TECHNOLOGY Co.,Ltd. Applicant after: STATE GRID CORPORATION OF CHINA Address before: No. 179, Weiyang Road, Yangzhou City, Yangzhou City, Jiangsu Province Applicant before: YANGZHOU POWER SUPPLY BRANCH OF STATE GRID JIANGSU ELECTRIC POWER Co.,Ltd. Applicant before: XIAMEN GREAT POWER GEO INFORMATION TECHNOLOGY Co.,Ltd. Applicant before: State Grid Corporation of China |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180619 |