CN105224651A - A kind of infosystem intranet and extranet database optimizing method based on read and write abruption - Google Patents
A kind of infosystem intranet and extranet database optimizing method based on read and write abruption Download PDFInfo
- Publication number
- CN105224651A CN105224651A CN201510638550.0A CN201510638550A CN105224651A CN 105224651 A CN105224651 A CN 105224651A CN 201510638550 A CN201510638550 A CN 201510638550A CN 105224651 A CN105224651 A CN 105224651A
- Authority
- CN
- China
- Prior art keywords
- database
- intranet
- read
- data
- write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A kind of infosystem intranet and extranet database optimizing method based on read and write abruption.It comprises intranet and extranet database purchase subregion, read-write point storehouse; Subregion is merged to the multiple storehouse of same instance after shining outer net soon; Optimum distribution formula data base querying; Optimum distribution formula database write is served; The stages such as record optimization daily record.The beneficial effect of the infosystem intranet and extranet database optimizing method based on read and write abruption provided by the invention: present inventive concept stores with data scatter, namely inquiry concurrence performance rope of serving as theme effectively has organized mass data, improve the query performance of mass data, be to present stage mass data, high concurrent searching platform creative optimal design.
Description
Technical field
The invention belongs to Enterprise Informatization Technology field, particularly relate to a kind of infosystem intranet and extranet database optimizing method based on read and write abruption.
Background technology
Under the overall background that informationization is popularized day by day, the maximization of infosystem and complicated be inexorable trend.Infosystem large-scale at present, mostly adopts read and write abruption technology in database aspect, a master library and multiple from storehouse.Master library is responsible for Data Update and real time data inquiry, is responsible for non-real-time data inquiry from storehouse.Because in the application of reality, database is all read to write that few (frequency namely reading data is high more, more the frequency of new data is relatively less), and system is more complicated, system data is more, query statement is more complicated, read the consuming time longer of data, the CPU taking database server is also more, thus reduces system performance, affects Consumer's Experience.And separation is just merely carried out to improve coincidence in read-write storehouse by existing data base read-write isolation technics, and it is inconvenient not solve data syn-chronization such as between intranet and extranet, the problem such as cause inquiry slow that body excessive and query statement in storehouse is complicated.
Summary of the invention
In order to solve the problem, the object of the present invention is to provide a kind of infosystem intranet and extranet database optimizing method based on read and write abruption.
In order to achieve the above object, the infosystem intranet and extranet database optimizing method based on read and write abruption provided by the invention comprises the following step carried out in order:
Step one: the S01 stage in intranet and extranet database purchase subregion, read-write point storehouse: infosystem intranet and extranet database is carried out read and write abruption, and data partition is stored;
Step 2: merge the S02 stage of subregion to the multiple storehouse of same instance after shining outer net soon: by step one be separated intranet data storehouse shine soon in step one be separated outer grid database in, then by the database combining under subregion different in step one in same instance;
Step 3: the S03 stage of Optimum distribution formula data base querying: by query decomposition in large data sets be the intranet and extranet database that can mate in step 2 and several small data set that inherent logic is associated carry out concurrent inquiry;
Step 4: the S04 stage of Optimum distribution formula database write service: the Optimizing Queries that integrating step three obtains, different data being write by concurrent program writing and the partitioning strategies in step one is assigned in different databases, finally obtains the database of read and write abruption and the optimization method of coupling read and write abruption database;
Step 5: the S05 stage of record optimization daily record: the optimization execution result daily record of the optimizing process daily record in the snapshot processes in recording step two and step 3, step 4 and step 3, step 4.
In step 3, the data base querying that CLIENT PROGRAM is submitted to is mainly resolved to many relatively simple query sentence of database by described distributed networks database query, and initiate inquiry request to the database be stored on different subregion simultaneously, query performance is promoted by concurrent speed advantage and the short advantage of simple statement query time, after having inquired about result set, Query Result has been organized into a data set and has returned to CLIENT PROGRAM by engine.
The beneficial effect of the infosystem intranet and extranet database optimizing method based on read and write abruption provided by the invention:
Present inventive concept stores with data scatter, and namely inquiry concurrence performance rope of serving as theme effectively has organized mass data, improves the query performance of mass data, be to present stage mass data, high concurrent searching platform creative optimal design.
Accompanying drawing explanation
Fig. 1 is the infosystem intranet and extranet database optimizing method process flow diagram based on read and write abruption provided by the invention.
Fig. 2 is infosystem intranet data storehouse horizontal partitioning schematic diagram.
Fig. 3 is the outer grid database schematic diagram of infosystem.
Embodiment
Below in conjunction with the drawings and specific embodiments, the infosystem intranet and extranet database optimizing method based on read and write abruption provided by the invention is described in detail.
As shown in Figure 1, the infosystem intranet and extranet database optimizing method based on read and write abruption provided by the invention comprises the following step carried out in order:
Step one: the S01 stage in intranet and extranet database purchase subregion, read-write point storehouse: infosystem intranet and extranet database is carried out read and write abruption, and data partition is stored;
Step 2: merge the S02 stage of subregion to the multiple storehouse of same instance after shining outer net soon: by step one be separated intranet data storehouse shine soon in step one be separated outer grid database in, then by the database combining under subregion different in step one in same instance;
Step 3: the S03 stage of Optimum distribution formula data base querying: by query decomposition in large data sets be the intranet and extranet database that can mate in step 2 and several small data set that inherent logic is associated carry out concurrent inquiry;
Step 4: the S04 stage of Optimum distribution formula database write service: the Optimizing Queries that integrating step three obtains, different data being write by concurrent program writing and the partitioning strategies in step one is assigned in different databases, finally obtains the database of read and write abruption and the optimization method of coupling read and write abruption database;
Step 5: the S05 stage of record optimization daily record: the optimization execution result daily record of the optimizing process daily record in the snapshot processes in recording step two and step 3, step 4 and step 3, step 4.
In step one, described outer grid database generally includes system-critical data and sensitive data, for adaptation data volume expansion needs and high-performance complete write and the inquiry of database; First the database of infosystem intranet and extranet is carried out read and write abruption, and data partition is stored; It is large for solving original infosystem write data volume that data base read-write is separated, and read operation is shared separately in a database, to avoid read operation to writing taking of database resource.Partitions of database stores to be intended to solve and stores, large-scale data dispersion to promote Query Efficiency.
In step 2, described snapshot creates the read-only static view with intranet data storehouse consistent affairs, returned on outer grid database by snapmap (FlashCopy), solve tertiary-structure network problem, set up effective passage that intranet data storehouse is updated to outer grid database, and by the database combining under different subregion in same instance, so that the one query of CLIENT PROGRAM can complete on of a load balancing machine.The inquiry service of infosystem outer net has been come based on snapshot completely.
In step 3, described distributed networks database query is the key promoting query performance.Query decomposition in large data sets is creatively the concurrent inquiry of several small data set that logic is relevant by the present invention, allows the time of one query significantly reduce in theory: the maximum query time of the query time=small data set after optimization.Hoc queries result after inquiry, with transparent single query feedback to CLIENT PROGRAM, can promote search efficiency greatly.
In step 4, described distributed data base writes service, is the partitioning strategies in conjunction with the solution formulation of data partitioned storage, the concurrent program writing for different subregion provided.The write of different data is assigned in different databases in conjunction with partitioning strategies by concurrent program writing.Whole process is transparent for CLIENT PROGRAM.
In step one, the method for designing of described intranet and extranet database is:
1.1) intranet data storehouse design
As shown in Figure 2, the intranet data storehouse of infosystem comprises critical data and sensitive data usually, exist magnanimity inquiry and magnanimity write this two parts business demand, and owing to there is a large amount of database write operations in intranet data storehouse, and data-base recording content is more, therefore the structural design in intranet data storehouse adopts the mode of database horizontal partitioning and carries out read and write abruption;
1.2) outer net database design
As shown in Figure 3, outer net database purchase critical data and sensitive data can inquire about version by outer net, primarily of the exchanges data transmission of Intranet, when inquiring about for infosystem; The present invention reaches more than 100,000 times according to the data access amount of infosystem every day, and data synchronously can cause the interruption of database certain hour to use from Intranet to outer net, are divided into multiple stage Query Database to carry out load balancing outer grid database.Every platform database server runs multiple database instance, a partition data in the corresponding intranet data storehouse of each database instance.
In step 3, the data base querying that CLIENT PROGRAM is submitted to is mainly resolved to many relatively simple query sentence of database by described distributed networks database query, and initiate inquiry request to the database be stored on different subregion simultaneously, query performance is promoted by concurrent speed advantage and the short advantage of simple statement query time, after having inquired about result set, Query Result has been organized into a data set and has returned to CLIENT PROGRAM by engine.Whole process is transparent for CLIENT PROGRAM, and CLIENT PROGRAM just have submitted a data base querying and returns a data set.About the initiation mode of data base querying request, storing process is passed through in present stage consideration.Based on the data base querying request of storing process, because parameter is clear and definite, the former data of subregion can be encapsulated well, so the repertoire of distributed data base can be realized well.Result returns to client in the mode of List<Entity>.
In step 4, the core mechanism that described distributed data base writes service mainly comprises two aspects:
(1) data store strategy, data all in guarantee system can be distributed on different subregions substantially fifty-fifty, and Strategy Design gos deep into gradual perfection by demand thereupon.
(2) integration across database affairs, when CLIENT PROGRAM submits to multiple database write operations to cover different partitions of database, need the affairs problem considering integration across database, in a database, the write of integration across database is unaffected to performance.Whole to write service be transparent to CLIENT PROGRAM, and CLIENT PROGRAM only needs write-in program to submit to database write service, and data are assigned in the database write operations of different subregion on backstage by writing service automatically, after complete operation, can return success status.
Present inventive concept stores with data scatter, and inquiry concurrence performance is served as theme rope, has namely effectively organized mass data, has improved the query performance of mass data, be to present stage mass data, high concurrent searching platform creative optimal design.
The present invention is separated into basis with data base read-write, in conjunction with the actual conditions of enterprise information system tertiary-structure network, the creative data scatter that proposes stores, inquires about the design philosophys such as concurrent, unification is write, the infosystem intranet and extranet database optimizing method of Erecting and improving, realizes the high stability of system, high maintainability, high security, high-performance.
Based on the infosystem intranet and extranet database design of above-mentioned optimization method, first the performance need of current infosystem business service is met, secondly the multi-level optimization such as data base architecture, database design and database manipulation is passed through, infosystem can be adapted in the data geometric sense growth of " 13 " period, for the research and extension of large data platform is groped in advance.
Claims (2)
1. based on an infosystem intranet and extranet database optimizing method for read and write abruption, it is characterized in that: the described infosystem intranet and extranet database optimizing method based on read and write abruption comprises the following step carried out in order:
Step one: the S01 stage in intranet and extranet database purchase subregion, read-write point storehouse: infosystem intranet and extranet database is carried out read and write abruption, and data partition is stored;
Step 2: merge the S02 stage of subregion to the multiple storehouse of same instance after shining outer net soon: by step one be separated intranet data storehouse shine soon in step one be separated outer grid database in, then by the database combining under subregion different in step one in same instance;
Step 3: the S03 stage of Optimum distribution formula data base querying: by query decomposition in large data sets be the intranet and extranet database that can mate in step 2 and several small data set that inherent logic is associated carry out concurrent inquiry;
Step 4: the S04 stage of Optimum distribution formula database write service: the Optimizing Queries that integrating step three obtains, different data being write by concurrent program writing and the partitioning strategies in step one is assigned in different databases, finally obtains the database of read and write abruption and the optimization method of coupling read and write abruption database;
Step 5: the S05 stage of record optimization daily record: the optimization execution result daily record of the optimizing process daily record in the snapshot processes in recording step two and step 3, step 4 and step 3, step 4.
2. the infosystem intranet and extranet database optimizing method based on read and write abruption according to claim 1, it is characterized in that: in step 3, the data base querying that CLIENT PROGRAM is submitted to is mainly resolved to many relatively simple query sentence of database by described distributed networks database query, and initiate inquiry request to the database be stored on different subregion simultaneously, query performance is promoted by concurrent speed advantage and the short advantage of simple statement query time, after having inquired about result set, Query Result has been organized into a data set and has returned to CLIENT PROGRAM by engine.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510638550.0A CN105224651A (en) | 2015-09-30 | 2015-09-30 | A kind of infosystem intranet and extranet database optimizing method based on read and write abruption |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510638550.0A CN105224651A (en) | 2015-09-30 | 2015-09-30 | A kind of infosystem intranet and extranet database optimizing method based on read and write abruption |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105224651A true CN105224651A (en) | 2016-01-06 |
Family
ID=54993619
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510638550.0A Pending CN105224651A (en) | 2015-09-30 | 2015-09-30 | A kind of infosystem intranet and extranet database optimizing method based on read and write abruption |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105224651A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106649828A (en) * | 2016-12-29 | 2017-05-10 | 中国银联股份有限公司 | Data query method and system |
| CN107679931A (en) * | 2017-08-17 | 2018-02-09 | 平安科技（深圳）有限公司 | Method, apparatus, storage medium and the terminal of order asynchronous process |
| CN112612831A (en) * | 2020-12-14 | 2021-04-06 | 南方电网数字电网研究院有限公司 | Operation and maintenance flow management performance optimization method of early warning system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104036007A (en) * | 2014-06-23 | 2014-09-10 | 北京京东尚科信息技术有限公司 | Method and device for querying distributed databases |
| CN104750729A (en) * | 2013-12-30 | 2015-07-01 | 中国移动通信集团公司 | Data management method and system based on journal file |
-
2015
- 2015-09-30 CN CN201510638550.0A patent/CN105224651A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104750729A (en) * | 2013-12-30 | 2015-07-01 | 中国移动通信集团公司 | Data management method and system based on journal file |
| CN104036007A (en) * | 2014-06-23 | 2014-09-10 | 北京京东尚科信息技术有限公司 | Method and device for querying distributed databases |
Non-Patent Citations (1)
| Title |
|---|
| 刘国斌等: "网络环境下的财务数据实时查询系统的设计和实施", 《中国管理信息化》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106649828A (en) * | 2016-12-29 | 2017-05-10 | 中国银联股份有限公司 | Data query method and system |
| CN106649828B (en) * | 2016-12-29 | 2019-12-24 | 中国银联股份有限公司 | Data query method and system |
| CN107679931A (en) * | 2017-08-17 | 2018-02-09 | 平安科技（深圳）有限公司 | Method, apparatus, storage medium and the terminal of order asynchronous process |
| CN112612831A (en) * | 2020-12-14 | 2021-04-06 | 南方电网数字电网研究院有限公司 | Operation and maintenance flow management performance optimization method of early warning system |
| CN112612831B (en) * | 2020-12-14 | 2023-10-17 | 南方电网数字电网研究院有限公司 | Operation and maintenance flow management performance optimization method of early warning system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11169981B2 (en) | Managing data with flexible schema | |
| US20220067025A1 (en) | Ordering transaction requests in a distributed database according to an independently assigned sequence | |
| US9792327B2 (en) | Self-described query execution in a massively parallel SQL execution engine | |
| US8935232B2 (en) | Query execution systems and methods | |
| US9501550B2 (en) | OLAP query processing method oriented to database and HADOOP hybrid platform | |
| CN103942342B (en) | Memory database OLTP and OLAP concurrency query optimization method | |
| US10387422B2 (en) | Incremental and collocated redistribution for expansion of online shared nothing database | |
| WO2016167999A1 (en) | Geo-scale analytics with bandwidth and regulatory constraints | |
| US9075858B2 (en) | Non-disruptive data movement and node rebalancing in extreme OLTP environments | |
| US20190018880A1 (en) | Distributed system with accelerator and catalog | |
| US9971820B2 (en) | Distributed system with accelerator-created containers | |
| WO2015152868A1 (en) | Parallelizing sql on distributed file systems | |
| CN108073696B (en) | GIS application method based on distributed memory database | |
| US11068504B2 (en) | Relational database storage system and method for supporting fast query processing with low data redundancy, and method for query processing based on the relational database storage method | |
| Cubukcu et al. | Citus: Distributed postgresql for data-intensive applications | |
| Borkar et al. | Have your data and query it too: From key-value caching to big data management | |
| US20180300147A1 (en) | Database Operating Method and Apparatus | |
| CN105224651A (en) | A kind of infosystem intranet and extranet database optimizing method based on read and write abruption | |
| CN114297173A (en) | Knowledge graph construction method and system for large-scale mass data | |
| CN108228725B (en) | GIS application system based on distributed database | |
| US11940972B2 (en) | Execution of operations on partitioned tables | |
| US11275737B2 (en) | Assignment of objects to processing engines for efficient database operations | |
| US10572483B2 (en) | Aggregate projection | |
| US11640399B2 (en) | Database query processing for data in a remote data store | |
| US20220197902A1 (en) | Range partitioned in-memory joins |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160106 |
|
| RJ01 | Rejection of invention patent application after publication |