CN103744979A - Automatic dividing method for domino database based system - Google Patents
Automatic dividing method for domino database based system Download PDFInfo
- Publication number
- CN103744979A CN103744979A CN201410016631.2A CN201410016631A CN103744979A CN 103744979 A CN103744979 A CN 103744979A CN 201410016631 A CN201410016631 A CN 201410016631A CN 103744979 A CN103744979 A CN 103744979A
- Authority
- CN
- China
- Prior art keywords
- storehouse
- database
- minute
- minute storehouse
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 20
- 230000000694 effects Effects 0.000 claims description 4
- 241001269238 Data Species 0.000 claims description 3
- 238000013461 design Methods 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000012010 growth Effects 0.000 description 2
- 235000006508 Nelumbo nucifera Nutrition 0.000 description 1
- 235000006510 Nelumbo pentapetala Nutrition 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007773 growth pattern Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides an automatic dividing method for a domino database based system. The method includes establishing a dividing strategy configuring module, a dynamic dividing engine, an index database and a scheduling module; allowing the dynamic dividing engine and the dividing strategy configuring module to be in charge of dynamic division of the database; allowing the index database and the scheduling module to be in charge of establishing and reading a compatible traditional manner after database division. According to the method, through data distributed storage and request sharing, the performance problem of the domino database based system is solved, and access speed of the database is increased.
Description
Technical field
The present invention relates to a kind of system based on domino database method in minute storehouse automatically.
Background technology
IBM(International Business Machine Corporation (IBM)) the office automation Lotus software under, (file suffixes is Notes database file: .NSF), this database file contains the data of application program, and its structure is comprised of list, territory, file, view and other Presentation Function parts (such as omniselector and database icon).The data that provide according to official, there is following restriction in Notes database: the largest amount of database: i.e. maximum OS file size limitation (being 64GB to the maximum); The largest amount of textview field: 15KB; The largest amount of single paragraph: 64KB in RTF textview field; In a hierarchical view, there is what response: 31 grades; In every one-level, there are several documents: 300,000 documents; A view index maximum can be: 130MB.
It is linear that Notes database connects.Along with data line becomes large, Notes database response institute's time spent is exponent increase.In Notes database and in relational database, the performance difference of connection table may be more than thousand times, especially true when using very large database.According to actual use experience, a Notes database is meeting under the prerequisite of system general performance and stability requirement, and the restriction of the size of database does not far reach official standard value.The document data of a general database reaches 20W when above, and system will become unstable or occur various faults.Data base view represent that duration with search response all can become oversize and the requirement that can not meet business.
The IT application in management system of enterprise-level, usually faces the document of big data quantity, and tens million of even more than one hundred million document sizes need Storage and Processing.A feasible and ripe scheme is divided storehouse to database exactly, but in original small data quantity situation, the mode of database being carried out once to manual minute storehouse every 1 year can also be dealt with problems, just no longer applicable under existing mass data growth pattern, need the design strategy in minute storehouse automatically, by task regularly, database is scanned to management, in needs, generate new database replace old storehouse, and the process need in minute storehouse guarantees the persistent service of system, before and after minute storehouse on system without impact.
Traditional Domino database use-pattern as shown in Figure 1, is all directly access and creates for the access of Domino database.Along with the growth of data, the data volume of Domino database (business library) can be increasing, and performance can decline gradually, even causes system crash.So traditional Domino database use-pattern cannot support the fast Large Information Systems of Data Growth.
Summary of the invention
The technical problem to be solved in the present invention, is to provide a kind of system based on domino database method in minute storehouse automatically, has solved the system performance problems based on domino database, has improved the access speed to database.
The present invention is achieved in that a kind of system based on domino database method in minute storehouse automatically, and described method is specially:
Set up one minute storehouse strategy configuration module, configure minute library information behind minute storehouse strategy configuration information and record minute storehouse;
Set up a dynamic minute storehouse engine, this dynamically minute storehouse engine be responsible for database to monitor the establishment with database, system is periodically processed the database transfiniting in a system minute storehouse according to a minute storehouse strategy configuration information for minute storehouse strategy configuration module by dynamic minute storehouse engine, and by dividing minute library information behind strategy configuration module record minute storehouse, storehouse;
Set up an index database, during minute storehouse is processed, the data of database increase, delete, change operation real-time synchronization in this index database, and index database is deposited critical field, is integrated with all document datas, and stores the index information in each divided data storehouse;
Set up a scheduler module, system is when needs create new data, and scheduler module reads minute library information behind minute storehouse, and according to index database module, data creation is carried out in the divided data storehouse of not transfiniting; Meanwhile, when system needs visit data, scheduler module reads required data by index database module in corresponding divided data storehouse.
Tool of the present invention has the following advantages: the present invention breaks the original module single database of domino or the manual pattern in minute storehouse, adopts the strategy in minute storehouse automatically, and use system self manages the domino database in system; The business function of modernization system can adapt to the situation in data minute storehouse, and minute storehouse is to meet big data quantity storage and situation about calculating under application system automatically; Improved the access speed to database.
Accompanying drawing explanation
Fig. 1 is the schematic diagram of the Domino database use-pattern of prior art.
Fig. 2 is the theory diagram of implementation method of the present invention.
Embodiment
Refer to shown in Fig. 2, the present invention is a kind of system based on domino database method in minute storehouse automatically, and described method is specially:
Set up one minute storehouse strategy configuration module, configure minute library information behind minute storehouse strategy configuration information and record minute storehouse;
Set up a dynamic minute storehouse engine, this dynamically minute storehouse engine be responsible for database to monitor the establishment with database, system is periodically processed the database transfiniting in a system minute storehouse according to a minute storehouse strategy configuration information for minute storehouse strategy configuration module by dynamic minute storehouse engine, and by dividing minute library information behind strategy configuration module record minute storehouse, storehouse;
Wherein, system is periodically processed the database transfiniting in a system minute storehouse according to a minute storehouse strategy configuration information for minute storehouse strategy configuration module by dynamic minute storehouse engine, be specially: minute storehouse strategy configuration information that dynamically minute storehouse engine reads configuration is monitored database, judge that whether physics size in database or number of documents are over threshold value; No, do not divide storehouse to process, be, to reaching the divided data storehouse of minute storehouse threshold value, be labeled as historical minute storehouse (no longer newly-increased data), increase the divided data storehouse of a same design sky simultaneously newly, and be labeled as movable minute storehouse.
Set up an index database, during minute storehouse is processed, the data of database increase, delete, change operation real-time synchronization in this index database, and index database is deposited critical field (being major key field), is integrated with all document datas, and stores the index information in each divided data storehouse; The actual database and the document that according to the index information of index database, point to when the view in database and Search Results actual access like this conduct interviews;
Set up a scheduler module, system is when needs create new data, and scheduler module reads minute library information behind minute storehouse, and according to index database module, data creation is carried out in the divided data storehouse of not transfiniting; Meanwhile, when system needs visit data, scheduler module reads required data by index database module in corresponding divided data storehouse.
In the present invention, described domino database is document data storehouse, and the evaluating standard whether excessive for database is: the physics size of database and the number of documents size of database; The number of documents that the physics size that the tactful configuration information in described configuration minute storehouse is configuration database reaches in first threshold minute storehouse or database reaches Second Threshold minute storehouse; The standard (threshold value) in minute storehouse can be that wherein any one index reaches threshold value, or any one index reaches threshold value, or two indexs all reach threshold value.Be tetra-kinds of A, B, A or B, A and B.Within described minute, library information comprises: the quantity in historical minute storehouse in database is, title, the path corresponding to each minute storehouse in the quantity in activity minute storehouse, each minute storehouse; Described history minute storehouse is the divided data storehouse being filled with, the divided data storehouse of movable minute storehouse for not being filled with; Minute library information carries out updating maintenance after changing by dynamic minute storehouse engine in minute storehouse.
Described index database implementation is Domino storehouse (a small amount of field can be carried more number of files), relevant database or NoSql database; Described index information comprises key business field, path, divided data storehouse, divided data title and document id etc.Described scheduler module is divided behind storehouse at database, when creating new data, need to select one of them database to create data; Read minute library information in minute storehouse strategy configuration module, an activity minute storehouse of choosing wherein creates new document.If there is a plurality of activities minute storehouse, not only need to select and enliven storehouse, also to use random or polling mode to select one with the whole machine balancing of assurance database data amount from each active minute storehouse.
Here it should be noted that: the present invention by a minute storehouse strategy configuration module, dynamically four parts of minute storehouse engine, index database and scheduler module cooperate mutually, form complete dynamically minute storehouse machine-processed.Dynamically the clocked flip of minute storehouse engine has guaranteed robotization, index database and scheduler module to external shield inner labyrinth, kept unified access entrance.Minute storehouse strategy configuration module dynamically updates minute details for storehouse situation, and the inquiry of scheduler module is provided.
In a word, the present invention is responsible for the dynamic resolution of database by dynamic minute storehouse engine and minute storehouse strategy configuration module, by index database and scheduler module, be responsible for the establishment behind database minute storehouse and read compatible traditional approach.Break the original module single database of domino or the manual pattern in minute storehouse, adopt the strategy in minute storehouse automatically, use system self manages the domino database in system; Improved the access speed to database.
The foregoing is only preferred embodiment of the present invention, all equalizations of doing according to the present patent application the scope of the claims change and modify, and all should belong to covering scope of the present invention.
Claims (4)
1. the system based on domino database is divided the method in storehouse automatically, it is characterized in that: described method is specially:
Set up one minute storehouse strategy configuration module, configure minute library information behind minute storehouse strategy configuration information and record minute storehouse;
Set up a dynamic minute storehouse engine, this dynamically minute storehouse engine be responsible for database to monitor the establishment with database, system is periodically processed the database transfiniting in a system minute storehouse according to a minute storehouse strategy configuration information for minute storehouse strategy configuration module by dynamic minute storehouse engine, and by dividing minute library information behind strategy configuration module record minute storehouse, storehouse;
Set up an index database, during minute storehouse is processed, the data of database increase, delete, change operation real-time synchronization in this index database, and index database is deposited critical field, is integrated with all document datas, and stores the index information in each divided data storehouse;
Set up a scheduler module, system is when needs create new data, and scheduler module reads minute library information behind minute storehouse, and according to index database module, data creation is carried out in the divided data storehouse of not transfiniting; Meanwhile, when system needs visit data, scheduler module reads required data by index database module in corresponding divided data storehouse.
2. the method that the system based on domino database according to claim 1 is divided storehouse automatically, is characterized in that:
Described domino database is document data storehouse, and the evaluating standard whether excessive for database is: the physics size of database and the number of documents size of database; The number of documents that the physics size that the tactful configuration information in described configuration minute storehouse is configuration database reaches in first threshold minute storehouse or database reaches Second Threshold minute storehouse; Within described minute, library information comprises: the quantity in historical minute storehouse in database is, title, the path corresponding to each minute storehouse in the quantity in activity minute storehouse, each minute storehouse; Described history minute storehouse is the divided data storehouse being filled with, the divided data storehouse of movable minute storehouse for not being filled with; Minute library information carries out updating maintenance after changing by dynamic minute storehouse engine in minute storehouse.
3. the method that the system based on domino database according to claim 2 is divided storehouse automatically, is characterized in that:
System is periodically processed the database transfiniting in a system minute storehouse according to a minute storehouse strategy configuration information for minute storehouse strategy configuration module by dynamic minute storehouse engine, be specially: minute storehouse strategy configuration information that dynamically minute storehouse engine reads configuration is monitored database, judge that whether physics size in database or number of documents are over threshold value; No, do not divide storehouse to process, be, to reaching the divided data storehouse of minute storehouse threshold value, be labeled as historical minute storehouse, increase the divided data storehouse of a same design sky simultaneously newly, and be labeled as movable minute storehouse.
4. the method that the system based on domino database according to claim 2 is divided storehouse automatically, is characterized in that:
Described index database implementation is Domino storehouse, relevant database or NoSql database; Described index information comprises key business field, path, divided data storehouse, divided data title and document id.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410016631.2A CN103744979A (en) | 2014-01-14 | 2014-01-14 | Automatic dividing method for domino database based system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410016631.2A CN103744979A (en) | 2014-01-14 | 2014-01-14 | Automatic dividing method for domino database based system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103744979A true CN103744979A (en) | 2014-04-23 |
Family
ID=50501997
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410016631.2A Pending CN103744979A (en) | 2014-01-14 | 2014-01-14 | Automatic dividing method for domino database based system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103744979A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113377876A (en) * | 2021-06-29 | 2021-09-10 | 中国农业银行股份有限公司 | Domino platform-based data sub-database processing method, device and platform |
-
2014
- 2014-01-14 CN CN201410016631.2A patent/CN103744979A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113377876A (en) * | 2021-06-29 | 2021-09-10 | 中国农业银行股份有限公司 | Domino platform-based data sub-database processing method, device and platform |
| CN113377876B (en) * | 2021-06-29 | 2024-05-28 | 中国农业银行股份有限公司 | Data database processing method, device and platform based on Domino platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9135071B2 (en) | Selecting processing techniques for a data flow task | |
| CN102141963B (en) | A kind of data analysing method and equipment | |
| CN111966677B (en) | Data report processing method and device, electronic equipment and storage medium | |
| CN105095056A (en) | Method for monitoring data in data warehouse | |
| US20140149568A1 (en) | Monitoring alerts in a computer landscape environment | |
| US9766906B2 (en) | Efficient sharing of artifacts between collaboration applications | |
| US20160170821A1 (en) | Performance assessment | |
| CN104111994A (en) | Label data screening method and device based on mixed data source | |
| US11726970B2 (en) | Incremental transfer of database segments | |
| CN105095392B (en) | A kind of method and device of Data Mart data sharing | |
| CN107748752A (en) | A kind of data processing method and device | |
| CN111694505B (en) | Data storage management method, device and computer readable storage medium | |
| CN103793382B (en) | A kind of data processing method and system of database | |
| CN102467525A (en) | Document associating method and system | |
| CN107341056A (en) | A kind of method and device of the thread distribution based on NFS | |
| CN104598540A (en) | Timing data migration device and using method thereof | |
| CN105260297B (en) | A kind of test data management system and method | |
| EP2662783A1 (en) | Data archiving approach leveraging database layer functionality | |
| CN103258047A (en) | Data organization method of data warehouse for controlling operation cost of medicine enterprise | |
| US20200153892A1 (en) | Groupware management | |
| CN103744979A (en) | Automatic dividing method for domino database based system | |
| CN105871659A (en) | Monitoring method and device of newly added server | |
| CN108959309B (en) | Method and device for data analysis | |
| US20160103842A1 (en) | Skeleton data point clustering | |
| CN104361112A (en) | Method for writing FRU information of server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140423 |