CN103761248A - Method and system for querying data through main memory database - Google Patents
Method and system for querying data through main memory database Download PDFInfo
- Publication number
- CN103761248A CN103761248A CN201310717225.4A CN201310717225A CN103761248A CN 103761248 A CN103761248 A CN 103761248A CN 201310717225 A CN201310717225 A CN 201310717225A CN 103761248 A CN103761248 A CN 103761248A
- Authority
- CN
- China
- Prior art keywords
- entity
- memory database
- loading
- load
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24534—Query rewriting; Transformation
- G06F16/24539—Query rewriting; Transformation using cached or materialised query results
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention specifically relates to a method and a system for querying data through main memory database, the method comprises the steps of 1) building monitoring service, setting an entity, loading strategy, and loading SQL for the entity in the monitoring service, the loading SQL is the SQL executed in the business base of the entity at the loading operation of the entity, the loading strategy is the data selection condition for the entity; 2) combining the corresponding loading strategy for each entity, using the corresponding loading SQL for selecting the corresponding data in the entity from the business base and loading the data in the main memory database, which adopts the different machine deploy mode; 3) receiving the querying command, finding the data corresponding to the querying command from the main memory database and returning. The method and system can speed up the data querying speed.
Description
[technical field]
The invention belongs to enterprise information management system field (ERP), relate in information management system field about checking, the normal data administrative skill of audit and associated monitoring business, be specifically related to utilize memory database to carry out the method and system of data query.
[background technology]
In data monitoring process, sometimes need the data of a plurality of business library to inquire about, business library is database, has at least one entity in business library, contains a plurality of business datums in entity; Current querying method in some schemes in the past generally in the following way:
1, in database (as Oracle), the tables of data of needs being inquired about by DBLINK is created to main business storehouse by the mode of chained list, because can taking database, the establishment of chained list connects, can cause database connection resource too much, in limited connection resource situation, producing too much connection can cause query performance low, serious situation can cause database collapse, creates chained list simultaneously and need to promote the authority of active user in Oracle, has certain potential safety hazard;
2, by the data query of a plurality of business library in internal memory, then in internal memory, a plurality of data sets are calculated, when data volume is too much, (as 1,000,000, ten million bar) can take a large amount of memory headrooms like this, affect the performance of production system, seriously can cause the delay situation of machine of production system, and need to process independent service logic for each inquiry, extendability is poor.
[summary of the invention]
First technical matters that the present invention will solve is to provide a kind of method of utilizing memory database to carry out data query, and it has accelerated the speed of data query.
Second technical matters that the present invention will solve is to provide a kind of system of utilizing memory database to carry out data query, and it has accelerated the speed of data query.
Above-mentioned first technical matters solves by the following technical programs:
Utilize memory database to carry out the method for data query, it is characterized in that, comprise the following steps:
1) set up monitor service, entity is set in monitor service, and the loading SQL of entity, load strategy, load SQL and be the SQL carrying out in the business library at this entity place when entity loads, loading strategy is the data selection condition of entity;
2) for each entity, in conjunction with corresponding loading strategy, use corresponding loading SQL from the business library at this entity place, to select corresponding data in this entity, be loaded in memory database, described interior database adopts different machine to dispose;
3) receive querying command, in memory database, find out with the corresponding data of querying command and return.
In step 2) in, in memory database, correspondingly generating an entity state table, this entity state souvenir is loaded with the attribute of each entity being loaded in memory database, and attribute comprises that physical name claims, loads SQL, loads strategy.
In step 2) each entity of being specially in monitor service carries out following operation: first according to entity title, judge this entity record whether in entity state table, if record, is not loaded into the corresponding data of this entity in memory database, at entity state souvenir, carry the attribute of this entity; If there is record, judge whether this time loading of this entity be tactful consistent with the loading strategy of correspondent entity in entity state table; If consistent, this entity this time loads and does not need to carry out; If inconsistent, the data of this entity in memory database are deleted, and this time need the data that load to be loaded in memory database this entity, upgrade the loading number of times of this entity in entity state table, nearest load time, loading SQL simultaneously, load strategy.
The loading sequence of entity is set in the monitor service in step 1); In step 2) in, by described loading sequence, each entity is regularly loaded.
In step 1) and step 2) between also comprise step 1 '): traversal entity list on all entities, the loading frequency that calculates each entity according to the nearest load time of entity, loading number of times, physical size, then loads entity into memory database from high in the end successively according to loading frequency.
Described attribute also comprises entity line item number, the entity line item size that loads entity; In step 2) in, before loading entity each time, judge whether the pre-load amount of entity and the current use amount sum of memory database are greater than internal memory threshold values, if be greater than, from memory database, the entity that meets deletion condition is deleted, in entity state table, the attribute corresponding with these entities deleted simultaneously; Entity line item number * entity line item size * error of calculation ratio of entity pre-load amount=current loading entity, error of calculation ratio=reality consumes the ratio of internal memory and calculated value.
Also comprise and regularly delete data step: at regular intervals, when determine the current use amount of memory database be greater than internal memory threshold values will meet from memory database the data of entity of deletion condition delete, in entity state table, by meeting with these attribute that entity of deletion condition is corresponding, delete simultaneously.
Above-mentioned second technical matters solves by the following technical programs:
Utilize memory database to carry out the system of data query, it is characterized in that, comprising:
1) module is set, for setting up monitor service, entity is set in monitor service, and the loading SQL of entity, load strategy, load SQL and be the SQL carrying out in the business library at this entity place when entity loads, loading strategy is the data selection condition of entity;
2) load-on module, for to each entity, in conjunction with corresponding loading strategy, is used corresponding loading SQL from the business library at this entity place, to select corresponding data in this entity, is loaded in memory database, and described interior database adopts different machine to dispose;
3) enquiry module for receiving querying command, is found out with the corresponding data of querying command and is returned in memory database.
In load-on module, in memory database, correspondingly generate an entity state table, this entity state souvenir is loaded with the attribute of each entity being loaded in memory database, and attribute comprises that physical name claims, loads SQL, loads strategy.
In load-on module for each entity in monitor service carries out following operation: first according to entity title, judge this entity record whether in entity state table, if record, is not loaded into the corresponding data of this entity in memory database, at entity state souvenir, carry the attribute of this entity; If there is record, judge whether this time loading of this entity be tactful consistent with the loading strategy of correspondent entity in entity state table; If consistent, this entity this time loads and does not need to carry out; If inconsistent, the data of this entity in memory database are deleted, and this time need the data that load to be loaded in memory database this entity, upgrade the loading number of times of this entity in entity state table, nearest load time, loading SQL simultaneously, load strategy.
The loading sequence of entity is set in the monitor service in module is set; In load step, by described loading sequence, each entity is regularly loaded.
Also comprise intelligent sequencing module: all entities on traversal entity list, the loading frequency that calculates each entity according to the nearest load time of entity, loading number of times, physical size, then loads entity into memory database from high in the end successively according to loading frequency.
Described attribute also comprises entity line item number, the entity line item size that loads entity; In load-on module, before loading entity each time, judge whether the pre-load amount of entity and the current use amount sum of memory database are greater than internal memory threshold values, if be greater than, from memory database, the entity that meets deletion condition is deleted, in entity state table, the attribute corresponding with these entities deleted simultaneously; Entity line item number * entity line item size * error of calculation ratio of entity pre-load amount=current loading entity, error of calculation ratio=reality consumes the ratio of internal memory and calculated value.
Also comprise and regularly delete data step: at regular intervals, when determine the current use amount of memory database be greater than internal memory threshold values will meet from memory database the data of entity of deletion condition delete, in entity state table, by meeting with these attribute that entity of deletion condition is corresponding, delete simultaneously.
As seen from the above technical solution, the present invention adopts memory database as middle database, before inquiry, the data of each business library are drawn into memory database by ETL, then in memory database, carry out inquiry, so just can accomplish the inter-library inquiry in multi-service storehouse, solve the slower problem of inter-library inquiry and query performance, as shown in Figure 4, experimental data proves, the inventive method is than the high several orders of magnitude of the efficiency of traditional rule service inquiry.By loading strategy, reduce the data volume that ETL extracts, promote query performance.Trans-sectoral business library inquiry, with the inter-library scheme comparison of tradition, adopt the mode efficiency of memory database higher, extensibility is better, and safer, in solution in the past, adopt the mode of DBLINK chained list because taken the connection resource of database, can cause database to connect too much and cannot respond new request, and user also will give the authority of renewal in addition, can there is potential safety hazard, and the scheme that adopts memory database can farthest alleviate the pressure of application server and Service Database, even if memory database is delayed, machine does not affect the normal operation of application service yet, only need to restart memory database service, and whole process operates by UI completely.Search efficiency, because involved business datum all, in memory database, has been saved the magnetic disc i/o time of traditional database, search efficiency is than the high several orders of magnitude of traditional database, as long as guarantee the high hit rate that data load, the query performance of system improves more greatly.
[accompanying drawing explanation]
Fig. 1 is the process flow diagram of embodiment mono-the inventive method;
Fig. 2 is the different machine deployment diagram of memory database;
Fig. 3 is the concrete exemplary plot of entity state table;
Fig. 4 is for adopting and not adopting memory database to carry out the result comparison diagram of data query;
Fig. 5 is the rule schema that entity loads;
Fig. 6 is the structured flowchart of embodiment bis-systems of the present invention.
[embodiment]
Embodiment mono-
As shown in Figure 1, utilize memory database to carry out the method for data query, comprise the following steps:
1) setting steps: set up monitor service, entity is set in monitor service, and the loading SQL of entity, load strategy, the SQL carrying out in the business library at this entity place when loading SQL is entity loading, loading strategy is the data selection condition of entity, loads strategy different and different according to current context environment;
2) load step: for each entity, in conjunction with corresponding loading strategy, use corresponding loading SQL to select corresponding data in this entity from the business library at this entity place, be loaded in memory database, described interior database adopts different machine to dispose;
3) query steps: receive querying command, find out in memory database with the corresponding data of querying command and return.The physics of above-mentioned memory database is disposed explanation: build special-purpose memory database server (conventionally, machines configurations requires that 64 bit manipulation systems, 32G internal memory are above, 4 cores and above main flow CPU, gigabit/10,000,000,000 network bandwidth, 64 JDK), all business datums that relate to across business library are all first loaded in memory database and (support the databases such as Oracle, SqlServer, DB2), and subsequent query is carried out at memory database.In practical application, memory database server supports embedded (being embedded into application service), same machine (from application service at same station server), different machine to dispose (with application service at two different servers) three kinds of patterns; See Fig. 2, the present invention adopts different machine to dispose, and disposes and supports Windows, linux system simultaneously.
Wherein, in step 2) in, in memory database, correspondingly generate an entity state table, this entity state souvenir is loaded with the attribute for each entity, as shown in Figure 3, attribute comprises that physical name claims, the last load time, load number of times, entity line item number, entity line item size, load strategy and load SQL, and entity title comprises entity English name, entity Chinese.
Entity English name: the English of entity is described can be called table name with physical name and set up entity list in memory database.
Entity Chinese: the Chinese of entity is described.
The last load time: the last time loading of entity.
Load number of times: the number of times that entity loads altogether.
Entity line item number: total how many line items of data that are loaded in this entity.
Entity line item size: the size of the line item of entity, computing method are that the field of entity all properties consumes big or small sum.
Load strategy: the loading strategy of recording this entity;
Load SQL: the SQL carrying out in the business library at its this entity place when entity loads.
In order to carry out better data query, in the present embodiment, from following aspect, improve.
One, improve and load entity efficiency
In actual motion, can exist some entity to need the situation of repeated loading, system is for promoting loading efficiency, save the load time, each can carry out following operation while loading entity, therefore, in above-mentioned steps 2) in can carry out following operation while loading entity at every turn: first according to entity title, judge this entity record whether in entity state table, if record, is not loaded into the corresponding data of this entity in memory database, at entity state souvenir, carry the attribute of this entity; If there is record, judge whether this time loading of this entity be tactful consistent with the loading strategy of correspondent entity in entity state table; If consistent, this entity this time loads and does not need to carry out; If inconsistent, the data of this entity in memory database are deleted, and this time need the data that load to be loaded in memory database this entity, upgrade the loading number of times of this entity in entity state table, nearest load time, loading SQL simultaneously, load strategy.
This step has avoided the tactful entity of identical loading to use then and no longer load in next time, reduces the data volume loading, and improves the efficiency of inquiring about.
Two, entity load mode
There are both schemes:
1, manually planning: the loading sequence that entity is set in the monitor service in step 1); In step 2) in, by described loading sequence, each entity is regularly loaded.This manually arranges as required, is convenient to tackle neatly needs.
2, intelligence arrangement: in step 1) and step 2) between, also comprise step 1 '): all entities on traversal entity list, according to the nearest load time of entity, load number of times, physical size calculates the loading frequency of each entity, then according to loading frequency, successively entity is loaded into memory database from high in the end, so just the relatively high entity of access frequency is all loaded into internal memory, this loading loads in the non-business activity phase (as evening) by scheduler program, at second day, just removed the process loading from like this, thereby shortening query time, promote search efficiency.
Three, the action of the garbage reclamation based on memory database
Because internal memory is limited resources, when loading, entity to have relevant security mechanism, to control the remaining amount of internal memory.Specifically there is two schemes:
When 3.1 entities load, delete: in step 2) in, before loading entity each time, judge whether the pre-load amount of entity and the current use amount sum of memory database are greater than internal memory threshold values (the maximum memory value that uses of memory database, can be according to actual conditions free setting), if be greater than, from memory database, the entity that meets deletion condition is deleted, in entity state table, the attribute corresponding with these entities deleted simultaneously; Entity line item number * entity line item size * error of calculation ratio of entity pre-load amount=current loading entity, error of calculation ratio=reality consumes the ratio of internal memory and calculated value.
3.2 regularly delete: (for example 10 minutes at regular intervals, can be according to actual conditions free setting), when determining the current use amount of memory database, be greater than internal memory threshold values, adopt " genetic algorithm " from memory database, the entity that meets deletion condition to be deleted, in entity state table, the attribute corresponding with these entities deleted simultaneously.
Above-mentioned deletion condition is normally set according to entity service time, access times, frequency of utilization, and object is that the entity being of little use is deleted.
Four, visual UI
The configuration of memory database, operation, attended operation are all visual, provide special UI interface to operate, improving user experiences, the operating console of memory database is provided simultaneously, be similar to PL/SQL, by user, login after mandate, can directly in memory database, carry out DML, DDL, DCL action statement, the load condition of while query entity, convenient enforcement/developer safeguards and issue track.
Embodiment bis-
As shown in Figure 6, utilize memory database to carry out the system of data query, comprising:
1) module is set, be used for setting up monitor service, entity is set in monitor service, and the loading SQL of entity, load strategy, the SQL carrying out in the business library at this entity place when loading SQL is entity loading, loading strategy is the data selection condition of entity, loads strategy different and different according to current context environment;
2) load-on module, for to each entity, in conjunction with corresponding loading strategy, is used corresponding loading SQL from the business library at this entity place, to select corresponding data in this entity, is loaded in memory database, and described interior database adopts different machine to dispose;
3) enquiry module for receiving querying command, is found out with the corresponding data of querying command and is returned in memory database.
The physics of above-mentioned memory database is disposed explanation: build special-purpose memory database server (conventionally, machines configurations requires that 64 bit manipulation systems, 32G internal memory are above, 4 cores and above main flow CPU, gigabit/10,000,000,000 network bandwidth, 64 JDK), all business datums that relate to across business library are all first loaded in memory database and (support the databases such as Oracle, SqlServer, DB2), and subsequent query is carried out at memory database.In practical application, memory database server supports embedded (being embedded into application service), same machine (from application service at same station server), different machine to dispose (with application service at two different servers) three kinds of patterns; See Fig. 2, the present invention adopts different machine to dispose, and disposes and supports Windows, linux system simultaneously.
Wherein, in load-on module, in memory database, correspondingly generate an entity state table, this entity state souvenir is loaded with the attribute for each entity, as shown in Figure 3, attribute comprises that physical name claims, the last load time, load number of times, entity line item number, entity line item size, load strategy and load SQL, and entity title comprises entity English name, entity Chinese.
Entity English name: the English of entity is described can be called table name with physical name and set up entity list in memory database.
Entity Chinese: the Chinese of entity is described.
The last load time: the last time loading of entity.
Load number of times: the number of times that entity loads altogether.
Entity line item number: total how many line items of data that are loaded in this entity.
Entity line item size: the size of the line item of entity, computing method are that the field of entity all properties consumes big or small sum.
Load strategy: the loading strategy of recording this entity;
Load SQL: the SQL carrying out in the business library at its this entity place when entity loads.
In order to carry out better data query, in the present embodiment, from following aspect, improve.
One, improve and load entity efficiency
In actual motion, can exist some entity to need the situation of repeated loading, system is for promoting loading efficiency, save the load time, each can carry out following operation while loading entity, therefore, can carry out following operation while loading entity at every turn in above-mentioned load-on module: first according to entity title, judge this entity record whether in entity state table, if record, is not loaded into the corresponding data of this entity in memory database, at entity state souvenir, carry the attribute of this entity; If there is record, judge whether this time loading of this entity be tactful consistent with the loading strategy of correspondent entity in entity state table; If consistent, this entity this time loads and does not need to carry out; If inconsistent, the data of this entity in memory database are deleted, and this time need the data that load to be loaded in memory database this entity, upgrade the loading number of times of this entity in entity state table, nearest load time, loading SQL simultaneously, load strategy.
This step has avoided the tactful entity of identical loading to use then and no longer load in next time, reduces the data volume loading, and improves the efficiency of inquiring about.
Two, entity load mode
There are both schemes:
1, manually planning: the loading sequence that entity is set in the monitor service in module is set; In load-on module, by described loading sequence, each entity is regularly loaded.This manually arranges as required, is convenient to tackle neatly needs.
2, intelligence arranges: also comprise intelligent sequencing module: all entities on traversal entity list, according to the nearest load time of entity, loading number of times, physical size, calculate the loading frequency of each entity, then according to loading frequency, successively entity is loaded into memory database from high in the end, so just the relatively high entity of access frequency is all loaded into internal memory, this loading loads in the non-business activity phase (as evening) by scheduler program, at second day, just remove the process loading from like this, thereby shortened query time, lifting search efficiency.
Three, the action of the garbage reclamation based on memory database
Because internal memory is limited resources, when loading, entity to have relevant security mechanism, to control the remaining amount of internal memory.Specifically there is two schemes:
When loading, delete 3.1 entities: in load-on module, before loading entity each time, judge whether the pre-load amount of entity and the current use amount sum of memory database are greater than internal memory threshold values (the maximum memory value that uses of memory database, can be according to actual conditions free setting), if be greater than, from memory database, the entity that meets deletion condition is deleted, in entity state table, the attribute corresponding with these entities deleted simultaneously; Entity line item number * entity line item size * error of calculation ratio of entity pre-load amount=current loading entity, error of calculation ratio=reality consumes the ratio of internal memory and calculated value.
3.2 regularly delete: (for example 10 minutes at regular intervals, can be according to actual conditions free setting), when determining the current use amount of memory database, be greater than internal memory threshold values, adopt " genetic algorithm " from memory database, the entity that meets deletion condition to be deleted, in entity state table, the attribute corresponding with these entities deleted simultaneously.
Above-mentioned deletion condition is normally set according to entity service time, access times, frequency of utilization, and object is that the entity being of little use is deleted.
Four, visual UI
The configuration of memory database, operation, attended operation are all visual, provide special UI interface to operate, improving user experiences, the operating console of memory database is provided simultaneously, be similar to PL/SQL, by user, login after mandate, can directly in memory database, carry out DML, DDL, DCL action statement, the load condition of while query entity, convenient enforcement/developer safeguards and issue track.
The present invention is not limited to above-described embodiment, based on simple replacement above-described embodiment, that do not make creative work, should belong to the scope that the present invention discloses.
Claims (14)
1. utilize memory database to carry out the method for data query, it is characterized in that, comprise the following steps:
1) set up monitor service, entity is set in monitor service, and the loading SQL of entity, load strategy, load SQL and be the SQL carrying out in the business library at this entity place when entity loads, loading strategy is the data selection condition of entity;
2) for each entity, in conjunction with corresponding loading strategy, use corresponding loading SQL from the business library at this entity place, to select corresponding data in this entity, be loaded in memory database, described interior database adopts different machine to dispose;
3) receive querying command, in memory database, find out with the corresponding data of querying command and return.
2. the method for utilizing memory database to carry out data query according to claim 1, it is characterized in that, in step 2) in, in memory database, correspondingly generate an entity state table, this entity state souvenir is loaded with the attribute of each entity being loaded in memory database, and attribute comprises that physical name claims, loads SQL, loads strategy.
3. the method for utilizing memory database to carry out data query according to claim 2, it is characterized in that, in step 2) each entity of being specially in monitor service carries out following operation: first according to entity title, judge this entity record whether in entity state table, if record, is not loaded into the corresponding data of this entity in memory database, at entity state souvenir, carry the attribute of this entity; If there is record, judge whether this time loading of this entity be tactful consistent with the loading strategy of correspondent entity in entity state table; If consistent, this entity this time loads and does not need to carry out; If inconsistent, the data of this entity in memory database are deleted, and this time need the data that load to be loaded in memory database this entity, upgrade the loading number of times of this entity in entity state table, nearest load time, loading SQL simultaneously, load strategy.
4. according to the memory database that utilizes described in claims 1 to 3 any one, carry out the method for data query, it is characterized in that, the loading sequence of entity is set in the monitor service in step 1); In step 2) in, by described loading sequence, each entity is regularly loaded.
5. according to the memory database that utilizes described in claims 1 to 3 any one, carry out the method for data query, it is characterized in that, in step 1) and step 2) between also comprise step 1 '): traversal entity list on all entities, the loading frequency that calculates each entity according to the nearest load time of entity, loading number of times, physical size, then loads entity into memory database from high in the end successively according to loading frequency.
6. the method for utilizing memory database to carry out data query according to claim 2, is characterized in that, described attribute also comprises entity line item number, the entity line item size that loads entity; In step 2) in, before loading entity each time, judge whether the pre-load amount of entity and the current use amount sum of memory database are greater than internal memory threshold values, if be greater than, from memory database, the entity that meets deletion condition is deleted, in entity state table, the attribute corresponding with these entities deleted simultaneously; Entity line item number * entity line item size * error of calculation ratio of entity pre-load amount=current loading entity, error of calculation ratio=reality consumes the ratio of internal memory and calculated value.
7. the method for utilizing memory database to carry out data query according to claim 1, it is characterized in that, also comprise and regularly delete data step: at regular intervals, when determine the current use amount of memory database be greater than internal memory threshold values will meet from memory database the data of entity of deletion condition delete, in entity state table, by meeting with these attribute that entity of deletion condition is corresponding, delete simultaneously.
8. utilize memory database to carry out the system of data query, it is characterized in that, comprising:
1) module is set, for setting up monitor service, entity is set in monitor service, and the loading SQL of entity, load strategy, load SQL and be the SQL carrying out in the business library at this entity place when entity loads, loading strategy is the data selection condition of entity;
2) load-on module, for to each entity, in conjunction with corresponding loading strategy, is used corresponding loading SQL from the business library at this entity place, to select corresponding data in this entity, is loaded in memory database, and described interior database adopts different machine to dispose;
3) enquiry module for receiving querying command, is found out with the corresponding data of querying command and is returned in memory database.
9. the system of utilizing memory database to carry out data query according to claim 8, it is characterized in that, in load-on module, in memory database, correspondingly generate an entity state table, this entity state souvenir is loaded with the attribute of each entity being loaded in memory database, and attribute comprises that physical name claims, loads SQL, loads strategy.
10. the system of utilizing memory database to carry out data query according to claim 9, it is characterized in that, in load-on module for each entity in monitor service carries out following operation: first according to entity title, judge this entity record whether in entity state table, if record, is not loaded into the corresponding data of this entity in memory database, at entity state souvenir, carry the attribute of this entity; If there is record, judge whether this time loading of this entity be tactful consistent with the loading strategy of correspondent entity in entity state table; If consistent, this entity this time loads and does not need to carry out; If inconsistent, the data of this entity in memory database are deleted, and this time need the data that load to be loaded in memory database this entity, upgrade the loading number of times of this entity in entity state table, nearest load time, loading SQL simultaneously, load strategy.
The memory database that utilizes described in 11. according to Claim 8 to 10 any one carries out the system of data query, it is characterized in that, the loading sequence of entity is set in the monitor service in module is set; In load step, by described loading sequence, each entity is regularly loaded.
The memory database that utilizes described in 12. according to Claim 8 to 10 any one carries out the system of data query, it is characterized in that, also comprise intelligent sequencing module: all entities on traversal entity list, the loading frequency that calculates each entity according to the nearest load time of entity, loading number of times, physical size, then loads entity into memory database from high in the end successively according to loading frequency.
13. systems of utilizing memory database to carry out data query according to claim 9, is characterized in that, described attribute also comprises entity line item number, the entity line item size that loads entity; In load-on module, before loading entity each time, judge whether the pre-load amount of entity and the current use amount sum of memory database are greater than internal memory threshold values, if be greater than, from memory database, the entity that meets deletion condition is deleted, in entity state table, the attribute corresponding with these entities deleted simultaneously; Entity line item number * entity line item size * error of calculation ratio of entity pre-load amount=current loading entity, error of calculation ratio=reality consumes the ratio of internal memory and calculated value.
14. systems of utilizing memory database to carry out data query according to claim 8, it is characterized in that, also comprise and regularly delete data step: at regular intervals, when determine the current use amount of memory database be greater than internal memory threshold values will meet from memory database the data of entity of deletion condition delete, in entity state table, by meeting with these attribute that entity of deletion condition is corresponding, delete simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310717225.4A CN103761248B (en) | 2013-12-23 | 2013-12-23 | The method and system of data query are carried out using memory database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310717225.4A CN103761248B (en) | 2013-12-23 | 2013-12-23 | The method and system of data query are carried out using memory database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103761248A true CN103761248A (en) | 2014-04-30 |
| CN103761248B CN103761248B (en) | 2018-04-17 |
Family
ID=50528487
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310717225.4A Active CN103761248B (en) | 2013-12-23 | 2013-12-23 | The method and system of data query are carried out using memory database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103761248B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106202080A (en) * | 2015-04-30 | 2016-12-07 | 中国移动通信集团公司 | A kind of webpage rendering intent, server and terminal unit |
| CN110888939A (en) * | 2018-09-06 | 2020-03-17 | 北京京东尚科信息技术有限公司 | Data management method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101221585A (en) * | 2008-02-03 | 2008-07-16 | 华为技术有限公司 | Data storage method and device |
| CN101320392A (en) * | 2008-07-17 | 2008-12-10 | 中兴通讯股份有限公司 | High-capacity data access method and device of internal memory database |
| CN101329685A (en) * | 2008-07-30 | 2008-12-24 | 烽火通信科技股份有限公司 | Implementing method of memory database on household gateway |
| CN101414917A (en) * | 2007-10-19 | 2009-04-22 | 华为技术有限公司 | Method for saving internal memory space, data management network element and network system |
| CN102054034A (en) * | 2010-12-27 | 2011-05-11 | 华中科技大学 | Implementation method for business basic data persistence of enterprise information system |
| CN102279885A (en) * | 2011-08-16 | 2011-12-14 | 中兴通讯股份有限公司 | Method and device for operating data by memory database |
| US20120096233A1 (en) * | 2005-12-05 | 2012-04-19 | Tianlong Chen | Apparatus and Method for On-Demand In-Memory Database Management Platform |
-
2013
- 2013-12-23 CN CN201310717225.4A patent/CN103761248B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120096233A1 (en) * | 2005-12-05 | 2012-04-19 | Tianlong Chen | Apparatus and Method for On-Demand In-Memory Database Management Platform |
| CN101414917A (en) * | 2007-10-19 | 2009-04-22 | 华为技术有限公司 | Method for saving internal memory space, data management network element and network system |
| CN101221585A (en) * | 2008-02-03 | 2008-07-16 | 华为技术有限公司 | Data storage method and device |
| CN101320392A (en) * | 2008-07-17 | 2008-12-10 | 中兴通讯股份有限公司 | High-capacity data access method and device of internal memory database |
| CN101329685A (en) * | 2008-07-30 | 2008-12-24 | 烽火通信科技股份有限公司 | Implementing method of memory database on household gateway |
| CN102054034A (en) * | 2010-12-27 | 2011-05-11 | 华中科技大学 | Implementation method for business basic data persistence of enterprise information system |
| CN102279885A (en) * | 2011-08-16 | 2011-12-14 | 中兴通讯股份有限公司 | Method and device for operating data by memory database |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106202080A (en) * | 2015-04-30 | 2016-12-07 | 中国移动通信集团公司 | A kind of webpage rendering intent, server and terminal unit |
| CN110888939A (en) * | 2018-09-06 | 2020-03-17 | 北京京东尚科信息技术有限公司 | Data management method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103761248B (en) | 2018-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109144994A (en) | Index updating method, system and relevant apparatus | |
| CN102930062B (en) | The method of the quick horizontal extension of a kind of database | |
| CN103365929B (en) | The management method of a kind of data base connection and system | |
| US9280568B2 (en) | Zero downtime schema evolution | |
| CN104462185B (en) | A kind of digital library's cloud storage system based on mixed structure | |
| CN102651007A (en) | Method and device for managing database indexes | |
| CN102663007B (en) | Data storage and query method supporting agile development and lateral spreading | |
| CN103593422A (en) | Virtual access management method of heterogeneous database | |
| CN103455540A (en) | System and method of generating in-memory models from data warehouse models | |
| CN103425762A (en) | Telecom operator mass data processing method based on Hadoop platform | |
| CN104216962A (en) | Mass network management data indexing design method based on HBase | |
| CN103544319A (en) | Multi-tenant database sharing method and multi-tenant database as-a-service system | |
| US9639544B1 (en) | Table data persistence | |
| CN105989015B (en) | Database capacity expansion method and device and method and device for accessing database | |
| CN105608224A (en) | Orthogonal multilateral Hash mapping indexing method for improving massive data inquiring performance | |
| Borkar et al. | Have your data and query it too: From key-value caching to big data management | |
| US20190362016A1 (en) | Frequent pattern analysis for distributed systems | |
| CN107153701A (en) | A kind of IT system management and method for managing and monitoring based on metadata | |
| CN109669975B (en) | Industrial big data processing system and method | |
| CN105608126A (en) | Method and apparatus for establishing secondary indexes for massive databases | |
| CN104376109A (en) | Multi-dimension data distribution method based on data distribution base | |
| US11249968B2 (en) | Large object containers with size criteria for storing mid-sized large objects | |
| CN106780157B (en) | Ceph-based power grid multi-temporal model storage and management system and method | |
| CN114968739A (en) | Operation and maintenance task management method, operation and maintenance method, device, equipment and medium | |
| CN105786877B (en) | A kind of date storage method, system and querying method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |