CN105183391A - Method and device for storing data under distributed data platform - Google Patents
Method and device for storing data under distributed data platform Download PDFInfo
- Publication number
- CN105183391A CN105183391A CN201510598396.9A CN201510598396A CN105183391A CN 105183391 A CN105183391 A CN 105183391A CN 201510598396 A CN201510598396 A CN 201510598396A CN 105183391 A CN105183391 A CN 105183391A
- Authority
- CN
- China
- Prior art keywords
- data
- class
- time
- subregion
- catalogue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and device for storing data under a distributed data platform. The data storage and data retrieval efficiency can be improved while data changes can be effectively recorded. The method for storing the data under the distributed data platform comprises the steps that changed data are classified by comparing intraday data and data in a data state changing table; the classified data are sorted into different catalogues and stored in corresponding partitions according to data storage rules of the catalogues; the data state changing table is updated.
Description
Technical field
The present invention relates to field of computer technology, the method and apparatus that particularly under a kind of distributed data platform, data store.
Background technology
Large data---people describe the epoch of current information blast with it, it not only shows the leap in data volume, and data storage kind also gets more and more, from traditional relational data, Key-Value data, to flat file, picture, audio frequency, video etc. that form is more various.Analyze so numerous and diverse data, higher requirement is proposed to the calculated performance of data platform and memory property.
Adopt distributed Hadoop system to do the common practice that the storage of large data and analysis are industries, what adopt due to distributed Hadoop system is the mode of file storage data, although improve memory space and the handling capacity of data, but but sacrifice the update mechanism of original relevant database, only support to insert, delete, the mode of operation of overlay text file, causes the accumulation of current data history can only adopt the mode of data snapshot.Every day a snapshot is preserved to the data stored in database, records complete data mode, and along with the storage of time integral history of forming data.When needing the historical track of reduction or the change of retrieve data state, need to scan historical data by full dose, the universe of carrying out different time points calculates comparison, finds out the difference of data, the data mode of recovery time point.
But there are some following shortcomings in existing technical scheme:
1. the storage scheme for relevant database is felt simply helpless to the process of big data quantity; And the mode that existing distributed file system takes snapshot to accumulate, sacrifice mass storage space, and in follow-up calculating, inefficiency;
2. data retrieval often needs to carry out full dose scanning, takies a large amount of system resource;
3., for data scene complicated and changeable on line, lack dirigibility.
But, in a large amount of application scenarioss, data often change through a lot of state from producing to wither away, correspondingly, data platform creates many parts of snapshots when recording data mode change, and data store meeting rapid expansion, and in data analysis process, often need tracking data being carried out to historical track, need to scan the reduction that a large amount of historical datas carries out state, inefficiency.Therefore, how designing a kind of mechanism and make data platform can either record data mode change and be convenient to analyze and reduction, is the major issue that pendulum needs solution badly in face of us.
Summary of the invention
In view of this, the invention provides the method and apparatus that data store under a kind of distributed data platform, can while effective record data variation, improve data and store and the efficiency of data retrieval.
For achieving the above object, according to an aspect of the present invention, a kind of method that data store under distributed data platform is provided.
The method that under distributed data platform, data store, comprising: by the data in the data on the same day and data mode change list being compared, classify to the data changed; By described sorted data acquisition under different catalogues, and under being stored in corresponding subregion according to the data storage rule of described catalogue; And upgrade described data mode change list.
Alternatively, described classification carries out according to the process of data life period, and comprise online class, expired class and filing class three types.
Alternatively, the step that the data changed are classified is comprised: by searching the key name of data, the data in the data on the same day and data mode change list are compared; If do not have described data in described data mode change list, and have described data in the data on the described same day, then described data are online class; If have described data in described data mode change list and the data on the described same day, but the key assignments of described data is different, then the described data in described data mode change list are expired class, and the described data on the same day are online class; And if have described data in described data mode change list, and do not have described data in the data on the same day, then described data are filing class.
Alternatively, described data storage rule comprises partition name, data time and data life 3 directory levels closing time.
Alternatively, described partition name comprises online subregion, expired subregion and filing subregion.
Alternatively, the step be stored under corresponding subregion according to the data storage rule of described catalogue comprises: the first class catalogue partition name of described online class data is online subregion, second-level directory data time is maximum time, and three grades of catalogue data life closing times are maximum time; The first class catalogue partition name of described expired class data is expired subregion, and second-level directory data time is transformation period, and three grades of catalogue data life closing times are transformation period; And the first class catalogue partition name of described filing class data is filing subregion, second-level directory data time is transformation period, and three grades of catalogue data life closing times are maximum time.
Alternatively, the step upgrading described data mode change list comprises: insert the key name of described online class data, key assignments, state change initial time and state change end time, wherein, described state change initial time is transformation period, and the described state change end time is maximum time; And the described state change end time of described expired class data is set to transformation period.
According to a further aspect in the invention, the device that data store under a kind of distributed data platform is provided.
The device that under distributed data platform, data store, comprising: data categorization module, for by the data in the data on the same day and data mode change list being compared, classifying to the data changed; Data memory module, for by described sorted data acquisition under different catalogues, and under being stored in corresponding subregion according to the data storage rule of described catalogue; And state updating module, for upgrading described data mode change list.
Alternatively, described classification carries out according to the process of the life cycle of data, and comprise online class, expired class and filing class three types.
Alternatively, described data categorization module also for: by searching the key name of data, the data in the data on the same day and data mode change list are compared; If do not have described data in described data mode change list, and have described data in the data on the described same day, then described data are online class; If have described data in described data mode change list and the data on the described same day, but the key assignments of described data is different, then the described data in described data mode change list are expired class, and the described data on the same day are online class; And if have described data in described data mode change list, and do not have described data in the data on the same day, then described data are filing class.
Alternatively, described data storage rule comprises partition name, data time and data life 3 directory levels closing time.
Alternatively, described partition name comprises online subregion, expired subregion and filing subregion.
Alternatively, described data memory module also for: the first class catalogue partition name of described online class data is online subregion, and second-level directory data time is maximum time, and three grades of catalogue data life closing times are maximum time; The first class catalogue partition name of described expired class data is expired subregion, and second-level directory data time is transformation period, and three grades of catalogue data life closing times are transformation period; And the first class catalogue partition name of described filing class data is filing subregion, second-level directory data time is transformation period, and three grades of catalogue data life closing times are maximum time.
Alternatively, described state updating module also for: insert the key name of described online class data, key assignments, state change initial time and state change the end time, wherein, described state change initial time is transformation period, and the described state change end time is maximum time; And the described state change end time of described expired class data is set to transformation period.
According to technical scheme of the present invention, only when data mode changes, just need to classify to these data, store and the operation such as state updating, for the data do not changed without the need to carrying out secondary storage or state updating, thus can while effectively recording data variation, improve the efficiency of data storage and data retrieval, effectively save data space, and also very easy and convenient to the cleaning of stale data.
Accompanying drawing explanation
Accompanying drawing is used for understanding the present invention better, does not form inappropriate limitation of the present invention.Wherein:
Fig. 1 is the key step schematic diagram of the method that under a kind of distributed data platform according to the embodiment of the present invention, data store;
Fig. 2 is the schematic diagram stored according to the data partition of the embodiment of the present invention;
Fig. 3 is the schematic diagram of the data scrubbing according to the embodiment of the present invention;
Fig. 4 is the schematic diagram of the data mode change list according to the embodiment of the present invention;
Fig. 5 is the main modular schematic diagram of the device that under a kind of distributed data platform according to the embodiment of the present invention, data store;
Fig. 6 is the storage effectiveness comparison schematic diagram of the embodiment of the present invention and prior art.
Embodiment
Below in conjunction with accompanying drawing, one exemplary embodiment of the present invention is explained, comprising the various details of the embodiment of the present invention to help understanding, they should be thought it is only exemplary.Therefore, those of ordinary skill in the art will be appreciated that, can make various change and amendment, and can not deviate from scope and spirit of the present invention to the embodiments described herein.Equally, for clarity and conciseness, the description to known function and structure is eliminated in following description.
The method that under a kind of distributed data platform of the present invention, data store, only when item status changes, just need to classify to this data item, store and the operation such as state updating, for the data item do not changed without the need to carrying out secondary storage or state updating, thus while effectively recording data variation, the efficiency of data storage and data retrieval can be improved.
Fig. 1 is the key step schematic diagram of the method that under a kind of distributed data platform according to the embodiment of the present invention, data store.As shown in Figure 1, the method that under a kind of distributed data platform of the present invention, data store mainly comprises following step S11 to step S13.
Step S11: by the data in the data on the same day and data mode change list are compared, the data changed are classified.In order to adapt to the feature of Hadoop file system, need uniform sequential to carry out deposit data to raise the efficiency.According to the process of data life period, data can be divided into three classes, i.e. online class (ACTIVE), expired class (EXPIRED), filing class (HISTORY).The effective data of the current meaning of online class data representation, likely can change; The data that the current meaning of expired class data representation had lost efficacy; Filing class data representation has been sealed up for safekeeping no longer to be changed, the lasting effective data of meaning.
When carrying out Data classification, according to predefined data processing rule, by searching the key name of data, the data in the data on the same day and data mode change list are compared the data determining to change; If do not have described data in described data mode change list, and have described data in the data on the described same day, then described data are online class; If have described data in described data mode change list and the data on the described same day, but the key assignments of described data is different, then the described data in described data mode change list are expired class, and the described data on the same day are online class; And if have described data in described data mode change list, and do not have described data in the data on the same day, then described data are filing class.
Step S12: by described sorted data acquisition under different catalogues, and under being stored in corresponding subregion according to the data storage rule of described catalogue.Wherein, described data storage rule comprises partition name, data time and data life 3 directory levels closing time.Data classification described in integrating step S11 is known, and described partition name comprises online subregion, expired subregion and filing subregion.
As shown in Figure 2, be the schematic diagram stored according to the data partition of the embodiment of the present invention.For the large enterprise of a stable operation, along with increasing progressively of time, the data volume in expired subregion and filing subregion also can steady-state growth, and the data volume of online subregion can keep relative stability while newly-increased to some extent as far as possible.As can be seen from Fig. 2, be to serve as theme axle with the time when data store, leave in equably in subregion corresponding to these 3 top-level directory as far as possible.
Classification for the ease of carrying out data stores and searches, and when carrying out data and storing, under being stored in corresponding subregion, correspondingly comprises following 3 kinds of situations for aforesaid 3 class data according to the data storage rule of described catalogue:
The first class catalogue partition name of described online class data is online subregion, and second-level directory data time is maximum time, and three grades of catalogue data life closing times are maximum time;
The first class catalogue partition name of described expired class data is expired subregion, and second-level directory data time is transformation period, and three grades of catalogue data life closing times are transformation period; And
The first class catalogue partition name of described filing class data is filing subregion, and second-level directory data time is transformation period, and three grades of catalogue data life closing times are maximum time.
Below, citing describes concrete data storage directory hierarchical structure.Such as:
For online class data, data storage directory hierarchical structure is dp=ACTIVE/dt=4712-12-31/end_date=4712-12-31;
For expired class data, data storage directory hierarchical structure is dp=EXPIRED/dt=2013-10-11/end_date=2013-10-11;
For filing class data, data storage directory hierarchical structure is dp=HISTORY/dt=2014-06-22/end_date=4712-12-31.Wherein, dp represents data partition datapartition, and dt represents data time datatime, and end_date represents that data life is by the time.To file class data instance, when storage one needs the data of filing, be first determine to be stored in " dp=HISTORY " this subregion; Afterwards, can be stored under this time is transferred to the data directory of this subregion according to the transformation period " dt=2014-06-22 " of these data; Finally, these data are preserved in corresponding data table according to data life by the time " end_date=4712-12-31 ".Wherein, because data filing representative data is sealed up for safekeeping, no longer change, the value of its attribute and meaning until permanent, so its " end_date " is maximum time " 4712-12-31 ".In actual applications, can setting data storage directory hierarchical structure as the case may be.
Fig. 3 is according to the schematic diagram of the data scrubbing of the embodiment of the present invention.Adopt partitioned storage mode as shown in Figure 2 to carry out data storage, the cleaning of historical data can be carried out very easily.As shown in Figure 3, for expired class data, its data attribute or tolerance there occurs change, and current data meaning lost efficacy, and only needed corresponding expired subregion to delete when clearing up it, simple to operation.
Step S13: upgrade described data mode change list.When the state of data changes, we need to upgrade the state of data.Integrating step S11 and step S12 is known, when upgrading data mode change list, need to insert the key name of described online class data, key assignments, state change initial time and state change end time, wherein, described state change initial time is transformation period, and the described state change end time is maximum time; And the described state change end time of described expired class data is set to transformation period.For the data that any change does not occur, without the need to carrying out state updating.
As shown in Figure 4, be the schematic diagram of the data mode change list according to the embodiment of the present invention.Table as upper left is the data on the 2014-01-01 same day, the table of upper right is the data on the 2014-01-02 same day, existing technical scheme is that the data of every day are carried out snapshot preservation, when certain data searched by needs or carry out the process such as calculating, need full dose to scan all snapshots, not only sacrifice a large amount of storage areas but also waste system resource.And the solution of the present invention is compared the data on table 2014-01-01 same day of the data on table 2014-01-02 same day of upper right and upper left, add record to the data item changed, the data item do not changed is without the need to changing.Simultaneously, when the structure of design data state change list, introduce the starting and ending time that audit field start_date/end_date carrys out the change of identification data state, and, in order to distinguish data better, the major key of tables of data will add audit field start_date.
In the diagram, after the data of the data of the table 2014-01-02 of upper right and the table 2014-01-01 of upper left are compared, the data mode change list mytable shown in can drawing below Fig. 4 arrow.In table mytable, major key comprises the initial time start_date of key name key and data mode change, distinguishes each data by major key.On line, data record has three kinds of operation: Insert usually, represents the generation of new record; Delete, represents the online termination be worth of record; Update, is equivalent to Delete/Update composition operation, represents the transition of recording status, namely records the end of previous state and the generation of new state.Such as, the data of 2014-01-02 can be found out compared with the data of 2014-01-01, key be 1 data there occurs change (Update), so in table mytable, according to major key key is 1 and the end_date that start_date is the data of 2014/1/1 is revised as transformation period, newly increase a record, major key is that key is 1 and start_date is transformation period simultaneously.Equally, be the data of 4 for key, in table mytable directly newly-increased (Insert).By the data in the data of every day and data mode change list are compared, the data changed can be found, according to the method for table mytable identification data state, snapshot preservation is carried out without the need to every day, thus effectively can save storage space, and ensure continuous in time, basis can be provided for follow-up retrieval analysis.
By the date storage method described in above step S11 to step S13, dividing according to data store organisation of the present invention and catalogue, according to the needs of data retrieval and calculating, directly inquiring about by writing SQL statement.Such as, if we will search the state of the 2014-01-01 same day " 1 " from the table mytable of Fig. 4, it is as follows to write SQL statement:
Select*frommytablewherestart_date<＝'2014-01-01'andend_date>'2014-01-01'and[key＝'1']；
If the state of " 1 " in during will searching this section of 2014-01-01 to 2014-01-02 from table mytable, it is as follows to write SQL statement:
Select*frommytablewherestart_date<＝'2014-01-02'andend_date>＝'2014-01-01'and[key＝'1']；
If will search " 1 " current last state from table mytable, it is as follows to write SQL statement:
Select*frommytablewheredp＝('ACTIVE'or[dp＝'HISTORY'])and[key＝'1']。
So, directly carrying out the inquiry of data mode by writing SQL statement, prescreen can be carried out to catalogue, all catalogues need not be traveled through, ensure under minimum resource use amount, complete data retrieval and calculating.
Fig. 5 is the main modular schematic diagram of the device that under a kind of distributed data platform according to the embodiment of the present invention, data store.As shown in Figure 5, the device 50 that under distributed data platform of the present invention, data store mainly comprises data categorization module 51, data memory module 52 and state updating module 53.
Data categorization module 51, for by the data in the data on the same day and data mode change list being compared, is classified to the data changed; Data memory module 52 for by described sorted data acquisition under different catalogues, and under being stored in corresponding subregion according to the data storage rule of described catalogue; And state updating module 53 is for upgrading described data mode change list.
Wherein, data categorization module 51 is carry out according to the process of the life cycle of data when carrying out Data classification, and comprises online class, expired class and filing class three types.
Data categorization module 51 can also be used for the key name by searching data, the data in the data on the same day and data mode change list is compared; If do not have described data in described data mode change list, and have described data in the data on the described same day, then described data are online class; If have described data in described data mode change list and the data on the described same day, but the key assignments of described data is different, then the described data in described data mode change list are expired class, and the described data on the same day are online class; And if have described data in described data mode change list, and do not have described data in the data on the same day, then described data are filing class.
Data memory module 52 is when carrying out data and storing, and the described data storage rule of foundation comprises partition name, data time and data life 3 directory levels closing time, and described partition name comprises online subregion, expired subregion and filing subregion.
Data memory module 52 can also be used for, and the first class catalogue partition name of described online class data is online subregion, and second-level directory data time is maximum time, and three grades of catalogue data life closing times are maximum time; The first class catalogue partition name of described expired class data is expired subregion, and second-level directory data time is transformation period, and three grades of catalogue data life closing times are transformation period; And the first class catalogue partition name of described filing class data is filing subregion, second-level directory data time is transformation period, and three grades of catalogue data life closing times are maximum time.
State updating module 53 can also be used for inserting the key name of described online class data, key assignments, state change initial time and state change end time, and wherein, described state change initial time is transformation period, and the described state change end time is maximum time; And the described state change end time of described expired class data is set to transformation period.
Fig. 6 is the storage effectiveness comparison schematic diagram of the embodiment of the present invention and prior art.Compared with the buildup of increments processing mode of prior art, data storage scheme of the present invention effectively can save data space.For the table of one hundred million DBMS, this table to increase newly every day and the data volume that changes about 1,000,000 ranks, its space saving rate can be obtained by following formulae discovery.
In above formula, base: radix (hundred million grades), N: number of days, C: increment every day (1,000,000 grades), M: every daily variation (1,000,000 grades).When N is tending towards infinity, saving rate in space is 1, that is: time span is longer, saves space more.In actual applications, space saving rate can to more than 90%.As can be seen here, adopt technical scheme of the present invention effectively can save data space, the historical rudiment of total data can be retained with minimum storage.
According to the technical scheme of the embodiment of the present invention, only when data mode changes, just need to classify to these data, store and the operation such as state updating, for the data do not changed without the need to carrying out secondary storage or state updating, thus can while effectively recording data variation, improve the efficiency of data storage and data retrieval, effectively save data space, and also very easy and convenient to the cleaning of stale data.
Above-mentioned embodiment, does not form limiting the scope of the invention.It is to be understood that depend on designing requirement and other factors, various amendment, combination, sub-portfolio can be there is and substitute in those skilled in the art.Any amendment done within the spirit and principles in the present invention, equivalent replacement and improvement etc., all should be included within scope.
Claims (14)
1. the method that under distributed data platform, data store, is characterized in that, comprising:
By the data in the data on the same day and data mode change list are compared, the data changed are classified;
By described sorted data acquisition under different catalogues, and under being stored in corresponding subregion according to the data storage rule of described catalogue; And
Upgrade described data mode change list.
2. method according to claim 1, is characterized in that, described classification carries out according to the process of data life period, and comprises online class, expired class and filing class three types.
3. method according to claim 1 and 2, is characterized in that, comprises the step that the data changed are classified:
By searching the key name of data, the data in the data on the same day and data mode change list are compared;
If do not have described data in described data mode change list, and have described data in the data on the described same day, then described data are online class;
If have described data in described data mode change list and the data on the described same day, but the key assignments of described data is different, then the described data in described data mode change list are expired class, and the described data on the same day are online class; And
If have described data in described data mode change list, and do not have described data in the data on the same day, then described data are filing class.
4. method according to claim 1, is characterized in that, described data storage rule comprises partition name, data time and data life 3 directory levels closing time.
5. method according to claim 4, is characterized in that, described partition name comprises online subregion, expired subregion and filing subregion.
6. method according to claim 1, is characterized in that, the step be stored under corresponding subregion according to the data storage rule of described catalogue comprises:
The first class catalogue partition name of described online class data is online subregion, and second-level directory data time is maximum time, and three grades of catalogue data life closing times are maximum time;
The first class catalogue partition name of described expired class data is expired subregion, and second-level directory data time is transformation period, and three grades of catalogue data life closing times are transformation period; And
The first class catalogue partition name of described filing class data is filing subregion, and second-level directory data time is transformation period, and three grades of catalogue data life closing times are maximum time.
7. method according to claim 1, is characterized in that, the step upgrading described data mode change list comprises:
Insert the key name of described online class data, key assignments, state change initial time and state change end time, wherein, described state change initial time is transformation period, and the described state change end time is maximum time; And
The described state change end time of described expired class data is set to transformation period.
8. the device that under distributed data platform, data store, is characterized in that, comprising:
Data categorization module, for by the data in the data on the same day and data mode change list being compared, classifies to the data changed;
Data memory module, for by described sorted data acquisition under different catalogues, and under being stored in corresponding subregion according to the data storage rule of described catalogue; And
State updating module, for upgrading described data mode change list.
9. device according to claim 8, is characterized in that, described classification carries out according to the process of the life cycle of data, and comprises online class, expired class and filing class three types.
10. device according to claim 8 or claim 9, is characterized in that, described data categorization module also for:
By searching the key name of data, the data in the data on the same day and data mode change list are compared;
If do not have described data in described data mode change list, and have described data in the data on the described same day, then described data are online class;
If have described data in described data mode change list and the data on the described same day, but the key assignments of described data is different, then the described data in described data mode change list are expired class, and the described data on the same day are online class; And
If have described data in described data mode change list, and do not have described data in the data on the same day, then described data are filing class.
11. devices according to claim 8, is characterized in that, described data storage rule comprises partition name, data time and data life 3 directory levels closing time.
12. devices according to claim 11, is characterized in that, described partition name comprises online subregion, expired subregion and filing subregion.
13. devices according to claim 8, is characterized in that, described data memory module also for:
The first class catalogue partition name of described online class data is online subregion, and second-level directory data time is maximum time, and three grades of catalogue data life closing times are maximum time;
The first class catalogue partition name of described expired class data is expired subregion, and second-level directory data time is transformation period, and three grades of catalogue data life closing times are transformation period; And
The first class catalogue partition name of described filing class data is filing subregion, and second-level directory data time is transformation period, and three grades of catalogue data life closing times are maximum time.
14. devices according to claim 8, is characterized in that, described state updating module also for:
Insert the key name of described online class data, key assignments, state change initial time and state change end time, wherein, described state change initial time is transformation period, and the described state change end time is maximum time; And
The described state change end time of described expired class data is set to transformation period.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510598396.9A CN105183391B (en) | 2015-09-18 | 2015-09-18 | The method and apparatus that data store under a kind of distributed data platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510598396.9A CN105183391B (en) | 2015-09-18 | 2015-09-18 | The method and apparatus that data store under a kind of distributed data platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105183391A true CN105183391A (en) | 2015-12-23 |
| CN105183391B CN105183391B (en) | 2018-12-28 |
Family
ID=54905500
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510598396.9A Active CN105183391B (en) | 2015-09-18 | 2015-09-18 | The method and apparatus that data store under a kind of distributed data platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105183391B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106897423A (en) * | 2017-02-24 | 2017-06-27 | 郑州云海信息技术有限公司 | A kind of cloud platform junk data processing method and system |
| CN108052281A (en) * | 2017-11-30 | 2018-05-18 | 平安科技（深圳）有限公司 | Business Information storage method, application server and computer storage media |
| CN109145052A (en) * | 2018-07-12 | 2019-01-04 | 北京炎黄盈动科技发展有限责任公司 | Data partition storage method, equipment, system, storage medium and electronic equipment |
| CN110347655A (en) * | 2019-06-12 | 2019-10-18 | 江苏富山软件科技有限公司 | A kind of distributed file system access frame |
| CN110865990A (en) * | 2019-10-12 | 2020-03-06 | 中国平安财产保险股份有限公司 | Non-real-time data exchange method, system and computer equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101876983A (en) * | 2009-04-30 | 2010-11-03 | 国际商业机器公司 | Method for partitioning database and system thereof |
| CN102141963A (en) * | 2010-01-28 | 2011-08-03 | 阿里巴巴集团控股有限公司 | Method and equipment for analyzing data |
| CN102567428A (en) * | 2010-12-30 | 2012-07-11 | 中国移动通信集团浙江有限公司 | Method and device for managing life cycle of online data |
| CN103186566A (en) * | 2011-12-28 | 2013-07-03 | 中国移动通信集团河北有限公司 | Data classification storage method, device and system |
| CN103838787A (en) * | 2012-11-27 | 2014-06-04 | 阿里巴巴集团控股有限公司 | Method and device for updating distributed data warehouse |
-
2015
- 2015-09-18 CN CN201510598396.9A patent/CN105183391B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101876983A (en) * | 2009-04-30 | 2010-11-03 | 国际商业机器公司 | Method for partitioning database and system thereof |
| CN102141963A (en) * | 2010-01-28 | 2011-08-03 | 阿里巴巴集团控股有限公司 | Method and equipment for analyzing data |
| CN102567428A (en) * | 2010-12-30 | 2012-07-11 | 中国移动通信集团浙江有限公司 | Method and device for managing life cycle of online data |
| CN103186566A (en) * | 2011-12-28 | 2013-07-03 | 中国移动通信集团河北有限公司 | Data classification storage method, device and system |
| CN103838787A (en) * | 2012-11-27 | 2014-06-04 | 阿里巴巴集团控股有限公司 | Method and device for updating distributed data warehouse |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106897423A (en) * | 2017-02-24 | 2017-06-27 | 郑州云海信息技术有限公司 | A kind of cloud platform junk data processing method and system |
| CN108052281A (en) * | 2017-11-30 | 2018-05-18 | 平安科技（深圳）有限公司 | Business Information storage method, application server and computer storage media |
| CN109145052A (en) * | 2018-07-12 | 2019-01-04 | 北京炎黄盈动科技发展有限责任公司 | Data partition storage method, equipment, system, storage medium and electronic equipment |
| CN109145052B (en) * | 2018-07-12 | 2021-10-08 | 北京炎黄盈动科技发展有限责任公司 | Data partition storage method, device, system, storage medium and electronic device |
| CN110347655A (en) * | 2019-06-12 | 2019-10-18 | 江苏富山软件科技有限公司 | A kind of distributed file system access frame |
| CN110865990A (en) * | 2019-10-12 | 2020-03-06 | 中国平安财产保险股份有限公司 | Non-real-time data exchange method, system and computer equipment |
| CN110865990B (en) * | 2019-10-12 | 2024-04-30 | 中国平安财产保险股份有限公司 | Non-real-time data exchange method, system and computer equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105183391B (en) | 2018-12-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101127034B (en) | Data organization, inquiry, presentation, documentation, recovery, deletion, refining method, device and system | |
| CN102930060B (en) | A kind of method of database quick indexing and device | |
| Mamoulis et al. | Mining, indexing, and querying historical spatiotemporal data | |
| CN105183391A (en) | Method and device for storing data under distributed data platform | |
| CN102339315B (en) | Index updating method and system of advertisement data | |
| CN102375853A (en) | Distributed database system, method for building index therein and query method | |
| Wu | FastBit: an efficient indexing technology for accelerating data-intensive science | |
| CN104933133A (en) | Metadata snap storage and access method in distributed file system | |
| US10108690B1 (en) | Rolling subpartition management | |
| CN110109910A (en) | Data processing method and system, electronic equipment and computer readable storage medium | |
| CN102467521A (en) | Easily-extensible multi-level classification search method and system | |
| KR20100015368A (en) | A method of data storage and management | |
| CN102779138A (en) | Hard disk access method of real time data | |
| CN110096509A (en) | Realize that historical data draws the system and method for storage of linked list modeling processing under big data environment | |
| US10055442B2 (en) | Efficient updates in non-clustered column stores | |
| CN102411632B (en) | Chain table-based memory database page type storage method | |
| CN101963993B (en) | Method for fast searching database sheet table record | |
| US10613988B2 (en) | Purging storage partitions of databases | |
| CN103186668A (en) | Method and device for processing data as well as data storage system based on key value data base | |
| US20140304266A1 (en) | Data base indexing | |
| CN101408882B (en) | Method and system for searching authorization document | |
| CN104391992A (en) | Asset data-oriented data processing system | |
| CN110245118B (en) | BIM data information three-dimensional gridding retrieval filing method and filing system thereof | |
| Nørvåg | Supporting temporal text-containment queries in temporal document databases | |
| CN104866508A (en) | Method and device for managing files in cloud environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |