CN108664499A - The method, apparatus and equipment of data storage - Google Patents
The method, apparatus and equipment of data storage Download PDFInfo
- Publication number
- CN108664499A CN108664499A CN201710200052.7A CN201710200052A CN108664499A CN 108664499 A CN108664499 A CN 108664499A CN 201710200052 A CN201710200052 A CN 201710200052A CN 108664499 A CN108664499 A CN 108664499A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- fractionated
- tables
- strategy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention discloses the method, apparatus and equipment of a kind of storage of data.This method includes：Read the configuration parameter of data staging storage；The data type of storage data to be fractionated is obtained from configuration parameter；Data staging storage strategy corresponding with data type is determined according to data type；According to data staging storage strategy, classification storage is carried out to storage data to be fractionated.The embodiment of the present invention can reduce the pressure and cost of data storage, improve the stability of system database, improve operational efficiency.
Description
Technical field
The present invention relates to method, apparatus and equipment that field of communication technology more particularly to a kind of data store.
Background technology
Currently, the critical data in more business event support system is frequently stored on the system database of relationship type, this
The retention cycle requirement of some data is persistence in a little critical datas.So over time, in system database
Store data scale it is more and more huger, carrying cost is also higher and higher, cause system operation efficiency can it is lower and lower under.
For lifting system operational efficiency, database and service deployment are split in the prior art.For example, according to region pair
Database and service deployment are split.But this mode only physically splits data, data Inner Constitution
Do not change, core data includes a large amount of invalid datas, it is easy to cause system database stability to decline, it can not be effective
Promote operational efficiency.
Invention content
An embodiment of the present invention provides the method, apparatus and equipment of a kind of storage of data, can solve database and industry
The problem of mode that business deployment is split causes system database stability to decline, can not effectively promote operational efficiency.
In a first aspect, an embodiment of the present invention provides a kind of methods of data storage, including：
Read the configuration parameter of data staging storage；
The data type of storage data to be fractionated is obtained from configuration parameter；
Data staging storage strategy corresponding with data type is determined according to data type；
According to data staging storage strategy, classification storage is carried out to storage data to be fractionated.
Second aspect, an embodiment of the present invention provides a kind of devices of data storage, including：
Reading unit, the configuration parameter for reading data staging storage；
Acquiring unit, the data type for obtaining storage data to be fractionated from configuration parameter；
Determination unit, for determining data staging storage strategy corresponding with data type according to data type；
Storage unit, for according to data staging storage strategy, classification storage to be carried out to storage data to be fractionated.
The third aspect, an embodiment of the present invention provides a kind of equipment of data storage, including：
Memory, processor, communication interface and bus；
Memory, processor and communication interface are connected by bus and complete mutual communication；
Memory is for storing program code；
Processor is corresponding with executable program code to run by reading the executable program code stored in memory
Program, the method for the method for executing a kind of storage of data, data storage includes：
Read the configuration parameter of data staging storage；
The data type of storage data to be fractionated is obtained from configuration parameter；
Data staging storage strategy corresponding with data type is determined according to data type；
According to data staging storage strategy, classification storage is carried out to storage data to be fractionated.
An embodiment of the present invention provides the method, apparatus and equipment of a kind of storage of data, in the embodiment of the present invention, read first
The configuration parameter for taking data staging to store, and the data type to be fractionated for storing data is obtained from configuration parameter；Then basis
Data type determines corresponding with data type data staging storage strategy, and then according to data staging storage strategy, treats point
Grade storage data carry out classification storage.The embodiment of the present invention configures parameter by setting, can be to the data of different types of data
Classification storage is carried out using different data staging storage strategies, and then reduces the pressure and cost of data storage, and is different
The data of type use different data staging storage strategies, store data according to data type, avoid various types number
The case where being stored according to mixing improves the stability of system database, improves operational efficiency.
Description of the drawings
In order to illustrate the technical solution of the embodiments of the present invention more clearly, will make below to required in the embodiment of the present invention
Attached drawing is briefly described, it should be apparent that, drawings described below is only some embodiments of the present invention, for
For those of ordinary skill in the art, without creative efforts, other are can also be obtained according to these attached drawings
Attached drawing.
Fig. 1 is the schematic flow chart of the method for the data storage provided according to one embodiment of the invention；
Fig. 2 is the schematic block diagram of the device of the data storage provided according to one embodiment of the invention；
Fig. 3 is the schematic block diagram of the equipment of the data storage provided according to one embodiment of the invention.
Specific implementation mode
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
The every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase
Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Data suitable for database of the embodiment of the present invention carry out the scene of classification storage.Due to what is stored in database
Data meet increase, in order to alleviate the storage pressure of database, can set the Data Migration of database purchase to other storages
The standby classification storage for realizing data.It is provided with configuration parameter in the embodiment of the present invention, starts number by configuring the setting of parameter
According to the task of classification storage.When executing the task of classification storage, the number of storage data to be fractionated is determined by configuring parameter
It according to type, and then determines the strategy of classification storage, and classification storage is carried out to storage data to be fractionated, to realize to data
The classification storage of data in library.
Fig. 1 shows the schematic flow chart of the method for the data storage provided according to one embodiment of the invention.Such as Fig. 1 institutes
Show, this approach includes the following steps.
110, read the configuration parameter of data staging storage.
Wherein, configuration parameter can be to pre-set, and it may include the number for executing the data that classification stores to configure in parameter
According to type, and the data staging storage strategy of classification storage is executed to different types of data.
It should be noted that can also include the time for executing data staging store tasks in configuration parameter, data be executed
The time of classification storage is usually chosen in business offpeak period, for example, zero.
120, the data type of storage data to be fractionated is obtained from configuration parameter.
Wherein, after reading configuration parameter, you can this data type for executing classification storage is obtained from configuration parameter,
Data type may include daily record data, subscriber data and assessment data.After determining data type, that is, indicate this data
It is to carry out classification storage to that data to be classified store tasks.
130, data staging storage strategy corresponding with data type is determined according to data type.
Wherein, due to the data of different types of data, the mode for being classified storage is different, so obtaining storage to be fractionated
After the data type of data, it is thus necessary to determine that go out data staging storage strategy corresponding with data type.
It should be noted that the data staging storage strategy of different types of data can be pre-set, such as join in configuration
The data staging storage strategy of different types of data is set in number.
140, according to data staging storage strategy, classification storage is carried out to storage data to be fractionated.
Wherein, after the data staging storage strategy that step 130 determines that this data staging store tasks to be executed, i.e.,
Classification storage can be carried out to storage data to be fractionated according to data staging storage strategy.
In the embodiment of the present invention, the configuration parameter of data staging storage is read first, and obtain and wait for point from configuration parameter
The data type of grade storage data；Then data staging storage strategy corresponding with data type is determined according to data type, into
And according to data staging storage strategy, classification storage is carried out to storage data to be fractionated.The embodiment of the present invention is configured by being arranged
Parameter can carry out classification storage using different data staging storage strategies to the data of different types of data, and then reduce
The pressure and cost of data storage, and different types of data use different data staging storage strategies, according to data class
Type stores data, the case where avoiding the storage of various types data mixing, improves the stability of system database, improves operation
Efficiency.
It should be noted that store tasks can be classified by configuring parameter setting data in the embodiment of the present invention, and
The processing data type of data staging store tasks, processing time, period etc. successively carry out data in database to realize
Periodical automatic classification storage, reduces the pressure and cost of data storage, improves the stability of system database, improves operation effect
Rate.
For example, indicating task names with TASK_NAME, the processing number of data staging store tasks is indicated with TASK_TYPE
According to type, including：Daily record data, subscriber data class and assessment data etc., task execution periodic group is indicated with CYCLETYPE
Type, including：It monthly executes, per diem execute and quarterly execute etc., the task execution interval period is indicated with INTERVAL, it can be with
It is used cooperatively with CYCLETYPE, next run time is indicated with NEXTDATE, is defaulted as same day system time, format for the first time
For YYYY/MM/DD HH24：MI：SS, can also manual configuration such as " 2,017,/02,/01 23：00：00 ", after executing a subtask
Automatically update next run time.
Example：If CYCLETYPE period types are set as monthly clearing up, INTERVAL gap periods are 1, then this
Business executes on 2 1st, 2,017 23 time：00：00, then next task execution time is 1 day 23 March in 2017：00：00.
It is understood that in embodiments of the present invention, data type may include daily record data, subscriber data and assessment
Data determine different data types different data staging storage strategies, so step 120 can also be executed specifically is
Following steps.
Step 121, data type is daily record data, determines that data staging storage strategy is daily record storage strategy；Step
122, data type is subscriber data, determines that data staging storage strategy is subscriber data storage strategy；Step 123, data class
Type is assessment data, determines that data staging storage strategy is assessment data store strategy.
Wherein, in the case that data type is daily record data, subscriber data or assessment data, different data are respectively adopted
It is classified storage strategy.
In this way, different types of data use different data staging storage strategies, data are stored according to data type,
The case where avoiding the storage of various types data mixing, improves the stability of system database, improves operational efficiency.
In one optional embodiment of the embodiment of the present invention, in embodiments of the present invention, log-rank storage strategy
It may include following process：Currently stored information and preset data based on configuration parameter acquiring storage data to be fractionated preserve week
Phase；Storage data to be fractionated are filtered out according to currently stored information and data retention cycle；Extremely by storage Data Migration to be fractionated
First storage device；Delete storage data to be fractionated.
Wherein, since daily record data has time effect, so can be preset when carrying out classification storage to daily record data
Data retention cycle.Configuring in parameter can also include that the currently stored information for storing data and preset data to be fractionated preserve week
Phase.The currently stored information of storage data to be fractionated illustrates that this data staging store tasks will be classified those data
Storage, can be from database by this storage data screening to be fractionated by currently stored information and preset data retention cycle
Out, then by the storage Data Migration to other storage devices to be fractionated screened, what is screened in database waits for
Classification storage data can be deleted.
Specifically, currently stored information includes field in the mark and tables of data for store the tables of data to be fractionated for storing data
Field value.
The tables of data for storing storage data to be fractionated indicates that this data staging store tasks is to data in this tables of data
It is handled, the field value of field indicates that field value is equal to setting field value in currently stored information in tables of data in tables of data
Field, the data corresponding to these fields are the storage number to be fractionated of this data staging store tasks storage of being classified
According to.
For example, it is assumed that in configuring parameter, the mark of the tables of data of currently stored information is expressed as BAKTAB, in tables of data
The field value of field is expressed as CONDITION, and preset data retention cycle is expressed as CYCLE.In the embodiment of the present invention, with
BAKTABLE values be " RECEPTION ", CONDITION values be " REGION=311 ", CYCLE values be " 3 months " for said
It is bright.BAKTABLE values are RECEPTION, indicate that the RECEPTION that is identified as of tables of data, i.e. this data staging store tasks are
It handles being identified as data in the tables of data of RECEPTION.CONDITION values are " REGION=311 ", indicate REGION
To be identified as the field in the tables of data of RECEPTION, 311 be the field value of RECEPTION fields, the storage of this data staging
Task is carried out to being identified as data corresponding when the field value of RECEPTION fields in the tables of data of RECEPTION is 311
Processing.CYCLE values are " 3 months ", indicate that this data staging store tasks is to 3rd month data before current month
It is handled.Assuming that current month is in March, 2017, then the in summary value of each parameter, you can obtain：It is identified as
In the tables of data of RECEPTION, the field value of RECEPTION fields be in data corresponding to 311 time be in December, 2016
Daily record data be storage data to be fractionated, this data staging store tasks is：To being identified as the tables of data of RECEPTION
In, the field values of RECEPTION fields be in data corresponding to 311 time be that the daily record data in December, 2016 is classified
Processing.
It should be noted that can storing daily record data, tables of data subregion can be by by way of subregion in database
It is defined according to districts and cities and month, being then based on parameter in appeal distance can show that storage data to be fractionated are to be identified as RECEPTION
Tables of data in 311201612 subregions data.The parameters being also based in the embodiment of the present invention in configuration parameter will
The storage data to be fractionated filtered out generate migration inventory, then carry out classification storage processing to the data in migration inventory.The
One storage device can be the storage devices such as tape library.
For daily record class data, the embodiment of the present invention can automatically generate cleaning script, can be according to the subregion of tables of data
Data staging storage is carried out, the migration in the storage devices such as Data Migration to tape library, realizing historical data and classification are stored.
In one optional embodiment of the embodiment of the present invention, in embodiments of the present invention, subscriber data storage strategy
It may include following process：Currently stored information based on configuration parameter acquiring storage data to be fractionated and storage data to be fractionated
Purpose store information；Go out storage data to be fractionated according to currently stored information sifting；It will be to be fractionated according to purpose storage information
Data Migration is stored to the second storage device；Delete storage data to be fractionated.
Wherein, configuring in parameter can also include that the currently stored information for storing data and purpose to be fractionated store information.
The currently stored information of storage data to be fractionated illustrates which data is this data staging store tasks will carry out classification and deposit
Storage, this storage data screening to be fractionated can be come out, then will screen by currently stored information from database
Storage Data Migration to other storage devices to be fractionated in, the storage data to be fractionated screened in database can carry out
It deletes.Purpose storage information refers to by storage letter when being stored in storage Data Migration to other storage devices to be fractionated
Breath, storage data to be fractionated can store information according to purpose in other storage devices and be stored.
Specifically, currently stored information include the marks of tables of data of storage storage data to be fractionated, the field of tables of data,
The Correlation Criteria of data in the mark and tables of data of the contingency table of tables of data；Purpose storage information includes being deposited in the second storage device
Store up the mark of the tables of data of storage data to be fractionated.
The tables of data for storing storage data to be fractionated indicates that this data staging store tasks is to data in this tables of data
It is handled, the field value of field indicates that field value is equal to setting field value in currently stored information in tables of data in tables of data
Field.Due to that may have incidence relation between each data in subscriber data, so also setting up association in the embodiment of the present invention
Table and Correlation Criteria, so as to filter out comprehensive subscriber data.The mark of contingency table is indicated to exist with tables of data and is associated with
The mark of the tables of data of system, the Correlation Criteria of data indicates Correlation Criteria existing for data in tables of data in tables of data, passes through pass
Bracing part can determine the data with Correlation Criteria existing for data in tables of data.To sum up, by storing storage number to be fractionated
According to the mark of tables of data, the field of tables of data, tables of data contingency table mark, and/or tables of data in data association item
Part can filter out the data to be migrated of this data staging store tasks.Storage data to be fractionated are stored in second storage device
Tables of data mark, indicate Data Migration to be migrated stored after the second storage device to purpose storage information setting mark
In tables of data.
For example, it is assumed that in configuring parameter, the mark of the tables of data of currently stored information is expressed as SOURCETAB, data
The field value of field is expressed as CONDITION in table, and the mark of the contingency table of tables of data is expressed as RELATETAB, in tables of data
The Correlation Criteria of data is expressed as RELATECON, and the mark that purpose stores tables of data in information is expressed as TARGETTAB.This hair
In bright embodiment, with TARGETTABLE values for " CUSTOMER ", CONDITION values are " REGION=311 ", RELATETAB values
For " ACCOUNT ", RELATECON1 values is for " A.CUSTID=B.ACCTID ".SOURCETAB values are CUSTOMER, are indicated
Tables of data is identified as CUSTOMER, i.e., this data staging store tasks is to being identified as data in the tables of data of CUSTOMER
It is handled.CONDITION values are " REGION=311 ", indicate that REGION is the word being identified as in the tables of data of CUSTOMER
Section, 311 be the field value of RECEPTION fields, this data staging store tasks is the tables of data to being identified as CUSTOMER
The field value of middle RECEPTION fields data corresponding when being 311 are handled.RELATETAB1 values are " ACCOUNT " table
Show and be identified as the tables of data of CUSTOMER there are the marks of the table of incidence relation, i.e., this data staging store tasks is also to mark
Know data in the associated data table for being ACCOUNT to be handled, RELATECON1 values indicate for " A.CUSTID=B.ACCTID "
The incidence relation of A.CUSTID and B.ACCTID is filtered out by this incidence relation from the associated data table for being identified as ACCOUNT
Associated data.It can show that the storage data to be fractionated of this data staging store tasks include being identified as CUSTOMER's on total
Corresponding data and the associated data table for being identified as ACCOUNT when the field value of RECEPTION fields is 311 in tables of data
In, with associated data that the data correlation condition in the tables of data of CUSTOMER of being identified as is A.CUSTID and B.ACCTID.
It should be noted that can be led to according to the process of configuration choice of parameters storage data to be fractionated in the embodiment of the present invention
Cross the execution of number collection process.TARGETTABLE values are screened in the example above is for " CUSTOMER " and CONDITION values
The data of " REGION=311 " can be expressed as " SELECT*FROM CUSTOMER by structured query language SQL
WHEREREGION=311 ", is further " ACCOUNT " in conjunction with RELATETAB1 values and RELATECON1 values are
When " A.CUSTID=B.ACCTID " two parameters, " SELECT B.*FROM can be expressed as by structured query language SQL
CUSTOMER A, ACCOUNT B WHERE A.REGION=311and A.CUSTIDID=B.ACCTID ".The present invention is implemented
The storage data to be fractionated filtered out are generated migration inventory by the parameters being also based in example in configuration parameter, then right
Data in migration inventory carry out classification storage processing.Second storage device can be the storage devices such as history library.
Production efficiency is influenced in compared with the prior art, or even needs to stop the mode that business service carries out data fractionation,
In the embodiment of the present invention, the Data Migration flow recycled according to cancellation class number is executed, and is needed according to contingency table and correlation
Data Migration to history library is realized classification processing by SQL processing.By making data staging Stored Procedure in order, data scrubbing is completed
Automation carries out asynchronous data cleaning on backstage according to classification Stored Procedure, does not influence production system effect for user's cancellation
Rate.
In one optional embodiment of the embodiment of the present invention, in embodiments of the present invention, data store strategy is assessed
It may include following process：Based on configuration parameter acquiring data access information, it is to be fractionated storage data currently stored information and
The purpose of storage data to be fractionated stores information；According to currently stored information and data access information, screened by assessment algorithm
Go out to meet the storage data to be fractionated for presetting evaluation index；Information is stored by storage Data Migration to be fractionated to second according to purpose
Storage device；Delete storage data to be fractionated.
Wherein, configure parameter in can also include data access information, it is to be fractionated storage data currently stored information and
Purpose stores information.The currently stored information of storage data to be fractionated illustrates which this data staging store tasks will count
According to classification storage is carried out, according to the data access information of these data, assessed by default assessment algorithm, then basis is commented
Estimate result and filters out the data for meeting and presetting evaluation index, storage data as to be fractionated.Then to be fractionated by what is screened
It stores in Data Migration to other storage devices, the storage data to be fractionated screened in database can be deleted.Mesh
Storage information refer to by it is to be fractionated storage Data Migration to other storage devices in stored when storage information, it is to be fractionated
Storing data can be stored in other storage devices according to purpose storage information.
It should be noted that default assessment algorithm includes mainly for the data assessment for not influencing to use in database
Middle table data, the data of personal user and interim table data.Least recently used principle may be used (such as in default assessment algorithm
At least use algorithm LRU in the recent period) assessed, specific algorithm it is the recently defined for how long, it is minimum using definition why
Frequency can be configured according to the concrete condition of actual scene.
Specifically, currently stored information includes the mark of the tables of data of storage storage data to be fractionated, data access information
The number of the data modification number of data query number, tables of data including tables of data, the data deletion number of tables of data, tables of data
According to the last data access time for increasing number and tables of data；Default evaluation index includes pre- within default access cycle
If visitation frequency；Purpose storage information includes the mark for the tables of data that storage data to be fractionated are stored in the second storage device.
The tables of data for storing storage data to be fractionated indicates that this data staging store tasks is to data in this tables of data
It is handled.Since data need to be assessed according to data access information in database, so the data of tables of data is needed to visit
It asks information, includes the information access, delete to data, increasing, changing, inquiring.Default evaluation index is included in default visit
Ask the default visitation frequency in the period, i.e., visitation frequency reaches the default data for accessing flat time within default access cycle in screening
As storage data to be fractionated.To sum up, by storing the marks of tables of data of storage data to be fractionated, data access information and pre-
If evaluation index can filter out the data to be migrated of this data staging store tasks.It is stored in second storage device to be fractionated
The mark for storing the tables of data of data indicates that Data Migration to be migrated is stored after the second storage device to purpose storage information and sets
It sets in the tables of data of mark.
For example, it is assumed that in configuring parameter, the mark of the tables of data of currently stored information is expressed as SOURCETAB, presets
Visitation frequency is expressed as FREQUENT, is expressed as CYCLE default access cycles, and the mark that purpose stores tables of data in information indicates
For TARGETTAB.In the embodiment of the present invention, with SOURCETAB values for " A ", FREQUENT values are " 0 ", and CYCLE values are " 6
Month " for.SOURCETAB values are A, indicate the A that is identified as of tables of data, i.e., this data staging store tasks is to being identified as A
Tables of data in data handled.FREQUENT values indicate that this data staging store tasks is the data to being identified as A for 0
The data that visitation frequency is 0 in table are handled, and are 6 months in conjunction with CYCLE values, then it is known that the storage of this data staging is appointed
Business is handled being identified as in the tables of data of A the data that visitation frequency is 0 in 6 months, i.e., storage data to be fractionated are
The data that visitation frequency is 0 in 6 months are identified as in the tables of data of A.
It should be noted that data access scenarios dictionary table can be arranged in tables of data in database, number in tables of data is recorded
According to access information, the table can when accessing tables of data using the increasing of trigger autographic recorder, the number deleted, change, inquired and
The time that last time accesses.For example, indicating the table name of tables of data with SOURCETAB, SELECT indicates the inquiry time of tables of data
Number, UPDATE indicate that the modification number of tables of data, DELETE indicate that the deletion number of tables of data, INSERT indicate the increasing of tables of data
Number, STATUSDATATIME is added to indicate the time that the last time of tables of data accesses.Default assessment algorithm is according to data access
The visitation frequency and last access time of content evaluation data in information, can then proceed in FREQUERNT's and CYCLE
Parameter value carries out " AND " operation, and then can produce the tables of data inventory of storage data to be fractionated.May be used also in the embodiment of the present invention
The storage data to be fractionated filtered out are generated migration inventory based on the parameters in configuration parameter, then to migrating inventory
In data carry out classification storage processing.Second storage device can be the storage devices such as history library.
It, can be by the embodiment of the present invention in compared with the existing technology in such a way that artificial judgment determines migrating data
It is based on lru algorithm according to least recently used principle and carries out data assessment, to assess the value of data, the low number of screening bid value
According to, and then determine whether to be classified storage by migrating, it would be desirable to the storage devices such as Data Migration of migration to history library are based on
The value of data automatically completes data staging storage.
It is right after determining the data staging storage strategy corresponding to the data of variant data type in the above-described embodiments
When storage data to be fractionated carry out classification storage, you can executed automatically according to the process in strategy that determines.It is moved in database deletion
After the storage data to be fractionated moved, it can also update the data and handle state recording processing daily record, or cleaning script is clear in storage facility located at processing plant
Reason has backed up history log.The method for carrying out classification storage for the data of different types of data is realized in above-described embodiment,
It plans the data staging storage strategy of existing and newly-increased entity, realizes that the definition of business historical data migration rules is moved with automation
It moves, realizes the classificationization storage of data.
It in embodiments of the present invention, can also be by being configured to configuration parameter, to record the letter of data staging storage
Breath, and then may be implemented to be monitored data staging storage and historical information is inquired.
Fig. 2 is the schematic block diagram of the device 200 of data storage according to an embodiment of the invention.As shown in Fig. 2, the dress
Setting 200 includes：
Reading unit 201, the configuration parameter for reading data staging storage；
Acquiring unit 202, the data type for obtaining storage data to be fractionated from the configuration parameter；
Determination unit 203, for determining that data staging corresponding with the data type stores according to the data type
Strategy；
Storage unit 204, for according to the data staging storage strategy, being classified to the storage data to be fractionated
Storage.
In the embodiment of the present invention, the configuration parameter of data staging storage is read first, and obtain and wait for point from configuration parameter
The data type of grade storage data；Then data staging storage strategy corresponding with data type is determined according to data type, into
And according to data staging storage strategy, classification storage is carried out to storage data to be fractionated.The embodiment of the present invention is configured by being arranged
Parameter can carry out classification storage using different data staging storage strategies to the data of different types of data, and then reduce
The pressure and cost of data storage, and different types of data use different data staging storage strategies, according to data class
Type stores data, the case where avoiding the storage of various types data mixing, improves the stability of system database, improves operation
Efficiency.
It is understood that determination unit 203 is specifically used for：
Data type is daily record data, determines that data staging storage strategy is daily record storage strategy；
Data type is subscriber data, determines that data staging storage strategy is subscriber data storage strategy；
Data type is assessment data, determines that data staging storage strategy is assessment data store strategy.
It is understood that log-rank storage strategy, including：
Currently stored information and preset data retention cycle based on configuration parameter acquiring storage data to be fractionated；
Storage data to be fractionated are filtered out according to currently stored information and data retention cycle；
By storage Data Migration to be fractionated to the first storage device；
Delete storage data to be fractionated.
It is understood that currently stored information includes the mark and tables of data of the tables of data of storage storage data to be fractionated
Field.
It is understood that subscriber data storage strategy, including：
The purpose of currently stored information and storage data to be fractionated based on configuration parameter acquiring storage data to be fractionated is deposited
Store up information；
Go out storage data to be fractionated according to currently stored information sifting；
Information is stored by storage Data Migration to be fractionated to the second storage device according to purpose；
Delete storage data to be fractionated.
It is understood that currently stored information includes mark, the tables of data of the tables of data of storage storage data to be fractionated
Field, tables of data contingency table mark and tables of data in data Correlation Criteria；
Purpose storage information includes the mark for the tables of data that storage data to be fractionated are stored in the second storage device.
It is understood that assessment data store strategy, including：
Based on configuration parameter acquiring data access information, the currently stored information of storage data to be fractionated and storage to be fractionated
The purpose of data stores information；
According to currently stored information and data access information, filters out to meet by assessment algorithm and preset waiting for for evaluation index
Classification storage data；
Information is stored by storage Data Migration to be fractionated to the second storage device according to purpose；
Delete storage data to be fractionated.
It is understood that currently stored information includes the mark of the tables of data of storage storage data to be fractionated, data are visited
Ask that information includes the data query number of tables of data, the data modification number of tables of data, the data deletion number of tables of data, data
The data of table increase the last data access time of number and tables of data；
Default evaluation index includes the default visitation frequency within default access cycle；
Purpose storage information includes the mark for the tables of data that storage data to be fractionated are stored in the second storage device.
The device 200 of data storage according to the ... of the embodiment of the present invention can correspond to data storage according to the ... of the embodiment of the present invention
Method in executive agent, and data storage device 200 in modules above and other operation and/or function
Respectively in order to realize the corresponding flow of each method in Fig. 1, for sake of simplicity, details are not described herein.
Fig. 3 is the schematic block diagram of the equipment 300 of data storage according to an embodiment of the invention.As shown in figure 3, equipment
300 include processor 301, memory 302 and communication interface 303, and memory 302 is located for storing executable program code
Reason device 301 runs journey corresponding with executable program code by reading the executable program code stored in memory 302
Sequence, communication interface 303 is used for and external device communication, and equipment 300 can also include bus 304, and bus 304 is used for connection processing
Device 301, memory 302 and communication interface 303, make processor 301, memory 302 and communication interface 303 by bus 304 into
Row is in communication with each other.
Specifically, processor 301 is additionally operable to execute a kind of method of data storage；Wherein, the method for data storage includes：
Read the configuration parameter of data staging storage；
The data type of storage data to be fractionated is obtained from configuration parameter；
Data staging storage strategy corresponding with data type is determined according to data type；
According to data staging storage strategy, classification storage is carried out to storage data to be fractionated.
The equipment 300 of data storage according to the ... of the embodiment of the present invention can correspond to data storage according to the ... of the embodiment of the present invention
Method in executive agent, and data storage equipment 300 in modules above and other operation and/or function
Respectively in order to realize the corresponding flow of each method in Fig. 1, for sake of simplicity, details are not described herein.
Those of ordinary skill in the art may realize that lists described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware
With the interchangeability of software, each exemplary composition and step are generally described according to function in the above description.This
A little functions are implemented in hardware or software actually, depend on the specific application and design constraint of technical solution.Specially
Industry technical staff can use different methods to achieve the described function each specific application, but this realization is not
It is considered as beyond the scope of this invention.
It is apparent to those skilled in the art that for convenience of description and succinctly, foregoing description is
The specific work process of system, device and unit, can refer to corresponding processes in the foregoing method embodiment, details are not described herein.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can readily occur in various equivalent modifications or replace
It changes, these modifications or substitutions should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with right
It is required that protection domain subject to.
Claims (17)
1. a kind of method of data storage, which is characterized in that including：
Read the configuration parameter of data staging storage；
The data type of storage data to be fractionated is obtained from the configuration parameter；
Data staging storage strategy corresponding with the data type is determined according to the data type；
According to the data staging storage strategy, classification storage is carried out to the storage data to be fractionated.
2. the method for data storage according to claim 1, which is characterized in that described according to data type determination and institute
The corresponding data staging storage strategy of data type is stated, including：
The data type is the daily record data, determines that the data staging storage strategy is daily record storage strategy；
The data type is the subscriber data, determines that the data staging storage strategy is subscriber data storage strategy；
The data type is the assessment data, determines that the data staging storage strategy is assessment data store strategy.
3. the method for data storage according to claim 2, which is characterized in that the log-rank storage strategy, including：
Currently stored information and preset data retention cycle based on storage data to be fractionated described in the configuration parameter acquiring；
The storage data to be fractionated are filtered out according to the currently stored information and the data retention cycle；
By the storage Data Migration to be fractionated to the first storage device；
Delete the storage data to be fractionated.
4. the method for data storage according to claim 3, which is characterized in that the currently stored information includes described in storage
The field value of field in the mark and the tables of data of the tables of data of storage data to be fractionated.
5. the method for data storage according to claim 2, which is characterized in that the subscriber data storage strategy, including：
Based on the currently stored information of storage data to be fractionated and the storage data to be fractionated described in the configuration parameter acquiring
Purpose store information；
Go out the storage data to be fractionated according to the currently stored information sifting；
Information is stored by the storage Data Migration to be fractionated to the second storage device according to the purpose；
Delete the storage data to be fractionated.
6. the method for data storage according to claim 5, which is characterized in that the currently stored information includes described in storage
The identifying of tables of data of storage data to be fractionated, the mark of the contingency table of the field of the tables of data, the tables of data and described
The Correlation Criteria of data in tables of data；
The purpose storage information includes the mark for the tables of data that the storage data to be fractionated are stored in second storage device
Know.
7. the method for data storage according to claim 2, which is characterized in that the assessment data store strategy, including：
Based on the configuration parameter acquiring data access information, the currently stored information of the storage data to be fractionated and described wait for
The purpose of classification storage data stores information；
According to the currently stored information and the data access information, satisfaction is filtered out by assessment algorithm and presets evaluation index
Storage data to be fractionated；
Information is stored by the storage Data Migration to be fractionated to the second storage device according to the purpose；
Delete the storage data to be fractionated.
8. the method for data storage according to claim 7, which is characterized in that the currently stored information includes described in storage
The mark of the tables of data of storage data to be fractionated, the data access information includes the data query number of the tables of data, institute
The data of the data modification number, the tables of data of stating tables of data delete number, the data of the tables of data increase number and institute
State the last data access time of tables of data；
The default evaluation index includes the default visitation frequency within default access cycle；
The purpose storage information includes the mark for the tables of data that the storage data to be fractionated are stored in second storage device
Know.
9. a kind of device of data storage, which is characterized in that including：
Reading unit, the configuration parameter for reading data staging storage；
Acquiring unit, the data type for obtaining storage data to be fractionated from the configuration parameter；
Determination unit, for determining data staging storage strategy corresponding with the data type according to the data type；
Storage unit, for according to the data staging storage strategy, classification storage to be carried out to the storage data to be fractionated.
10. the device of data storage according to claim 9, which is characterized in that the determination unit is specifically used for：
The data type is the daily record data, determines that the data staging storage strategy is daily record storage strategy；
The data type is the subscriber data, determines that the data staging storage strategy is subscriber data storage strategy；
The data type is the assessment data, determines that the data staging storage strategy is assessment data store strategy.
11. the device of data storage according to claim 10, which is characterized in that the log-rank storage strategy, including：
Currently stored information and preset data retention cycle based on storage data to be fractionated described in the configuration parameter acquiring；
The storage data to be fractionated are filtered out according to the currently stored information and the data retention cycle；
By the storage Data Migration to be fractionated to the first storage device；
Delete the storage data to be fractionated.
12. the device stored according to data described in claim 11, which is characterized in that the currently stored information includes storage institute
State the field of the mark and the tables of data of the tables of data of storage data to be fractionated.
13. the device of data storage according to claim 10, which is characterized in that the subscriber data storage strategy, including：
Based on the currently stored information of storage data to be fractionated and the storage data to be fractionated described in the configuration parameter acquiring
Purpose store information；
Go out the storage data to be fractionated according to the currently stored information sifting；
Information is stored by the storage Data Migration to be fractionated to the second storage device according to the purpose；
Delete the storage data to be fractionated.
14. the device stored according to data described in claim 13, which is characterized in that the currently stored information includes storage institute
State the identifying of tables of data, the mark of the contingency table of the field of the tables of data, the tables of data and the institute of storage data to be fractionated
State the Correlation Criteria of data in tables of data；
The purpose storage information includes the mark for the tables of data that the storage data to be fractionated are stored in second storage device
Know.
15. the device of data storage according to claim 10, which is characterized in that the assessment data store strategy, including：
Based on the configuration parameter acquiring data access information, the currently stored information of the storage data to be fractionated and described wait for
The purpose of classification storage data stores information；
According to the currently stored information and the data access information, satisfaction is filtered out by assessment algorithm and presets evaluation index
Storage data to be fractionated；
Information is stored by the storage Data Migration to be fractionated to the second storage device according to the purpose；
Delete the storage data to be fractionated.
16. the device stored according to data described in claim 15, which is characterized in that the currently stored information includes storage institute
State it is to be fractionated storage data tables of data mark, the data access information include the tables of data data query number,
The data modification number of the tables of data, the tables of data data delete number, the data of the tables of data increase number and
The last data access time of the tables of data；
The default evaluation index includes the default visitation frequency within default access cycle；
The purpose storage information includes the mark for the tables of data that the storage data to be fractionated are stored in second storage device
Know.
17. a kind of equipment of data storage, which is characterized in that including：
Memory, processor, communication interface and bus；
Memory, processor and communication interface are connected by bus and complete mutual communication；
Memory is for storing program code；
Processor runs journey corresponding with executable program code by reading the executable program code stored in memory
Sequence, for executing a kind of method of data storage, the method for the data storage includes：
Read the configuration parameter of data staging storage；
The data type of storage data to be fractionated is obtained from the configuration parameter；
Data staging storage strategy corresponding with the data type is determined according to the data type；
According to the data staging storage strategy, classification storage is carried out to the storage data to be fractionated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710200052.7A CN108664499B (en) | 2017-03-29 | 2017-03-29 | Data storage method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710200052.7A CN108664499B (en) | 2017-03-29 | 2017-03-29 | Data storage method, device and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108664499A true CN108664499A (en) | 2018-10-16 |
| CN108664499B CN108664499B (en) | 2021-04-23 |
Family
ID=63786996
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710200052.7A Active CN108664499B (en) | 2017-03-29 | 2017-03-29 | Data storage method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108664499B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110081A (en) * | 2019-04-09 | 2019-08-09 | 国家计算机网络与信息安全管理中心 | Classification processing method and system for mobile Internet magnanimity monitoring data |
| CN110263044A (en) * | 2019-06-21 | 2019-09-20 | 深圳前海微众银行股份有限公司 | Date storage method, device, equipment and computer readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100185793A1 (en) * | 2009-01-22 | 2010-07-22 | Fujitsu Limited | Storage device and data transfer method for the same |
| CN102667772A (en) * | 2010-03-01 | 2012-09-12 | 株式会社日立制作所 | File level hierarchical storage management system, method, and apparatus |
| CN103902593A (en) * | 2012-12-27 | 2014-07-02 | 中国移动通信集团河南有限公司 | Data transfer method and device |
| CN105677650A (en) * | 2014-11-18 | 2016-06-15 | 中国移动通信集团公司 | Cache data storage method and device |
-
2017
- 2017-03-29 CN CN201710200052.7A patent/CN108664499B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100185793A1 (en) * | 2009-01-22 | 2010-07-22 | Fujitsu Limited | Storage device and data transfer method for the same |
| CN102667772A (en) * | 2010-03-01 | 2012-09-12 | 株式会社日立制作所 | File level hierarchical storage management system, method, and apparatus |
| CN103902593A (en) * | 2012-12-27 | 2014-07-02 | 中国移动通信集团河南有限公司 | Data transfer method and device |
| CN105677650A (en) * | 2014-11-18 | 2016-06-15 | 中国移动通信集团公司 | Cache data storage method and device |
Non-Patent Citations (1)
| Title |
|---|
| 王艳云: "《分级存储系统中数据迁移技术的研究》", 《万方数据库》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110081A (en) * | 2019-04-09 | 2019-08-09 | 国家计算机网络与信息安全管理中心 | Classification processing method and system for mobile Internet magnanimity monitoring data |
| CN110110081B (en) * | 2019-04-09 | 2021-05-11 | 国家计算机网络与信息安全管理中心 | Hierarchical classification processing method and system for mobile internet mass monitoring data |
| CN110263044A (en) * | 2019-06-21 | 2019-09-20 | 深圳前海微众银行股份有限公司 | Date storage method, device, equipment and computer readable storage medium |
| CN110263044B (en) * | 2019-06-21 | 2023-03-31 | 深圳前海微众银行股份有限公司 | Data storage method, device, equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108664499B (en) | 2021-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103390066B (en) | A kind of database overall automation optimizes prior-warning device and disposal route thereof | |
| US10572156B2 (en) | Capacity forecasting based on capacity policies and transactions | |
| US20060059005A1 (en) | Systems and methods for managing data in an advanced planning environment | |
| US20060059060A1 (en) | Systems and methods for executing planning services | |
| US20060059059A1 (en) | Systems and methods for managing the execution of services | |
| US8626703B2 (en) | Enterprise resource planning (ERP) system change data capture | |
| CN102542382A (en) | Method and device for managing business rule | |
| CN109271435A (en) | A kind of data pick-up method and system for supporting breakpoint transmission | |
| US20140350993A1 (en) | Information management device and method | |
| CN110134689B (en) | Target group screening method and system based on main body object label change and computer equipment | |
| van Dongen et al. | EMiT: A process mining tool | |
| CN109298929A (en) | Timing task carrying-out time recommended method, device, equipment and storage medium | |
| CN108664499A (en) | The method, apparatus and equipment of data storage | |
| CN105868216A (en) | Method for realizing object expiration operation and device and equipment | |
| CN107506380B (en) | A kind of method and server updating business data | |
| CN206421382U (en) | A kind of data handling system | |
| US8190459B1 (en) | Customizable workflow reporter | |
| KR20230077262A (en) | SaaS(Software-as-a-Service) system of open source-based intelligent manufacturing process workflow | |
| CN112506957A (en) | Method and device for determining workflow dependency relationship | |
| CN110096518A (en) | Knowledge base metadata sending method and device, readable storage medium storing program for executing | |
| CN108256911A (en) | A kind of processing method and processing device for dividing customer type | |
| CN117252345B (en) | Charging method and system based on charging logic sequence | |
| CN110888928B (en) | Visual control method based on ETL tool service assembly | |
| JP2001187477A (en) | Data base system having hierarchic link table | |
| JP2009123038A (en) | Project operation achievement management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |