CN110532262A - A kind of data storage rule auto recommending method, device, equipment and readable storage medium storing program for executing - Google Patents
A kind of data storage rule auto recommending method, device, equipment and readable storage medium storing program for executing Download PDFInfo
- Publication number
- CN110532262A CN110532262A CN201910696205.0A CN201910696205A CN110532262A CN 110532262 A CN110532262 A CN 110532262A CN 201910696205 A CN201910696205 A CN 201910696205A CN 110532262 A CN110532262 A CN 110532262A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- optimized
- tables
- scheme
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of data storage rule auto recommending method, device, equipment and readable storage medium storing program for executing, comprising: requests in response to data-optimized storage, extracts multiple tables of data in data warehouse；User is obtained for the storage rule of each tables of data configuration；Storing the multiple tables of data again according to the storage rule is tables of data to be optimized；Obtain the attribute of the tables of data to be optimized；According to the attribute of the tables of data to be optimized, determine that the multiple of tables of data to be optimized optimize storage scheme；Calculate each storage benefit that can optimize storage scheme；Using it is described storage it is most effective described in can optimize storage scheme as optimal storage scheme, recommend the user.Solve the problems, such as that the prior art is based on artificial experience selection prioritization scheme inefficiency and human cost is higher.
Description
Technical field
The present invention relates to technical field of website design, more particularly to a kind of data storage rule auto recommending method, dress
It sets, equipment and readable storage medium storing program for executing.
Background technique
With the rise of big data memory module, big data, which carries out storage according to scientific and effective configuration in cloud platform, is
The vital issue that must be faced.Firstly, user it should be understood that big data storage mode, further according to different demands using optimal
The scheme of change is stored again, efficiently to utilize resource and efficient output data.
In the prior art, most of data-optimized scheme is all based on artificial experience and is chosen, time-consuming and laborious and cost
Excessively high, another scheme is to extract important candidate parameter collection according to the ranking of candidate parameter；Training number is generated according to candidate parameter
According to library and establish prediction model；Extract the load characteristic of destination application；According to prediction model to the target application journey
The load characteristic of sequence is handled, and is generated and is exported the value distributed parameter rationally and respectively distribute parameter rationally.This method is directed to
Different application carries out different configurations, lacks unified standard, efficiency not can guarantee, and feasibility and income estimate inaccuracy.
Summary of the invention
In view of the above problems, it proposes the embodiment of the present invention and overcomes the above problem or at least partly in order to provide one kind
A kind of data storage rule auto recommending method, device, equipment and the readable storage medium storing program for executing to solve the above problems.
According to the first aspect of the invention, the embodiment of the invention discloses a kind of data storage rule auto recommending method,
It specifically includes:
It is requested in response to data-optimized storage, extracts multiple tables of data in data warehouse；
User is obtained for the storage rule of each tables of data configuration；
Storing the multiple tables of data again according to the storage rule is tables of data to be optimized；
Obtain the attribute of the tables of data to be optimized；
According to the attribute of the tables of data to be optimized, determine that the multiple of tables of data to be optimized optimize storage scheme；
Calculate each storage benefit that can optimize storage scheme；
Using it is described storage it is most effective described in can optimize storage scheme as optimal storage scheme, recommend the use
Family.
According to the second aspect of the invention, the embodiment of the invention discloses a kind of automatic recommendation apparatus of data storage rule,
It specifically includes:
Tables of data extraction module extracts multiple tables of data in data warehouse for requesting in response to data-optimized storage；
Storage rule obtains module, for obtaining user for the storage rule of each tables of data configuration；
Tables of data memory module to be optimized is to excellent for storing the multiple tables of data again according to the storage rule
Change tables of data；
Data Table Properties to be optimized obtain module, for obtaining the attribute of the tables of data to be optimized；
Storage scheme determining module determines the tables of data to be optimized for the attribute according to the tables of data to be optimized
Multiple optimize storage scheme；
Benefit calculation module is stored, for calculating each storage benefit that can optimize storage scheme；
Recommending module, for using the storage it is most effective described in can optimize storage scheme as optimal storage scheme,
Recommend the user.
According to the third aspect of the invention we, a kind of equipment is provided, comprising: processor, memory and be stored in described
On memory and the computer program that can run on the processor, which is characterized in that the processor executes described program
Shi Shixian data storage rule auto recommending method as the aforementioned.
According to the fourth aspect of the invention, provide a kind of readable storage medium storing program for executing, when the instruction in the storage medium by
When the processor of electronic equipment executes, so that electronic equipment can be realized data storage rule auto recommending method above-mentioned.
The embodiment of the present invention includes following advantages to be extracted more in data warehouse by requesting in response to data-optimized storage
A tables of data；User is obtained for the storage rule of each tables of data configuration；Described in being stored again according to the storage rule
Multiple tables of data are tables of data to be optimized；Obtain the attribute of the tables of data to be optimized；According to the category of the tables of data to be optimized
Property, determine that the multiple of tables of data to be optimized optimize storage scheme；Calculate each storage effect that can optimize storage scheme
Benefit；Using it is described storage it is most effective described in can optimize storage scheme as optimal storage scheme, recommend the user.It can needle
Scheme is optimized to different configurations to automatically select, and recommends user, is solved the prior art and is chosen based on artificial experience
The higher problem of prioritization scheme inefficiency, human cost or other be based on model calculation optimization scheme cannot be answered according to difference
The problem of inefficient low-quality caused by flexible calculation optimization scheme.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below by institute in the description to the embodiment of the present invention
Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the invention
Example, for those of ordinary skill in the art, without any creative labor, can also be according to these attached drawings
Obtain other attached drawings.
Fig. 1 is a kind of step flow chart of data storage rule auto recommending method embodiment of the invention；
Fig. 2 is a kind of step flow chart of data storage rule auto recommending method embodiment of the invention；
Fig. 2A is a kind of data storage rule auto recommending method data flow diagram of the invention；
Fig. 2 B is prioritization scheme schematic diagram of calculation flow of the invention；
Fig. 2 C is model optimization scheme schematic diagram；
Fig. 3 is a kind of structural block diagram of the automatic recommendation apparatus embodiment of data storage rule of the invention；
Fig. 4 is a kind of structural block diagram of the automatic recommendation apparatus embodiment of data storage rule of the invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall within the protection scope of the present invention.
Embodiment one
Referring to Fig.1, a kind of step flow chart of data storage rule auto recommending method embodiment of the invention is shown,
It can specifically include following steps:
Step 101, it is requested in response to data-optimized storage, extracts multiple tables of data in data warehouse；
In the embodiment of the present invention, the optimization storage request for a certain data bins is received in client, then extracts the number
According to the multiple tables of data stored in storehouse, wherein the tables of data for optimizing storage is typically more than two, and pair as optimization storage
As to further calculate the benefit for needing the storage index optimized.
Step 102, user is obtained for the storage rule of each tables of data configuration；
In the embodiment of the present invention, for the multiple tables of data obtained in above-mentioned steps, obtains user and be directed to each tables of data
The rule and index of configuration, the i.e. specific rules and index of storing data, zoning ordinance, partitioned storage amount comprising data storage
Deng.
It will of course be understood that, the storage rule of tables of data configuration is not limited to foregoing description, the embodiment of the present invention to this not
It limits.
Step 103, storing the multiple tables of data again according to the storage rule is tables of data to be optimized；
In the embodiment of the present invention, according to the storage rule of above-mentioned user configuration, above-mentioned multiple tables of data are re-stored as
One new table, tables of data as to be optimized.
Wherein it is possible to be interpreted as, tables of data to be optimized is in the above-mentioned multiple tables of data obtained, and each tables of data is based on one
The data of a storage index, and then summarize for a new table, the total data being not necessarily in multiple tables of data be stored as to
Optimize tables of data, in this regard, formulating according to specific requirements of the user to data store optimization, the embodiment of the present invention is not limited
System.
Step 104, the attribute of the tables of data to be optimized is obtained；
In the embodiment of the present invention, extract the specific object information in tables of data to be optimized, as data type, file size,
The information such as field redundancies between table.
Certainly, attribute information is not limited to foregoing description, for different tables of data, obtains relevant to prioritization scheme calculating
Data all may be defined as attribute information, and the embodiments of the present invention are not limited thereto.
Step 105, according to the attribute of the tables of data to be optimized, determine that multiple optimize of the tables of data to be optimized is deposited
Storage scheme；
In the embodiment of the present invention, according to the attribute information of tables of data to be optimized, that is, the storage of storage table to be optimized can determine
Type is further obtained for such preset optimization storage scheme.
For example, can be chosen such as snapshot table: rule subregion filing, compression storage, model optimization；Full dose table can be chosen:
Compress the rules such as storage, model optimization；Increment list can be chosen: compression storage, subregion filing, model optimization, cold and hot subregion etc. are excellent
Change rule.
Certainly, the storage class of tables of data and corresponding optimization storage scheme are not limited to foregoing description, the embodiment of the present invention
It is without restriction to this.
Step 106, each storage benefit that can optimize storage scheme is calculated；
In the embodiment of the present invention, system carries out regular calculating and income calculation according to the principle of optimality that previous step is chosen.
Such as when subregion filing rule, the continuous zoning that subregion inquiry temperature is less than definite value L is calculated, it is continuous according to these
Subregion calculates storage income；When compressing storage rule: calculating Repeating Field and repetitive rate, it is corresponding to calculate Repeating Field
Store income；When model optimization rule: model optimization be same particle sizes table between calculate, need to calculate the institute between table and table
There is redundant field, calculates redundant field and occupy storage.
Step 107, using it is described storage it is most effective described in can optimize storage scheme as optimal storage scheme, recommend
The user.
In the embodiment of the present invention, according to the calculating of above-mentioned storage income, it is small and high-efficient to select storage redundancy, that is, deposits
It stores up most effective optimal storage scheme and recommends user.
In embodiments of the present invention, by requesting in response to data-optimized storage, multiple tables of data in data warehouse are extracted；
User is obtained for the storage rule of each tables of data configuration；The multiple tables of data is stored again according to the storage rule
For tables of data to be optimized；Obtain the attribute of the tables of data to be optimized；According to the attribute of the tables of data to be optimized, determine described in
The multiple of tables of data to be optimized optimize storage scheme；Calculate each storage benefit that can optimize storage scheme；It is deposited described
The most effective storage scheme that optimizes is stored up as optimal storage scheme, recommends the user.It realizes according to user
Specified storage rule determines data table types, and determines storage optimization scheme and calculate each optimization storage scheme automatically and deposit
The purpose of benefit is stored up, the selection efficiency of storage optimization scheme is improved, solves and prioritization scheme low efficiency is chosen based on artificial experience
The lower and higher problem of human cost.
Embodiment two
Referring to Fig. 2, a kind of step flow chart of data storage rule auto recommending method embodiment of the invention is shown,
It can specifically include following steps:
Step 201, it is requested in response to data-optimized storage, extracts multiple tables of data in data warehouse；
This step is identical as step 101, and this will not be detailed here.
Step 202, user is obtained for the storage rule of each tables of data configuration；
This step is identical as step 102, and this will not be detailed here.
Step 203, corresponding metadata in each tables of data is extracted according to the storage rule；
In the embodiment of the present invention, as shown in Figure 2 A, when the multiple tables of data of selection and user are directed to the configuration of each tables of data
After storage rule, the metadata in each tables of data is obtained, wherein metadata includes subregion the inquiry temperature, field weight in tables of data
The indexs one or more therein such as field redundancies, partitioned storage amount between multiple record, table.
It is to be appreciated that metadata is not limited to foregoing description, depending on specific tables of data and user setting rule, this
Inventive embodiments are not limited this.
Step 204, the metadata is re-stored as tables of data to be optimized.
Preferably, step 204 further comprises:
Sub-step 2041 extracts the metadata index of the metadata；The metadata index include subregion temperature storage,
Field repeats one of field redundancies storage between record storage, table, file size storage or a variety of；
Specifically, according to the metadata of each tables of data of acquisition, extract the index in metadata, i.e., it is as shown in Figure 2 B
Index structureization storage, including the storage of subregion temperature, field repeat field redundancies storage, file size between record storage, table and deposit
One of storage is a variety of.
The metadata is stored according to the subregion temperature and/or the field repeats record and deposits by sub-step 2042
Field redundancies storage and/or the file size are stored as snapshot table to be optimized or full dose to be optimized between storage and/or the table
Table or increment list to be optimized.
Further, structured storage table is generated as the lower half portion Fig. 2 B table is shown in index structureization storage,
Middle first row includes field redundancies, file size between subregion temperature, field repetition record, table, and second is classified as table name, and third is classified as
Temperature is inquired, as unit of day, wherein including field, granularity and amount of storage, the 4th nearest query time of column, including repetition
Rate and redundancy rate.
Step 205, according to the metadata index, the Table Properties of the tables of data to be optimized are determined；The Table Properties packet
Include snapshot table or full dose table or increment list.
Specifically, the structured storage table according to obtained in above-mentioned steps can determine the type of tables of data to be optimized, i.e.,
Snapshot table or full dose table or increment list.
It step 206, is the snapshot table to be optimized in pre-set level scheme corresponding relationship list or described to be optimized complete
Scale or the increment list selection to be optimized can optimize subregion archival solution, or can optimize compression storage scheme or can optimize mould
One of type prioritization scheme is a variety of.
Specifically, the structured storage index of corresponding tables of data to be optimized, is corresponding with preset prioritization scheme, is stored in pre-
If in index scheme corresponding relationship list, thus in this table available correspondence data table memory to be optimized the side of optimization
Case can optimize subregion archival solution for snapshot table to be optimized or the selection of full dose table to be optimized or increment list to be optimized, or
Compression storage scheme can be optimized or can one of Optimized model prioritization scheme or a variety of.
Step 207, when can optimize subregion archival solution described in extraction use, the subregion of the tables of data to be optimized inquires heat
Degree is less than the continuous zoning of preset threshold；
Specifically, system carries out regular calculating and income calculation according to the principle of optimality that previous step is chosen.Such as subregion is returned
When shelves rule: calculating the continuous zoning that subregion inquiry temperature is less than definite value L, calculate storage according to these continuous zonings and receive
Benefit.
Step 208, the storage income for calculating the continuous zoning is determined as the storage income of the tables of data to be optimized；
In another embodiment of the invention, income calculation step is stored, can also include:
Step A1 is extracted when can optimize compression storage scheme described in using, the Repeating Field in the tables of data to be optimized
And corresponding repetitive rate；
Step A2 calculates the storage income of the Repeating Field according to the repetitive rate, is determined as the data to be optimized
The storage income of table；
Wherein, when compressing storage rule, Repeating Field and repetitive rate are calculated, the corresponding storage of Repeating Field is calculated and receives
Benefit.
In another embodiment of the invention, income calculation step is stored, can also include:
Step B1, extract described in using can Optimized model prioritization scheme when, the same particle sizes in the tables of data to be optimized
Table；
Step B2 extracts the redundant field in the same particle sizes table；
Step B3 calculates the storage income of the redundant field, is determined as the storage income of the tables of data to be optimized.
Wherein, when model optimization rule: model optimization be same particle sizes table between calculate, need to calculate between table and table
All redundant fields, calculate redundant field occupy storage.
Specifically, as shown in Figure 2 C, be one it is to be optimized store benefit calculation example schematic, wherein inputting table
Name, the optimization storage scheme that can be used to the determining table are subregion filing, compression storage, model optimization etc., the company of calculating separately
The storage income of continuous subregion, Repeating Field and redundant field, last comprehensive judgement are stored using model optimization scheme, are deposited
Storage income is 249.48MB, and scarce word field is added model to delete table by the optimization operation of recommendation.
Step 209, using it is described storage it is most effective described in can optimize storage scheme as optimal storage scheme, recommend
The user.
Wherein, it is compared according to profitable result and chooses optimal case, be pushed to user optimization suggestion.Such as previous step calculates
Income, compression storage income and the model optimization income of subregion filing, compare pushing away as Optimizing Suggestions for three Income Maximums
User is given, comprising recommending prioritization scheme, estimated revenue, recommending optimization detail operation.
In embodiments of the present invention, by requesting in response to data-optimized storage, multiple tables of data in data warehouse are extracted；
User is obtained for the storage rule of each tables of data configuration；It is extracted according to the storage rule corresponding in each tables of data
Metadata；The metadata is re-stored as tables of data to be optimized.Obtain the attribute of the tables of data to be optimized；According to institute
The attribute for stating tables of data to be optimized determines that the multiple of tables of data to be optimized optimize storage scheme；Calculate it is each it is described can be excellent
Change the storage benefit of storage scheme；Using the storage it is most effective described in can optimize storage scheme as optimal storage scheme,
Recommend the user.The storage rule specified according to user is realized, the metadata of each storage table is obtained, it is true according to metadata
Determine data table types, and determine storage optimization scheme and calculate the purpose of each optimization storage scheme storage benefit automatically, improves
The selection efficiency of storage optimization scheme.
Embodiment three
Referring to Fig. 3, a kind of structural block diagram of the automatic recommendation apparatus embodiment of data storage rule of the invention is shown, is had
Body may include following module:
Tables of data extraction module 301 extracts multiple data in data warehouse for requesting in response to data-optimized storage
Table；
Storage rule obtains module 302, for obtaining user for the storage rule of each tables of data configuration；
Tables of data memory module 303 to be optimized is for storing the multiple tables of data again according to the storage rule
Tables of data to be optimized；
Data Table Properties to be optimized obtain module 304, for obtaining the attribute of the tables of data to be optimized；
Storage scheme determining module 305 determines the data to be optimized for the attribute according to the tables of data to be optimized
The multiple of table optimize storage scheme；
Benefit calculation module 306 is stored, for calculating each storage benefit that can optimize storage scheme；
Recommending module 307, for using it is described storage it is most effective described in can optimize storage scheme as optimal storage side
Case recommends the user.
In embodiments of the present invention, number is extracted for requesting in response to data-optimized storage by tables of data extraction module
According to tables of data multiple in warehouse；Storage rule obtains module, for obtaining user for the storage rule of each tables of data configuration
Then；Tables of data memory module to be optimized is number to be optimized for storing the multiple tables of data again according to the storage rule
According to table；Data Table Properties to be optimized obtain module, for obtaining the attribute of the tables of data to be optimized；Storage scheme determines mould
Block determines that the multiple of tables of data to be optimized optimize storage scheme for the attribute according to the tables of data to be optimized；It deposits
Benefit calculation module is stored up, for calculating each storage benefit that can optimize storage scheme；Recommending module is used for the storage
The most effective storage scheme that optimizes recommends the user as optimal storage scheme.It realizes and is referred to according to user
Fixed storage rule determines data table types, and determines storage optimization scheme and calculate each optimization storage scheme storage automatically
The purpose of benefit improves the selection efficiency of storage optimization scheme, solves and chooses prioritization scheme inefficiency based on artificial experience
And the higher problem of human cost.
Example IV
Referring to Fig. 4, a kind of structural block diagram of the automatic recommendation apparatus embodiment of data storage rule of the invention is shown, is had
Body may include following module:
Tables of data extraction module 401 extracts multiple data in data warehouse for requesting in response to data-optimized storage
Table；
Storage rule obtains module 402, for obtaining user for the storage rule of each tables of data configuration；
Tables of data memory module 403 to be optimized is for storing the multiple tables of data again according to the storage rule
Tables of data to be optimized；
Preferably, the tables of data memory module 403 to be optimized, further, comprising:
Metadata acquisition submodule 4031, for extracting corresponding first number in each tables of data according to the storage rule
According to；
Tables of data sub-module stored 4032 to be optimized, for the metadata to be re-stored as tables of data to be optimized.
Preferably, the tables of data sub-module stored 4032 to be optimized further comprises:
Metadata index lifts unit, for extracting the metadata index of the metadata；The metadata index includes
The storage of subregion temperature, field repeat one of field redundancies storage between record storage, table, file size storage or a variety of；
Metadata storage unit is used to store the metadata according to the subregion temperature and/or the field repeats
Field redundancies storage and/or the file size are stored as snapshot table to be optimized between record storage and/or the table, or to excellent
Change full dose table or increment list to be optimized.
Data Table Properties to be optimized obtain module 404, for obtaining the attribute of the tables of data to be optimized；
Preferably, the data Table Properties to be optimized obtain module 404, further comprise:
Table Properties determine submodule 4041, for determining the table of the tables of data to be optimized according to the metadata index
Attribute；The Table Properties include snapshot table or full dose table or increment list.
Storage scheme determining module 405 determines the data to be optimized for the attribute according to the tables of data to be optimized
The multiple of table optimize storage scheme；
Preferably, the storage scheme determining module 405 further comprises:
Storage scheme determines submodule 4051, for being described to be optimized fast in pre-set level scheme corresponding relationship list
Subregion archival solution can be optimized according to table or the full dose table to be optimized or the increment list selection to be optimized, or compression can be optimized
Storage scheme can one of Optimized model prioritization scheme or a variety of.
Benefit calculation module 406 is stored, for calculating each storage benefit that can optimize storage scheme；
Preferably, the storage benefit calculation module 406 further comprises:
Continuous zoning extracting sub-module 4061, it is described to excellent when can optimize subregion archival solution described in use for extracting
The subregion inquiry temperature for changing tables of data is less than the continuous zoning of preset threshold；
Income calculation submodule 4062 is stored, for calculating the storage income of the continuous zoning, is determined as described to excellent
Change the storage income of tables of data；
Or,
Repetitive rate extracting sub-module, when can optimize compression storage scheme described in use for extracting, the data to be optimized
Repeating Field and corresponding repetitive rate in table；
Storage income determines submodule, for calculating the storage income of the Repeating Field according to the repetitive rate, determines
For the storage income of the tables of data to be optimized；
Or,
Same particle sizes table extracting sub-module, for extract use described in can Optimized model prioritization scheme when, it is described to be optimized
Same particle sizes table in tables of data；
Redundant field extracting sub-module, for extracting the redundant field in the same particle sizes table；
Income calculation submodule is determined as the tables of data to be optimized for calculating the storage income of the redundant field
Storage income.
Recommending module 407, for using it is described storage it is most effective described in can optimize storage scheme as optimal storage side
Case recommends the user.
For device embodiment, since it is basically similar to the method embodiment, related so being described relatively simple
Place illustrates referring to the part of embodiment of the method.
The embodiment of the present invention also provides a kind of equipment, comprising: processor, memory and is stored on the memory simultaneously
The computer program that can be run on the processor, which is characterized in that the processor is realized as above when executing described program
Data storage rule described in the one or more stated is recommended automatically.
The embodiment of the present invention also provides a kind of readable storage medium storing program for executing, when the instruction in the storage medium is by electronic equipment
When processor executes, so that electronic equipment is able to carry out data storage rule as mentioned and recommends automatically.
In conclusion in embodiments of the present invention, by tables of data extraction module, for being asked in response to data-optimized storage
It asks, extracts multiple tables of data in data warehouse；Storage rule obtains module, for obtaining user for each tables of data configuration
Storage rule；Metadata acquisition submodule, for extracting corresponding first number in each tables of data according to the storage rule
According to；Tables of data sub-module stored to be optimized, for the metadata to be re-stored as tables of data to be optimized.Tables of data to be optimized
Attribute obtains module, for obtaining the attribute of the tables of data to be optimized；Storage scheme determining module, it is described to excellent for basis
The attribute for changing tables of data, determines that the multiple of tables of data to be optimized optimize storage scheme；Benefit calculation module is stored, is used for
Calculate each storage benefit that can optimize storage scheme；Recommending module, for by it is described storage it is most effective described in can be excellent
Change storage scheme as optimal storage scheme, recommends the user.The storage rule specified according to user is realized, is obtained each
The metadata of storage table determines data table types according to metadata, and determines storage optimization scheme and calculate each optimization automatically
Storage scheme stores the purpose of benefit, improves the selection efficiency of storage optimization scheme.It has the advantages that
One, unifies optimisation criteria: having unified principle of optimality Choice.
Two, are improved efficiency: can disposably obtain estimated revenue to avoid repetitive operation.
Three, save cost: artificial and system effectively combines, and uses manpower and material resources sparingly.
All the embodiments in this specification are described in a progressive manner, the highlights of each of the examples are with
The difference of other embodiments, the same or similar parts between the embodiments can be referred to each other.
It should be understood by those skilled in the art that, the embodiment of the embodiment of the present invention can provide as method, apparatus or calculate
Machine program product.Therefore, the embodiment of the present invention can be used complete hardware embodiment, complete software embodiment or combine software and
The form of the embodiment of hardware aspect.Moreover, the embodiment of the present invention can be used one or more wherein include computer can
With in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code
The form of the computer program product of implementation.
The embodiment of the present invention be referring to according to the method for the embodiment of the present invention, terminal device (system) and computer program
The flowchart and/or the block diagram of product describes.It should be understood that flowchart and/or the block diagram can be realized by computer program instructions
In each flow and/or block and flowchart and/or the block diagram in process and/or box combination.It can provide these
Computer program instructions are set to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing terminals
Standby processor is to generate a machine, so that being held by the processor of computer or other programmable data processing terminal devices
Capable instruction generates for realizing in one or more flows of the flowchart and/or one or more blocks of the block diagram
The device of specified function.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing terminal devices
In computer-readable memory operate in a specific manner, so that instruction stored in the computer readable memory generates packet
The manufacture of command device is included, which realizes in one side of one or more flows of the flowchart and/or block diagram
The function of being specified in frame or multiple boxes.
These computer program instructions can also be loaded into computer or other programmable data processing terminal devices, so that
Series of operation steps are executed on computer or other programmable terminal equipments to generate computer implemented processing, thus
The instruction executed on computer or other programmable terminal equipments is provided for realizing in one or more flows of the flowchart
And/or in one or more blocks of the block diagram specify function the step of.
Although the preferred embodiment of the embodiment of the present invention has been described, once a person skilled in the art knows bases
This creative concept, then additional changes and modifications can be made to these embodiments.So the following claims are intended to be interpreted as
Including preferred embodiment and fall into all change and modification of range of embodiment of the invention.
Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that process, method, article or terminal device including a series of elements not only wrap
Those elements are included, but also including other elements that are not explicitly listed, or further includes for this process, method, article
Or the element that terminal device is intrinsic.In the absence of more restrictions, being wanted by what sentence "including a ..." limited
Element, it is not excluded that there is also other identical elements in process, method, article or the terminal device for including the element.
Above to a kind of data storage rule auto recommending method provided by the present invention, device, equipment and readable storage
Medium is described in detail, and used herein a specific example illustrates the principle and implementation of the invention, with
The explanation of upper embodiment is merely used to help understand method and its core concept of the invention；Meanwhile for the general of this field
Technical staff, according to the thought of the present invention, there will be changes in the specific implementation manner and application range, in conclusion
The contents of this specification are not to be construed as limiting the invention.
Claims (9)
1. a kind of data storage rule auto recommending method characterized by comprising
It is requested in response to data-optimized storage, extracts multiple tables of data in data warehouse；
User is obtained for the storage rule of each tables of data configuration；
Storing the multiple tables of data again according to the storage rule is tables of data to be optimized；
Obtain the attribute of the tables of data to be optimized；
According to the attribute of the tables of data to be optimized, determine that the multiple of tables of data to be optimized optimize storage scheme；
Calculate each storage benefit that can optimize storage scheme；
Using it is described storage it is most effective described in can optimize storage scheme as optimal storage scheme, recommend the user.
2. the method according to claim 1, wherein it is described stored again according to the storage rule it is the multiple
Tables of data is tables of data to be optimized, comprising:
Corresponding metadata in each tables of data is extracted according to the storage rule；
The metadata is re-stored as tables of data to be optimized.
3. according to the method described in claim 2, it is characterized in that, described be re-stored as data to be optimized for the metadata
Table, comprising:
Extract the metadata index of the metadata；The metadata index includes that subregion temperature stores, field repeats record and deposits
One of field redundancies storage, file size storage or a variety of between storage, table；
The metadata is stored according to the subregion temperature and/or the field repeats between record storage and/or the table
Field redundancies storage and/or the file size are stored as snapshot table to be optimized or full dose table to be optimized or increment to be optimized
Table.
4. according to the method described in claim 3, it is characterized in that, the attribute for obtaining the tables of data to be optimized, comprising:
According to the metadata index, the Table Properties of the tables of data to be optimized are determined；The Table Properties include snapshot table, or complete
Scale or increment list.
5. according to the method described in claim 4, it is characterized in that, the attribute according to the tables of data to be optimized, determines
The multiple of tables of data to be optimized optimize storage scheme, comprising:
It is the snapshot table to be optimized or the full dose table to be optimized or described in pre-set level scheme corresponding relationship list
Increment list to be optimized selection can optimize subregion archival solution, or can optimize compression storage scheme or can Optimized model prioritization scheme
One of or it is a variety of.
6. according to the method described in claim 4, it is characterized in that, described calculate each storage effect that can optimize storage scheme
Benefit, comprising:
When can optimize subregion archival solution described in extraction use, the subregion inquiry temperature of the tables of data to be optimized is less than default threshold
The continuous zoning of value；
The storage income for calculating the continuous zoning is determined as the storage income of the tables of data to be optimized；
Or,
It extracts when can optimize compression storage scheme described in using, Repeating Field in the tables of data to be optimized and corresponding heavy
Multiple rate；
The storage income that the Repeating Field is calculated according to the repetitive rate, the storage for being determined as the tables of data to be optimized are received
Benefit；
Or,
Extract described in using can Optimized model prioritization scheme when, the same particle sizes table in the tables of data to be optimized；
Extract the redundant field in the same particle sizes table；
The storage income for calculating the redundant field is determined as the storage income of the tables of data to be optimized.
7. a kind of automatic recommendation apparatus of data storage rule characterized by comprising
Tables of data extraction module extracts multiple tables of data in data warehouse for requesting in response to data-optimized storage；
Storage rule obtains module, for obtaining user for the storage rule of each tables of data configuration；
Tables of data memory module to be optimized is number to be optimized for storing the multiple tables of data again according to the storage rule
According to table；
Data Table Properties to be optimized obtain module, for obtaining the attribute of the tables of data to be optimized；
Storage scheme determining module determines the more of the tables of data to be optimized for the attribute according to the tables of data to be optimized
It is a to optimize storage scheme；
Benefit calculation module is stored, for calculating each storage benefit that can optimize storage scheme；
Recommending module, for using the storage it is most effective described in can optimize storage scheme as optimal storage scheme, recommendation
To the user.
8. a kind of equipment characterized by comprising
Processor, memory and it is stored in the computer program that can be run on the memory and on the processor,
It is characterized in that, the processor realizes described in any item data storage rules such as claim 1-6 when executing described program
Auto recommending method.
9. a kind of readable storage medium storing program for executing, which is characterized in that when the instruction in the storage medium is held by the processor of electronic equipment
When row, so that electronic equipment can be realized described in any item measured data storage rules side of recommendation automatically such as claim 1-6
Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910696205.0A CN110532262B (en) | 2019-07-30 | 2019-07-30 | Automatic data storage rule recommendation method, device and equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910696205.0A CN110532262B (en) | 2019-07-30 | 2019-07-30 | Automatic data storage rule recommendation method, device and equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110532262A true CN110532262A (en) | 2019-12-03 |
| CN110532262B CN110532262B (en) | 2021-02-05 |
Family
ID=68661111
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910696205.0A Active CN110532262B (en) | 2019-07-30 | 2019-07-30 | Automatic data storage rule recommendation method, device and equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110532262B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101501623A (en) * | 2006-05-03 | 2009-08-05 | 数据机器人技术公司 | Filesystem-aware block storage system, apparatus, and method |
| US8214355B2 (en) * | 2010-02-09 | 2012-07-03 | Yahoo! Inc. | Small table: multitenancy for lots of small tables on a cloud database |
| US20160019115A1 (en) * | 2007-04-27 | 2016-01-21 | Gary Stephen Shuster | Flexible data storage system |
| CN108108436A (en) * | 2017-12-20 | 2018-06-01 | 东软集团股份有限公司 | Date storage method, device, storage medium and electronic equipment |
| CN109299088A (en) * | 2018-08-22 | 2019-02-01 | 中国平安人寿保险股份有限公司 | Mass data storage means, device, storage medium and electronic equipment |
-
2019
- 2019-07-30 CN CN201910696205.0A patent/CN110532262B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101501623A (en) * | 2006-05-03 | 2009-08-05 | 数据机器人技术公司 | Filesystem-aware block storage system, apparatus, and method |
| US20160019115A1 (en) * | 2007-04-27 | 2016-01-21 | Gary Stephen Shuster | Flexible data storage system |
| US8214355B2 (en) * | 2010-02-09 | 2012-07-03 | Yahoo! Inc. | Small table: multitenancy for lots of small tables on a cloud database |
| CN108108436A (en) * | 2017-12-20 | 2018-06-01 | 东软集团股份有限公司 | Date storage method, device, storage medium and electronic equipment |
| CN109299088A (en) * | 2018-08-22 | 2019-02-01 | 中国平安人寿保险股份有限公司 | Mass data storage means, device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110532262B (en) | 2021-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110363456B (en) | Method and device for putting articles on shelves | |
| US11347776B2 (en) | Index mechanism for report generation | |
| Henn et al. | Metaheuristics for order batching and sequencing in manual order picking systems | |
| US10162855B2 (en) | Systems and methods for optimizing data analysis | |
| EP2924594A1 (en) | Data encoding and corresponding data structure in a column-store database | |
| KR102125119B1 (en) | Data handling method and device | |
| CN104111936B (en) | Data query method and system | |
| WO2017076297A1 (en) | Finding method and device for equipment storage location | |
| CN103377232A (en) | Headline keyword recommendation method and system | |
| EP3217296A1 (en) | Data query method and apparatus | |
| CN110580246B (en) | Method, device, computer equipment and storage medium for migrating data | |
| CN105022763B (en) | Realize the method and system of data query | |
| CN107229718A (en) | The method and apparatus for handling report data | |
| CN110019551A (en) | A kind of Building Method of Data Warehouse and device | |
| CN113568940B (en) | Method, device, equipment and storage medium for data query | |
| CN107203532A (en) | Construction method, the implementation method of search and the device of directory system | |
| CN104636349A (en) | Method and equipment for compression and searching of index data | |
| CN103475730B (en) | The web services system of selection that under a kind of cloud environment, user's degree of certainty guides | |
| CN108389152B (en) | Graph processing method and device for graph structure perception | |
| CN106547805A (en) | The method and apparatus of optimization database index | |
| CN101901246A (en) | Neighbor searching apparatus | |
| CN111752986A (en) | Data query method and device, equipment and storage medium | |
| US8577902B1 (en) | Data organization and indexing related technology | |
| CN106096065B (en) | A kind of similar to search method and device of multimedia object | |
| CN110532262A (en) | A kind of data storage rule auto recommending method, device, equipment and readable storage medium storing program for executing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |