CN110209345A - The method and device of data storage - Google Patents
The method and device of data storage Download PDFInfo
- Publication number
- CN110209345A CN110209345A CN201811616160.3A CN201811616160A CN110209345A CN 110209345 A CN110209345 A CN 110209345A CN 201811616160 A CN201811616160 A CN 201811616160A CN 110209345 A CN110209345 A CN 110209345A
- Authority
- CN
- China
- Prior art keywords
- business
- configuration information
- temperature
- data
- hot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Debugging And Monitoring (AREA)
Abstract
This application provides a kind of method and devices of data storage, wherein this method comprises: monitoring configuration information for the multiple temperatures of the first business configuration, the temperature of the first business is monitored according to the configuration in each temperature monitoring configuration information, obtain the corresponding hot value of each temperature monitoring configuration information, then the first business corresponding data position of storage is selected according to multiple hot value, such as solid state hard disk or mechanical hard disk, it can be and the first business corresponding data is migrated after comprehensively considering multiple hot values, it is also possible to independently migrate the first business corresponding data according to a hot value, using the above scheme, one business configuration has multiple temperature monitoring configuration informations, can the more accurate hot spot data for migrating the business in time to solid state hard disk, classification storage efficiency is substantially improved, it solves in the related technology The problem for causing hot spot data classification storage effect undesirable since hot value statistical is single.
Description
Technical field
This application involves but be not limited to field of data storage, in particular to a kind of method and device of data storage.
Background technique
In the related art, usual distributed memory system framework is made of following three parts: file access client mould
Block, meta data server module and storage server modules.Fig. 1 is according to distributed memory system structure mould in the related technology
Type figure provides application file behaviour as shown in Figure 1, file access client is the agency of application program access file system
Make interface, the functions such as hot statistics report;Meta data server module has the management of configuration data management and file metadata
With hierarchical storage management function;Storage server modules actual storage file data within the storage system.
Storage system includes metadata and file content, both metadata and file content with document form storing data
Be to separate storage: metadata (comprising the file information and data block location) is by meta data server module management, file content
It is to store according to fixed size fragment into storage server.Each fragment has multiple redundancy pairs on different volumes in system
This, ensures fragment reliability.Such as a file size 100M, system configuration fragment size 64M, then this file has 2 points
Piece.
The universal mixed insertion mechanical hard disk of distributed memory system (Distribute Storage System, referred to as DSS) and
SSD (Solid State Drives, solid state hard disk) flash memory, to meet large capacity and high performance demands.Novel SSD dodges in recent years
It deposits, such as NVMe protocol type, even more has the characteristics that very high performance, ultralow delay, also gradually answered extensively in enterprise-level storage
With.Storage system uses hierarchical storage management different type hard disk, balanced storage performance and capacity requirement.SSD in classification storage
Flash memory main function is the caching as hot spot data, to store the newest or most hot data of current business.Data are cold and hot
Judgment basis mainly has: the indexs such as data value, data access frequency, retention time, data access size, referred to as data
Access temperature.Classification stores in summary element, the copy of fragment is stored into different type hard disk, and in different type
Autonomic Migration Framework is carried out according to hot spot situation between hard disk.
It is undesirable for causing hot spot data classification to store effect since hot value statistical is single in the related technology
Problem, there is presently no effective solution schemes.
Summary of the invention
The embodiment of the present application provides a kind of method and device of data storage, at least to solve in the related technology due to heat
The single problem for causing hot spot data classification storage effect undesirable of angle value statistical.
According to one embodiment of the application, a kind of method of data storage is provided, comprising: be retrieved as the first business and set
The multiple temperatures monitoring configuration information set;Monitor the temperature of first business respectively according to each temperature monitoring configuration information
Value, wherein the hot value is used to indicate the accessed frequency of first business;Match confidence according to the monitoring of the multiple temperature
Corresponding multiple hot values are ceased, selection stores the position of the first business corresponding data, and stores the data.
According to another embodiment of the application, a kind of device of data storage is additionally provided, comprising: first obtains mould
Block, multiple temperatures for being retrieved as the setting of the first business monitor configuration information;Second obtains module, for according to each temperature
Monitoring configuration information monitors the hot value of first business respectively, wherein the hot value is used to indicate first business
Accessed frequency;Selecting module, for according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection to be deposited
The position of the first business corresponding data is stored up, and stores the data.
According to another embodiment of the application, a kind of storage medium is additionally provided, meter is stored in the storage medium
Calculation machine program, wherein the computer program is arranged to execute the step in any of the above-described embodiment of the method when operation.
According to another embodiment of the application, a kind of electronic device, including memory and processor are additionally provided, it is described
Computer program is stored in memory, the processor is arranged to run the computer program to execute any of the above-described
Step in embodiment of the method.
By the application, it is that the multiple temperatures of the first business configuration monitor configuration information, matches confidence according to the monitoring of each temperature
Configuration in breath is monitored the temperature of the first business, obtains the corresponding hot value of each temperature monitoring configuration information, then
The first business corresponding data position of storage, such as solid state hard disk or mechanical hard disk are selected according to multiple hot value, can be
Comprehensively consider multiple hot values later to migrate the first business corresponding data, be also possible to independently according to a hot value
First business corresponding data is migrated, using the above scheme, a business configuration there are multiple temperature monitoring configuration informations, can
With the more accurate hot spot data for migrating the business in time to solid state hard disk, classification storage efficiency is substantially improved, solves phase
The problem for causing hot spot data classification storage effect undesirable since hot value statistical is single in the technology of pass.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present application, constitutes part of this application, this Shen
Illustrative embodiments and their description please are not constituted an undue limitation on the present application for explaining the application.In the accompanying drawings:
Fig. 1 is according to distributed memory system structural model figure in the related technology;
Fig. 2 is according to memory hierarchy illustraton of model in the related technology;
Fig. 3 is a kind of hardware block diagram of the terminal of the method for data storage of the embodiment of the present application;
Fig. 4 is the flow chart according to the method for the data of the embodiment of the present application storage;
Fig. 5 is to be classified storage according to the multi-service of the embodiment of the present application to improve module interaction figure;
Fig. 6 is to store newly-increased module interaction figure according to the multi-service of the embodiment of the present application classification;
Fig. 7 is to monitor configuration information interface schematic diagram according to the multi-service temperature of the application example one;
Fig. 8 is to store multi-service list schematic diagram according to the classification of the application example two;
Fig. 9 is the weight management process schematic diagram according to the another example three of the application;
Figure 10 is that more catalogue configuration temperature management and superseded structure chart are stored according to the classification of the application example four;
Figure 11 is to eliminate main flow schematic diagram according to the fragment of the application example four.
Specific embodiment
The application is described in detail below with reference to attached drawing and in conjunction with the embodiments.It should be noted that not conflicting
In the case of, the features in the embodiments and the embodiments of the present application can be combined with each other.
It should be noted that the description and claims of this application and term " first " in above-mentioned attached drawing, "
Two " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.
Classification storage architecture main functional modules are as follows: file access client hot statistics and reporting;Metadata Service
Device configuration management module, temperature management module, temperature scheduler module, statistical module, Fig. 2 are deposited according to classification in the related technology
Storage structure illustraton of model, as shown in Figure 2, including access client, meta data server, storage server, temperature configuration module,
Temperature management module, hot statistics module, fragment eliminate module, temperature scheduler module, weight management module, coordinated scheduling mould
Block.
Classification storage temperature manages general flow are as follows:
(1) when application call interface (such as read, sendfile) access file fragmentation, file access client system
Meter reports the information such as fragment read-write number, read-write byte number to give meta data server temperature management module.
(2) meta data server, which receives, currently reports fragment raw information, in conjunction with history temperature and currently reports temperature,
The fragment temperature is calculated according to formula and is saved in metadata.
(3) fragment of temperature management module timing scan metadata, if fragment temperature is greater than configuration heat degree threshold and divides
All copies of piece are respectively positioned on mechanical hard disk, then associated metadata are inserted into list to be upgraded, and again by column to be upgraded
List sorting.If fragment hot value is less than heat degree threshold and has copy on SSD flash memory, associated metadata is inserted into wait drop
Grade list, and list to be degraded of resequencing;Heat degree threshold refers to that data access temperature is more than that the fragment of this value can be made herein
SSD flash memory is upgraded to for candidate fragment.List to be upgraded, which refers to, have been sequenced sequence from big to small using temperature as keyword and has included to meet
Burst information beyond heat degree threshold;Degradation list refers to has sequenced sequence by keyword of temperature from small to large, and temperature is less than temperature
The burst information of threshold value.
(3) temperature scheduler module regular check system configuration takes out list to be upgraded and wait eligible in the list that degrades
Fragment to storage server modules assign fragment copy migration instruction.
(4) after the success of storage server migration fragment copy, meta data server is reported;
(5) new hard disk position after the migration of meta data server modification fragment copy.
The relevant technologies are that statistics file or object temperature are anti-to predict as history temperature in several historical time sections
The temperature of file in following a period of time is reflected, accordingly as classification storage temperature judgment basis, different temperature file migrations are arrived
On the hard disk of different performance.
There are more limitations for classification memory technology in the related technology, first is that Supporting multi-services are poor, a set of storage is often
Need to provide storage service for multiple business, different business has different Hot Contents and hot spot period, general based on going through
The statistics of history file access temperature, it will cause hot spot not hot, the effect for being classified storage is undesirable;Second is different time sections heat
It is poor that point is supported, even same business, section often has different Hot Contents in different times, single based on the passing time
The statistics of section, will lead to hot spot dislocation, and the efficiency for being classified storage is had a greatly reduced quality;Third is that focus statistics period assignment management is tired
Difficulty is difficult to adapt to the variation of Hot Contents and period by the artificial setting hot spot period.
Embodiment one
Embodiment of the method provided by the embodiment of the present application one can be in terminal or similar arithmetic unit
It executes.For running on computer terminals, Fig. 3 is that a kind of computer of the method for data storage of the embodiment of the present application is whole
The hardware block diagram at end, as shown in figure 3, terminal may include one or more (only showing one in Fig. 3) processing
Device 302 (processing unit that processor 302 can include but is not limited to Micro-processor MCV or programmable logic device FPGA etc.) and
Memory 304 for storing data, optionally, above-mentioned terminal can also include the transmitting device for communication function
306 and input-output equipment 308.It will appreciated by the skilled person that structure shown in Fig. 3 is only to illustrate, simultaneously
The structure of above-mentioned terminal is not caused to limit.For example, terminal may also include than shown in Fig. 3 more or more
Few component, or with the configuration different from shown in Fig. 3.
Memory 304 can be used for storing the software program and module of application software, such as the data in the embodiment of the present application
Corresponding program instruction/the module of the method for storage, processor 302 by the software program that is stored in memory 304 of operation with
And module realizes above-mentioned method thereby executing various function application and data processing.Memory 304 may include high speed
Random access memory may also include nonvolatile memory, such as one or more magnetic storage device, flash memory or other are non-
Volatile solid-state.In some instances, memory 304 can further comprise remotely located relative to processor 302
Memory, these remote memories can pass through network connection to terminal.The example of above-mentioned network includes but is not limited to
Internet, intranet, local area network, mobile radio communication and combinations thereof.
Transmitting device 306 is used to that data to be received or sent via a network.Above-mentioned network specific example may include
The wireless network that the communication providers of terminal provide.In an example, transmitting device 306 includes a Network adaptation
Device (Network Interface Controller, NIC), can be connected by base station with other network equipments so as to it is mutual
Networking is communicated.In an example, transmitting device 306 can be radio frequency (Radio Frequency, RF) module, use
In wirelessly being communicated with internet.
A kind of method of data storage for running on above-mentioned terminal is provided in the present embodiment, and Fig. 4 is basis
The flow chart of the method for the data storage of the embodiment of the present application, as shown in figure 4, the process includes the following steps:
Step S402 is retrieved as multiple temperatures monitoring configuration information of the first business setting;
Step S404 monitors the hot value of first business according to each temperature monitoring configuration information, wherein institute respectively
It states hot value and is used to indicate the accessed frequency of first business;
Step S406, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described the
The position of one business corresponding data, and store the data;
Modification metadata information can be corresponded to after change storage location.
It through the above steps, is that the multiple temperatures of the first business configuration monitor configuration information, according to each heat by the application
Configuration in degree monitoring configuration information is monitored the temperature of the first business, and it is corresponding to obtain each temperature monitoring configuration information
Then hot value selects the first business corresponding data position of storage, such as solid state hard disk or machinery according to multiple hot value
Hard disk, can be to comprehensively consider and is migrated after multiple hot values to the first business corresponding data, be also possible to independently according to
The first business corresponding data is migrated according to a hot value, using the above scheme, a business configuration there are multiple temperatures to supervise
Survey configuration information, can the more accurate hot spot data for migrating the business in time to solid state hard disk, classification storage is substantially improved
Efficiency, solving leads to hot spot data classification storage since hot value statistical is single in the related technology effect is undesirable asks
Topic.
Optionally, it is retrieved as multiple temperatures monitoring configuration information of the first business setting, comprising: obtain the temperature monitoring
At least one following information for including in configuration information: at the end of temperature update cycle, hot statistics initial time, hot statistics
Between.
Optionally, the hot value of first business is monitored respectively according to each temperature monitoring configuration information, comprising: every
A temperature monitored in the configuration information corresponding hot statistics time started to hot statistics end time, counts each temperature and updates
First time accessed number of first business described in period;Each temperature, which is obtained, according to described first time number monitors configuration information pair
The hot value for first business answered.
Optionally, the hot value of first business is monitored respectively according to each temperature monitoring configuration information, comprising: in institute
State the first operation list that the first temperature monitoring configuration information in multiple temperature monitoring configuration informations is directed to first business
When, the temperature of one or more data fragmentations in first operation list is counted according to first temperature monitoring configuration information
Value.
Optionally, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described first
The position of business corresponding data, and store the data, comprising: monitoring configuration information in the multiple temperature is associated temperature
When monitoring configuration information, the product of each temperature monitoring configuration information corresponding hot value and default weight is obtained;Described in acquisition
The product of multiple temperatures monitoring configuration informations and value, the position of the first business corresponding data is stored according to described and value selection
It sets, and stores the data.
Optionally, the position of the first business corresponding data is stored according to described and value selection, and stores the data,
It include: to be migrated by mechanical hard disk the corresponding data of first business hard to solid-state when described and value is greater than heat degree threshold
Disk;When described and value is less than heat degree threshold, the corresponding data of first business are migrated by solid state hard disk to mechanical hard disk.
Optionally, selection stores the position of the first business corresponding data, and stores the data, comprising: selection is deposited
Store up the solid state hard disk or mechanical hard disk of the copy of the first data fragmentation of first business;The copy is stored to selected
Solid state hard disk or mechanical hard disk.
Optionally, the copy is migrated to solid state hard disk, within a temperature update cycle, described in statistics execution
The solid state hard disk is read when the first business and reads the number ratio of mechanical hard disk;It is lower than preset ratio in the number ratio
When, the default weight of the multiple temperature monitoring configuration information is adjusted to increase corresponding number ratio of next temperature update cycle
Example.
Optionally, the default weight of the multiple temperature monitoring configuration information is adjusted to increase next temperature update cycle
After corresponding number ratio, after the adjustment by the default weight of multiple temperature update cycles, the number ratio is detected
Reach maximum value;When the maximum value is still less than the preset ratio, generates statistical report and alert.
Optionally, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described first
The position of business corresponding data, comprising: monitoring configuration information in the multiple temperature is temperature monitoring configuration independent of each other
When information, the first business corresponding data is stored according to the corresponding hot value selection of each temperature monitoring configuration information respectively
Position.
Optionally, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described first
The position of business corresponding data, and after storing the data, second time accessed number of the first business described in real-time statistics,
When second number meets preset condition, the second temperature monitoring configuration information of first business is automatically generated.It is detecting
To after the current multiple temperature monitoring configuration informations for executing first business, the data of the first business fail efficient calling
Afterwards, the second temperature monitoring configuration information is automatically generated, in the subsequent temperature monitoring to the first business, which to be supervised
The concrete configuration for surveying configuration information can be to the monitoring configuration information study of the temperature of other business.
Optionally, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described first
The position of business corresponding data, and after storing the data, meet in the storage state for the first hard disk for being stored with data pre-
If when state, at least one the in the following manner memory space of release first hard disk: will be stored on first hard disk
Hot value go out lower than heat degree threshold or the smallest second business migration of hot value;Will stored on first hard disk
The smallest data fragmentation of the hot value of two business migrates out.
It is further illustrated below with reference to another embodiment of the application.
In view of the limitation of above-mentioned the relevant technologies, this application discloses classification storage is improved in a kind of distributed memory system
The method of efficiency.Be applicable in multi-service scene, and by it is adaptive while are carried out multiple periods with the statistics of hot spot respectively
Analysis, very good solution distributed memory system classification storage in the above scenario the problem of.
The application technical problems to be solved are: a set of distributed memory system bearing multiple service, different business have
Different access hot spots and rush hour section, and whether history temperature or current temperature, temperature tribute in different time periods
Value is offered to be different.When, there are many business and section of different rush hours, the relevant technologies are deposited in temperature management aspect in storage system
In the low problem of dispatching efficiency.It therefore, can be flexible in view of the above-mentioned problems, this programme proposes a kind of classification storage method and device
Be deployed in distributed memory system, it supports multi-service that different rush hours section is arranged, and carries out independent temperature management, utilizes
Business different time sections temperature and performance data automatically generate a variety of association temperature monitoring configuration informations in different time periods, change
Temperature management is stored into classification, and generates statistical data automatically according to hot statistics, adjust automatically associated configuration weight is provided
Method, simplify operation maintenance personnel burden.
Technical solution:
This programme increases several functional modules (meta data server dotted line frame mould in Fig. 2 on the basis of above-mentioned architecture
Block) and the multiple modules realizations of optimization, flexible temperature management and improvement hot spot fragment rush hour section when realizing Supporting multi-services
Scheduling problem proposes to support:
(1) multi-service carries out independent temperature management and scheduling in classification storage;
(2) storage system can be each business according to statistical data according to operating condition, automatically generate multiple times
Duan Guanlian temperature monitors configuration information;Multiple temperature monitoring configuration informations can be separate configurations in some business, carry out independent
Temperature management or associated configuration carry out shared temperature management.
(3) business association temperature monitoring configuration information in different time periods provides a kind of automatic in system operation
The method for adjusting associated configuration weight.
Hierarchical stor supports multiple business and multiple periods to carry out temperature management and scheduling, needs for existing frame
Structure adjusts optimization, and temperature monitoring configuration information module, temperature management, temperature scheduler module, heat are discussed in detail in turn below
Spend statistical module, fragment eliminates module contents:
To support this programme, temperature monitors configuration information module and extends several fields, and single business can increase multiple heat
Degree monitoring configuration information, distinguishes different configurations with configuration number;Hierarchical stor is supported to configure multiple business simultaneously.Thus
Each temperature monitoring configuration information relevant field includes that service identification, temperature scheduling time, temperature calculation formula, fragment exist
SSD flash memory retention time, SSD flash memory maximum occupying space, hot statistics initial time, hot statistics end time.
Table 1 is to monitor each primary fields meaning in configuration information according to the temperature of the application to illustrate table, such as 1 institute of table
Show:
Table 1
This configuration basis field is the combination of service identification, temperature renewal time section, weight.Service identification described herein is
As the mark of service operation used resource within the storage system, business can by directory name, complete trails, relative path,
File prefix or suffix format etc. distinguish different service types, can be used as this programme examples of implementation.One service identification
It may include multiple catalogues or complete trails.Temperature renewal time section can be some time in one day, such as 10. -14 points,
Also festivals or holidays (Sunday, May Day, 11 on every Saturdays) etc. are configurable to.The same business different period can be configured to solely
Vertical configuration is managed independently, and shared temperature management can also be carried out by system automatically generated associated configuration.Associated configuration power
Weight can be with manual configuration, can also be in system operation, when automatically generating associated configuration, and system assigns initial value automatically,
And it is automatically adjusted.It can also be calculated comprising preferred field correlation tag, fragment in SSD retention time, compileable temperature
Formula etc. is combined, this temperature Managed Solution is improved.
Such as hierarchical stor 2 separate traffics catalogue HOT and TV, table 2 are supervised according to the temperature of the embodiment of the present application
Configuration information schematic table is surveyed, as shown in the table, 4 temperature monitoring configuration informations are as follows:
Table 2
Configuration 1, configuration 2 be associated configuration, all act on operation list HOT, share it is same it is to be upgraded, wait degrade column
Table and the same temperature management role.Configuration 3 and configuration 4 be separate configurations, it is each configuration all have it is independent it is to be upgraded, to
Degradation list and individual temperature management role.Structure chart such as Figure 10.
Associated configuration in different time periods can be automatically generated in the process of running with system.Create-rule is as follows: automatic raw
At premise be configuration in have related service catalogue configuration.It obtains operation list reading performance height according to hot statistics module
Some time.Performance in time period exceeds 1 times or 2 times of usually operation preset value.It, can in system operation
With the configuration and time period one associated configuration of generation according to time operation list, and initial weight is set.This business exists in this way
Configuration information is monitored comprising multiple temperatures in storage system, each temperature, which monitors configuration information, has certain weight.Storage system
System obtains data in multiple measurement periods according to hot statistics module, can be with adjust automatically associated configuration weight.
Temperature management module:
There are multiple operation lists in hierarchical stor, each operation list can configure multiple temperature monitorings with confidence
Breath.For some operation list, the temperature data for needing to update at some time point and saving multiple configuration generations are just formed, are
This meta data server increases several original temperature fields (such as h1, h2, h3) in file fragmentation associated metadata, to store
Different temperatures monitor configuration information original temperature information in the same report cycle;Increase several temperature monitoring configuration information marks
It signs (such as tag1, tag2, tag3), which temperature monitoring configuration information is corresponding original temperature field correspond to.
When application program reads file by interfaces such as read and sendfile, file access client calculates original reading
Number, read-write fragment byte number are write, meta data server is sent to.Meta data server, which receives, updates fragment temperature message,
Current time is read, the affiliated catalogue of respective file, and then recursive lookup upper directory is searched, is checked whether for every first class catalogue
Configuration service catalogue temperature monitors configuration information, obtains configuration number of the current time within the scope of hot statistics.Fragment is related
An idle temperature field, the temperature that filling is currently configured number and is calculated according to this configuration are obtained in metadata.
The same operation list associated configuration can have multiple, their shared temperature management roles.Temperature manages mould
Block meeting timing scan temperature monitors configuration information, starts an individual temperature management role for each separate configurations, and
Only need to start a temperature management role for the management of associated configuration temperature.After temperature management role enters runing time,
The fragment associated metadata under current business catalogue is scanned, obtains current time, such as current time, in 9. -12 points, temperature is every
Hour updates once, and when temperature updates task run, configuration 1 comes into effect with configuration 2, according to calculation formula calculating heat
Degree, is indicated with benefit1, bennfit2.So practical temperature benefit of current slice is repaired by following formula (1)
Just:
Benefit=benefit1*w1+benefit2*w2, formula (1)
In above-mentioned formula, wherein w1 is 1 associated configuration weight of configuration, and w2 is 2 associated configuration weights of configuration.At the beginning of w1, w2
Initial value is 0.5, i.e., default association configuration 1 and configures as 2 status are.
The weight of each configuration can pass through system adjust automatically in associated configuration.It is real when calculating practical temperature
Temperature in border temperature and relevant configuration is closest, and this configuration statistics numbers is increased by 1.It is complete when hot statistics this period of module
It after operation, counts this business and reads SSD flash memory and mechanical hard disk performance data, show that epicycle temperature dispatches actual efficiency
(business SSD flash memory actual read data amount/business can be used and always read data volume).Scheduling actual efficiency is adjusted with pre- imagination
Degree efficiency such as 80% compares, if actual efficiency is lower than preset schedule efficiency, configuration most associated in associated configuration is weighed
10% is raised again.So after the scheduling of several period temperatures and hot statistics, according to adjustment rule, adjustment in each period
Associated configuration weight.The scheduling of (i.e. most associated weights reach 1) temperature and real data statistics discovery are adjusted after several cycles of operation
It spends efficiency and is less than preset schedule efficiency, then generate statistical report and alarm, warning operation maintenance personnel needs to reappraise scheduling scheme:
Adjust hot statistics time, calculation formula.
After temperature management role calculates current slice temperature, judge whether temperature is greater than heat degree threshold, if meeting condition,
Then it is added into list to be upgraded.Whether temperature management role is also handled simultaneously has been upgraded to the fragment temperature of SSD flash memory and has been less than
Queue to be degraded is added in heat degree threshold if meeting condition.Details are not described herein again.
Temperature scheduler module
Each temperature monitoring configuration information is taken out in the timing of this module, and it is corresponding wait rise to first look at temperature monitoring configuration information
Grade list, successively takes out the highest burst information of temperature, checks that all copies of fragment whether all only on mechanical hard disk, will expire
One copy of sufficient promotion condition fragment sends copy to storage server and moves to SSD flash request from mechanical disk;Copy liter
After the completion of grade, the current update time point of this fragment is set.Then, burst information is taken out from list to be degraded, checks the pair of fragment
Whether this has been downgraded to mechanical hard disk, and whether alreadys exceed the SSD retention time, will meet one copy of condition fragment
Copy, which is sent, to storage server moves to mechanical disk request from SSD flash memory.
The function that hot statistics module has is as follows, counts each industry in each temperature dispatching cycle monitoring configuration information
All fragments under catalogue of being engaged in read mechanical hard disk, the number of SSD flash memory and reading size;It calculates in temperature monitoring configuration information catalogue
Read the percentage of fragment hit SSD flash memory, i.e. temperature dispatching efficiency;The space different business catalogue SSD and fragment in output system
The space hold in SSD.Above-mentioned statistical information is used to assessment classification storage efficiency, and feeds back to temperature management module and improve heat
Degree monitoring configuration information.
Fragment eliminates module
The multiple operation lists of hierarchical stor carry out having multiple temperature scheduling to appoint under temperature scheduling and a catalogue simultaneously
Business, and SSD flash memory space is limited, it may appear that SSD flash memory space is full, causes some operation lists that temperature is needed to dispatch, but
It is the problem of space SSD is occupied by other business, led to insufficient memory.There are two types of solutions:
1. monitoring configuration information for each business or temperature, SSD maximum is arranged by manual assignment mode and is occupied
Space.Guarantee that the accumulated value of all configuration SSD space hold maximum values is less than the entirety space SSD.Such method needs to advise in advance
Draw requirement of the business to storage system.
2. multiple business can not be accurate using hierarchical stor or association of multiple periods temperature monitoring configuration information
SSD flash memory space occupies, using only plan of operation maximum space when, the use of SSD flash memory space can be more than SSD in storage system
When capacity-threshold, needs to start pressure and eliminate function.Such as storage system SSD flash memory space is 24T, HOT plan of operation SSD is empty
Between maximum occupy be 13T, TV plan of operation SSD space maximum 14T;Or the more space a associated configuration SSD maximums of TV occupy greatly
In 24T.When the space storage system SSD is practical occupies over SSD capacity-threshold, need for all business and temperature in system
Fragment occupancy carries out analysis and Free up Memory in monitoring configuration information.Replacement policy can there are many, preferably eliminate each industry
Temperature is lower than the fragment of heat degree threshold in business, next eliminates the fragment that hot value is small in each business.
Illustrate a kind of method that fragment is superseded under multi-service and more temperatures monitoring configuration information below, mainly comprises the processes of
(1) the current operation list more than SSD capacity-threshold is first looked at, all temperatures monitoring of traversing directories configuration is matched
Confidence breath.Each temperature is monitored into the fragment in configuration information in degradation list, fragment is added to and eliminates module.Immediately triggering
Create new temperature scheduler task.
(2) if SSD space hold is unsatisfactory for condition, all temperatures monitoring of other operation lists is searched with confidence
Breath.Repeat the first step.
(3) when SSD space hold is still unsatisfactory for condition, need to eliminate part fragment not out of date in SSD flash memory.
After the sequence of operation list SSD space hold, the fragment of the file in catalogue is successively searched, it will be more than the SSD retention time
Fragment is added fragment and eliminates module.
(4) last successively to eliminate the fragment small more than temperature in SSD capacity-threshold.
Coordinated scheduling
The temperature management of different business and temperature scheduling are independent from each other, and such different business makes simultaneously in the same time
Use hierarchical stor.Their sharing CPUs, SSD flash memory, mechanical hard disk, Internet resources.Such as business access height is corresponded in HOT
Peak time section carries out a large amount of TV catalogues and corresponds to a large amount of fragment temperature scheduling of business, will affect the stability of HOT catalogue.Cause
This carries out coordinated scheduling to multiple independent temperature management, prevents from influencing service stability because of other business backstage scheduling reason.
Major function has 2:
(1) it receives hot statistics module and notifies the peak traffic period, check that all temperatures of business monitor configuration information, from
The dynamic association temperature that generates monitors configuration information, initializes initial weight.
(2) when hot statistics module finds that SSD flash memory or mechanical hard disk IO ability reach threshold value in some period,
Or storage system is notified that each traffic scheduling program carries out fragment copy migration velocity when performance being reported to reach performance threshold
Control.
Fig. 5 is to be classified storage according to the multi-service of the embodiment of the present application to improve module interaction figure, as shown in figure 5, this programme
By increasing several management modules and optimization function, more preferably to support multi-service using same hierarchical stor, and according to
Statistical module obtains different time sections and automatically generates association temperature monitoring configuration information, provides associated configuration weight adjust automatically side
Method, to simplify O&M complexity and promote dispatching efficiency.After meta data server optimizes for correlation module, a variety of industry are supported
Business temperature management, main flow are described as follows (see Fig. 5):
(1) after meta data server receives fragment temperature information, the affiliated operation list of file of fragment is searched, is read current
Time, recursive lookup upper directory checks whether catalogue carries out temperature monitoring configuration information, and then obtains current time in temperature
The configuration of scope of statistics is numbered, and the corresponding temperature of this configuration is updated.
(2) fragment of temperature management module timing scan metadata obtains the affiliated service identification of file of fragment, and current
Time, and then obtain all independences of business and associated configuration.Check that current time comes into force is separate configurations or associated configuration,
And then calculate fragment temperature.
(3) configuration information is monitored according to service identification and current temperature, searches configuration respective upgrades, degradation list.It checks
Fragment temperature be greater than configuration heat degree threshold and all copies of fragment be respectively positioned on mechanical hard disk, then by associated metadata be inserted into
Upgrade list, and again by list ordering to be upgraded.If fragment hot value is less than heat degree threshold and has copy to dodge in SSD
It deposits, then associated metadata is inserted into list to be degraded, and list to be degraded of resequencing.
(4) configuration information is monitored for each group of independence temperature, phase is examined successively in several temperature scheduler tasks of start by set date
It should upgrade, degradation list.Fragment copy migration request is sent to storage server.
Fig. 6 is to store newly-increased module interaction figure according to the multi-service of the embodiment of the present application classification, as shown in fig. 6, this programme
Newly-increased weight management, coordinated scheduling, the superseded module of fragment are background functions, and each functions of modules realizes that front has been described, existing
Showing that increasing module and existing module newly interacts process.Each newly-increased module interaction, as shown in Figure 6:
Weight management module and hot statistics, temperature monitor configuration information interactive step:
(1) business statistics information in the hot statistics execution cycle, sends notification to weight management module;
(2) the relevant temperature monitoring configuration information of weight management acquisition business institute, the corresponding focus statistics data of retrieval service,
Calculate associated configuration weight;
(3) associated configuration weight is updated, and is stored in temperature monitoring configuration information, persistent storage is carried out.
Coordinated scheduling module is interacted with hot statistics, temperature scheduling, temperature monitoring configuration information, major function process:
(1) regular check checks performance of storage system, SSD, mechanical hard disk hit situation, when system is busy, notifies institute
There is business carrying out temperature scheduler task, reduces migration velocity.
(2) there is portfolio peak period in hot statistics task discovery, is arranged beyond threshold value, notifies coordinated scheduling module.
(3) coordinated scheduling module obtains the peak traffic period, checks that all temperatures of business monitor configuration information, automatic raw
Configuration information is monitored at association temperature, and initializes initial weight, in deposit temperature monitoring configuration information.
Technical problems to be solved in this application are: a set of distributed memory system bearing multiple service, different business tool
There are different access hot spot and rush hour section, and whether history temperature or current temperature, temperature in different time periods
Contribution margin is different.When, there are many business and section of different rush hours, the prior art is in temperature management aspect in storage system
Shortcomings and the low problem of dispatching efficiency.Therefore distributed memory system is in view of the above-mentioned problems, propose a kind of classification storage dress
It sets, can flexibly be deployed in distributed memory system, it supports multi-service, and automatically generates a variety of in different time periods
It is associated with temperature and monitors configuration information, improve classification storage temperature management, and generate association temperature prison automatically according to hot statistics
Configuration information is surveyed, the method for adjust automatically associated configuration weight is provided, simplifies operation maintenance personnel burden.
Example one, multi-service temperature monitor configuration information and management
Above-mentioned hierarchical stor can also carry webpage video cache, small other than the program request of big video, live broadcast service
The business such as program application, mailbox backup.These business and video on demand user group, the flat rate of access, peak access time section etc. have
Many differences.They cannot carry out the migration of fragment copy according to unified temperature management.So match according to each operation list
It sets a basic temperature monitoring configuration information and is associated with temperature monitoring configuration information with several.Service identification described herein is as industry
Business operates in resource identification used in storage system, business can also by complete trails, relative path, file prefix or
The file of the differentiation different service types such as suffix format.In addition the period not only can daily certain section of time interval it is (every
Its 9. -11 point), festivals or holidays can also be configured to according to day, such as Saturday, Sunday, National Day (October 1 to October 7).Example
As being directed to another mailbox service, increase following configuration in same storage system:
Configuration 4 monitors configuration information, hot statistics period daily 8. -18 point, heat as MAIL application foundation temperature
Spending the update cycle is each hour.
Configuration 5, as the associated configuration of configuration 4, the hot statistics period is 8 points early-early 9 thirty, temperature update cycle
It is every 30 minutes.
Concrete configuration mode passes through human-computer interaction order or interactive interface.When storage system Added Business, in addition to increasing
Outside service path, it is also necessary to execute multi-service classification storage temperature monitoring configuration information.Multi-service in storage system is described below
Temperature monitors configuration information temperature interactive interface, such as when increase business TV, increases temperature monitoring configuration information partial parameters and matches
It sets and is illustrated in fig. 7 shown below, Fig. 7 is to monitor configuration information interface schematic diagram according to the multi-service temperature of the application example one.
Fig. 8 is to store multi-service list schematic diagram according to the classification of the application example two, is deposited as shown in figure 8, being shown below
It include multiple business configuration lists in storage system.
It is Mail business that temperature, which monitors configuration information 1, and configuration 2,3 configures for TV, is association temperature monitoring configuration information.Match
Setting 1 is independent temperature monitoring configuration information, and configuration 2,3 is that associated configuration shares temperature management.Temperature pipe between different business
Reason and temperature scheduling are independent from each other, and such different business can use hierarchical stor simultaneously.Hierarchical stor
In order to provide stable access performance, and better control system hardware, need to carry out multiple independent temperature management
Coordinated scheduling.Multiple business sharing CPU, SSD flash memory, mechanical hard disk, Internet resources within the storage system, cannot be because of backstage temperature
Scheduling reason causes service operation stability to decline.
When hot statistics module finds that SSD flash memory or mechanical hard disk IO ability reach threshold value in some period, or
When person's storage system reports performance to reach performance threshold, it is notified that each traffic scheduling program carries out the migration velocity control of fragment copy
System.More common factor is in some peak traffic phase, and if televiewer is in 19. -20 request programs, other business are herein
The speed for needing to reduce temperature management and scheduling in time.
Example two, association temperature monitoring configuration information generate
By taking content distributing network as an example, it usually provides the business such as user live broadcast, program request, using hierarchical stor to mention
IO and large capacity ability are read for high-performance.Business has storage system major requirement: a large amount of tape reading are wide, lower delay and compared with
Large storage capacity.The common scene of business: typical time section spectators watch it is more steady with demand TV program, but daily it is several
Viewing program is concentrated in the particular times such as a period and weekend, can trigger storage system peak traffic.It is with operation list HOT
Example, such as user, often in 11. -12 points and the request program of evening 19-21 point, storage system pressure is larger at this time.If energy will very
The fragment of heat is dispatched in SSD flash memory, then the handling capacity of storage system can be improved and compared with low delay.This period we
Referred to as peak period.Other times section user's request program, the business of storage system are steady.The temperature management of peak period and usually industry
Temperature of being engaged in has a great difference, cannot be determined with a set of standard.Using this programme, 3 or more can be configured for HOT catalogue
Temperature monitor configuration information, it is as follows:
Configuration 1, hot statistics period (initial time, end time, similarly hereinafter) are configured to daily 11-12 point, and temperature updates
Time is per half an hour, and calculation formula etc. does not do specified otherwise, by taking default configuration as an example.
Configuration 2 monitors configuration information as HOT catalogue basis temperature, mainly using it is usual when section business, when hot statistics
Between section be early 8 points -23 points of evening, temperature renewal time is each hour.
Configuration 3, hot statistics period are evening 18-22 point, and temperature renewal time is per half an hour.
Illustrate: configuration 1 is as separate configurations.Configuration 2, configuration 3 are set as associated configuration, and initial weight is respectively 0.2 He
0.8.SSD memory space is occupied according to plan of operation, and same business association temperature monitoring configuration information makes it is not necessary that this value is accurately arranged
With same configuration data.Other configurations repeat no more.
HOT business 3 configurations in systems logical construction as shown in figure 8, operation list HOT by above-mentioned with postponing, deposit
Storage system distributes respective resources: generating corresponding list to be upgraded, list to be degraded, creation scheduler task etc..Wherein configuration 1 has
Individual list to be upgraded, to be degraded and temperature management role.A list to be upgraded, to be degraded is shared in configuration 2, configuration 3, and
And they have a public temperature management role that can execute with configuration 2,3 rule of configuration.
This example is also provided in a kind of system operation, after storage system perception service rush hour section, automatically generates
It is associated with temperature and monitors configuration information.When the existing basic temperature of operation list monitors configuration information, system is according to hot statistics module
It statistical service peak period, generates the newly-increased associated configuration of operation list, and existing temperature monitoring configuration information and new is set
The weight of gain of heat degree monitoring configuration information.It can help customer analysis to go out the peak traffic period, and generate new association heat
Degree monitoring configuration information, it is automatic to carry out temperature scheduling, simplify operation maintenance personnel configuration complexity.Key step has:
(1) after system runs a complete temperature dispatching cycle and measurement period, there is the peak traffic period, beyond flat
When N times of preset value of amount of access.And temperature monitoring configuration information is traversed, does not find the association temperature prison of relevant time period
Survey configuration information.
(2) it notifies coordinated scheduling module, generates new associated configuration.
(3) coordinated scheduling module obtains this operation list and already present temperature monitoring configuration information and time period
Statistical information generates a newly-increased associated configuration.The newly-increased associated configuration hot statistics time is set as the peak business period, heat
It spends the parameters such as renewal time and monitors configuration information referring to existing temperature, the weight of newly-increased associated configuration is set.
(4) newly-increased associated configuration is added in allocation list by coordinated scheduling module.
Example three, association temperature monitor the management of configuration information weight
The same operation list associated configuration weight is specified when storage system initializes, and can both repair in O&M
Change, can also system in the process of running, according to hot statistics module data, be automatically adjusted.It, can after this exemplary application
To reduce parameter adjustment and frequent upgraded version in O&M.
Operation list occupies the space SSD and mechanical hard disk space in hot statistics module measurement period, and business reads SSD flash memory
With number, the byte number of mechanical hard disk etc., each associated configuration calculates the upgrading fragment number etc. obtained.
The weight value range of associated configuration is [0,1], and initialization weight default value is equal to 1/ associated configuration number.Fig. 9
Be according to the weight management process schematic diagram of the another example three of the application, as shown in Figure 9 the following steps are included:
Step 1, initial weight;
Step 2, hot statistics task are completed to start weight monitor task to after whole system items statistics;
Step 3 is searched in temperature configuration, most related temperature configuration in each group of associated configuration;
Step 4, the most related temperature configuration weight of setting is original value+increment weight Wd;
Step 5, next hot statistics period repeat the above steps, when some temperature configuration weight reaches threshold value (such as
1), but with preset schedule efficiency, generate statistical report or alarm.
The detailed process of weight management process may include: storage system statistical module notice coordinated scheduling module, starting
Weight monitor task, with the weight of fixed mode adjustment associated configuration.Such as it is adjusted with fixed step size 0.1, is searched and is closed
Most related temperature monitors configuration information in this measurement period in connection temperature monitoring configuration information.Most related temperature monitors configuration information
Refer in a preset measurement period, the upgrading fragment number and this temperature management role being calculated in some configuration are practical
The immediate configuration of fragment number of upgrading.Then most related temperature is monitored into increment weight w in configuration informationd, increase by 0.1.
In next measurement period, hot statistics data are analyzed, weight is adjusted.It (is most associated with when final several cycles of operation
Weight reach 1) temperature scheduling and real data statistics discovery dispatching efficiency be less than preset schedule efficiency, then generate statistical report and
Alarm automatically generates the associated configuration of rush hour section.
It additionally supports when some operation list temperature dispatching efficiency is more steady, when being needed beyond service feature, its phase
It closes associated configuration weight to be set as not needing to adjust in some period, is applicable in fixed value.
Example four, demonstration fragment eliminate module.
This programme supports multiple business and the multiple temperatures of single business configuration to monitor configuration information.Their actual moving process
In share SSD flash memory, and there is independent temperature management and temperature to dispatch, can make that the space SSD uses and release causes some problems.
Therefore increase to distribute and eliminate module as auxiliary, smoothly adapt to multiple temperature management and temperature scheduling.
Figure 11 is to eliminate main flow schematic diagram according to the fragment of the application example four, as shown in figure 11, storage system point
It is as follows that piece eliminates module basic procedure:
Step 1, all temperature configurations of traversal current business catalogue and hot statistics, with the sequence of SSD space hold.Take SSD
The configuration of space hold maximum temperature is set as current temperature configuration.
Step 2 traverses current temperature configuration degradation list, fragment is sorted according to temperature, is added into wait eliminate column
Table.
Step 3, the new temperature scheduling of triggering creation immediately one.After finishing scheduling, the release of SSD flash memory occupied space is checked
Meet condition, that is, exits.
Step 4, it is when the release of the space SSD is unsatisfactory for condition, all configuration temperature config directories of storage system are empty by SSD
Between sort.It repeats the above steps for each temperature catalogue.
Whether step 5, the operation list of traversal temperature configuration search fragment in SSD flash memory, and the retention time is more than to match
The SSD retention time is set, expired fragment is added wait eliminate expired list;Not out of date fragment is added not out of date list, and calculates and account for
With the space SSD.
Not out of date list is added in step 6, not out of date fragment, and calculates and occupy the space SSD.Judging SSD flash memory occupancy is
It is no to meet condition, otherwise, successively minimum temperature fragment is added and eliminates list, the new temperature scheduling of triggering creation one.
Fragment, which eliminates process, can further include following steps:
(1) all temperatures monitoring configuration information and hot statistics for traversing current business catalogue occupy SSD sky according to practical
Between sort.
(2) it takes and occupies the maximum temperature monitoring configuration information of SSD space hold.Traverse queue to be degraded, by fragment according to
Temperature sequence, and by more than the fragment of SSD retention time, queue to be eliminated is added, and (eliminating queue referring to Figure 10, Figure 10 is basis
The classification of the application example four stores more catalogue configuration temperature management and superseded structure chart).
(3) triggering immediately creates new temperature scheduling, and by temperature scheduler module, it is moved to machinery from SSD flash memory
Hard disk.
(4) lower temperature monitoring configuration information of current business catalogue is taken, second step is repeated.
(5) all classifications storage temperature monitoring configuration information catalogue is sorted according to practical SSD occupied space size;Traversal
Operation list after sequence takes one of operation list to be set as current business catalogue.Repeat the first step.
(6) triggering creates new temperature scheduling.The release of SSD occupied space meets condition, that is, exits.
(7) the current most operation lists of SSD space hold are taken, the fragment of file in catalogue is searched, check that fragment copy is
It is no on SSD flash memory, and compare copy update time and whether the SSD retention time expires.By the fragment that copy is expired, it is added
Wait eliminate expired candidate queue;Not out of date fragment is added wait eliminate not out of date candidate queue, and calculates and occupies the space SSD, is pressed
It is arranged from small to large according to temperature.
(8) fragment is taken out from wait eliminate expired candidate queue.The degradation queue of temperature scheduler module is added.Turn step 6.
(9) fragment is taken out from wait eliminate not out of date candidate queue, needs to eliminate when the fragment space of this queue is greater than to meet
The fragment of temperature minimum inside queue is eliminated come out every time by space size.Turn step 6.
Through the above description of the embodiments, those skilled in the art can be understood that according to above-mentioned implementation
The method of example can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but it is very much
In the case of the former be more preferably embodiment.Based on this understanding, the technical solution of the application is substantially in other words to existing
The part that technology contributes can be embodied in the form of software products, which is stored in a storage
In medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that a terminal device (can be mobile phone, calculate
Machine, server or network equipment etc.) execute method described in each embodiment of the application.
Embodiment two
A kind of device of data storage is additionally provided in the present embodiment, and the device is for realizing above-described embodiment and preferably
Embodiment, the descriptions that have already been made will not be repeated.As used below, predetermined function may be implemented in term " module "
The combination of software and/or hardware.Although device described in following embodiment is preferably realized with software, hardware, or
The realization of the combination of person's software and hardware is also that may and be contemplated.
According to another embodiment of the application, a kind of device of data storage is additionally provided, comprising:
First obtains module, and multiple temperatures for being retrieved as the setting of the first business monitor configuration information;
Second obtains module, for monitoring the temperature of first business respectively according to each temperature monitoring configuration information
Value, wherein the hot value is used to indicate the accessed frequency of first business;
Selecting module, for according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage institute
The position of the first business corresponding data is stated, and stores the data.
By the application, it is that the multiple temperatures of the first business configuration monitor configuration information, matches confidence according to the monitoring of each temperature
Configuration in breath is monitored the temperature of the first business, obtains the corresponding hot value of each temperature monitoring configuration information, then
The first business corresponding data position of storage, such as solid state hard disk or mechanical hard disk are selected according to multiple hot value, can be
Comprehensively consider multiple hot values later to migrate the first business corresponding data, be also possible to independently according to a hot value
First business corresponding data is migrated, using the above scheme, a business configuration there are multiple temperature monitoring configuration informations, can
With the more accurate hot spot data for migrating the business in time to solid state hard disk, classification storage efficiency is substantially improved, solves phase
The problem for causing hot spot data classification storage effect undesirable since hot value statistical is single in the technology of pass.
It should be noted that above-mentioned modules can be realized by software or hardware, for the latter, Ke Yitong
Following manner realization is crossed, but not limited to this: above-mentioned module is respectively positioned in same processor;Alternatively, above-mentioned modules are with any
Combined form is located in different processors.
Embodiment three
Embodiments herein additionally provides a kind of storage medium.Optionally, in the present embodiment, above-mentioned storage medium can
To be arranged to store the program code for executing following steps:
S1 is retrieved as multiple temperatures monitoring configuration information of the first business setting;
S2 monitors the hot value of first business according to each temperature monitoring configuration information, wherein the temperature respectively
Value is used to indicate the accessed frequency of first business;
S3, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection stores first business
The position of corresponding data, and store the data.
Optionally, in the present embodiment, above-mentioned storage medium can include but is not limited to: USB flash disk, read-only memory (ROM,
Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or
The various media that can store program code such as CD.
Embodiments herein additionally provides a kind of electronic device, including memory and processor, stores in the memory
There is computer program, which is arranged to run computer program to execute the step in any of the above-described embodiment of the method
Suddenly.
Optionally, above-mentioned electronic device can also include transmitting device and input-output equipment, wherein the transmitting device
It is connected with above-mentioned processor, which connects with above-mentioned processor.
Optionally, in the present embodiment, above-mentioned processor can be set to execute following steps by computer program:
S1 is retrieved as multiple temperatures monitoring configuration information of the first business setting;
S2 monitors the hot value of first business according to each temperature monitoring configuration information, wherein the temperature respectively
Value is used to indicate the accessed frequency of first business;
S3, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection stores first business
The position of corresponding data, and store the data.
Optionally, the specific example in the present embodiment can be with reference to described in above-described embodiment and optional embodiment
Example, details are not described herein for the present embodiment.
Optionally, the specific example in the present embodiment can be with reference to described in above-described embodiment and optional embodiment
Example, details are not described herein for the present embodiment.
Obviously, those skilled in the art should be understood that each module of above-mentioned the application or each step can be with general
Computing device realize that they can be concentrated on a single computing device, or be distributed in multiple computing devices and formed
Network on, optionally, they can be realized with the program code that computing device can perform, it is thus possible to which they are stored
It is performed by computing device in the storage device, and in some cases, it can be to be different from shown in sequence execution herein
Out or description the step of, perhaps they are fabricated to each integrated circuit modules or by them multiple modules or
Step is fabricated to single integrated circuit module to realize.It is combined in this way, the application is not limited to any specific hardware and software.
The foregoing is merely preferred embodiment of the present application, are not intended to limit this application, for the skill of this field
For art personnel, various changes and changes are possible in this application.Within the spirit and principles of this application, made any to repair
Change, equivalent replacement, improvement etc., should be included within the scope of protection of this application.
Claims (15)
1. a kind of method of data storage characterized by comprising
It is retrieved as multiple temperatures monitoring configuration information of the first business setting;
Monitor the hot value of first business respectively according to each temperature monitoring configuration information, wherein the hot value is used for
Indicate the accessed frequency of first business;
According to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection stores the first business corresponding data
Position, and store the data.
2. the method according to claim 1, wherein being retrieved as multiple temperatures monitoring configuration of the first business setting
Information, comprising:
Obtain at least one the following information for including in the temperature monitoring configuration information:
Temperature update cycle, hot statistics initial time, hot statistics end time.
3. the method according to claim 1, wherein according to each temperature monitoring configuration information monitor respectively described in
The hot value of first business, comprising:
Within each temperature monitoring configuration information corresponding hot statistics time started to hot statistics end time, statistics is each
First time accessed number of first business described in the temperature update cycle;
The hot value of corresponding first business of each temperature monitoring configuration information is obtained according to described first time number.
4. the method according to claim 1, wherein according to each temperature monitoring configuration information monitor respectively described in
The hot value of first business, comprising:
The first temperature monitoring configuration information in the multiple temperature monitoring configuration information is directed to the first of first business
When operation list, one or more data point in first operation list are counted according to first temperature monitoring configuration information
The hot value of piece.
5. the method according to claim 1, wherein corresponding more according to the multiple temperature monitoring configuration information
A hot value, selection store the position of the first business corresponding data, and store the data, comprising:
When the multiple temperature monitoring configuration information is that associated temperature monitors configuration information, each temperature monitoring configuration is obtained
The product of information corresponding hot value and default weight;
Obtain the product of the multiple temperature monitoring configuration information and value, according to described and value selection storage first business
The position of corresponding data, and store the data.
6. according to the method described in claim 5, it is characterized in that, according to described corresponding with value selection storage first business
The position of data, and store the data, comprising:
When described and value is greater than heat degree threshold, the corresponding data of first business are migrated by mechanical hard disk hard to solid-state
Disk;
When described and value is less than heat degree threshold, the corresponding data of first business are migrated by solid state hard disk to mechanical hard
Disk.
7. the method according to claim 1, wherein selection stores the position of the first business corresponding data,
And store the data, comprising:
Selection stores the solid state hard disk or mechanical hard disk of the copy of the first data fragmentation of first business;
The copy is stored to selected solid state hard disk or mechanical hard disk.
8. the method according to the description of claim 7 is characterized in that the copy is migrated to solid state hard disk, the side
Method further include:
Within a temperature update cycle, statistics reads the solid state hard disk and reads mechanical hard disk when executing first business
Number ratio;
When the number ratio is lower than preset ratio, the default weight of the multiple temperature monitoring configuration information is adjusted to increase
Next temperature update cycle corresponding number ratio.
9. according to the method described in claim 8, it is characterized in that, adjusting the default power of the multiple temperature monitoring configuration information
After weight is to increase corresponding number ratio of next temperature update cycle, which comprises
After adjustment by the default weight of multiple temperature update cycles, detect that the number ratio reaches maximum value;
When the maximum value is still less than the preset ratio, generates statistical report and alert.
10. the method according to claim 1, wherein corresponding according to the multiple temperature monitoring configuration information
Multiple hot values, selection store the position of the first business corresponding data, comprising:
When the multiple temperature monitoring configuration information is temperature monitoring configuration information independent of each other, respectively according to each heat
The corresponding hot value selection of degree monitoring configuration information stores the position of the first business corresponding data.
11. the method according to claim 1, wherein corresponding according to the multiple temperature monitoring configuration information
Multiple hot values, selection store the position of the first business corresponding data, and after storing the data, the method is also wrapped
It includes:
Second time accessed number of first business described in real-time statistics, it is automatic raw when second number meets preset condition
Configuration information is monitored at the second temperature of first business.
12. the method according to claim 1, wherein corresponding according to the multiple temperature monitoring configuration information
Multiple hot values, selection store the position of the first business corresponding data, and after storing the data, the method is also wrapped
It includes:
When the storage state for the first hard disk for being stored with data meets preset state, at least one in the following manner release institute
State the memory space of the first hard disk:
The hot value stored on first hard disk is gone out lower than heat degree threshold or the smallest second business migration of hot value;
The smallest data fragmentation of hot value of the second business stored on first hard disk is migrated out.
13. a kind of device of data storage characterized by comprising
First obtains module, and multiple temperatures for being retrieved as the setting of the first business monitor configuration information;
Second obtains module, for monitoring the hot value of first business respectively according to each temperature monitoring configuration information,
In, the hot value is used to indicate the accessed frequency of first business;
Selecting module, for according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described the
The position of one business corresponding data, and store the data.
14. a kind of storage medium, which is characterized in that be stored with computer program in the storage medium, wherein the computer
Program is arranged to execute method described in any one of claim 1 to 12 when operation.
15. a kind of electronic device, including memory and processor, which is characterized in that be stored with computer journey in the memory
Sequence, the processor are arranged to run the computer program to execute described in any one of claim 1 to 12
Method.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811616160.3A CN110209345A (en) | 2018-12-27 | 2018-12-27 | The method and device of data storage |
PCT/CN2019/115774 WO2020134609A1 (en) | 2018-12-27 | 2019-11-05 | Data storage method and apparatus |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811616160.3A CN110209345A (en) | 2018-12-27 | 2018-12-27 | The method and device of data storage |
Publications (1)
Publication Number | Publication Date |
---|---|
CN110209345A true CN110209345A (en) | 2019-09-06 |
Family
ID=67780027
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201811616160.3A Pending CN110209345A (en) | 2018-12-27 | 2018-12-27 | The method and device of data storage |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN110209345A (en) |
WO (1) | WO2020134609A1 (en) |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111309251A (en) * | 2020-01-21 | 2020-06-19 | 青梧桐有限责任公司 | Data storage method, system, electronic device and readable storage medium |
WO2020134609A1 (en) * | 2018-12-27 | 2020-07-02 | 中兴通讯股份有限公司 | Data storage method and apparatus |
CN111400318A (en) * | 2020-03-09 | 2020-07-10 | 北京易华录信息技术股份有限公司 | Method and device for generating scheduling strategy of data storage |
CN111427969A (en) * | 2020-03-18 | 2020-07-17 | 清华大学 | Data replacement method of hierarchical storage system |
CN112559504A (en) * | 2020-12-09 | 2021-03-26 | 北京思特奇信息技术股份有限公司 | Data cleaning method and device based on data heat and storage medium |
CN112734103A (en) * | 2021-01-05 | 2021-04-30 | 烽火通信科技股份有限公司 | Video cold picture prediction method and device based on space-time sequence |
CN113032369A (en) * | 2021-03-26 | 2021-06-25 | 山东英信计算机技术有限公司 | Data migration method, device and medium |
CN113297005A (en) * | 2020-07-27 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data processing method, device and equipment |
CN113885797A (en) * | 2021-09-24 | 2022-01-04 | 济南浪潮数据技术有限公司 | Data storage method, device, equipment and storage medium |
CN114020828A (en) * | 2021-09-27 | 2022-02-08 | 南京云创大数据科技股份有限公司 | Distributed hierarchical storage system |
CN114666121A (en) * | 2022-03-21 | 2022-06-24 | 山东鼎夏智能科技有限公司 | Data monitoring method and device |
CN116189896A (en) * | 2023-04-24 | 2023-05-30 | 北京快舒尔医疗技术有限公司 | Cloud-based diabetes health data early warning method and system |
CN114020828B (en) * | 2021-09-27 | 2024-05-31 | 南京云创大数据科技股份有限公司 | Distributed hierarchical storage system |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103186350A (en) * | 2011-12-31 | 2013-07-03 | 北京快网科技有限公司 | Hybrid storage system and hot spot data block migration method |
CN104133643A (en) * | 2014-08-04 | 2014-11-05 | 浪潮电子信息产业股份有限公司 | Method for improving data transfer efficiency under automatic data hierarchical storage frame |
US20150286418A1 (en) * | 2013-01-22 | 2015-10-08 | International Business Machines Corporation | Tiered caching and migration in differing granularities |
CN106709068A (en) * | 2017-01-22 | 2017-05-24 | 郑州云海信息技术有限公司 | Hotspot data identification method and device |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103150347B (en) * | 2013-02-07 | 2015-10-21 | 浙江大学 | Based on the dynamic replication management method of file temperature |
CN108121802A (en) * | 2017-12-22 | 2018-06-05 | 东软集团股份有限公司 | The thermodynamic analysis method, apparatus and its equipment of web page access |
CN110209345A (en) * | 2018-12-27 | 2019-09-06 | 中兴通讯股份有限公司 | The method and device of data storage |
-
2018
- 2018-12-27 CN CN201811616160.3A patent/CN110209345A/en active Pending
-
2019
- 2019-11-05 WO PCT/CN2019/115774 patent/WO2020134609A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103186350A (en) * | 2011-12-31 | 2013-07-03 | 北京快网科技有限公司 | Hybrid storage system and hot spot data block migration method |
US20150286418A1 (en) * | 2013-01-22 | 2015-10-08 | International Business Machines Corporation | Tiered caching and migration in differing granularities |
CN104133643A (en) * | 2014-08-04 | 2014-11-05 | 浪潮电子信息产业股份有限公司 | Method for improving data transfer efficiency under automatic data hierarchical storage frame |
CN106709068A (en) * | 2017-01-22 | 2017-05-24 | 郑州云海信息技术有限公司 | Hotspot data identification method and device |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020134609A1 (en) * | 2018-12-27 | 2020-07-02 | 中兴通讯股份有限公司 | Data storage method and apparatus |
CN111309251A (en) * | 2020-01-21 | 2020-06-19 | 青梧桐有限责任公司 | Data storage method, system, electronic device and readable storage medium |
CN111400318B (en) * | 2020-03-09 | 2023-09-15 | 北京易华录信息技术股份有限公司 | Method and device for generating scheduling policy of data storage |
CN111400318A (en) * | 2020-03-09 | 2020-07-10 | 北京易华录信息技术股份有限公司 | Method and device for generating scheduling strategy of data storage |
CN111427969A (en) * | 2020-03-18 | 2020-07-17 | 清华大学 | Data replacement method of hierarchical storage system |
CN111427969B (en) * | 2020-03-18 | 2022-05-27 | 清华大学 | Data replacement method of hierarchical storage system |
CN113297005A (en) * | 2020-07-27 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data processing method, device and equipment |
CN113297005B (en) * | 2020-07-27 | 2024-01-05 | 阿里巴巴集团控股有限公司 | Data processing method, device and equipment |
CN112559504A (en) * | 2020-12-09 | 2021-03-26 | 北京思特奇信息技术股份有限公司 | Data cleaning method and device based on data heat and storage medium |
CN112734103A (en) * | 2021-01-05 | 2021-04-30 | 烽火通信科技股份有限公司 | Video cold picture prediction method and device based on space-time sequence |
CN113032369A (en) * | 2021-03-26 | 2021-06-25 | 山东英信计算机技术有限公司 | Data migration method, device and medium |
CN113885797A (en) * | 2021-09-24 | 2022-01-04 | 济南浪潮数据技术有限公司 | Data storage method, device, equipment and storage medium |
CN113885797B (en) * | 2021-09-24 | 2023-12-22 | 济南浪潮数据技术有限公司 | Data storage method, device, equipment and storage medium |
CN114020828A (en) * | 2021-09-27 | 2022-02-08 | 南京云创大数据科技股份有限公司 | Distributed hierarchical storage system |
CN114020828B (en) * | 2021-09-27 | 2024-05-31 | 南京云创大数据科技股份有限公司 | Distributed hierarchical storage system |
CN114666121A (en) * | 2022-03-21 | 2022-06-24 | 山东鼎夏智能科技有限公司 | Data monitoring method and device |
CN116189896A (en) * | 2023-04-24 | 2023-05-30 | 北京快舒尔医疗技术有限公司 | Cloud-based diabetes health data early warning method and system |
CN116189896B (en) * | 2023-04-24 | 2023-08-08 | 北京快舒尔医疗技术有限公司 | Cloud-based diabetes health data early warning method and system |
Also Published As
Publication number | Publication date |
---|---|
WO2020134609A1 (en) | 2020-07-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110209345A (en) | The method and device of data storage | |
CN104935482B (en) | Distributed monitoring system and method | |
EP3279794A1 (en) | Time-based node election method and apparatus | |
US20100185768A1 (en) | Resource allocation and modification using statistical analysis | |
CN103827858A (en) | Caching in mobile networks | |
CN106888381B (en) | A kind of data resource storage method and device | |
US10326854B2 (en) | Method and apparatus for data caching in a communications network | |
CN109861878A (en) | The monitoring method and relevant device of the topic data of kafka cluster | |
CN109190070A (en) | A kind of data processing method, device, system and application server | |
CN109379425A (en) | Distributed cluster deployment management method and device | |
CN110213203A (en) | Network dispatching method, device and computer storage medium | |
CN112165508B (en) | Resource allocation method for multi-tenant cloud storage request service | |
CN102098170B (en) | Data acquisition optimization method and system | |
US10891849B1 (en) | System for suppressing false service outage alerts | |
CN100505871C (en) | Video-frequency data processing method, video-frequency collecting equipment and video-frequency management equipment | |
US11099921B2 (en) | Predictive system resource allocation | |
CN112506926A (en) | Monitoring data storage and query method and corresponding device, equipment and medium | |
US20230251789A1 (en) | Record information management based on self-describing attributes | |
CN111324459A (en) | Calendar-based resource scheduling method and device, electronic equipment and storage medium | |
CN113965538B (en) | Equipment state message processing method, device and storage medium | |
CN113315836B (en) | File access request scheduling method and device, electronic equipment and storage medium | |
CN114090201A (en) | Resource scheduling method, device, equipment and storage medium | |
CN106357735A (en) | Method and device for operating infrastructure layer of cloud computing architecture | |
US11026108B2 (en) | Fault monitoring in a utility supply network | |
EP4365751A1 (en) | Dynamic data retention policies for iot platforms |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190906 |
|
RJ01 | Rejection of invention patent application after publication |