CN110209345A - The method and device of data storage - Google Patents

The method and device of data storage Download PDF

Info

Publication number
CN110209345A
CN110209345A CN201811616160.3A CN201811616160A CN110209345A CN 110209345 A CN110209345 A CN 110209345A CN 201811616160 A CN201811616160 A CN 201811616160A CN 110209345 A CN110209345 A CN 110209345A
Authority
CN
China
Prior art keywords
business
configuration information
temperature
data
hot
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811616160.3A
Other languages
Chinese (zh)
Inventor
王波
屠要峰
黄震江
韩银俊
洪建峰
郭斌
丁毅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to CN201811616160.3A priority Critical patent/CN110209345A/en
Publication of CN110209345A publication Critical patent/CN110209345A/en
Priority to PCT/CN2019/115774 priority patent/WO2020134609A1/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604Improving or facilitating administration, e.g. storage management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Debugging And Monitoring (AREA)

Abstract

This application provides a kind of method and devices of data storage, wherein this method comprises: monitoring configuration information for the multiple temperatures of the first business configuration, the temperature of the first business is monitored according to the configuration in each temperature monitoring configuration information, obtain the corresponding hot value of each temperature monitoring configuration information, then the first business corresponding data position of storage is selected according to multiple hot value, such as solid state hard disk or mechanical hard disk, it can be and the first business corresponding data is migrated after comprehensively considering multiple hot values, it is also possible to independently migrate the first business corresponding data according to a hot value, using the above scheme, one business configuration has multiple temperature monitoring configuration informations, can the more accurate hot spot data for migrating the business in time to solid state hard disk, classification storage efficiency is substantially improved, it solves in the related technology The problem for causing hot spot data classification storage effect undesirable since hot value statistical is single.

Description

The method and device of data storage
Technical field
This application involves but be not limited to field of data storage, in particular to a kind of method and device of data storage.
Background technique
In the related art, usual distributed memory system framework is made of following three parts: file access client mould Block, meta data server module and storage server modules.Fig. 1 is according to distributed memory system structure mould in the related technology Type figure provides application file behaviour as shown in Figure 1, file access client is the agency of application program access file system Make interface, the functions such as hot statistics report;Meta data server module has the management of configuration data management and file metadata With hierarchical storage management function;Storage server modules actual storage file data within the storage system.
Storage system includes metadata and file content, both metadata and file content with document form storing data Be to separate storage: metadata (comprising the file information and data block location) is by meta data server module management, file content It is to store according to fixed size fragment into storage server.Each fragment has multiple redundancy pairs on different volumes in system This, ensures fragment reliability.Such as a file size 100M, system configuration fragment size 64M, then this file has 2 points Piece.
The universal mixed insertion mechanical hard disk of distributed memory system (Distribute Storage System, referred to as DSS) and SSD (Solid State Drives, solid state hard disk) flash memory, to meet large capacity and high performance demands.Novel SSD dodges in recent years It deposits, such as NVMe protocol type, even more has the characteristics that very high performance, ultralow delay, also gradually answered extensively in enterprise-level storage With.Storage system uses hierarchical storage management different type hard disk, balanced storage performance and capacity requirement.SSD in classification storage Flash memory main function is the caching as hot spot data, to store the newest or most hot data of current business.Data are cold and hot Judgment basis mainly has: the indexs such as data value, data access frequency, retention time, data access size, referred to as data Access temperature.Classification stores in summary element, the copy of fragment is stored into different type hard disk, and in different type Autonomic Migration Framework is carried out according to hot spot situation between hard disk.
It is undesirable for causing hot spot data classification to store effect since hot value statistical is single in the related technology Problem, there is presently no effective solution schemes.
Summary of the invention
The embodiment of the present application provides a kind of method and device of data storage, at least to solve in the related technology due to heat The single problem for causing hot spot data classification storage effect undesirable of angle value statistical.
According to one embodiment of the application, a kind of method of data storage is provided, comprising: be retrieved as the first business and set The multiple temperatures monitoring configuration information set;Monitor the temperature of first business respectively according to each temperature monitoring configuration information Value, wherein the hot value is used to indicate the accessed frequency of first business;Match confidence according to the monitoring of the multiple temperature Corresponding multiple hot values are ceased, selection stores the position of the first business corresponding data, and stores the data.
According to another embodiment of the application, a kind of device of data storage is additionally provided, comprising: first obtains mould Block, multiple temperatures for being retrieved as the setting of the first business monitor configuration information;Second obtains module, for according to each temperature Monitoring configuration information monitors the hot value of first business respectively, wherein the hot value is used to indicate first business Accessed frequency;Selecting module, for according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection to be deposited The position of the first business corresponding data is stored up, and stores the data.
According to another embodiment of the application, a kind of storage medium is additionally provided, meter is stored in the storage medium Calculation machine program, wherein the computer program is arranged to execute the step in any of the above-described embodiment of the method when operation.
According to another embodiment of the application, a kind of electronic device, including memory and processor are additionally provided, it is described Computer program is stored in memory, the processor is arranged to run the computer program to execute any of the above-described Step in embodiment of the method.
By the application, it is that the multiple temperatures of the first business configuration monitor configuration information, matches confidence according to the monitoring of each temperature Configuration in breath is monitored the temperature of the first business, obtains the corresponding hot value of each temperature monitoring configuration information, then The first business corresponding data position of storage, such as solid state hard disk or mechanical hard disk are selected according to multiple hot value, can be Comprehensively consider multiple hot values later to migrate the first business corresponding data, be also possible to independently according to a hot value First business corresponding data is migrated, using the above scheme, a business configuration there are multiple temperature monitoring configuration informations, can With the more accurate hot spot data for migrating the business in time to solid state hard disk, classification storage efficiency is substantially improved, solves phase The problem for causing hot spot data classification storage effect undesirable since hot value statistical is single in the technology of pass.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present application, constitutes part of this application, this Shen Illustrative embodiments and their description please are not constituted an undue limitation on the present application for explaining the application.In the accompanying drawings:
Fig. 1 is according to distributed memory system structural model figure in the related technology;
Fig. 2 is according to memory hierarchy illustraton of model in the related technology;
Fig. 3 is a kind of hardware block diagram of the terminal of the method for data storage of the embodiment of the present application;
Fig. 4 is the flow chart according to the method for the data of the embodiment of the present application storage;
Fig. 5 is to be classified storage according to the multi-service of the embodiment of the present application to improve module interaction figure;
Fig. 6 is to store newly-increased module interaction figure according to the multi-service of the embodiment of the present application classification;
Fig. 7 is to monitor configuration information interface schematic diagram according to the multi-service temperature of the application example one;
Fig. 8 is to store multi-service list schematic diagram according to the classification of the application example two;
Fig. 9 is the weight management process schematic diagram according to the another example three of the application;
Figure 10 is that more catalogue configuration temperature management and superseded structure chart are stored according to the classification of the application example four;
Figure 11 is to eliminate main flow schematic diagram according to the fragment of the application example four.
Specific embodiment
The application is described in detail below with reference to attached drawing and in conjunction with the embodiments.It should be noted that not conflicting In the case of, the features in the embodiments and the embodiments of the present application can be combined with each other.
It should be noted that the description and claims of this application and term " first " in above-mentioned attached drawing, " Two " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.
Classification storage architecture main functional modules are as follows: file access client hot statistics and reporting;Metadata Service Device configuration management module, temperature management module, temperature scheduler module, statistical module, Fig. 2 are deposited according to classification in the related technology Storage structure illustraton of model, as shown in Figure 2, including access client, meta data server, storage server, temperature configuration module, Temperature management module, hot statistics module, fragment eliminate module, temperature scheduler module, weight management module, coordinated scheduling mould Block.
Classification storage temperature manages general flow are as follows:
(1) when application call interface (such as read, sendfile) access file fragmentation, file access client system Meter reports the information such as fragment read-write number, read-write byte number to give meta data server temperature management module.
(2) meta data server, which receives, currently reports fragment raw information, in conjunction with history temperature and currently reports temperature, The fragment temperature is calculated according to formula and is saved in metadata.
(3) fragment of temperature management module timing scan metadata, if fragment temperature is greater than configuration heat degree threshold and divides All copies of piece are respectively positioned on mechanical hard disk, then associated metadata are inserted into list to be upgraded, and again by column to be upgraded List sorting.If fragment hot value is less than heat degree threshold and has copy on SSD flash memory, associated metadata is inserted into wait drop Grade list, and list to be degraded of resequencing;Heat degree threshold refers to that data access temperature is more than that the fragment of this value can be made herein SSD flash memory is upgraded to for candidate fragment.List to be upgraded, which refers to, have been sequenced sequence from big to small using temperature as keyword and has included to meet Burst information beyond heat degree threshold;Degradation list refers to has sequenced sequence by keyword of temperature from small to large, and temperature is less than temperature The burst information of threshold value.
(3) temperature scheduler module regular check system configuration takes out list to be upgraded and wait eligible in the list that degrades Fragment to storage server modules assign fragment copy migration instruction.
(4) after the success of storage server migration fragment copy, meta data server is reported;
(5) new hard disk position after the migration of meta data server modification fragment copy.
The relevant technologies are that statistics file or object temperature are anti-to predict as history temperature in several historical time sections The temperature of file in following a period of time is reflected, accordingly as classification storage temperature judgment basis, different temperature file migrations are arrived On the hard disk of different performance.
There are more limitations for classification memory technology in the related technology, first is that Supporting multi-services are poor, a set of storage is often Need to provide storage service for multiple business, different business has different Hot Contents and hot spot period, general based on going through The statistics of history file access temperature, it will cause hot spot not hot, the effect for being classified storage is undesirable;Second is different time sections heat It is poor that point is supported, even same business, section often has different Hot Contents in different times, single based on the passing time The statistics of section, will lead to hot spot dislocation, and the efficiency for being classified storage is had a greatly reduced quality;Third is that focus statistics period assignment management is tired Difficulty is difficult to adapt to the variation of Hot Contents and period by the artificial setting hot spot period.
Embodiment one
Embodiment of the method provided by the embodiment of the present application one can be in terminal or similar arithmetic unit It executes.For running on computer terminals, Fig. 3 is that a kind of computer of the method for data storage of the embodiment of the present application is whole The hardware block diagram at end, as shown in figure 3, terminal may include one or more (only showing one in Fig. 3) processing Device 302 (processing unit that processor 302 can include but is not limited to Micro-processor MCV or programmable logic device FPGA etc.) and Memory 304 for storing data, optionally, above-mentioned terminal can also include the transmitting device for communication function 306 and input-output equipment 308.It will appreciated by the skilled person that structure shown in Fig. 3 is only to illustrate, simultaneously The structure of above-mentioned terminal is not caused to limit.For example, terminal may also include than shown in Fig. 3 more or more Few component, or with the configuration different from shown in Fig. 3.
Memory 304 can be used for storing the software program and module of application software, such as the data in the embodiment of the present application Corresponding program instruction/the module of the method for storage, processor 302 by the software program that is stored in memory 304 of operation with And module realizes above-mentioned method thereby executing various function application and data processing.Memory 304 may include high speed Random access memory may also include nonvolatile memory, such as one or more magnetic storage device, flash memory or other are non- Volatile solid-state.In some instances, memory 304 can further comprise remotely located relative to processor 302 Memory, these remote memories can pass through network connection to terminal.The example of above-mentioned network includes but is not limited to Internet, intranet, local area network, mobile radio communication and combinations thereof.
Transmitting device 306 is used to that data to be received or sent via a network.Above-mentioned network specific example may include The wireless network that the communication providers of terminal provide.In an example, transmitting device 306 includes a Network adaptation Device (Network Interface Controller, NIC), can be connected by base station with other network equipments so as to it is mutual Networking is communicated.In an example, transmitting device 306 can be radio frequency (Radio Frequency, RF) module, use In wirelessly being communicated with internet.
A kind of method of data storage for running on above-mentioned terminal is provided in the present embodiment, and Fig. 4 is basis The flow chart of the method for the data storage of the embodiment of the present application, as shown in figure 4, the process includes the following steps:
Step S402 is retrieved as multiple temperatures monitoring configuration information of the first business setting;
Step S404 monitors the hot value of first business according to each temperature monitoring configuration information, wherein institute respectively It states hot value and is used to indicate the accessed frequency of first business;
Step S406, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described the The position of one business corresponding data, and store the data;
Modification metadata information can be corresponded to after change storage location.
It through the above steps, is that the multiple temperatures of the first business configuration monitor configuration information, according to each heat by the application Configuration in degree monitoring configuration information is monitored the temperature of the first business, and it is corresponding to obtain each temperature monitoring configuration information Then hot value selects the first business corresponding data position of storage, such as solid state hard disk or machinery according to multiple hot value Hard disk, can be to comprehensively consider and is migrated after multiple hot values to the first business corresponding data, be also possible to independently according to The first business corresponding data is migrated according to a hot value, using the above scheme, a business configuration there are multiple temperatures to supervise Survey configuration information, can the more accurate hot spot data for migrating the business in time to solid state hard disk, classification storage is substantially improved Efficiency, solving leads to hot spot data classification storage since hot value statistical is single in the related technology effect is undesirable asks Topic.
Optionally, it is retrieved as multiple temperatures monitoring configuration information of the first business setting, comprising: obtain the temperature monitoring At least one following information for including in configuration information: at the end of temperature update cycle, hot statistics initial time, hot statistics Between.
Optionally, the hot value of first business is monitored respectively according to each temperature monitoring configuration information, comprising: every A temperature monitored in the configuration information corresponding hot statistics time started to hot statistics end time, counts each temperature and updates First time accessed number of first business described in period;Each temperature, which is obtained, according to described first time number monitors configuration information pair The hot value for first business answered.
Optionally, the hot value of first business is monitored respectively according to each temperature monitoring configuration information, comprising: in institute State the first operation list that the first temperature monitoring configuration information in multiple temperature monitoring configuration informations is directed to first business When, the temperature of one or more data fragmentations in first operation list is counted according to first temperature monitoring configuration information Value.
Optionally, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described first The position of business corresponding data, and store the data, comprising: monitoring configuration information in the multiple temperature is associated temperature When monitoring configuration information, the product of each temperature monitoring configuration information corresponding hot value and default weight is obtained;Described in acquisition The product of multiple temperatures monitoring configuration informations and value, the position of the first business corresponding data is stored according to described and value selection It sets, and stores the data.
Optionally, the position of the first business corresponding data is stored according to described and value selection, and stores the data, It include: to be migrated by mechanical hard disk the corresponding data of first business hard to solid-state when described and value is greater than heat degree threshold Disk;When described and value is less than heat degree threshold, the corresponding data of first business are migrated by solid state hard disk to mechanical hard disk.
Optionally, selection stores the position of the first business corresponding data, and stores the data, comprising: selection is deposited Store up the solid state hard disk or mechanical hard disk of the copy of the first data fragmentation of first business;The copy is stored to selected Solid state hard disk or mechanical hard disk.
Optionally, the copy is migrated to solid state hard disk, within a temperature update cycle, described in statistics execution The solid state hard disk is read when the first business and reads the number ratio of mechanical hard disk;It is lower than preset ratio in the number ratio When, the default weight of the multiple temperature monitoring configuration information is adjusted to increase corresponding number ratio of next temperature update cycle Example.
Optionally, the default weight of the multiple temperature monitoring configuration information is adjusted to increase next temperature update cycle After corresponding number ratio, after the adjustment by the default weight of multiple temperature update cycles, the number ratio is detected Reach maximum value;When the maximum value is still less than the preset ratio, generates statistical report and alert.
Optionally, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described first The position of business corresponding data, comprising: monitoring configuration information in the multiple temperature is temperature monitoring configuration independent of each other When information, the first business corresponding data is stored according to the corresponding hot value selection of each temperature monitoring configuration information respectively Position.
Optionally, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described first The position of business corresponding data, and after storing the data, second time accessed number of the first business described in real-time statistics, When second number meets preset condition, the second temperature monitoring configuration information of first business is automatically generated.It is detecting To after the current multiple temperature monitoring configuration informations for executing first business, the data of the first business fail efficient calling Afterwards, the second temperature monitoring configuration information is automatically generated, in the subsequent temperature monitoring to the first business, which to be supervised The concrete configuration for surveying configuration information can be to the monitoring configuration information study of the temperature of other business.
Optionally, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described first The position of business corresponding data, and after storing the data, meet in the storage state for the first hard disk for being stored with data pre- If when state, at least one the in the following manner memory space of release first hard disk: will be stored on first hard disk Hot value go out lower than heat degree threshold or the smallest second business migration of hot value;Will stored on first hard disk The smallest data fragmentation of the hot value of two business migrates out.
It is further illustrated below with reference to another embodiment of the application.
In view of the limitation of above-mentioned the relevant technologies, this application discloses classification storage is improved in a kind of distributed memory system The method of efficiency.Be applicable in multi-service scene, and by it is adaptive while are carried out multiple periods with the statistics of hot spot respectively Analysis, very good solution distributed memory system classification storage in the above scenario the problem of.
The application technical problems to be solved are: a set of distributed memory system bearing multiple service, different business have Different access hot spots and rush hour section, and whether history temperature or current temperature, temperature tribute in different time periods Value is offered to be different.When, there are many business and section of different rush hours, the relevant technologies are deposited in temperature management aspect in storage system In the low problem of dispatching efficiency.It therefore, can be flexible in view of the above-mentioned problems, this programme proposes a kind of classification storage method and device Be deployed in distributed memory system, it supports multi-service that different rush hours section is arranged, and carries out independent temperature management, utilizes Business different time sections temperature and performance data automatically generate a variety of association temperature monitoring configuration informations in different time periods, change Temperature management is stored into classification, and generates statistical data automatically according to hot statistics, adjust automatically associated configuration weight is provided Method, simplify operation maintenance personnel burden.
Technical solution:
This programme increases several functional modules (meta data server dotted line frame mould in Fig. 2 on the basis of above-mentioned architecture Block) and the multiple modules realizations of optimization, flexible temperature management and improvement hot spot fragment rush hour section when realizing Supporting multi-services Scheduling problem proposes to support:
(1) multi-service carries out independent temperature management and scheduling in classification storage;
(2) storage system can be each business according to statistical data according to operating condition, automatically generate multiple times Duan Guanlian temperature monitors configuration information;Multiple temperature monitoring configuration informations can be separate configurations in some business, carry out independent Temperature management or associated configuration carry out shared temperature management.
(3) business association temperature monitoring configuration information in different time periods provides a kind of automatic in system operation The method for adjusting associated configuration weight.
Hierarchical stor supports multiple business and multiple periods to carry out temperature management and scheduling, needs for existing frame Structure adjusts optimization, and temperature monitoring configuration information module, temperature management, temperature scheduler module, heat are discussed in detail in turn below Spend statistical module, fragment eliminates module contents:
To support this programme, temperature monitors configuration information module and extends several fields, and single business can increase multiple heat Degree monitoring configuration information, distinguishes different configurations with configuration number;Hierarchical stor is supported to configure multiple business simultaneously.Thus Each temperature monitoring configuration information relevant field includes that service identification, temperature scheduling time, temperature calculation formula, fragment exist SSD flash memory retention time, SSD flash memory maximum occupying space, hot statistics initial time, hot statistics end time.
Table 1 is to monitor each primary fields meaning in configuration information according to the temperature of the application to illustrate table, such as 1 institute of table Show:
Table 1
This configuration basis field is the combination of service identification, temperature renewal time section, weight.Service identification described herein is As the mark of service operation used resource within the storage system, business can by directory name, complete trails, relative path, File prefix or suffix format etc. distinguish different service types, can be used as this programme examples of implementation.One service identification It may include multiple catalogues or complete trails.Temperature renewal time section can be some time in one day, such as 10. -14 points, Also festivals or holidays (Sunday, May Day, 11 on every Saturdays) etc. are configurable to.The same business different period can be configured to solely Vertical configuration is managed independently, and shared temperature management can also be carried out by system automatically generated associated configuration.Associated configuration power Weight can be with manual configuration, can also be in system operation, when automatically generating associated configuration, and system assigns initial value automatically, And it is automatically adjusted.It can also be calculated comprising preferred field correlation tag, fragment in SSD retention time, compileable temperature Formula etc. is combined, this temperature Managed Solution is improved.
Such as hierarchical stor 2 separate traffics catalogue HOT and TV, table 2 are supervised according to the temperature of the embodiment of the present application Configuration information schematic table is surveyed, as shown in the table, 4 temperature monitoring configuration informations are as follows:
Table 2
Configuration 1, configuration 2 be associated configuration, all act on operation list HOT, share it is same it is to be upgraded, wait degrade column Table and the same temperature management role.Configuration 3 and configuration 4 be separate configurations, it is each configuration all have it is independent it is to be upgraded, to Degradation list and individual temperature management role.Structure chart such as Figure 10.
Associated configuration in different time periods can be automatically generated in the process of running with system.Create-rule is as follows: automatic raw At premise be configuration in have related service catalogue configuration.It obtains operation list reading performance height according to hot statistics module Some time.Performance in time period exceeds 1 times or 2 times of usually operation preset value.It, can in system operation With the configuration and time period one associated configuration of generation according to time operation list, and initial weight is set.This business exists in this way Configuration information is monitored comprising multiple temperatures in storage system, each temperature, which monitors configuration information, has certain weight.Storage system System obtains data in multiple measurement periods according to hot statistics module, can be with adjust automatically associated configuration weight.
Temperature management module:
There are multiple operation lists in hierarchical stor, each operation list can configure multiple temperature monitorings with confidence Breath.For some operation list, the temperature data for needing to update at some time point and saving multiple configuration generations are just formed, are This meta data server increases several original temperature fields (such as h1, h2, h3) in file fragmentation associated metadata, to store Different temperatures monitor configuration information original temperature information in the same report cycle;Increase several temperature monitoring configuration information marks It signs (such as tag1, tag2, tag3), which temperature monitoring configuration information is corresponding original temperature field correspond to.
When application program reads file by interfaces such as read and sendfile, file access client calculates original reading Number, read-write fragment byte number are write, meta data server is sent to.Meta data server, which receives, updates fragment temperature message, Current time is read, the affiliated catalogue of respective file, and then recursive lookup upper directory is searched, is checked whether for every first class catalogue Configuration service catalogue temperature monitors configuration information, obtains configuration number of the current time within the scope of hot statistics.Fragment is related An idle temperature field, the temperature that filling is currently configured number and is calculated according to this configuration are obtained in metadata.
The same operation list associated configuration can have multiple, their shared temperature management roles.Temperature manages mould Block meeting timing scan temperature monitors configuration information, starts an individual temperature management role for each separate configurations, and Only need to start a temperature management role for the management of associated configuration temperature.After temperature management role enters runing time, The fragment associated metadata under current business catalogue is scanned, obtains current time, such as current time, in 9. -12 points, temperature is every Hour updates once, and when temperature updates task run, configuration 1 comes into effect with configuration 2, according to calculation formula calculating heat Degree, is indicated with benefit1, bennfit2.So practical temperature benefit of current slice is repaired by following formula (1) Just:
Benefit=benefit1*w1+benefit2*w2, formula (1)
In above-mentioned formula, wherein w1 is 1 associated configuration weight of configuration, and w2 is 2 associated configuration weights of configuration.At the beginning of w1, w2 Initial value is 0.5, i.e., default association configuration 1 and configures as 2 status are.
The weight of each configuration can pass through system adjust automatically in associated configuration.It is real when calculating practical temperature Temperature in border temperature and relevant configuration is closest, and this configuration statistics numbers is increased by 1.It is complete when hot statistics this period of module It after operation, counts this business and reads SSD flash memory and mechanical hard disk performance data, show that epicycle temperature dispatches actual efficiency (business SSD flash memory actual read data amount/business can be used and always read data volume).Scheduling actual efficiency is adjusted with pre- imagination Degree efficiency such as 80% compares, if actual efficiency is lower than preset schedule efficiency, configuration most associated in associated configuration is weighed 10% is raised again.So after the scheduling of several period temperatures and hot statistics, according to adjustment rule, adjustment in each period Associated configuration weight.The scheduling of (i.e. most associated weights reach 1) temperature and real data statistics discovery are adjusted after several cycles of operation It spends efficiency and is less than preset schedule efficiency, then generate statistical report and alarm, warning operation maintenance personnel needs to reappraise scheduling scheme: Adjust hot statistics time, calculation formula.
After temperature management role calculates current slice temperature, judge whether temperature is greater than heat degree threshold, if meeting condition, Then it is added into list to be upgraded.Whether temperature management role is also handled simultaneously has been upgraded to the fragment temperature of SSD flash memory and has been less than Queue to be degraded is added in heat degree threshold if meeting condition.Details are not described herein again.
Temperature scheduler module
Each temperature monitoring configuration information is taken out in the timing of this module, and it is corresponding wait rise to first look at temperature monitoring configuration information Grade list, successively takes out the highest burst information of temperature, checks that all copies of fragment whether all only on mechanical hard disk, will expire One copy of sufficient promotion condition fragment sends copy to storage server and moves to SSD flash request from mechanical disk;Copy liter After the completion of grade, the current update time point of this fragment is set.Then, burst information is taken out from list to be degraded, checks the pair of fragment Whether this has been downgraded to mechanical hard disk, and whether alreadys exceed the SSD retention time, will meet one copy of condition fragment Copy, which is sent, to storage server moves to mechanical disk request from SSD flash memory.
The function that hot statistics module has is as follows, counts each industry in each temperature dispatching cycle monitoring configuration information All fragments under catalogue of being engaged in read mechanical hard disk, the number of SSD flash memory and reading size;It calculates in temperature monitoring configuration information catalogue Read the percentage of fragment hit SSD flash memory, i.e. temperature dispatching efficiency;The space different business catalogue SSD and fragment in output system The space hold in SSD.Above-mentioned statistical information is used to assessment classification storage efficiency, and feeds back to temperature management module and improve heat Degree monitoring configuration information.
Fragment eliminates module
The multiple operation lists of hierarchical stor carry out having multiple temperature scheduling to appoint under temperature scheduling and a catalogue simultaneously Business, and SSD flash memory space is limited, it may appear that SSD flash memory space is full, causes some operation lists that temperature is needed to dispatch, but It is the problem of space SSD is occupied by other business, led to insufficient memory.There are two types of solutions:
1. monitoring configuration information for each business or temperature, SSD maximum is arranged by manual assignment mode and is occupied Space.Guarantee that the accumulated value of all configuration SSD space hold maximum values is less than the entirety space SSD.Such method needs to advise in advance Draw requirement of the business to storage system.
2. multiple business can not be accurate using hierarchical stor or association of multiple periods temperature monitoring configuration information SSD flash memory space occupies, using only plan of operation maximum space when, the use of SSD flash memory space can be more than SSD in storage system When capacity-threshold, needs to start pressure and eliminate function.Such as storage system SSD flash memory space is 24T, HOT plan of operation SSD is empty Between maximum occupy be 13T, TV plan of operation SSD space maximum 14T;Or the more space a associated configuration SSD maximums of TV occupy greatly In 24T.When the space storage system SSD is practical occupies over SSD capacity-threshold, need for all business and temperature in system Fragment occupancy carries out analysis and Free up Memory in monitoring configuration information.Replacement policy can there are many, preferably eliminate each industry Temperature is lower than the fragment of heat degree threshold in business, next eliminates the fragment that hot value is small in each business.
Illustrate a kind of method that fragment is superseded under multi-service and more temperatures monitoring configuration information below, mainly comprises the processes of
(1) the current operation list more than SSD capacity-threshold is first looked at, all temperatures monitoring of traversing directories configuration is matched Confidence breath.Each temperature is monitored into the fragment in configuration information in degradation list, fragment is added to and eliminates module.Immediately triggering Create new temperature scheduler task.
(2) if SSD space hold is unsatisfactory for condition, all temperatures monitoring of other operation lists is searched with confidence Breath.Repeat the first step.
(3) when SSD space hold is still unsatisfactory for condition, need to eliminate part fragment not out of date in SSD flash memory. After the sequence of operation list SSD space hold, the fragment of the file in catalogue is successively searched, it will be more than the SSD retention time Fragment is added fragment and eliminates module.
(4) last successively to eliminate the fragment small more than temperature in SSD capacity-threshold.
Coordinated scheduling
The temperature management of different business and temperature scheduling are independent from each other, and such different business makes simultaneously in the same time Use hierarchical stor.Their sharing CPUs, SSD flash memory, mechanical hard disk, Internet resources.Such as business access height is corresponded in HOT Peak time section carries out a large amount of TV catalogues and corresponds to a large amount of fragment temperature scheduling of business, will affect the stability of HOT catalogue.Cause This carries out coordinated scheduling to multiple independent temperature management, prevents from influencing service stability because of other business backstage scheduling reason. Major function has 2:
(1) it receives hot statistics module and notifies the peak traffic period, check that all temperatures of business monitor configuration information, from The dynamic association temperature that generates monitors configuration information, initializes initial weight.
(2) when hot statistics module finds that SSD flash memory or mechanical hard disk IO ability reach threshold value in some period, Or storage system is notified that each traffic scheduling program carries out fragment copy migration velocity when performance being reported to reach performance threshold Control.
Fig. 5 is to be classified storage according to the multi-service of the embodiment of the present application to improve module interaction figure, as shown in figure 5, this programme By increasing several management modules and optimization function, more preferably to support multi-service using same hierarchical stor, and according to Statistical module obtains different time sections and automatically generates association temperature monitoring configuration information, provides associated configuration weight adjust automatically side Method, to simplify O&M complexity and promote dispatching efficiency.After meta data server optimizes for correlation module, a variety of industry are supported Business temperature management, main flow are described as follows (see Fig. 5):
(1) after meta data server receives fragment temperature information, the affiliated operation list of file of fragment is searched, is read current Time, recursive lookup upper directory checks whether catalogue carries out temperature monitoring configuration information, and then obtains current time in temperature The configuration of scope of statistics is numbered, and the corresponding temperature of this configuration is updated.
(2) fragment of temperature management module timing scan metadata obtains the affiliated service identification of file of fragment, and current Time, and then obtain all independences of business and associated configuration.Check that current time comes into force is separate configurations or associated configuration, And then calculate fragment temperature.
(3) configuration information is monitored according to service identification and current temperature, searches configuration respective upgrades, degradation list.It checks Fragment temperature be greater than configuration heat degree threshold and all copies of fragment be respectively positioned on mechanical hard disk, then by associated metadata be inserted into Upgrade list, and again by list ordering to be upgraded.If fragment hot value is less than heat degree threshold and has copy to dodge in SSD It deposits, then associated metadata is inserted into list to be degraded, and list to be degraded of resequencing.
(4) configuration information is monitored for each group of independence temperature, phase is examined successively in several temperature scheduler tasks of start by set date It should upgrade, degradation list.Fragment copy migration request is sent to storage server.
Fig. 6 is to store newly-increased module interaction figure according to the multi-service of the embodiment of the present application classification, as shown in fig. 6, this programme Newly-increased weight management, coordinated scheduling, the superseded module of fragment are background functions, and each functions of modules realizes that front has been described, existing Showing that increasing module and existing module newly interacts process.Each newly-increased module interaction, as shown in Figure 6:
Weight management module and hot statistics, temperature monitor configuration information interactive step:
(1) business statistics information in the hot statistics execution cycle, sends notification to weight management module;
(2) the relevant temperature monitoring configuration information of weight management acquisition business institute, the corresponding focus statistics data of retrieval service, Calculate associated configuration weight;
(3) associated configuration weight is updated, and is stored in temperature monitoring configuration information, persistent storage is carried out.
Coordinated scheduling module is interacted with hot statistics, temperature scheduling, temperature monitoring configuration information, major function process:
(1) regular check checks performance of storage system, SSD, mechanical hard disk hit situation, when system is busy, notifies institute There is business carrying out temperature scheduler task, reduces migration velocity.
(2) there is portfolio peak period in hot statistics task discovery, is arranged beyond threshold value, notifies coordinated scheduling module.
(3) coordinated scheduling module obtains the peak traffic period, checks that all temperatures of business monitor configuration information, automatic raw Configuration information is monitored at association temperature, and initializes initial weight, in deposit temperature monitoring configuration information.
Technical problems to be solved in this application are: a set of distributed memory system bearing multiple service, different business tool There are different access hot spot and rush hour section, and whether history temperature or current temperature, temperature in different time periods Contribution margin is different.When, there are many business and section of different rush hours, the prior art is in temperature management aspect in storage system Shortcomings and the low problem of dispatching efficiency.Therefore distributed memory system is in view of the above-mentioned problems, propose a kind of classification storage dress It sets, can flexibly be deployed in distributed memory system, it supports multi-service, and automatically generates a variety of in different time periods It is associated with temperature and monitors configuration information, improve classification storage temperature management, and generate association temperature prison automatically according to hot statistics Configuration information is surveyed, the method for adjust automatically associated configuration weight is provided, simplifies operation maintenance personnel burden.
Example one, multi-service temperature monitor configuration information and management
Above-mentioned hierarchical stor can also carry webpage video cache, small other than the program request of big video, live broadcast service The business such as program application, mailbox backup.These business and video on demand user group, the flat rate of access, peak access time section etc. have Many differences.They cannot carry out the migration of fragment copy according to unified temperature management.So match according to each operation list It sets a basic temperature monitoring configuration information and is associated with temperature monitoring configuration information with several.Service identification described herein is as industry Business operates in resource identification used in storage system, business can also by complete trails, relative path, file prefix or The file of the differentiation different service types such as suffix format.In addition the period not only can daily certain section of time interval it is (every Its 9. -11 point), festivals or holidays can also be configured to according to day, such as Saturday, Sunday, National Day (October 1 to October 7).Example As being directed to another mailbox service, increase following configuration in same storage system:
Configuration 4 monitors configuration information, hot statistics period daily 8. -18 point, heat as MAIL application foundation temperature Spending the update cycle is each hour.
Configuration 5, as the associated configuration of configuration 4, the hot statistics period is 8 points early-early 9 thirty, temperature update cycle It is every 30 minutes.
Concrete configuration mode passes through human-computer interaction order or interactive interface.When storage system Added Business, in addition to increasing Outside service path, it is also necessary to execute multi-service classification storage temperature monitoring configuration information.Multi-service in storage system is described below Temperature monitors configuration information temperature interactive interface, such as when increase business TV, increases temperature monitoring configuration information partial parameters and matches It sets and is illustrated in fig. 7 shown below, Fig. 7 is to monitor configuration information interface schematic diagram according to the multi-service temperature of the application example one.
Fig. 8 is to store multi-service list schematic diagram according to the classification of the application example two, is deposited as shown in figure 8, being shown below It include multiple business configuration lists in storage system.
It is Mail business that temperature, which monitors configuration information 1, and configuration 2,3 configures for TV, is association temperature monitoring configuration information.Match Setting 1 is independent temperature monitoring configuration information, and configuration 2,3 is that associated configuration shares temperature management.Temperature pipe between different business Reason and temperature scheduling are independent from each other, and such different business can use hierarchical stor simultaneously.Hierarchical stor In order to provide stable access performance, and better control system hardware, need to carry out multiple independent temperature management Coordinated scheduling.Multiple business sharing CPU, SSD flash memory, mechanical hard disk, Internet resources within the storage system, cannot be because of backstage temperature Scheduling reason causes service operation stability to decline.
When hot statistics module finds that SSD flash memory or mechanical hard disk IO ability reach threshold value in some period, or When person's storage system reports performance to reach performance threshold, it is notified that each traffic scheduling program carries out the migration velocity control of fragment copy System.More common factor is in some peak traffic phase, and if televiewer is in 19. -20 request programs, other business are herein The speed for needing to reduce temperature management and scheduling in time.
Example two, association temperature monitoring configuration information generate
By taking content distributing network as an example, it usually provides the business such as user live broadcast, program request, using hierarchical stor to mention IO and large capacity ability are read for high-performance.Business has storage system major requirement: a large amount of tape reading are wide, lower delay and compared with Large storage capacity.The common scene of business: typical time section spectators watch it is more steady with demand TV program, but daily it is several Viewing program is concentrated in the particular times such as a period and weekend, can trigger storage system peak traffic.It is with operation list HOT Example, such as user, often in 11. -12 points and the request program of evening 19-21 point, storage system pressure is larger at this time.If energy will very The fragment of heat is dispatched in SSD flash memory, then the handling capacity of storage system can be improved and compared with low delay.This period we Referred to as peak period.Other times section user's request program, the business of storage system are steady.The temperature management of peak period and usually industry Temperature of being engaged in has a great difference, cannot be determined with a set of standard.Using this programme, 3 or more can be configured for HOT catalogue Temperature monitor configuration information, it is as follows:
Configuration 1, hot statistics period (initial time, end time, similarly hereinafter) are configured to daily 11-12 point, and temperature updates Time is per half an hour, and calculation formula etc. does not do specified otherwise, by taking default configuration as an example.
Configuration 2 monitors configuration information as HOT catalogue basis temperature, mainly using it is usual when section business, when hot statistics Between section be early 8 points -23 points of evening, temperature renewal time is each hour.
Configuration 3, hot statistics period are evening 18-22 point, and temperature renewal time is per half an hour.
Illustrate: configuration 1 is as separate configurations.Configuration 2, configuration 3 are set as associated configuration, and initial weight is respectively 0.2 He 0.8.SSD memory space is occupied according to plan of operation, and same business association temperature monitoring configuration information makes it is not necessary that this value is accurately arranged With same configuration data.Other configurations repeat no more.
HOT business 3 configurations in systems logical construction as shown in figure 8, operation list HOT by above-mentioned with postponing, deposit Storage system distributes respective resources: generating corresponding list to be upgraded, list to be degraded, creation scheduler task etc..Wherein configuration 1 has Individual list to be upgraded, to be degraded and temperature management role.A list to be upgraded, to be degraded is shared in configuration 2, configuration 3, and And they have a public temperature management role that can execute with configuration 2,3 rule of configuration.
This example is also provided in a kind of system operation, after storage system perception service rush hour section, automatically generates It is associated with temperature and monitors configuration information.When the existing basic temperature of operation list monitors configuration information, system is according to hot statistics module It statistical service peak period, generates the newly-increased associated configuration of operation list, and existing temperature monitoring configuration information and new is set The weight of gain of heat degree monitoring configuration information.It can help customer analysis to go out the peak traffic period, and generate new association heat Degree monitoring configuration information, it is automatic to carry out temperature scheduling, simplify operation maintenance personnel configuration complexity.Key step has:
(1) after system runs a complete temperature dispatching cycle and measurement period, there is the peak traffic period, beyond flat When N times of preset value of amount of access.And temperature monitoring configuration information is traversed, does not find the association temperature prison of relevant time period Survey configuration information.
(2) it notifies coordinated scheduling module, generates new associated configuration.
(3) coordinated scheduling module obtains this operation list and already present temperature monitoring configuration information and time period Statistical information generates a newly-increased associated configuration.The newly-increased associated configuration hot statistics time is set as the peak business period, heat It spends the parameters such as renewal time and monitors configuration information referring to existing temperature, the weight of newly-increased associated configuration is set.
(4) newly-increased associated configuration is added in allocation list by coordinated scheduling module.
Example three, association temperature monitor the management of configuration information weight
The same operation list associated configuration weight is specified when storage system initializes, and can both repair in O&M Change, can also system in the process of running, according to hot statistics module data, be automatically adjusted.It, can after this exemplary application To reduce parameter adjustment and frequent upgraded version in O&M.
Operation list occupies the space SSD and mechanical hard disk space in hot statistics module measurement period, and business reads SSD flash memory With number, the byte number of mechanical hard disk etc., each associated configuration calculates the upgrading fragment number etc. obtained.
The weight value range of associated configuration is [0,1], and initialization weight default value is equal to 1/ associated configuration number.Fig. 9 Be according to the weight management process schematic diagram of the another example three of the application, as shown in Figure 9 the following steps are included:
Step 1, initial weight;
Step 2, hot statistics task are completed to start weight monitor task to after whole system items statistics;
Step 3 is searched in temperature configuration, most related temperature configuration in each group of associated configuration;
Step 4, the most related temperature configuration weight of setting is original value+increment weight Wd
Step 5, next hot statistics period repeat the above steps, when some temperature configuration weight reaches threshold value (such as 1), but with preset schedule efficiency, generate statistical report or alarm.
The detailed process of weight management process may include: storage system statistical module notice coordinated scheduling module, starting Weight monitor task, with the weight of fixed mode adjustment associated configuration.Such as it is adjusted with fixed step size 0.1, is searched and is closed Most related temperature monitors configuration information in this measurement period in connection temperature monitoring configuration information.Most related temperature monitors configuration information Refer in a preset measurement period, the upgrading fragment number and this temperature management role being calculated in some configuration are practical The immediate configuration of fragment number of upgrading.Then most related temperature is monitored into increment weight w in configuration informationd, increase by 0.1. In next measurement period, hot statistics data are analyzed, weight is adjusted.It (is most associated with when final several cycles of operation Weight reach 1) temperature scheduling and real data statistics discovery dispatching efficiency be less than preset schedule efficiency, then generate statistical report and Alarm automatically generates the associated configuration of rush hour section.
It additionally supports when some operation list temperature dispatching efficiency is more steady, when being needed beyond service feature, its phase It closes associated configuration weight to be set as not needing to adjust in some period, is applicable in fixed value.
Example four, demonstration fragment eliminate module.
This programme supports multiple business and the multiple temperatures of single business configuration to monitor configuration information.Their actual moving process In share SSD flash memory, and there is independent temperature management and temperature to dispatch, can make that the space SSD uses and release causes some problems. Therefore increase to distribute and eliminate module as auxiliary, smoothly adapt to multiple temperature management and temperature scheduling.
Figure 11 is to eliminate main flow schematic diagram according to the fragment of the application example four, as shown in figure 11, storage system point It is as follows that piece eliminates module basic procedure:
Step 1, all temperature configurations of traversal current business catalogue and hot statistics, with the sequence of SSD space hold.Take SSD The configuration of space hold maximum temperature is set as current temperature configuration.
Step 2 traverses current temperature configuration degradation list, fragment is sorted according to temperature, is added into wait eliminate column Table.
Step 3, the new temperature scheduling of triggering creation immediately one.After finishing scheduling, the release of SSD flash memory occupied space is checked Meet condition, that is, exits.
Step 4, it is when the release of the space SSD is unsatisfactory for condition, all configuration temperature config directories of storage system are empty by SSD Between sort.It repeats the above steps for each temperature catalogue.
Whether step 5, the operation list of traversal temperature configuration search fragment in SSD flash memory, and the retention time is more than to match The SSD retention time is set, expired fragment is added wait eliminate expired list;Not out of date fragment is added not out of date list, and calculates and account for With the space SSD.
Not out of date list is added in step 6, not out of date fragment, and calculates and occupy the space SSD.Judging SSD flash memory occupancy is It is no to meet condition, otherwise, successively minimum temperature fragment is added and eliminates list, the new temperature scheduling of triggering creation one.
Fragment, which eliminates process, can further include following steps:
(1) all temperatures monitoring configuration information and hot statistics for traversing current business catalogue occupy SSD sky according to practical Between sort.
(2) it takes and occupies the maximum temperature monitoring configuration information of SSD space hold.Traverse queue to be degraded, by fragment according to Temperature sequence, and by more than the fragment of SSD retention time, queue to be eliminated is added, and (eliminating queue referring to Figure 10, Figure 10 is basis The classification of the application example four stores more catalogue configuration temperature management and superseded structure chart).
(3) triggering immediately creates new temperature scheduling, and by temperature scheduler module, it is moved to machinery from SSD flash memory Hard disk.
(4) lower temperature monitoring configuration information of current business catalogue is taken, second step is repeated.
(5) all classifications storage temperature monitoring configuration information catalogue is sorted according to practical SSD occupied space size;Traversal Operation list after sequence takes one of operation list to be set as current business catalogue.Repeat the first step.
(6) triggering creates new temperature scheduling.The release of SSD occupied space meets condition, that is, exits.
(7) the current most operation lists of SSD space hold are taken, the fragment of file in catalogue is searched, check that fragment copy is It is no on SSD flash memory, and compare copy update time and whether the SSD retention time expires.By the fragment that copy is expired, it is added Wait eliminate expired candidate queue;Not out of date fragment is added wait eliminate not out of date candidate queue, and calculates and occupies the space SSD, is pressed It is arranged from small to large according to temperature.
(8) fragment is taken out from wait eliminate expired candidate queue.The degradation queue of temperature scheduler module is added.Turn step 6.
(9) fragment is taken out from wait eliminate not out of date candidate queue, needs to eliminate when the fragment space of this queue is greater than to meet The fragment of temperature minimum inside queue is eliminated come out every time by space size.Turn step 6.
Through the above description of the embodiments, those skilled in the art can be understood that according to above-mentioned implementation The method of example can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but it is very much In the case of the former be more preferably embodiment.Based on this understanding, the technical solution of the application is substantially in other words to existing The part that technology contributes can be embodied in the form of software products, which is stored in a storage In medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that a terminal device (can be mobile phone, calculate Machine, server or network equipment etc.) execute method described in each embodiment of the application.
Embodiment two
A kind of device of data storage is additionally provided in the present embodiment, and the device is for realizing above-described embodiment and preferably Embodiment, the descriptions that have already been made will not be repeated.As used below, predetermined function may be implemented in term " module " The combination of software and/or hardware.Although device described in following embodiment is preferably realized with software, hardware, or The realization of the combination of person's software and hardware is also that may and be contemplated.
According to another embodiment of the application, a kind of device of data storage is additionally provided, comprising:
First obtains module, and multiple temperatures for being retrieved as the setting of the first business monitor configuration information;
Second obtains module, for monitoring the temperature of first business respectively according to each temperature monitoring configuration information Value, wherein the hot value is used to indicate the accessed frequency of first business;
Selecting module, for according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage institute The position of the first business corresponding data is stated, and stores the data.
By the application, it is that the multiple temperatures of the first business configuration monitor configuration information, matches confidence according to the monitoring of each temperature Configuration in breath is monitored the temperature of the first business, obtains the corresponding hot value of each temperature monitoring configuration information, then The first business corresponding data position of storage, such as solid state hard disk or mechanical hard disk are selected according to multiple hot value, can be Comprehensively consider multiple hot values later to migrate the first business corresponding data, be also possible to independently according to a hot value First business corresponding data is migrated, using the above scheme, a business configuration there are multiple temperature monitoring configuration informations, can With the more accurate hot spot data for migrating the business in time to solid state hard disk, classification storage efficiency is substantially improved, solves phase The problem for causing hot spot data classification storage effect undesirable since hot value statistical is single in the technology of pass.
It should be noted that above-mentioned modules can be realized by software or hardware, for the latter, Ke Yitong Following manner realization is crossed, but not limited to this: above-mentioned module is respectively positioned in same processor;Alternatively, above-mentioned modules are with any Combined form is located in different processors.
Embodiment three
Embodiments herein additionally provides a kind of storage medium.Optionally, in the present embodiment, above-mentioned storage medium can To be arranged to store the program code for executing following steps:
S1 is retrieved as multiple temperatures monitoring configuration information of the first business setting;
S2 monitors the hot value of first business according to each temperature monitoring configuration information, wherein the temperature respectively Value is used to indicate the accessed frequency of first business;
S3, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection stores first business The position of corresponding data, and store the data.
Optionally, in the present embodiment, above-mentioned storage medium can include but is not limited to: USB flash disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or The various media that can store program code such as CD.
Embodiments herein additionally provides a kind of electronic device, including memory and processor, stores in the memory There is computer program, which is arranged to run computer program to execute the step in any of the above-described embodiment of the method Suddenly.
Optionally, above-mentioned electronic device can also include transmitting device and input-output equipment, wherein the transmitting device It is connected with above-mentioned processor, which connects with above-mentioned processor.
Optionally, in the present embodiment, above-mentioned processor can be set to execute following steps by computer program:
S1 is retrieved as multiple temperatures monitoring configuration information of the first business setting;
S2 monitors the hot value of first business according to each temperature monitoring configuration information, wherein the temperature respectively Value is used to indicate the accessed frequency of first business;
S3, according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection stores first business The position of corresponding data, and store the data.
Optionally, the specific example in the present embodiment can be with reference to described in above-described embodiment and optional embodiment Example, details are not described herein for the present embodiment.
Optionally, the specific example in the present embodiment can be with reference to described in above-described embodiment and optional embodiment Example, details are not described herein for the present embodiment.
Obviously, those skilled in the art should be understood that each module of above-mentioned the application or each step can be with general Computing device realize that they can be concentrated on a single computing device, or be distributed in multiple computing devices and formed Network on, optionally, they can be realized with the program code that computing device can perform, it is thus possible to which they are stored It is performed by computing device in the storage device, and in some cases, it can be to be different from shown in sequence execution herein Out or description the step of, perhaps they are fabricated to each integrated circuit modules or by them multiple modules or Step is fabricated to single integrated circuit module to realize.It is combined in this way, the application is not limited to any specific hardware and software.
The foregoing is merely preferred embodiment of the present application, are not intended to limit this application, for the skill of this field For art personnel, various changes and changes are possible in this application.Within the spirit and principles of this application, made any to repair Change, equivalent replacement, improvement etc., should be included within the scope of protection of this application.

Claims (15)

1. a kind of method of data storage characterized by comprising
It is retrieved as multiple temperatures monitoring configuration information of the first business setting;
Monitor the hot value of first business respectively according to each temperature monitoring configuration information, wherein the hot value is used for Indicate the accessed frequency of first business;
According to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection stores the first business corresponding data Position, and store the data.
2. the method according to claim 1, wherein being retrieved as multiple temperatures monitoring configuration of the first business setting Information, comprising:
Obtain at least one the following information for including in the temperature monitoring configuration information:
Temperature update cycle, hot statistics initial time, hot statistics end time.
3. the method according to claim 1, wherein according to each temperature monitoring configuration information monitor respectively described in The hot value of first business, comprising:
Within each temperature monitoring configuration information corresponding hot statistics time started to hot statistics end time, statistics is each First time accessed number of first business described in the temperature update cycle;
The hot value of corresponding first business of each temperature monitoring configuration information is obtained according to described first time number.
4. the method according to claim 1, wherein according to each temperature monitoring configuration information monitor respectively described in The hot value of first business, comprising:
The first temperature monitoring configuration information in the multiple temperature monitoring configuration information is directed to the first of first business When operation list, one or more data point in first operation list are counted according to first temperature monitoring configuration information The hot value of piece.
5. the method according to claim 1, wherein corresponding more according to the multiple temperature monitoring configuration information A hot value, selection store the position of the first business corresponding data, and store the data, comprising:
When the multiple temperature monitoring configuration information is that associated temperature monitors configuration information, each temperature monitoring configuration is obtained The product of information corresponding hot value and default weight;
Obtain the product of the multiple temperature monitoring configuration information and value, according to described and value selection storage first business The position of corresponding data, and store the data.
6. according to the method described in claim 5, it is characterized in that, according to described corresponding with value selection storage first business The position of data, and store the data, comprising:
When described and value is greater than heat degree threshold, the corresponding data of first business are migrated by mechanical hard disk hard to solid-state Disk;
When described and value is less than heat degree threshold, the corresponding data of first business are migrated by solid state hard disk to mechanical hard Disk.
7. the method according to claim 1, wherein selection stores the position of the first business corresponding data, And store the data, comprising:
Selection stores the solid state hard disk or mechanical hard disk of the copy of the first data fragmentation of first business;
The copy is stored to selected solid state hard disk or mechanical hard disk.
8. the method according to the description of claim 7 is characterized in that the copy is migrated to solid state hard disk, the side Method further include:
Within a temperature update cycle, statistics reads the solid state hard disk and reads mechanical hard disk when executing first business Number ratio;
When the number ratio is lower than preset ratio, the default weight of the multiple temperature monitoring configuration information is adjusted to increase Next temperature update cycle corresponding number ratio.
9. according to the method described in claim 8, it is characterized in that, adjusting the default power of the multiple temperature monitoring configuration information After weight is to increase corresponding number ratio of next temperature update cycle, which comprises
After adjustment by the default weight of multiple temperature update cycles, detect that the number ratio reaches maximum value;
When the maximum value is still less than the preset ratio, generates statistical report and alert.
10. the method according to claim 1, wherein corresponding according to the multiple temperature monitoring configuration information Multiple hot values, selection store the position of the first business corresponding data, comprising:
When the multiple temperature monitoring configuration information is temperature monitoring configuration information independent of each other, respectively according to each heat The corresponding hot value selection of degree monitoring configuration information stores the position of the first business corresponding data.
11. the method according to claim 1, wherein corresponding according to the multiple temperature monitoring configuration information Multiple hot values, selection store the position of the first business corresponding data, and after storing the data, the method is also wrapped It includes:
Second time accessed number of first business described in real-time statistics, it is automatic raw when second number meets preset condition Configuration information is monitored at the second temperature of first business.
12. the method according to claim 1, wherein corresponding according to the multiple temperature monitoring configuration information Multiple hot values, selection store the position of the first business corresponding data, and after storing the data, the method is also wrapped It includes:
When the storage state for the first hard disk for being stored with data meets preset state, at least one in the following manner release institute State the memory space of the first hard disk:
The hot value stored on first hard disk is gone out lower than heat degree threshold or the smallest second business migration of hot value;
The smallest data fragmentation of hot value of the second business stored on first hard disk is migrated out.
13. a kind of device of data storage characterized by comprising
First obtains module, and multiple temperatures for being retrieved as the setting of the first business monitor configuration information;
Second obtains module, for monitoring the hot value of first business respectively according to each temperature monitoring configuration information, In, the hot value is used to indicate the accessed frequency of first business;
Selecting module, for according to the corresponding multiple hot values of the multiple temperature monitoring configuration information, selection storage described the The position of one business corresponding data, and store the data.
14. a kind of storage medium, which is characterized in that be stored with computer program in the storage medium, wherein the computer Program is arranged to execute method described in any one of claim 1 to 12 when operation.
15. a kind of electronic device, including memory and processor, which is characterized in that be stored with computer journey in the memory Sequence, the processor are arranged to run the computer program to execute described in any one of claim 1 to 12 Method.
CN201811616160.3A 2018-12-27 2018-12-27 The method and device of data storage Pending CN110209345A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201811616160.3A CN110209345A (en) 2018-12-27 2018-12-27 The method and device of data storage
PCT/CN2019/115774 WO2020134609A1 (en) 2018-12-27 2019-11-05 Data storage method and apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811616160.3A CN110209345A (en) 2018-12-27 2018-12-27 The method and device of data storage

Publications (1)

Publication Number Publication Date
CN110209345A true CN110209345A (en) 2019-09-06

Family

ID=67780027

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811616160.3A Pending CN110209345A (en) 2018-12-27 2018-12-27 The method and device of data storage

Country Status (2)

Country Link
CN (1) CN110209345A (en)
WO (1) WO2020134609A1 (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111309251A (en) * 2020-01-21 2020-06-19 青梧桐有限责任公司 Data storage method, system, electronic device and readable storage medium
WO2020134609A1 (en) * 2018-12-27 2020-07-02 中兴通讯股份有限公司 Data storage method and apparatus
CN111400318A (en) * 2020-03-09 2020-07-10 北京易华录信息技术股份有限公司 Method and device for generating scheduling strategy of data storage
CN111427969A (en) * 2020-03-18 2020-07-17 清华大学 Data replacement method of hierarchical storage system
CN112559504A (en) * 2020-12-09 2021-03-26 北京思特奇信息技术股份有限公司 Data cleaning method and device based on data heat and storage medium
CN112734103A (en) * 2021-01-05 2021-04-30 烽火通信科技股份有限公司 Video cold picture prediction method and device based on space-time sequence
CN113032369A (en) * 2021-03-26 2021-06-25 山东英信计算机技术有限公司 Data migration method, device and medium
CN113297005A (en) * 2020-07-27 2021-08-24 阿里巴巴集团控股有限公司 Data processing method, device and equipment
CN113885797A (en) * 2021-09-24 2022-01-04 济南浪潮数据技术有限公司 Data storage method, device, equipment and storage medium
CN114020828A (en) * 2021-09-27 2022-02-08 南京云创大数据科技股份有限公司 Distributed hierarchical storage system
CN114666121A (en) * 2022-03-21 2022-06-24 山东鼎夏智能科技有限公司 Data monitoring method and device
CN116189896A (en) * 2023-04-24 2023-05-30 北京快舒尔医疗技术有限公司 Cloud-based diabetes health data early warning method and system
CN114020828B (en) * 2021-09-27 2024-05-31 南京云创大数据科技股份有限公司 Distributed hierarchical storage system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103186350A (en) * 2011-12-31 2013-07-03 北京快网科技有限公司 Hybrid storage system and hot spot data block migration method
CN104133643A (en) * 2014-08-04 2014-11-05 浪潮电子信息产业股份有限公司 Method for improving data transfer efficiency under automatic data hierarchical storage frame
US20150286418A1 (en) * 2013-01-22 2015-10-08 International Business Machines Corporation Tiered caching and migration in differing granularities
CN106709068A (en) * 2017-01-22 2017-05-24 郑州云海信息技术有限公司 Hotspot data identification method and device

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103150347B (en) * 2013-02-07 2015-10-21 浙江大学 Based on the dynamic replication management method of file temperature
CN108121802A (en) * 2017-12-22 2018-06-05 东软集团股份有限公司 The thermodynamic analysis method, apparatus and its equipment of web page access
CN110209345A (en) * 2018-12-27 2019-09-06 中兴通讯股份有限公司 The method and device of data storage

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103186350A (en) * 2011-12-31 2013-07-03 北京快网科技有限公司 Hybrid storage system and hot spot data block migration method
US20150286418A1 (en) * 2013-01-22 2015-10-08 International Business Machines Corporation Tiered caching and migration in differing granularities
CN104133643A (en) * 2014-08-04 2014-11-05 浪潮电子信息产业股份有限公司 Method for improving data transfer efficiency under automatic data hierarchical storage frame
CN106709068A (en) * 2017-01-22 2017-05-24 郑州云海信息技术有限公司 Hotspot data identification method and device

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020134609A1 (en) * 2018-12-27 2020-07-02 中兴通讯股份有限公司 Data storage method and apparatus
CN111309251A (en) * 2020-01-21 2020-06-19 青梧桐有限责任公司 Data storage method, system, electronic device and readable storage medium
CN111400318B (en) * 2020-03-09 2023-09-15 北京易华录信息技术股份有限公司 Method and device for generating scheduling policy of data storage
CN111400318A (en) * 2020-03-09 2020-07-10 北京易华录信息技术股份有限公司 Method and device for generating scheduling strategy of data storage
CN111427969A (en) * 2020-03-18 2020-07-17 清华大学 Data replacement method of hierarchical storage system
CN111427969B (en) * 2020-03-18 2022-05-27 清华大学 Data replacement method of hierarchical storage system
CN113297005A (en) * 2020-07-27 2021-08-24 阿里巴巴集团控股有限公司 Data processing method, device and equipment
CN113297005B (en) * 2020-07-27 2024-01-05 阿里巴巴集团控股有限公司 Data processing method, device and equipment
CN112559504A (en) * 2020-12-09 2021-03-26 北京思特奇信息技术股份有限公司 Data cleaning method and device based on data heat and storage medium
CN112734103A (en) * 2021-01-05 2021-04-30 烽火通信科技股份有限公司 Video cold picture prediction method and device based on space-time sequence
CN113032369A (en) * 2021-03-26 2021-06-25 山东英信计算机技术有限公司 Data migration method, device and medium
CN113885797A (en) * 2021-09-24 2022-01-04 济南浪潮数据技术有限公司 Data storage method, device, equipment and storage medium
CN113885797B (en) * 2021-09-24 2023-12-22 济南浪潮数据技术有限公司 Data storage method, device, equipment and storage medium
CN114020828A (en) * 2021-09-27 2022-02-08 南京云创大数据科技股份有限公司 Distributed hierarchical storage system
CN114020828B (en) * 2021-09-27 2024-05-31 南京云创大数据科技股份有限公司 Distributed hierarchical storage system
CN114666121A (en) * 2022-03-21 2022-06-24 山东鼎夏智能科技有限公司 Data monitoring method and device
CN116189896A (en) * 2023-04-24 2023-05-30 北京快舒尔医疗技术有限公司 Cloud-based diabetes health data early warning method and system
CN116189896B (en) * 2023-04-24 2023-08-08 北京快舒尔医疗技术有限公司 Cloud-based diabetes health data early warning method and system

Also Published As

Publication number Publication date
WO2020134609A1 (en) 2020-07-02

Similar Documents

Publication Publication Date Title
CN110209345A (en) The method and device of data storage
CN104935482B (en) Distributed monitoring system and method
EP3279794A1 (en) Time-based node election method and apparatus
US20100185768A1 (en) Resource allocation and modification using statistical analysis
CN103827858A (en) Caching in mobile networks
CN106888381B (en) A kind of data resource storage method and device
US10326854B2 (en) Method and apparatus for data caching in a communications network
CN109861878A (en) The monitoring method and relevant device of the topic data of kafka cluster
CN109190070A (en) A kind of data processing method, device, system and application server
CN109379425A (en) Distributed cluster deployment management method and device
CN110213203A (en) Network dispatching method, device and computer storage medium
CN112165508B (en) Resource allocation method for multi-tenant cloud storage request service
CN102098170B (en) Data acquisition optimization method and system
US10891849B1 (en) System for suppressing false service outage alerts
CN100505871C (en) Video-frequency data processing method, video-frequency collecting equipment and video-frequency management equipment
US11099921B2 (en) Predictive system resource allocation
CN112506926A (en) Monitoring data storage and query method and corresponding device, equipment and medium
US20230251789A1 (en) Record information management based on self-describing attributes
CN111324459A (en) Calendar-based resource scheduling method and device, electronic equipment and storage medium
CN113965538B (en) Equipment state message processing method, device and storage medium
CN113315836B (en) File access request scheduling method and device, electronic equipment and storage medium
CN114090201A (en) Resource scheduling method, device, equipment and storage medium
CN106357735A (en) Method and device for operating infrastructure layer of cloud computing architecture
US11026108B2 (en) Fault monitoring in a utility supply network
EP4365751A1 (en) Dynamic data retention policies for iot platforms

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20190906

RJ01 Rejection of invention patent application after publication