WO2014183514A1 - 一种分级存储方法、装置和计算机存储介质 - Google Patents
一种分级存储方法、装置和计算机存储介质 Download PDFInfo
- Publication number
- WO2014183514A1 WO2014183514A1 PCT/CN2014/075259 CN2014075259W WO2014183514A1 WO 2014183514 A1 WO2014183514 A1 WO 2014183514A1 CN 2014075259 W CN2014075259 W CN 2014075259W WO 2014183514 A1 WO2014183514 A1 WO 2014183514A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- file
- upgrade
- migration
- migrated
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000013508 migration Methods 0.000 claims abstract description 100
- 230000005012 migration Effects 0.000 claims abstract description 100
- 230000007246 mechanism Effects 0.000 claims abstract description 14
- 238000005065 mining Methods 0.000 claims description 20
- 238000013523 data management Methods 0.000 claims description 14
- 230000008901 benefit Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000005516 engineering process Methods 0.000 claims description 6
- 230000000717 retained effect Effects 0.000 claims description 5
- 238000012423 maintenance Methods 0.000 claims description 4
- 230000000593 degrading effect Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000007726 management method Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000000875 corresponding effect Effects 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 4
- 238000013138 pruning Methods 0.000 description 4
- 230000001960 triggered effect Effects 0.000 description 4
- 230000015556 catabolic process Effects 0.000 description 3
- 238000006731 degradation reaction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000011263 association rule mining technique Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013403 standard screening design Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/185—Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
Definitions
- the present invention relates to the field of computer storage systems, and more particularly to a hierarchical storage method, apparatus, and computer storage medium. Background technique
- Hierarchical storage management refers to the physical storage storage device is divided into different categories according to price, performance or other attributes, and the data is dynamically migrated between different types of storage devices according to its access activity or other characteristics. system. Compared to traditional storage systems, tiered storage systems combine devices with different performance, capacity, and price to provide a high-performance, high-capacity, low-cost storage environment.
- online storage uses high-performance disks such as SSDs and FC disks.
- Online storage stores a small amount of data with high value and access frequency. In comparison, these storage devices have good performance, fast access speed, and access to data in online storage to meet the high performance requirements of the application.
- Nearline storage stores less active data, and SATA disks are suitable for nearline storage because of their larger capacity, lower price, and lower performance.
- the upper application has less access to nearline storage, so it has little effect on the overall performance of the system.
- Offline storage devices typically use tape or tape libraries whose primary purpose is to store backed up or archived data for data in online and nearline storage. The reading of offline stored data often takes a long time, and almost no access to offline stored data.
- the embodiment of the present invention provides a hierarchical storage method, a device, and a computer storage medium, which are used to solve the technical problems such as the above technical problems.
- a first aspect of the embodiments of the present invention provides a hierarchical storage method, where the method includes: triggering data migration based on a data dynamic classification policy;
- Determining whether to perform data migration on the data to be migrated based on the migration rate control mechanism includes: performing a data grading operation based on the data dynamic grading policy; wherein the data grading operation includes a file upgrading operation and a file degrading operation;
- Data migration is triggered based on data grading operations.
- the file upgrade operation is performed based on the data dynamic classification policy, including:
- the file degrading operation based on the data dynamic grading policy comprises:
- the degraded thread fetches the coldest file in the LRU queue at a specified time as a degraded object
- a file downgrade operation is performed on the downgraded object.
- the method further includes: a step of closing files according to file association rule mining technology;
- the file association rule mining technology, associating files with each other includes:
- determining the migrated data is:
- the migrated data is determined based on strong association rules between files.
- the determining, according to the migration rate control mechanism, whether to perform data migration on the data to be migrated includes:
- the data migration is limited.
- the method when performing the data migration, the method further includes: The migration speed of the data migration is determined according to the length of the I/O access queue.
- a second aspect of the embodiments of the present invention provides a hierarchical storage device, where the device includes: a triggering module configured to trigger data migration based on a data dynamic classification policy;
- the data to be migrated determining module is configured to determine the data to be migrated according to the mutual association of the files; and the control module is configured to perform data migration on the data to be migrated based on the migration rate control mechanism.
- the trigger module includes:
- the grading unit is configured to perform a data grading operation based on the data dynamic grading policy, where the data grading operation includes a file upgrading operation and a file degrading operation;
- the processing unit configured to trigger data migration based on data classification operations.
- the upgrading unit comprises:
- the first revenue efficiency calculation sub-unit is configured to measure the revenue efficiency of the file upgrade according to the amount of data accessed per unit time after the file is upgraded;
- a second revenue efficiency calculation sub-unit configured to determine a unit cost benefit efficiency of the data upgrade according to the revenue efficiency of the file upgrade and the cost of the file upgrade;
- the upgrade operation sub-unit is configured to determine whether the upgrade utility value of the file is higher than the upgrade threshold according to the unit cost-benefit efficiency of the data upgrade; when the file upgrade Xiao Yongzhi is higher than the upgrade threshold, perform an upgrade operation on the file.
- the grading unit further includes:
- a file maintenance subunit configured to maintain all files on the fast storage device in a memory block LRU queue according to data access conditions
- the downgrade operation subunit is configured to perform a file downgrade operation on the demoted object.
- the device further includes an association module;
- the association module includes:
- the frequent sequence mining unit is configured to use the mining algorithm BIDE to mine frequent sequences;
- the strong association rule determining unit is configured to convert frequent sequences into association rules and determine strong association rules;
- the redundancy rule judging unit is configured to determine whether the strong association rule is a redundancy rule; when the strong association rule is a redundancy regulation, the strong association rule is eliminated; when the strong association rule is a non-redundant regulation When the strong association rule is retained;
- the data to be migrated determining module is configured to determine the migrated data according to a strong association rule between the files.
- control module includes:
- the obtaining unit is configured to obtain a load of the front-end application in the data management client, and determine whether the file is migrated;
- the migration processing unit is configured to perform data migration when the load is lower than a first specified threshold; and restrict data migration when the load is higher than a second specified threshold.
- the control module is further configured to determine a migration speed of the data migration according to an I/O access queue length.
- a third aspect of the invention provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, the computer executable instructions for performing the method of any of the above.
- data migration is triggered based on the data dynamic classification policy; data to be migrated is determined based on file correlation; and the migration rate control mechanism is used to determine whether to migrate data flexibly between migrations, thereby enabling more flexible control
- the trigger of the relocation can flexibly configure the hierarchical management strategy to flexibly migrate data between online storage and near-line storage.
- FIG. 1 is a flowchart of a hierarchical storage method according to an embodiment of the present invention.
- FIG. 2 is a schematic diagram of a system hardware architecture according to an embodiment of the present invention.
- FIG. 3 is a schematic diagram of operation of rate control according to an embodiment of the present invention.
- FIG. 4 is a structural block diagram of a hierarchical storage device according to an embodiment of the present invention. detailed description
- the embodiment of the present invention provides a hierarchical storage method and device, and further, the present invention is further described below with reference to the accompanying drawings and embodiments. Detailed description. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
- FIG. 1 is a flowchart of a hierarchical storage method according to an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps (step S102-step S106):
- Step S102 Trigger data migration based on a data dynamic classification policy.
- the data grading operation is performed based on the data dynamic grading policy; wherein the data grading operation includes a file upgrading operation and a file degrading operation; and the data migrating is triggered based on the data grading operation.
- file upgrade operations include:
- the unit cost-benefit efficiency of the data upgrade determine whether the upgrade utility value of the file is higher than Upgrade threshold; if yes, perform an upgrade operation on the file.
- file degradation operations include:
- the degraded thread fetches the coldest file in the LRU queue at a specified time as a degraded object; performs a file downgrade operation on the degraded object.
- the degraded thread may be one or more.
- the LRU queue is a storage queue managed by the Least Recently Used algorithm.
- Step S104 Determine data to be migrated according to the mutual association of the files.
- the files are related to each other.
- the files A and B have been correlated according to the file association rules. Therefore, when migrating file A, file B can be migrated at the same time; if file A has been migrated before, according to the correlation between file A and file B, file B is determined to be migrated at the current moment to realize automatic prefetching of file.
- the data in the file B to be migrated is the data to be migrated.
- the association between the files may be established in real time according to the historical information of the access or processing data, or may be operated periodically.
- the method further includes: a step of associating files with each other based on a file association rule mining technique; and the details are as follows:
- Mining mining algorithm BIDE mining frequent sequences; converting frequent sequences into association rules, and determining strong association rules;
- the file to be migrated is determined according to the strong association rule; the data included in the file to be migrated is the data to be migrated.
- Step S106 Determine, according to the migration rate control mechanism, whether to migrate the data to be migrated.
- the step S106 may specifically be:
- the first specified threshold is typically less than the second specified threshold in a particular implementation. Both the first specified threshold and the second specified threshold represent corresponding load values.
- the method further includes:
- the method further includes:
- the migration speed of the data migration is determined according to the length of the I/O access queue.
- the I/O access queue is long, indicating that there are many tasks to be processed in the current system, and the system is busy. If the data is migrated again, the task of the system will be further cumbersome; the I/O response is delayed, and the user experience is poor; The current I/O access queue is short, indicating that the system has fewer tasks to be processed, and the system has more idle resources, which is suitable for rapidly migrating data.
- the speed of data migration is determined according to the length of the I/O access queue, and the response of the system is optimized.
- the maximum migration speed can be determined according to the urgency of the current data migration and the length of the I/O access queue.
- the urgency of the data migration may be represented by a priority; the confirmation of the preferred level may be determined according to parameters such as a user indication or an access frequency of data to be migrated.
- data migration is triggered; file-based correlation is used to determine migration data; and based on the migration rate control mechanism, whether to perform data migration data is determined, thereby enabling more flexible control of the trigger of the migration Flexible configuration of hierarchical management policies to flexibly migrate data between online storage and nearline storage.
- the hierarchical storage method provided by this embodiment may also be referred to as a data automatic migration method FlexMig (note: the name of the automatic data migration method), and the FlexMig is mainly composed of three parts:
- Rate control during migration looking for a reasonable trade-off between front-end I/O performance impact and data migration completion deadlines.
- the data grading evaluation in FlexMig's data dynamic grading strategy includes file upgrade evaluation and document degradation evaluation.
- FlexMig determines whether to perform an upgrade operation on a file based on the unit cost benefit efficiency of the data upgrade.
- the revenue efficiency of a file upgrade is measured by the amount of data accessed per unit time after the file is upgraded.
- AS and AF respectively indicate the file access size and file access frequency after file upgrade
- the performance benefit efficiency after file upgrade is AS X AF.
- the cost of file upgrades can be measured using the file size FS.
- the average value of the access size of the FlexMig statistics file is used as the estimated value of the future access size AS; the value of AF is the statistical file. AS and AF may introduce smoothing factors to fit a more reasonable value.
- the basic idea of the FlexMig degrading algorithm is to maintain all files on the fast storage device in the LRU queue according to the access situation.
- a degraded thread fetches the coldest file from the LRU queue at a certain time as a degraded object, and the length of the degradation interval is The space idle rate of a fast storage device is related.
- FlexMig's data upgrade algorithm takes into account both file access history and file size. It not only makes the file migration cost relatively small, but also ensures that the file I/O performance gains after migration are high. The determination of the degraded interval ensures that the fast storage device always has enough free space. Calculate the utility value of the upgrade migration when the file stored on the slow storage device is accessed; when storing When the file on the fast device is accessed, the corresponding LRU queue status is updated. The data dynamic grading algorithm does not need to scan all files periodically for file value evaluation, so the added computational overhead is not large.
- FlexMig uses data mining techniques to effectively identify file associations in the system. It maps a file into an item and maps an access sequence to a sequence in the sequence database. A frequent subsequence indicates that related files are often accessed together.
- a file can be accessed in a variety of ways, except when it is turned off, it can also be executed as a process.
- FlexMig builds a long access trace by logging these system calls. FlexMig uses a simple cut to cut long traces into many short sequences. FlexMig turned the problem into mining frequent closed sequence problems, using the mining algorithm BIDE, and made some improvements.
- the BIDE algorithm essentially uses a depth-first approach to construct a frequent subtree while checking for closure and pruning.
- the two key tasks implemented by the BIDE algorithm are: 1) closedness checking; 2) search space pruning.
- the BIDE algorithm uses a two-way extension mode: forward extension for the closure check of the growth prefix pattern and prefix mode; backwards for the closure check for the prefix pattern and search space pruning.
- For the current sequence BIDE scans each mapping sequence forward to find local frequency items. For each local frequent item, check if it can be pruned. If you cannot pruning, expand the current sequence forward.
- FlexMig To convert frequent sequences into association rules, FlexMig generates some rules from each sequence. FlexMig specifies that the number of items to the right of a rule is 1, because this is sufficient for file prefetching. To limit the number of rules, FlexMig constrains the number of items to the left of a rule to no more than two.
- FlexMig also uses credibility parameters to measure the dependability of rules.
- the credibility of the rule indicates the accuracy of the prediction.
- FlexMig uses a minimum confidence threshold to filter low-quality association rules, and the remaining rules are called strong association rules.
- FlexMig takes two approaches to avoid the performance impact of file prefetching.
- For the associated small file group it is stored centrally on the low-level device, and the pre-fetching method is adopted in the way of upgrading the tape.
- For slightly larger association files use the rate control mechanism described below to ensure data prefetching is performed when the front load is light.
- FIG. 2 is a schematic diagram of a system hardware architecture according to an embodiment of the present invention.
- a data management client, a data management server, a data management backend server, and a system administrator are connected through an internal network (intranet), and data management is performed.
- Back-end servers are used for online storage, nearline storage, and offline storage, respectively.
- the system performs different management operations, such as data migration, data backup, and data compression, according to the established strategy.
- Data migration and data backup require access to the main Store files in the pool and store the data over the network into a low-level storage pool. Since the data management operations are performed online, the data migration and front-end applications will share the I/O and network resources in the application competition data management client.
- the system provides a rate control mechanism to maximize data migration speed without affecting the performance of the front-end application.
- the system uses logical on/off states to adjust the data migration speed. Whether the file migration is performed or not is determined by the load of the front-end application in the data management client. If the front-end load is relatively low, data migration occurs, and if the load is too high, file migration is limited.
- the system monitors the length of the I/O queues in the data management client. As shown in FIG. 3, the operation diagram of the rate control according to the embodiment of the present invention determines that the front end load is high when the I/O queue length is higher than the set threshold T. In general, the I/O response time increases as the length of the I/O queue grows. The system calculates the length of time w that the data migration should wait by formula,
- W Ex(L-T) ; where E is a constant, L is the I/O queue length, and T is the set threshold. The values of E and T are based on empirical values.
- FIG. 4 is a structural block diagram of a hierarchical storage device according to an embodiment of the present invention. As shown in FIG. 4, the device includes: a trigger module 10, a data determination module 20 to be migrated, and a control module 30. The structure is described in detail below.
- the triggering module 10 is configured to trigger data migration based on a data dynamic ranking policy.
- the trigger module includes:
- the grading unit is configured to perform a data grading operation based on the data dynamic grading policy, where the data grading operation includes a file upgrading operation and a file degrading operation;
- the processing unit configured to trigger data migration based on data classification operations.
- the grading unit may include:
- the first revenue efficiency calculation sub-unit is configured to measure the revenue efficiency of the file upgrade according to the amount of data accessed per unit time after the file is upgraded;
- a second revenue efficiency calculation subunit configured to determine a unit cost benefit efficiency of the data upgrade according to the revenue efficiency of the file upgrade and the cost of the file upgrade; the upgrade operation subunit, the unit cost benefit efficiency for upgrading according to the data , determine whether the upgrade utility value of the file is higher than the upgrade threshold; if yes, perform an upgrade operation on the file.
- the grading unit further includes:
- a file maintenance subunit configured to maintain all files on the fast storage device in a memory block LRU queue according to data access conditions
- the degraded object determining subunit is configured to cause a degraded thread to fetch the coldest file in the LRU queue at a specified time as a degraded object; and the degrading operation subunit is configured to perform a file degrading operation on the degraded object.
- the data to be migrated determining module 20 is configured to determine the data to be migrated according to the mutual association of the files;
- the apparatus further includes an association module configured to associate the files with each other based on a file association rule mining technique; wherein the associated files are used for automatic prefetching.
- the association module may include:
- Frequent sequence mining unit configuration mining algorithm BIDE, mining frequent sequences
- strong association rule determining unit configured to convert frequent sequences into association rules, and determine strong association rules
- the redundancy rule determining unit is configured to determine whether the strong association rule is a redundancy rule; if yes, the strong association rule is eliminated; if not, the strong association rule is retained.
- the data to be migrated determining module 20 determines the data to be migrated according to the strong association rule.
- the control module 30 is configured to determine, according to the migration rate control mechanism, whether the to-be-migrated Data is migrated.
- control module 30 may include:
- a migration unit configured to obtain a load of a front-end application in the data management client
- the migration processing unit is configured to perform data migration when the load is lower than a first specified threshold; and limit data migration when the load is higher than a second specified threshold.
- the first specified threshold is typically less than the second specified threshold in a particular implementation.
- the first specified threshold and the second specified threshold each represent a corresponding load value.
- the control module 30 is further configured to determine a migration speed of the data migration according to an I/O access queue length.
- the control module is specifically configured to calculate the length of time that the data migration should wait by formula calculation.
- the calculation formula of W is as follows:
- W Ex(L-T) ; where E is a constant, L is the I/O queue length, and T is the set threshold.
- E is a constant
- L is the I/O queue length
- T is the set threshold.
- the values of E and T are available based on empirical values. In specific operations, you can maximize the migration speed as needed.
- the specific structure of the trigger module 10, the data to be migrated determining module 20, and the control module 30 may correspond to a storage medium and a memory; the storage medium stores executable instructions; and the processor passes the bus or the communication.
- a connection manner such as an interface is connected to the storage medium, and by executing the executable instruction, a function corresponding to each module can be executed.
- the triggering module 10, the data to be migrated determining module 20, and the control module 30 may respectively correspond to different processors, or may be integrated to correspond to the same processor.
- the processor can be an electronic component having processing functions such as a central processing unit, a microprocessor, a single chip microcomputer, a digital signal processor, or a programmable array.
- the device in this embodiment further includes at least one external communication interface, respectively, and the storage medium and the communication bridge; for example, receiving data access from the user equipment.
- the communication interface may be a wired communication interface, such as an RJ45, a coaxial cable connection interface, or a fiber connection interface.
- the communication interface may also be a wireless communication interface, such as a transceiver antenna or a transceiver antenna array.
- the triggering module 10 determines a trigger for data migration under different conditions based on the data dynamic grading policy; the data to be migrated determining module 20 determines the data to be migrated based on the mutual association of files; and the control module 30 determines whether based on the migration rate control mechanism.
- the problem of data migration and migration of live migration data enables flexible control of the trigger of the migration, flexible configuration of the hierarchical management strategy, and flexible migration of data between online storage and nearline storage.
- the technical solution of the present invention can more flexibly control the trigger of the relocation, and can flexibly configure the hierarchical management strategy to flexibly migrate data between the online storage and the near-line storage.
- the embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are used to execute the method according to any one of the foregoing hierarchical storage methods.
- the executable instructions may be computer identifiable instructions, such as executable instructions constructed based on a mechanical language of 0 and 1, based on C, java or . #High-level language instructions or instruction sets.
- the storage medium is preferably a non-transitory storage medium such as an optical disk, a flash drive, a magnetic disk, or the like.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
RU2016124001A RU2651216C2 (ru) | 2013-11-19 | 2014-04-14 | Способ, устройство и компьютерный носитель данных для перемещения данных |
EP14797086.7A EP3073383A4 (en) | 2013-11-19 | 2014-04-14 | Method, device, and computer storage medium for hierarchical storage |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201310586012.2A CN104657286B (zh) | 2013-11-19 | 2013-11-19 | 一种分级存储方法及装置 |
CN201310586012.2 | 2013-11-19 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2014183514A1 true WO2014183514A1 (zh) | 2014-11-20 |
Family
ID=51897668
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2014/075259 WO2014183514A1 (zh) | 2013-11-19 | 2014-04-14 | 一种分级存储方法、装置和计算机存储介质 |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP3073383A4 (zh) |
CN (1) | CN104657286B (zh) |
RU (1) | RU2651216C2 (zh) |
WO (1) | WO2014183514A1 (zh) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111930303A (zh) * | 2020-07-02 | 2020-11-13 | 苏州浪潮智能科技有限公司 | 一种基于分布式对象存储系统分级迁移优化方法与系统 |
CN112965957A (zh) * | 2021-03-22 | 2021-06-15 | 广州汇量信息科技有限公司 | 一种数据迁移方法、装置、设备及存储介质 |
CN113515238A (zh) * | 2021-07-27 | 2021-10-19 | 华云数据控股集团有限公司 | 一种基于分层存储的数据调度方法、系统及电子设备 |
WO2022062965A1 (en) * | 2020-09-22 | 2022-03-31 | International Business Machines Corporation | Storage tiering within a unified storage environment |
CN116257884A (zh) * | 2023-03-20 | 2023-06-13 | 杭州霖芮科技有限公司 | 一种基于流量分析的电商平台客户数据处理方法与系统 |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106484368B (zh) * | 2016-09-20 | 2019-08-16 | 深圳大学 | 一种基于Spark语义的数据重用方法及其系统 |
CN108009041B (zh) * | 2017-12-17 | 2020-02-14 | 华中科技大学 | 一种基于数据关联性感知的闪存阵列校验更新方法 |
CN109857528B (zh) * | 2019-01-10 | 2021-08-27 | 北京三快在线科技有限公司 | 数据迁移的速度调整方法、装置、存储介质和移动终端 |
CN110175068A (zh) * | 2019-04-16 | 2019-08-27 | 平安科技(深圳)有限公司 | 分布式系统中主机数量弹性伸缩方法、装置和计算机设备 |
CN111858469B (zh) * | 2020-07-24 | 2024-01-26 | 成都成信高科信息技术有限公司 | 一种基于时间滑动窗口的自适应分级存储的方法 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102521138A (zh) * | 2011-11-28 | 2012-06-27 | 成都市华为赛门铁克科技有限公司 | 数据迁移触发方法和装置 |
CN102882928A (zh) * | 2012-08-31 | 2013-01-16 | 浪潮电子信息产业股份有限公司 | 一种云存储系统的分级存储管理方法 |
US20130046920A1 (en) * | 2011-08-17 | 2013-02-21 | Samsung Electronics Co., Ltd. | Nonvolatile memory system with migration manager |
CN103078933A (zh) * | 2012-12-29 | 2013-05-01 | 深圳先进技术研究院 | 一种确定数据迁移时机的方法和装置 |
CN103106047A (zh) * | 2013-01-29 | 2013-05-15 | 浪潮(北京)电子信息产业有限公司 | 一种基于对象的存储系统及其存储方法 |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5530932A (en) * | 1994-12-23 | 1996-06-25 | Intel Corporation | Cache coherent multiprocessing computer system with reduced power operating features |
US8006111B1 (en) * | 2007-09-21 | 2011-08-23 | Emc Corporation | Intelligent file system based power management for shared storage that migrates groups of files based on inactivity threshold |
-
2013
- 2013-11-19 CN CN201310586012.2A patent/CN104657286B/zh active Active
-
2014
- 2014-04-14 WO PCT/CN2014/075259 patent/WO2014183514A1/zh active Application Filing
- 2014-04-14 RU RU2016124001A patent/RU2651216C2/ru active
- 2014-04-14 EP EP14797086.7A patent/EP3073383A4/en not_active Withdrawn
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130046920A1 (en) * | 2011-08-17 | 2013-02-21 | Samsung Electronics Co., Ltd. | Nonvolatile memory system with migration manager |
CN102521138A (zh) * | 2011-11-28 | 2012-06-27 | 成都市华为赛门铁克科技有限公司 | 数据迁移触发方法和装置 |
CN102882928A (zh) * | 2012-08-31 | 2013-01-16 | 浪潮电子信息产业股份有限公司 | 一种云存储系统的分级存储管理方法 |
CN103078933A (zh) * | 2012-12-29 | 2013-05-01 | 深圳先进技术研究院 | 一种确定数据迁移时机的方法和装置 |
CN103106047A (zh) * | 2013-01-29 | 2013-05-15 | 浪潮(北京)电子信息产业有限公司 | 一种基于对象的存储系统及其存储方法 |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111930303A (zh) * | 2020-07-02 | 2020-11-13 | 苏州浪潮智能科技有限公司 | 一种基于分布式对象存储系统分级迁移优化方法与系统 |
CN111930303B (zh) * | 2020-07-02 | 2022-08-12 | 苏州浪潮智能科技有限公司 | 一种基于分布式对象存储系统分级迁移优化方法与系统 |
WO2022062965A1 (en) * | 2020-09-22 | 2022-03-31 | International Business Machines Corporation | Storage tiering within a unified storage environment |
GB2614668A (en) * | 2020-09-22 | 2023-07-12 | Ibm | Storage tiering within a unified storage environment |
US12019867B2 (en) | 2020-09-22 | 2024-06-25 | International Business Machines Corporation | Storage tiering within a unified storage environment |
CN112965957A (zh) * | 2021-03-22 | 2021-06-15 | 广州汇量信息科技有限公司 | 一种数据迁移方法、装置、设备及存储介质 |
CN113515238A (zh) * | 2021-07-27 | 2021-10-19 | 华云数据控股集团有限公司 | 一种基于分层存储的数据调度方法、系统及电子设备 |
CN113515238B (zh) * | 2021-07-27 | 2024-02-06 | 华云数据控股集团有限公司 | 一种基于分层存储的数据调度方法、系统及电子设备 |
CN116257884A (zh) * | 2023-03-20 | 2023-06-13 | 杭州霖芮科技有限公司 | 一种基于流量分析的电商平台客户数据处理方法与系统 |
CN116257884B (zh) * | 2023-03-20 | 2023-09-05 | 杭州霖芮科技有限公司 | 一种基于流量分析的电商平台客户数据处理方法与系统 |
Also Published As
Publication number | Publication date |
---|---|
EP3073383A1 (en) | 2016-09-28 |
RU2651216C2 (ru) | 2018-04-18 |
RU2016124001A (ru) | 2017-12-20 |
CN104657286B (zh) | 2019-05-10 |
EP3073383A4 (en) | 2017-08-16 |
CN104657286A (zh) | 2015-05-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2014183514A1 (zh) | 一种分级存储方法、装置和计算机存储介质 | |
US9513817B2 (en) | Free space collection in log structured storage systems | |
US11960441B2 (en) | Retention management for data streams | |
US9454321B1 (en) | Workload-driven storage configuration management | |
US9773011B2 (en) | On-demand caching in a WAN separated distributed file system or clustered file system cache | |
US10509739B1 (en) | Optimized read IO for mix read/write scenario by chunking write IOs | |
US8904377B2 (en) | Reconfiguration of computer system to allow application installation | |
CN111309732B (zh) | 数据处理方法、装置、介质和计算设备 | |
RU2671049C2 (ru) | Выгрузка в файловой системе | |
US10915461B2 (en) | Multilevel cache eviction management | |
US10719245B1 (en) | Transactional IO scheduler for storage systems with multiple storage devices | |
US10223270B1 (en) | Predicting future access requests by inverting historic access requests in an object storage system | |
US11914894B2 (en) | Using scheduling tags in host compute commands to manage host compute task execution by a storage device in a storage system | |
CN112088364A (zh) | 使用机器学习模块来确定是否执行存储单元的错误检查 | |
US9460001B2 (en) | Systems and methods for identifying access rate boundaries of workloads | |
US10877893B2 (en) | Adaptive pre-fetch | |
CN110175070B (zh) | 分布式数据库的管理方法、装置、系统、介质及电子设备 | |
US10222994B2 (en) | Storing selected data in a stub file in a hierarchical storage management system | |
US8775786B1 (en) | Boot caching for boot acceleration within data storage systems | |
CN112612728B (zh) | 缓存管理方法及装置、设备 | |
CN117235088B (zh) | 一种存储系统的缓存更新方法、装置、设备、介质及平台 | |
US9652155B2 (en) | Computer system, cash data management method, and computer | |
US9594689B2 (en) | Designated cache data backup during system operation | |
US10599340B1 (en) | Policy driven IO scheduler to improve read IO performance in hybrid storage systems | |
WO2022152086A1 (zh) | 数据缓存方法、装置、设备及计算机可读存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14797086 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2016124001 Country of ref document: RU Kind code of ref document: A |
|
REEP | Request for entry into the european phase |
Ref document number: 2014797086 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2014797086 Country of ref document: EP |