CN112783423A

CN112783423A - Data object storage method and device, electronic equipment and computer readable medium

Info

Publication number: CN112783423A
Application number: CN201911089732.1A
Authority: CN
Inventors: 董元元; 杨濮源
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-11-08
Filing date: 2019-11-08
Publication date: 2021-05-11

Abstract

The embodiment of the application provides a data object storage method and device, electronic equipment and a computer readable medium, and relates to the technical field of computers. Wherein the method comprises the following steps: determining a first category to which a data object corresponding to a data object request belongs and a frequency at which the data object is accessed in a first future time period based on data object characteristic data carried in the data object request; determining a second category to which a storage device with service performance meeting the service performance requirement of the data object belongs based on the first category; based on the frequency and the second category, a storage device suitable for storing the data object is determined. Through the embodiment of the application, the data object can be stored on the corresponding appropriate storage equipment more accurately on the premise of meeting the performance requirement of the data object, so that the overall utilization rate of the storage equipment is improved.

Description

Data object storage method and device, electronic equipment and computer readable medium

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to a data object storage method and device, electronic equipment and a computer readable medium.

Background

In a large-scale multi-level heterogeneous storage system, storage devices are currently classified into Solid State Disks (SSD) and Hard Disk Drives (HDD). The HDD has a low price per unit storage capacity and a large overall capacity, but has low data random access performance. SSDs have high read and write performance, but are expensive per unit of capacity, have small overall capacity, and have a limited lifetime. The data object storage strategy is used as an important component of the cluster storage system, and corresponding storage nodes are allocated to the data objects so as to use storage equipment on the storage nodes to store the data objects. The storage strategy of the data object directly determines the utilization capacity of the cluster storage system to storage space resources, I/O bandwidth resources and energy consumption.

In a conventional data object storage strategy, all access processes of a data object are from top to bottom. In particular, data objects must initially be stored on the upper layer, i.e., fast devices, and then demoted to slow devices by eviction. When a data object is read and lost in the fast device, the data object in the slow device needs to be upgraded to the fast device. When there is a new data object and the storage space of the fast device is not enough, a common replacement algorithm is used to swap out the data object. The algorithm treats the newly created data objects as hot data, considers that the hot data objects have high probability of being accessed in a later period of time, and then directly places the hot data objects on a quick device. However, if the probability that a newly created data object is accessed for a long period of time is low, the data object occupies the scarce storage space of the fast device from the time when the data object is put into the fast device to the time when the data object is replaced, which results in the waste of the storage space, the hit rate of the fast device is decreased, and the utilization rate of the fast device is also decreased. In addition, after a data object is read or updated, it is considered to be accessed again in the next period of time according to the principle of time locality, and the data object is promoted from the device in which it is located to the fast device. If the data object is not accessed any more or is accessed again with a low probability for a later period of time, swapping it into a fast device may not only introduce data pollution, but may also replace data objects with a high probability of access. Therefore, how to more accurately store the data objects on the corresponding proper storage devices becomes a technical problem to be solved urgently at present.

Disclosure of Invention

The present application aims to provide a data object storage method, an apparatus, an electronic device, and a computer-readable medium, which are used to solve the technical problem in the prior art of how to more accurately store a data object on a corresponding appropriate storage device.

According to a first aspect of the embodiments of the present application, a method for storing a data object is provided. The method comprises the following steps: determining a first category to which a data object corresponding to a data object request belongs and a frequency at which the data object is accessed in a first future time period based on data object characteristic data carried in the data object request; determining a second category to which a storage device with service performance meeting the service performance requirement of the data object belongs based on the first category; based on the frequency and the second category, a storage device suitable for storing the data object is determined.

According to a second aspect of embodiments of the present application, there is provided a storage device for data objects. The device comprises: a first determining module, configured to determine, based on data object feature data carried in a data object request, a first category to which a data object corresponding to the data object request belongs and a frequency at which the data object is accessed in a first future time period; a second determining module, configured to determine, based on the first category, a second category to which a storage device whose service performance meets the service performance requirement of the data object belongs; a third determination module to determine a storage device suitable for storing the data object based on the frequency and the second category.

According to a third aspect of embodiments of the present application, there is provided an electronic apparatus, including: one or more processors; a computer readable medium configured to store one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method of depositing a data object as described in the first aspect of the embodiments above.

According to a fourth aspect of embodiments of the present application, there is provided a computer-readable medium, on which a computer program is stored, which when executed by a processor, implements the method of depositing a data object as described in the first aspect of the embodiments above.

According to the technical scheme provided by the embodiment of the application, based on the data object characteristic data carried in the data object request, the first class to which the data object corresponding to the data object request belongs and the frequency of the data object accessed in the first future time period are determined; determining a second class to which the storage equipment with the service performance meeting the service performance requirement of the data object belongs based on the first class; and determining storage equipment suitable for storing the data objects based on the frequency and the second category, wherein compared with the traditional data object storage strategy, the storage equipment does not follow the principle that all access processes of the data objects in the traditional data object storage strategy need to be from top to bottom, and on the premise of meeting the performance requirement of the data objects, the data objects stored in the fast equipment are all data objects with high access frequency, and the data objects stored in the slow equipment are all data objects with low access frequency, so that the data objects can be stored on the corresponding proper storage equipment more accurately, thereby not only improving the hit rate of the fast equipment, but also improving the overall utilization rate of the storage equipment.

Drawings

Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:

FIG. 1A is a flowchart illustrating steps of a method for storing data objects according to an embodiment of the present disclosure;

FIG. 1B is a schematic diagram of an online or offline training model according to an embodiment of the present disclosure;

fig. 1C is a schematic view of an overall flow of a storage method for data objects according to an embodiment of the present application;

FIG. 2A is a flowchart illustrating a method for storing data objects according to a second embodiment of the present application;

FIG. 2B is a schematic diagram of a process for creating a data object according to the second embodiment of the present application;

FIG. 2C is a schematic flowchart of reading or updating a data object according to the second embodiment of the present application;

FIG. 3 is a schematic structural diagram of a storage device for data objects according to a third embodiment of the present application;

FIG. 4 is a schematic structural diagram of a storage device for data objects according to a fourth embodiment of the present application;

FIG. 5 is a schematic diagram of a storage device for data objects according to a fifth embodiment of the present application;

fig. 6 is a schematic structural diagram of an electronic device according to a sixth embodiment of the present application;

fig. 7 is a hardware structure of an electronic device according to a seventh embodiment of the present application.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

Referring to fig. 1A, a flowchart illustrating steps of a data object storage method according to a first embodiment of the present application is shown.

Specifically, the storage method of the data object of the embodiment includes the following steps:

in step S101, based on data object feature data carried in a data object request, a first category to which a data object corresponding to the data object request belongs and a frequency at which the data object is accessed in a first future time period are determined.

In an embodiment of the present application, the data object request includes a data object creation request, a data object update request, or a data object read request. The data object characteristic data comprises at least one of: a data object generation source, a data object creation time, a data object size, type data identified by a user for the data object, a frequency with which the data object is accessed within a historical period of time, and data of a service performance requirement of the data object for the storage device. Wherein, the data object generation source may be a server connected to the client in communication, a client running on the terminal device, and the like, the type data identified by the user for the data object may be hot data, warm data, cold data, and the like, and the service performance requirement data may include at least one of the following: data that is required for storage latency of a storage device, data that is required for throughput or throughput rate of a storage device, and data that is required for storage latency fluctuation of a storage device. The data object can be understood as various service data on a terminal device or various service data on a server and the like. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In an embodiment of the present application, the frequency with which data objects are accessed during the historical period of time includes the frequency with which data objects are written during the historical period of time and the frequency with which data objects are read during the historical period of time. Compared with a method for dynamically determining the storage position of a data object by tracking the condition that the data object is accessed at a certain moment or a certain time point in a traditional LRU (Least Recently Used) based data object storage strategy, the embodiment can classify the data object by the frequency of the data object being accessed in a historical time period, and can more accurately store the data object to a corresponding proper storage device. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a large-scale multi-level heterogeneous storage system for storing data objects, certain characteristics exist for each data object. For example, a data object is frequently accessed on day 1, less frequently accessed on days 2 to 4, and substantially not accessed after day 5. As another example, a data object may be accessed every 3 days. The frequency with which each data object is accessed changes over time, as does the location in which it is stored. However, there are billions of data objects stored in the storage system, and the value tracks of each data object are different, some are very different, and some are very small, so that the data objects with small value track difference can be classified into one class, and the data objects in the class have similar attributes. Therefore, the data objects can be classified and treated, and the data objects can be stored on the corresponding proper storage devices more accurately.

Specifically, when a first category to which a data object corresponding to a data object request belongs and a frequency at which the data object is accessed in a first future time period are determined based on data object characteristic data carried in the data object request, a data object prediction model to be trained is trained based on category marking data of a data object sample and frequency marking data at which the data object sample is accessed in a second future time period; predicting the first class and the frequency based on the data object feature data through the trained data object prediction model. The data object prediction model may be any suitable neural network model that can implement feature extraction or target object detection, including but not limited to a convolutional neural network, an reinforcement learning neural network, a generation network in an antagonistic neural network, and the like. The specific structure of the neural network can be set by those skilled in the art according to actual requirements, such as the number of convolutional layers, the size of convolutional core, the number of channels, and the like. Therefore, the first category to which the data object corresponding to the data object request belongs and the frequency of the data object being accessed in the first future time period can be accurately predicted through the data object prediction model. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In some optional embodiments, before training the data object prediction model to be trained based on the class labeling data of the data object sample and the frequency labeling data of the data object sample accessed in the second future time period, the method further comprises: determining the category label data based on the frequency with which the data object sample is accessed within a first historical time period, the source of generation of the data object sample, the time of creation of the data object sample, and the type data identified by the user for the data object sample; determining the frequency marking data based on distribution data of the frequency of the data object samples accessed in a second historical time period, wherein the second historical time period has a corresponding relationship with the second future time period. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In one specific example, off-line trace log data of the data object samples is collected, and the collected trace log data is analyzed and counted to obtain the frequency of the data object samples being accessed in the first historical period of time, the generation source of the data object samples, the creation time of the data object samples, and the type data identified by the user for the data object samples. Then, the data object samples which have the same generation source, creation time and type data and are accessed in the same range in the first historical time period are divided into one class, and class labels are marked on the data object samples of the class, so that class marking data of the data object samples are obtained. Of course, the data object samples whose generation source and type data are the same, whose creation time is in the same time period, and whose frequency of being accessed in the first history time period is in the same range may also be classified into one type. In addition, statistical analysis is performed on the distribution data of the frequency of the data object samples accessed in the second historical time period, so that frequency marking data of the data object samples accessed in the second future time period can be obtained. For example, the frequency with which a data object sample is accessed from 0 to 6 points per day in the past 10 days is obtained, and then the frequency with which a data object sample is accessed from 0 to 6 points per day in the past 10 days is averaged to predict the frequency with which a data object sample is accessed from 0 to 6 points in the following day, thereby obtaining frequency annotation data for the data object sample. For another example, the frequency with which the data object samples are accessed from 12 to 17 points per day in the past 8 days is obtained, and then the frequency with which the data object samples are accessed from 12 to 17 points per day in the past 8 days is subjected to mode calculation to predict the frequency with which the data object samples are accessed from 12 to 17 points in the following day, so as to obtain the frequency labeling data of the data object samples. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In step S102, a second class to which a storage device whose service performance satisfies the service performance requirement of the data object belongs is determined based on the first class.

In the embodiment of the application, the storage devices in the large-scale multi-level heterogeneous storage system can be classified according to the performance of the storage devices. The performance of the storage device includes, but is not limited to, configuration of the storage device, peak read-write performance, random IOPS, storage capacity, price of the storage capacity, and the like. Since the static performance characteristics of a storage device are already determined at the time of shipment, it can be classified according to its actual configuration and nominal performance. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In some optional embodiments, when determining that the storage device whose service performance meets the service performance requirement of the data object belongs to the second category based on the first category, training a service performance prediction model to be trained based on service performance marking data when a data object sample of the same category is stored on the storage device of the same category; obtaining service performance prediction data when the data objects belonging to the first category are stored on the storage equipment of each category based on the first category through the trained service performance prediction model; and determining a second category to which the storage device with the service performance meeting the service performance requirement of the data object belongs based on the service performance prediction data. The service performance prediction model may be any suitable neural network model that can implement feature extraction or target object detection, including but not limited to a convolutional neural network, an reinforcement learning neural network, a generation network in an antagonistic neural network, and the like. The specific structure of the neural network can be set by those skilled in the art according to actual requirements, such as the number of convolutional layers, the size of convolutional core, the number of channels, and the like. The service performance prediction data comprises at least one of: the storage delay data of the storage device, the storage throughput or throughput of the storage device, and the fluctuation data of the storage delay of the storage device. Therefore, the service performance data of the first class of data objects stored in the storage equipment of each class is predicted through the service performance prediction model, and the second class of the storage equipment with the service performance meeting the service performance requirement of the data objects can be accurately determined. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, before the service performance prediction model to be trained is trained, offline trace log data of the data object samples are collected, and analysis statistics is performed on the collected trace log data to obtain frequency of access of the data object samples in a first historical time period, generation sources of the data object samples, creation time of the data object samples, type data identified by a user for the data object samples, and service performance data of the data object samples when the data object samples are stored on the storage device in the first historical time period. The data-object samples are then classified based on the frequency with which the data-object samples were accessed within the first historical period of time, the source of generation of the data-object samples, the time of creation of the data-object samples, the type data identified by the user for the data-object samples to determine the class to which the data-object samples belong, and the storage devices are classified based on their actual configuration and nominal performance to determine the class to which the storage devices belong. And finally, determining a service performance label, namely service performance marking data, of the data object sample of the same class when the data object sample of the same class is stored on the storage device of the same class based on the class to which the storage device belongs, the class to which the data object sample belongs, and the service performance data of the data object sample when the data object sample is stored on the storage device in the first historical time period. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, as shown in fig. 2B, during the operation of the large-scale multi-level heterogeneous storage system, new trace record data is generated, and the model trainer (the to-be-trained data object prediction model or the to-be-trained service performance prediction model) retrains the model with the online newly generated trace record data at intervals, so as to ensure that the data object prediction model or the service performance prediction model is suitable for the current load. Specifically, the model trainer first trains based on the offline trace record data of the collected data object samples to obtain a trained data object prediction model or a service performance prediction model. When a data object request is completed, the metadata server adds the trace record data corresponding to the data object request into the online trace record table, and the model trainer trains based on the trace record data in the online trace record table at intervals (assumed to be 1 day) to obtain a trained data object prediction model or service performance prediction model, so that the data object prediction model or service performance prediction model is continuously updated. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, when the data object characteristic data further includes service performance requirement data of the data object for the storage device, the determining, based on the service performance prediction data, a second class to which a storage device whose service performance meets the service performance requirement of the data object belongs includes: determining the class to which the storage device for which the service performance prediction data meets the service performance requirement data belongs to is the second class. For example, when the predicted storage delay data is less than or equal to the storage delay data required by the data object, the predicted storage delay fluctuation data is less than or equal to the storage delay fluctuation data required by the data object, or the predicted storage throughput is greater than or equal to the storage throughput required by the data object, it may be understood that the service performance prediction data satisfies the service performance requirement data. When the data object characteristic data does not include service performance requirement data of the data object for the storage device, determining all classes to which the storage devices in the large-scale multi-level heterogeneous storage system belong as the second class. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In step S103, a storage device suitable for storing the data object is determined based on the frequency and the second class.

In some optional embodiments, when the data object request comprises a data object creation request, the determining a storage device suitable for storing the data object based on the frequency and the second category comprises: calculating a deposit cost of the data object on the storage device of the second class based on the frequency and the second class through a cost function for calculating a deposit cost of the data object on a storage device of the same class; determining a storage device suitable for storing the data object based on the storage device of the second category with the least storage cost. Thereby, by means of the cost function, a storage device suitable for storing the data object can be accurately determined. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, when the data object request includes a data object creation request, the migration of the data object is not involved, only the cost function is used to calculate the storage cost of the data object when the data object is stored on the storage device of each class, the storage device class with the lowest storage cost is selected, and then a specific storage device is selected from the storage device class according to the storage principle. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, when the cost function is used to calculate the storage cost of the data object on the second type of storage device based on the frequency and the second type, the cost function is used to calculate the usage cost of the data object occupying the storage space in the second type of storage device in the first future time period based on the second type; calculating, by the cost function, a deposit value resulting from the data object occupying storage space in the second class of storage devices for the first future period of time based on the frequency; calculating, by the cost function, a deposit cost of the data object on the second class of storage devices for the first future period of time based on the usage cost and the deposit value. In particular, the deposit value is subtracted from the usage cost by the cost function to obtain a deposit cost of the data object on the second class of storage devices for the first future time period. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, when determining the storage device suitable for storing the data object based on the storage device of the second category with the least storage cost, the storage device suitable for storing the data object is determined based on the storage device of the second category with the least storage cost by using a load balancing principle or a load centralizing principle. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In one specific example, the load balancing principle may be understood as that the number of data objects in the second category of storage devices with the least storage cost is balanced. When the storage device suitable for storing the data object is determined, the number of the stored data objects in each storage device belonging to the second category is determined, the data object to be currently allocated is allocated to the storage device with the least number of the stored data objects, and after the data object is added, the performance of the storage device can meet the performance requirement of each internal data object, and the used storage capacity does not exceed the total capacity of the storage device. The load concentrating principle may be understood as that a storage device belonging to the second class is preferentially filled with data objects, and if the storage space of the storage device is not enough, the data objects are stored in the next storage device. When the storage device suitable for storing the data object is determined, the number of the stored data objects in each storage device belonging to the second category is determined, the data object to be currently allocated is allocated to the storage device with the largest number of the stored data objects, and after the data object is added, the performance of the storage device can meet the performance requirement of each internal data object, and the used storage capacity does not exceed the total capacity of the storage device. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In some optional embodiments, when the data object request comprises a data object update request or a data object read request, the determining, based on the frequency and the second class, a storage device suitable for storing the data object comprises: calculating a deposit cost of the data object on the storage device of the second class based on the frequency and the second class through a cost function for calculating a deposit cost of the data object on a storage device of the same class; sorting the second category based on the storage cost to obtain a sorting result of the second category; and determining the storage equipment suitable for storing the data object based on the sorting result of the second category and the third category to which the storage equipment currently storing the data object belongs. Therefore, the storage device suitable for storing the data object can be accurately determined according to the sorting result of the second category and the third category to which the storage device currently storing the data object belongs. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, the specific implementation of calculating the storage cost of the data object on the storage device of the second category based on the frequency and the second category by the cost function for calculating the storage cost of the data object on the storage device of the same category is similar to the specific implementation of calculating the storage cost of the data object on the storage device of the second category based on the frequency and the second category by the cost function for calculating the storage cost of the data object on the storage device of the same category, which is not described herein again. And when the second category is sorted, sorting the second category according to the sorting from low to high of the storage cost so as to obtain a sorting result of the second category. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, when determining a storage device suitable for storing the data object based on the sorting result of the second category and a third category to which a storage device currently storing the data object belongs, if it is determined that the third category is not the second category currently rotated in the sorting result, determining a migration benefit when the data object is migrated from the storage device currently storing the data object to the storage device currently rotated to the second category, and calculating a migration cost when the data object is migrated from the storage device currently storing the data object to the storage device currently rotated to the second category by using a migration function for calculating a migration cost when the data object is migrated from one category of storage devices to another category of storage devices, determining to migrate the data object from the storage device currently storing the data object to the storage device currently rotated to the second category of storage devices based on the migration cost and the migration benefit And when the storage device of the second category is rotated, determining a storage device suitable for storing the data object based on the currently rotated storage device of the second category. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, when determining migration profit of the data object when the data object is migrated from the storage device currently storing the data object to the currently rotated storage device of the second category, calculating storage cost of the data object on the storage device of the third category and storage cost of the data object on the currently rotated storage device of the second category through the cost function; determining the migration benefit based on a storage cost of the data object on the third class of storage devices and a storage cost of the data object on the currently rotated second class of storage devices. Specifically, the storage cost of the data object on the storage device of the third category is subtracted by the storage cost of the data object on the currently rotated storage device of the second category to obtain the migration benefit. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, before determining the storage device suitable for storing the data object based on the currently rotated storage device of the second category, the method further comprises: and if the difference value between the migration benefit and the migration cost is larger than or equal to a set threshold value, determining to migrate the data object from the storage device currently storing the data object to the currently rotated storage device of the second category. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, when determining the storage device suitable for storing the data object based on the currently rotated storage device of the second category, a load balancing principle or a load centralizing principle is adopted to determine the storage device suitable for storing the data object based on the currently rotated storage device of the second category. The load balancing principle and the load concentrating principle are similar to the load balancing principle and the load concentrating principle, and are not described herein again. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In one particular example, when a data object needs to be read or updated, it may be that the data object is migrated from the original storage device to a new storage device, requiring both the cost function and the migration function to be used. The specific process steps are as follows: (1) the cost function calculates the storage cost of the data objects stored on each type of storage equipment, sorts the types of the storage equipment according to the storage cost from low to high, and obtains the sequence of the types of the corresponding storage equipment: SD_i∈{SD₀，SD₁，……，SD_n-1H, let i equal to 0,turning to (2); (2) determining class SD to which primary storage device of data object belongs_tWhether or not it is SD_i(the current rotated second category in the sequencing result), if yes, turning to (6), and if not, turning to (3); (3)

first computing data object in class SD_iCost (i) when deposited on the storage device of (D) and data object in class SD_tThe cost (t) of the storage device(s) of (1), the migration gain (t) cost (i), and the data object from the category SD is calculated_tStorage device migration to class SD_iIf the gain-penalty (ti) is greater than or equal to a certain threshold, migration is feasible, and the storage device is in the SD according to a storage principle (a load balancing principle or a load centralizing principle)_iSelecting one storage device from the storage devices of the categories, sending the identification of the storage device to a metadata server, turning to (4), and turning to (5) if the identification is not feasible; (4) the metadata server informs the storage nodes to carry out data object migration, and then (6); (5) i +1, go (2); (6) and (6) ending. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, as shown in fig. 1C, the overall flow of the data object storage method is briefly described. Specifically, the model trainer trains based on the off-line trace record data of the collected data object samples or the on-line newly generated trace record data of the data object samples to obtain a trained data object prediction model or a service performance prediction model. When a suitable storage position is selected for a certain data object, firstly, feature extraction is carried out on the behavior of the data object to obtain data object feature data. For example, a data object creation request, a data object update request, or a data object read request is parsed to obtain data object characterization data. Then, through a data object prediction model, the class to which the data object belongs and the frequency with which the data object is accessed in a future time period are predicted based on the data object feature data. Then, the class to which the data object belongs is input into a service performance prediction model, and a list of classes to which storage devices with service performance meeting the service performance requirement of the data object belong is obtained through the service performance prediction model. And finally, inputting a list of categories to which the storage equipment with the frequency of accessing the data object in a future time period and the service performance meeting the service performance requirement of the data object belongs into the cost model, selecting the storage equipment category with the lowest storage cost through the cost model (cost function and migration function), and selecting one storage equipment for storing the data object from the storage equipment of the category according to a storage principle. Wherein the cost model is used for evaluating the cost of the data object stored on each type of storage device. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

According to the data object storage method provided by the embodiment of the application, based on the data object characteristic data carried in the data object request, the first class to which the data object corresponding to the data object request belongs and the frequency of the data object being accessed in the first future time period are determined; determining a second class to which the storage equipment with the service performance meeting the service performance requirement of the data object belongs based on the first class; and determining storage equipment suitable for storing the data objects based on the frequency and the second category, wherein compared with the traditional data object storage strategy, the storage equipment does not follow the principle that all access processes of the data objects in the traditional data object storage strategy need to be from top to bottom, and on the premise of meeting the performance requirement of the data objects, the data objects stored in the fast equipment are all data objects with high access frequency, and the data objects stored in the slow equipment are all data objects with low access frequency, so that the data objects can be stored on the corresponding proper storage equipment more accurately, thereby not only improving the hit rate of the fast equipment, but also improving the overall utilization rate of the storage equipment.

The method for storing the data object of the present embodiment may be executed by any suitable device with data processing capability, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, in-vehicle devices, entertainment devices, advertising devices, Personal Digital Assistants (PDAs), tablet computers, notebook computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices (such as Google Glass, Oculus rise, Hololens, Gear VR), and the like.

Referring to fig. 2A, a flowchart illustrating steps of a method for storing a data object according to a second embodiment of the present application is shown.

in step S201, based on the data object feature data carried in the data object request, a first category to which a data object corresponding to the data object request belongs and a frequency at which the data object is accessed in a first future time period are determined.

In some optional embodiments, when determining a first category to which a data object corresponding to the data object request belongs and a frequency with which the data object is accessed within a first future time period, if it is determined that the data object feature data is similar to the feature data of a predetermined data object sample belonging to the same category, determining that the same category is the first category to which the data object belongs, and determining the frequency with which the data object is accessed within the first future time period based on distribution data of the frequency with which the data object sample of the same category is accessed within a third history time period, wherein the third history time period has a corresponding relationship with the first future time period. Therefore, by comparing the characteristic data of the data object with the characteristic data of the data object sample, the first class to which the data object corresponding to the data object request belongs and the frequency of the data object being accessed in the first future time period can be accurately determined. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, before the determination that the same category is the first category to which the data object belongs, trace record data of data object samples is collected, and analysis statistics is performed on the collected trace record data to obtain frequency of access of the data object samples in a fourth historical time period, a generation source of the data object samples, creation time of the data object samples, and type data identified by a user for the data object samples. Then, data object samples whose generation sources, creation times, and type data are the same, and whose frequencies of being accessed in the fourth history period are within the same range are classified into one type. Of course, the data object samples whose generation source and type data are the same, whose creation time is in the same time period, and whose frequency of being accessed in the fourth history time period is in the same range may also be classified into one type.

In a specific example, when the data object characteristic data comprises at least one of: when a data object generation source, a data object creation time, type data identified by a user for the data object, and a frequency with which the data object is accessed within a fourth historical period of time, the determining that the same category is before the first category to which the data object belongs, the method further includes: and if the data object generation source, the data object creation time and the type data are determined to be respectively the same as the generation source of the data object sample, the creation time of the data object sample and the type data identified by the user for the data object sample in the feature data, and the frequency of the data object being accessed in the fourth historical time period and the frequency of the data object sample in the feature data being accessed in the fourth historical time period are within a preset range, determining that the data object feature data is similar to the predetermined feature data of the data object sample belonging to the same category. Of course, the present embodiment is not limited thereto, and if it is determined that the data object generation source and the type data are respectively the same as the generation source of the data object sample in the feature data and the type data identified by the user for the data object sample, the frequency with which the data object is accessed in the fourth history period is within a preset range as the frequency with which the data object sample in the feature data is accessed in the fourth history period, and the data object creation time is within the same period as the creation time of the data object sample, the data object feature data is determined to be similar to the predetermined feature data of the data object sample belonging to the same category. Alternatively, if it is determined that the data object generation source and the type data are respectively the same as the generation source of the data object sample in the feature data and the type data identified by the user for the data object sample, the proportion of the data object being read and written in the fourth history period is within a preset range as the proportion of the data object sample in the feature data being read and written in the fourth history period, and the data object creation time is within the same period as the creation time of the data object sample, then it is determined that the data object feature data is similar to the predetermined feature data of the data object sample belonging to the same category. The preset range may be set by a person skilled in the art according to actual needs, and the embodiment of the present application is not limited in any way. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, when determining the frequency of the data object being accessed in the first future time period based on the distribution data of the frequency of the data object sample of the same category being accessed in the third history time period, performing a statistical analysis on the distribution data of the frequency of the data object sample of the same category being accessed in the third history time period may obtain the frequency of the data object being accessed in the first future time period. For example, the frequency of 8 to 12 accesses in the past 1 day of multiple data object samples of the same category is obtained, and then the frequency of 8 to 12 accesses in the past 1 day of multiple data object samples is averaged to predict the frequency of 8 to 12 accesses in the next day of data objects. For another example, the frequency with which a plurality of data object samples of the same category are accessed from 19 to 23 points in the past 1 day is obtained, and then the frequency with which the data object samples are accessed from 19 to 23 points in the past 1 day is subjected to a mode to predict the frequency with which the data object is accessed from 19 to 23 points in the following day. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In step S202, based on the service performance data when the sample of the data object belonging to the first category is deposited on the storage device of the same category in the fifth history period, the service performance prediction data when the data object belonging to the first category is deposited on the storage device of the same category is obtained.

In a specific example, service performance data of a plurality of data object samples belonging to the first category when 8 to 12 points are stored on the same category of storage devices in the past 1 day is obtained, and then, the service performance data is averaged or subjected to mode calculation to obtain service performance prediction data of the data object belonging to the first category when stored on the same category of storage devices. Wherein the service performance data comprises at least one of: the storage delay data of the storage device, the storage throughput or throughput of the storage device, and the fluctuation data of the storage delay of the storage device. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In step S203, a second class to which a storage device whose service performance satisfies the service performance requirement of the data object belongs is determined based on the service performance prediction data.

In this embodiment, the specific implementation manner of step S203 is similar to the specific implementation manner of the second category to which the storage device whose service performance meets the service performance requirement of the data object is determined based on the service performance prediction data, and is not described herein again.

In step S204, a storage device suitable for storing the data object is determined based on the frequency and the second class.

Since step S204 is similar to step S103, it is not described herein again.

In a specific example, as shown in fig. 2B, when a user needs to create a data object, the metadata server receives a data object creation request sent by the user and inputs the data object creation request into the data object prediction model, and the data object prediction model performs data object predictionThe test model extracts the data object characteristic data carried in the data object creating request, and outputs the category of the data object and the frequency of the data object to be accessed in the next period of time according to the data object characteristic data. When the performance model receives the data object type output by the data object prediction model, service performance data of the data object on each type of storage equipment is predicted, if the data object characteristic data contains service performance requirement data of the storage equipment, the storage equipment type meeting the service performance requirement data is selected and added into a list; and if no service performance requirement data exists, adding the categories of all the storage devices into the list, and outputting the list of the categories of the storage devices. When the cost model receives the frequency that the data object will be accessed in the next period of time and the class list { SD of storage devices that satisfy the data object's service performance requirement data for the storage device }₀，SD₁，……，SD_n-1And when the capacity of the data object is enough, selecting the storage device type with the lowest cost, selecting one storage device from the storage devices of the type according to a load balancing principle, outputting the identifier of the storage device to a metadata server, and returning the identifier to the user by the metadata server. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

In a specific example, as shown in fig. 2C, when a user needs to read or update a data object, the metadata server receives a data object read request or a data object update request sent by the user, and stores the class SD to which the storage device of the data object belongs_tAnd returning the request to the user, inputting the request into the data object prediction model, wherein the data object prediction model extracts the characteristic data of the requested data object and outputs the category of the data object and the frequency of the data object to be accessed in the next period of time according to the characteristic data of the data object. Predicting when the performance model receives the data object class output by the data object prediction modelIf the data object feature data contains service performance requirement data of the storage equipment, selecting the storage equipment type meeting the service performance requirement data and adding the selected storage equipment type into a list; and if no service performance requirement data exists, adding the categories of all the storage devices into the list, and outputting the list of the categories of the storage devices. When the cost model receives the frequency with which data objects will be accessed in the next period of time and the class list { SD0, SD1, … …, SDn-1} of storage devices that satisfy the data objects' service performance requirements for the storage devices, the storage devices suitable for hosting the data objects are determined. The specific process steps are as follows: (1) the cost function calculates the storage cost of the data objects stored on each type of storage equipment, sorts the types of the storage equipment according to the storage cost from low to high, and obtains the sequence of the types of the corresponding storage equipment: SD_i∈{SD₀，SD₁，……，SD_n-1J, let i equal to 0, turn (2); (2) determining class SD to which primary storage device of data object belongs_tWhether or not it is SD_iIf yes, turning to (6), otherwise turning to (3); (3) first computing data object in class SD_iCost (i) when deposited on the storage device of (D) and data object in class SD_tThe cost (t) of the storage device(s) of (1), the migration gain (t) cost (i), and the data object from the category SD is calculated_tStorage device migration to class SD_iIf the gain-penalty (ti) is greater than or equal to a certain threshold, migration is feasible, and the storage device is in the SD according to a storage principle (a load balancing principle or a load centralizing principle)_iSelecting one storage device from the storage devices of the categories, sending the identification of the storage device to a metadata server, turning to (4), and turning to (5) if the identification is not feasible; (4) the metadata server informs the storage nodes to carry out data object migration, and then (6); (5) i +1, go (2); (6) and (6) ending. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.

On the basis of the first embodiment, based on the service performance data of the data object sample belonging to the first category when stored on the storage devices of the same category in the fifth historical time period, obtaining service performance prediction data of the data object belonging to the first category when stored on the storage devices of the same category; and then, based on the service performance prediction data, determining a second class to which the storage device of which the service performance meets the service performance requirement of the data object belongs, and compared with other existing modes, determining a second class to which the storage device of which the service performance meets the service performance requirement of the data object belongs based on the service performance prediction data when the data object of which the service performance meets the service performance requirement of the data object is stored on the storage device of the same class, so that the storage device of which the service performance meets the service performance requirement of the data object belongs can be accurately determined.

Referring to fig. 3, a schematic structural diagram of a storage apparatus for data objects in a third embodiment of the present application is shown.

The storage device for data objects of the embodiment comprises: a first determining module 301, configured to determine, based on data object feature data carried in a data object request, a first category to which a data object corresponding to the data object request belongs and a frequency at which the data object is accessed in a first future time period; a second determining module 302, configured to determine, based on the first category, a second category to which a storage device whose service performance meets the service performance requirement of the data object belongs; a third determining module 303, configured to determine, based on the frequency and the second category, a storage device suitable for storing the data object.

The storage apparatus for data objects of this embodiment is used to implement the storage method for corresponding data objects in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein again.

Referring to fig. 4, a schematic structural diagram of a storage apparatus for data objects in the fourth embodiment of the present application is shown.

The storage device for data objects of the embodiment comprises: a first determining module 401, configured to determine, based on data object feature data carried in a data object request, a first category to which a data object corresponding to the data object request belongs and a frequency at which the data object is accessed in a first future time period; a second determining module 402, configured to determine, based on the first category, a second category to which a storage device whose service performance meets the service performance requirement of the data object belongs; a third determining module 403, configured to determine, based on the frequency and the second category, a storage device suitable for storing the data object.

Optionally, the first determining module 401 includes: the first training submodule 4013 is configured to train a data object prediction model to be trained, based on class labeling data of a data object sample and frequency labeling data of the data object sample accessed in a second future time period; a first prediction sub-module 4014, configured to predict the first class and the frequency based on the data object feature data through the trained data object prediction model.

Optionally, before the first training sub-module 4013, the first determining module 401 further includes: a first determining sub-module 4011, configured to determine the category marking data based on a frequency with which the data object sample is accessed in a first historical time period, a generation source of the data object sample, a creation time of the data object sample, and type data identified by a user for the data object sample; the second determining sub-module 4012 is configured to determine the frequency labeling data based on distribution data of frequencies of the data object samples visited in a second historical time period, where the second historical time period and the second future time period have a corresponding relationship.

Optionally, the first determining module 401 includes: a third determining sub-module 4016, configured to, if it is determined that the data object feature data is similar to the predetermined feature data of the data object sample belonging to the same category, determine that the same category is the first category to which the data object belongs, and determine, based on distribution data of frequencies at which the data object sample of the same category is visited in a third history time period, the frequency at which the data object is visited in the first future time period, where the third history time period has a corresponding relationship with the first future time period.

Optionally, the data object characteristic data comprises at least one of: before the third determining sub-module 4016, the first determining module 401 further includes: a fourth determining sub-module 4015, configured to determine that the data object feature data is similar to the predetermined feature data of the data object samples belonging to the same category if it is determined that the data object generation source, the data object creation time, and the type data are respectively the same as the generation source of the data object samples in the feature data, the creation time of the data object samples, and the type data identified by the user for the data object samples, and the frequency with which the data object is accessed in the fourth history period is within a preset range as compared with the frequency with which the data object samples in the feature data are accessed in the fourth history period.

Optionally, the data object request includes a data object creation request, and the third determining module 403 includes: a first calculating submodule 4031, configured to calculate, based on the frequency and the second category, a storage cost of the data object on a storage device of the second category through a cost function for calculating a storage cost of the data object on a storage device of the same category; a sixth determining sub-module 4032, configured to determine, based on the storage device of the second category with the least storage cost, a storage device suitable for storing the data object.

Optionally, the first calculation sub-module 4031 is specifically configured to: calculating, by the cost function, based on the second class, a usage cost incurred by the data object occupying storage space in the second class of storage devices for the first future time period; calculating, by the cost function, a deposit value resulting from the data object occupying storage space in the second class of storage devices for the first future period of time based on the frequency; calculating, by the cost function, a deposit cost of the data object on the second class of storage devices for the first future period of time based on the usage cost and the deposit value.

Optionally, the sixth determining sub-module 4032 is specifically configured to: and determining the storage equipment suitable for storing the data object based on the storage equipment of the second class with the least storage cost by adopting a load balancing principle or a load concentration principle.

Referring to fig. 5, a schematic structural diagram of a storage apparatus for data objects in the fifth embodiment of the present application is shown.

The storage device for data objects of the embodiment comprises: a first determining module 501, configured to determine, based on data object feature data carried in a data object request, a first category to which a data object corresponding to the data object request belongs and a frequency at which the data object is accessed in a first future time period; a second determining module 502, configured to determine, based on the first category, a second category to which a storage device whose service performance meets the service performance requirement of the data object belongs; a third determining module 503, configured to determine, based on the frequency and the second category, a storage device suitable for storing the data object.

Optionally, the second determining module 502 includes: the second training submodule 5021 is used for training a service performance prediction model to be trained on the basis of service performance marking data when data object samples of the same type are stored in storage equipment of the same type; the second prediction sub-module 5022 is configured to obtain, based on the first class, service performance prediction data when the data objects belonging to the first class are stored on the storage device of each class through the trained service performance prediction model; a fifth determining submodule 5023, configured to determine, based on the service performance prediction data, a second category to which a storage device whose service performance meets the service performance requirement of the data object belongs.

Optionally, the second determining module 502 includes: a third prediction sub-module 5024, configured to obtain service performance prediction data when the data object belonging to the first category is stored on the storage device of the same category based on the service performance data when the data object sample belonging to the first category is stored on the storage device of the same category in a fifth historical time period; a fifth determining submodule 5023, configured to determine, based on the service performance prediction data, a second category to which a storage device whose service performance meets the service performance requirement of the data object belongs.

Optionally, the data object feature data further includes data of service performance requirements of the data object for the storage device, and the fifth determining sub-module 5023 is specifically configured to: determining the class to which the storage device for which the service performance prediction data meets the service performance requirement data belongs to is the second class.

Optionally, the service performance prediction data comprises at least one of: the storage delay data of the storage device, the storage throughput or throughput of the storage device, and the fluctuation data of the storage delay of the storage device.

Optionally, the data object request includes a data object update request or a data object read request, and the third determining module 503 includes: a second calculating sub-module 5031, configured to calculate, based on the frequency and the second class, a storage cost of the data object on the storage device of the second class through a cost function for calculating a storage cost of the data object on the storage device of the same class; a sorting sub-module 5032, configured to sort the second category based on the storage cost to obtain a sorting result of the second category; a seventh determining sub-module 5033, configured to determine, based on the sorting result of the second category and a third category to which the storage device currently storing the data object belongs, a storage device suitable for storing the data object.

Optionally, the seventh determining sub-module 5033 comprises: a first determining unit 5034, configured to determine, if it is determined that the third category is not the currently rotated second category in the sorting result, a migration benefit when the data object is migrated from the storage device currently storing the data object to the currently rotated second category of storage device, and calculate, through a migration function for calculating a migration cost when the data object is migrated from one category of storage device to another category of storage device, a migration cost when the data object is migrated from the storage device currently storing the data object to the currently rotated second category of storage device, and when determining, based on the migration cost and the migration benefit, to migrate the data object from the storage device currently storing the data object to the currently rotated second category of storage device, based on the currently rotated second category of storage device, a storage device suitable for storing the data object is determined.

Optionally, the first determining unit 5034 is specifically configured to: calculating, by the cost function, a storage cost of the data object on the third class of storage devices and a storage cost of the data object on the currently rotated second class of storage devices; determining the migration benefit based on a storage cost of the data object on the third class of storage devices and a storage cost of the data object on the currently rotated second class of storage devices.

Optionally, the first determining unit 5034 is specifically configured to: and determining the storage equipment suitable for storing the data object based on the currently rotated storage equipment of the second class by adopting a load balancing principle or a load concentration principle.

Fig. 6 is a schematic structural diagram of an electronic device according to a sixth embodiment of the present application; the electronic device may include:

one or more processors 601;

a computer-readable medium 602, which may be configured to store one or more programs,

when the one or more programs are executed by the one or more processors, the one or more processors implement the method for storing data objects according to the first embodiment or the second embodiment.

Fig. 7 is a hardware structure of an electronic device according to a seventh embodiment of the present application; as shown in fig. 7, the hardware structure of the electronic device may include: a processor 701, a communication interface 702, a computer-readable medium 703 and a communication bus 704;

wherein the processor 701, the communication interface 702, and the computer-readable medium 703 are in communication with each other via a communication bus 704;

alternatively, the communication interface 702 may be an interface of a communication module, such as an interface of a GSM module;

the processor 701 may be specifically configured to: determining a first category to which a data object corresponding to a data object request belongs and a frequency at which the data object is accessed in a first future time period based on data object characteristic data carried in the data object request; determining a second category to which a storage device with service performance meeting the service performance requirement of the data object belongs based on the first category; based on the frequency and the second category, a storage device suitable for storing the data object is determined.

The Processor 701 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The computer-readable medium 703 may be, but is not limited to, a Random Access Memory (RAM), a Read-Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code configured to perform the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section, and/or installed from a removable medium. The computer program, when executed by a Central Processing Unit (CPU), performs the above-described functions defined in the method of the present application. It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access storage media (RAM), a read-only storage media (ROM), an erasable programmable read-only storage media (EPROM or flash memory), an optical fiber, a portable compact disc read-only storage media (CD-ROM), an optical storage media piece, a magnetic storage media piece, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code configured to carry out operations for the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may operate over any of a variety of networks: including a Local Area Network (LAN) or a Wide Area Network (WAN) -to the user's computer, or alternatively, to an external computer (e.g., through the internet using an internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions configured to implement the specified logical function(s). In the above embodiments, specific precedence relationships are provided, but these precedence relationships are only exemplary, and in particular implementations, the steps may be fewer, more, or the execution order may be modified. That is, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present application may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a first determination module, a second determination module, and a third determination module. For example, the first determining module may be further described as a "module that determines, based on data object characteristic data carried in a data object request, a first category to which a data object corresponding to the data object request belongs and a frequency with which the data object is accessed in a first future time period.

As another aspect, the present application also provides a computer-readable medium on which a computer program is stored, the program, when executed by a processor, implementing the method for storing data objects as described in the first or second embodiment.

As another aspect, the present application also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be present separately and not assembled into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the apparatus to: determining a first category to which a data object corresponding to a data object request belongs and a frequency at which the data object is accessed in a first future time period based on data object characteristic data carried in the data object request; determining a second category to which a storage device with service performance meeting the service performance requirement of the data object belongs based on the first category; based on the frequency and the second category, a storage device suitable for storing the data object is determined.

The expressions "first", "second", "said first" or "said second" used in various embodiments of the present disclosure may modify various components regardless of order and/or importance, but these expressions do not limit the respective components. The above description is only configured for the purpose of distinguishing elements from other elements. For example, the first user equipment and the second user equipment represent different user equipment, although both are user equipment. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of the present disclosure.

When an element (e.g., a first element) is referred to as being "operably or communicatively coupled" or "connected" (operably or communicatively) to "another element (e.g., a second element) or" connected "to another element (e.g., a second element), it is understood that the element is directly connected to the other element or the element is indirectly connected to the other element via yet another element (e.g., a third element). In contrast, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (a second element), no element (e.g., a third element) is interposed therebetween.

The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims

1. A method for storing a data object, the method comprising:

determining a first category to which a data object corresponding to a data object request belongs and a frequency at which the data object is accessed in a first future time period based on data object characteristic data carried in the data object request;

determining a second category to which a storage device with service performance meeting the service performance requirement of the data object belongs based on the first category;

based on the frequency and the second category, a storage device suitable for storing the data object is determined.

2. The method according to claim 1, wherein the determining, based on data object characteristic data carried in a data object request, a first class to which a data object corresponding to the data object request belongs and a frequency with which the data object is accessed in a first future time period includes:

training a data object prediction model to be trained based on the category marking data of the data object sample and the frequency marking data of the data object sample accessed in a second future time period;

predicting the first class and the frequency based on the data object feature data through the trained data object prediction model.

3. The method of claim 2, wherein prior to training the data object prediction model to be trained based on the class label data for the data object samples and the frequency label data with which the data object samples are accessed over the second future time period, the method further comprises:

determining the category label data based on the frequency with which the data object sample is accessed within a first historical time period, the source of generation of the data object sample, the time of creation of the data object sample, and the type data identified by the user for the data object sample;

determining the frequency marking data based on distribution data of the frequency of the data object samples accessed in a second historical time period, wherein the second historical time period has a corresponding relationship with the second future time period.

4. The method according to claim 1, wherein the determining, based on data object characteristic data carried in a data object request, a first class to which a data object corresponding to the data object request belongs and a frequency with which the data object is accessed in a first future time period includes:

if the data object characteristic data is determined to be similar to the predetermined characteristic data of the data object sample belonging to the same category, determining that the same category is the first category to which the data object belongs, and determining the frequency of the data object being accessed in the first future time period based on the distribution data of the frequency of the data object sample of the same category being accessed in a third history time period, wherein the third history time period has a corresponding relationship with the first future time period.

5. The method of claim 4, wherein the data object characteristic data comprises at least one of:

a data object generation source, a data object creation time, type data identified by a user for the data object, a frequency with which the data object is accessed within a fourth historical period of time,

before the determining that the same class is the first class to which the data object belongs, the method further comprises:

and if the data object generation source, the data object creation time and the type data are determined to be respectively the same as the generation source of the data object sample, the creation time of the data object sample and the type data identified by the user for the data object sample in the feature data, and the frequency of the data object being accessed in the fourth historical time period and the frequency of the data object sample in the feature data being accessed in the fourth historical time period are within a preset range, determining that the data object feature data is similar to the predetermined feature data of the data object sample belonging to the same category.

6. The method of claim 1, wherein determining, based on the first class, a second class to which a storage device having a service performance that meets the service performance requirement of the data object belongs comprises:

training a service performance prediction model to be trained on the basis of service performance marking data when data object samples of the same type are stored on storage equipment of the same type;

obtaining service performance prediction data when the data objects belonging to the first category are stored on the storage equipment of each category based on the first category through the trained service performance prediction model;

and determining a second category to which the storage device with the service performance meeting the service performance requirement of the data object belongs based on the service performance prediction data.

7. The method of claim 1, wherein determining, based on the first class, a second class to which a storage device having a service performance that meets the service performance requirement of the data object belongs comprises:

obtaining service performance prediction data when the data object belonging to the first class is deposited on the storage device of the same class based on the service performance data when the data object sample belonging to the first class is deposited on the storage device of the same class in a fifth historical time period;

8. The method according to claim 6 or 7, wherein the data object characteristic data further comprises service performance requirement data of the data object for the storage device,

the determining, based on the service performance prediction data, a second class to which a storage device whose service performance satisfies the service performance requirement of the data object belongs includes:

determining the class to which the storage device for which the service performance prediction data meets the service performance requirement data belongs to is the second class.

9. The method according to claim 6 or 7, wherein the service performance prediction data comprises at least one of:

the storage delay data of the storage device, the storage throughput or throughput of the storage device, and the fluctuation data of the storage delay of the storage device.

10. The method of claim 1, wherein the data object request comprises a data object creation request,

said determining a storage device suitable for storing said data object based on said frequency and said second class comprises:

calculating a deposit cost of the data object on the storage device of the second class based on the frequency and the second class through a cost function for calculating a deposit cost of the data object on a storage device of the same class;

determining a storage device suitable for storing the data object based on the storage device of the second category with the least storage cost.

11. The method of claim 10, wherein calculating the deposit cost of the data object on the second class of storage devices based on the frequency and the second class via a cost function for calculating the deposit cost of the data object on the same class of storage devices comprises:

calculating, by the cost function, based on the second class, a usage cost incurred by the data object occupying storage space in the second class of storage devices for the first future time period;

calculating, by the cost function, a deposit value resulting from the data object occupying storage space in the second class of storage devices for the first future period of time based on the frequency;

calculating, by the cost function, a deposit cost of the data object on the second class of storage devices for the first future period of time based on the usage cost and the deposit value.

12. The method of claim 10, wherein determining a storage device suitable for storing the data object based on the second category of storage devices having the least storage cost comprises:

and determining the storage equipment suitable for storing the data object based on the storage equipment of the second class with the least storage cost by adopting a load balancing principle or a load concentration principle.

13. The method of claim 1, wherein the data object request comprises a data object update request or a data object read request,

sorting the second category based on the storage cost to obtain a sorting result of the second category;

and determining the storage equipment suitable for storing the data object based on the sorting result of the second category and the third category to which the storage equipment currently storing the data object belongs.

14. The method of claim 13, wherein determining a storage device suitable for storing the data object based on the sorting result of the second category and a third category to which a storage device currently storing the data object belongs comprises:

if the third category is judged not to be the currently rotated second category in the sequencing result, determining migration benefit when the data object is migrated from the storage device currently storing the data object to the currently rotated storage device of the second category, calculating migration cost when the data object is migrated from the storage device currently storing the data object to the currently rotated storage device of the second category through a migration function for calculating migration cost when the data object is migrated from one type of storage device to another type of storage device, and based on the currently rotated storage device when determining to migrate the data object from the storage device currently storing the data object to the currently rotated storage device of the second category based on the migration cost and the migration benefit, a storage device suitable for storing the data object is determined.

15. The method of claim 14, wherein determining a migration benefit for the data object when migrating from the storage device currently storing the data object to the storage device of the second category currently rotated to comprises:

calculating, by the cost function, a storage cost of the data object on the third class of storage devices and a storage cost of the data object on the currently rotated second class of storage devices;

determining the migration benefit based on a storage cost of the data object on the third class of storage devices and a storage cost of the data object on the currently rotated second class of storage devices.

16. The method of claim 14, wherein determining a storage device suitable for storing the data object based on the currently rotated storage device of the second class comprises:

and determining the storage equipment suitable for storing the data object based on the currently rotated storage equipment of the second class by adopting a load balancing principle or a load concentration principle.

17. An apparatus for storing data objects, the apparatus comprising:

a first determining module, configured to determine, based on data object feature data carried in a data object request, a first category to which a data object corresponding to the data object request belongs and a frequency at which the data object is accessed in a first future time period;

a second determining module, configured to determine, based on the first category, a second category to which a storage device whose service performance meets the service performance requirement of the data object belongs;

a third determination module to determine a storage device suitable for storing the data object based on the frequency and the second category.

18. The apparatus of claim 17, wherein the first determining module comprises:

the first training submodule is used for training a data object prediction model to be trained on the basis of class marking data of a data object sample and frequency marking data of the data object sample accessed in a second future time period;

a first prediction sub-module, configured to predict the first category and the frequency based on the data object feature data through the trained data object prediction model.

19. The apparatus of claim 18, wherein the first training submodule is preceded by the first determining module further comprising:

a first determining sub-module, configured to determine the category labeling data based on a frequency with which the data object sample is accessed within a first historical time period, a generation source of the data object sample, a creation time of the data object sample, and type data identified by a user for the data object sample;

a second determining submodule, configured to determine the frequency labeling data based on distribution data of frequencies at which the data object samples are visited in a second historical time period, where the second historical time period and the second future time period have a corresponding relationship.

20. The apparatus of claim 17, wherein the first determining module comprises:

a third determining sub-module, configured to determine that the same class is the first class to which the data object belongs if it is determined that the data object feature data is similar to predetermined feature data of a data object sample belonging to the same class, and determine a frequency with which the data object is accessed in a first future time period based on distribution data of frequencies with which the data object sample of the same class is accessed in the third future time period, where the third future time period has a corresponding relationship with the first future time period.

21. The apparatus of claim 20, wherein the data object characteristic data comprises at least one of:

before the third determining submodule, the first determining module further includes:

a fourth determination submodule, configured to determine that the data object feature data is similar to predetermined feature data of data object samples belonging to the same category if it is determined that the data object generation source, the data object creation time, and the type data are respectively the same as the generation source of the data object sample, the creation time of the data object sample, and the type data identified by the user for the data object sample, and the frequency with which the data object is accessed in the fourth history period is within a preset range from the frequency with which the data object sample in the feature data is accessed in the fourth history period.

22. The apparatus of claim 17, wherein the second determining module comprises:

the second training submodule is used for training a service performance prediction model to be trained on the basis of service performance marking data when data object samples of the same type are stored on storage equipment of the same type;

the second prediction submodule is used for obtaining service performance prediction data when the data objects belonging to the first category are stored on the storage equipment of each category based on the first category through the trained service performance prediction model;

and the fifth determining submodule is used for determining the second category of the storage equipment of which the service performance meets the service performance requirement of the data object based on the service performance prediction data.

23. The apparatus of claim 17, wherein the second determining module comprises:

a third prediction sub-module, configured to obtain, based on service performance data when a sample of data objects belonging to the first category is stored on a storage device of the same category in a fifth historical time period, service performance prediction data when a data object belonging to the first category is stored on a storage device of the same category;

24. The apparatus according to claim 22 or 23, wherein the data object characteristic data further comprises service performance requirement data of the data object for the storage device,

the fifth determining submodule is specifically configured to:

25. The apparatus according to claim 22 or 23, wherein the service performance prediction data comprises at least one of:

26. The apparatus of claim 17, wherein the data object request comprises a data object creation request,

the third determining module includes:

a first calculation submodule, configured to calculate, based on the frequency and the second category, a storage cost of the data object on a storage device of the second category through a cost function for calculating a storage cost of the data object on a storage device of the same category;

a sixth determining sub-module, configured to determine, based on the storage device of the second category with the least storage cost, a storage device suitable for storing the data object.

27. The apparatus according to claim 26, wherein the first computation submodule is specifically configured to:

28. The apparatus according to claim 26, wherein the sixth determining submodule is specifically configured to:

29. The apparatus of claim 17, wherein the data object request comprises a data object update request or a data object read request,

the third determining module includes:

a second calculating sub-module, configured to calculate, based on the frequency and the second class, a storage cost of the data object on a storage device of the second class through a cost function for calculating a storage cost of the data object on a storage device of the same class;

the sorting submodule is used for sorting the second category based on the storage cost so as to obtain a sorting result of the second category;

and the seventh determining submodule is used for determining the storage equipment suitable for storing the data object based on the sorting result of the second category and the third category to which the storage equipment currently storing the data object belongs.

30. The apparatus of claim 29, wherein the seventh determination submodule comprises:

a first determining unit, configured to determine, if it is determined that the third category is not the currently rotated second category in the sorting result, migration benefit when the data object is migrated from a storage device currently storing the data object to the currently rotated second category of storage devices, and calculate, through a migration function for calculating migration cost when the data object is migrated from one category of storage devices to another category of storage devices, migration cost when the data object is migrated from a storage device currently storing the data object to the currently rotated second category of storage devices, and when determining, based on the migration cost and the migration benefit, the data object is migrated from a storage device currently storing the data object to the currently rotated second category of storage devices, based on the currently rotated second category of storage devices, a storage device suitable for storing the data object is determined.

31. The apparatus according to claim 30, wherein the first determining unit is specifically configured to:

32. The apparatus according to claim 30, wherein the first determining unit is specifically configured to:

33. An electronic device, comprising:

one or more processors;

a computer readable medium configured to store one or more programs,

when executed by the one or more processors, cause the one or more processors to implement a method of depositing a data object as claimed in any one of claims 1 to 16.

34. A computer-readable medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out a method of storing a data object according to any one of claims 1 to 16.