CN107783734B - Resource allocation method, device and terminal based on super-fusion storage system - Google Patents

Resource allocation method, device and terminal based on super-fusion storage system Download PDF

Info

Publication number
CN107783734B
CN107783734B CN201711147145.4A CN201711147145A CN107783734B CN 107783734 B CN107783734 B CN 107783734B CN 201711147145 A CN201711147145 A CN 201711147145A CN 107783734 B CN107783734 B CN 107783734B
Authority
CN
China
Prior art keywords
storage
data
stored
storage resource
resource
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711147145.4A
Other languages
Chinese (zh)
Other versions
CN107783734A (en
Inventor
彭江
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Lenovo Super Fusion Technology Co Ltd
Original Assignee
Beijing Lenovo Super Fusion Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Lenovo Super Fusion Technology Co Ltd filed Critical Beijing Lenovo Super Fusion Technology Co Ltd
Priority to CN201711147145.4A priority Critical patent/CN107783734B/en
Publication of CN107783734A publication Critical patent/CN107783734A/en
Application granted granted Critical
Publication of CN107783734B publication Critical patent/CN107783734B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • G06F3/0631Configuration or reconfiguration of storage systems by allocating resources to storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The invention discloses a resource allocation method, a resource allocation device and a resource allocation terminal based on a super-fusion storage system, relates to the technical field of storage, and aims to solve the problem that in the prior art, when storage resources are allocated for data, the allocation mode is not reasonable enough, so that the allocation efficiency of the storage resources in the super-fusion storage system is low. The method of the invention comprises the following steps: calculating the storage requirement of the data to be stored according to the storage information of the data to be stored; acquiring a storage resource list; according to the storage requirements, filtering the corresponding storage resources in the storage resource list by using preset rules and classification labels of the storage resources to obtain an available storage resource list, wherein the classification labels are used for classifying the storage resources according to different dimensions, and at least comprise one or more of position labels and performance labels; and allocating storage resources corresponding to the available storage resource list for the data to be stored. The method is suitable for being applied to the allocation of the storage resources in the super-fusion storage system.

Description

Resource allocation method, device and terminal based on super-fusion storage system
Technical Field
The invention relates to the technical field of storage, in particular to a resource allocation method, a resource allocation device and a resource allocation terminal based on a super-fusion storage system.
Background
With the continuous progress of network technology and the continuous increase of data volume in the network, the data volume in the ultra-converged storage system is gradually increased. The system aims to realize the integration of resources such as computation, network, storage, virtualization servers and the like in the system so as to achieve the effects of aggregating multiple nodes through the network, realizing modularized seamless transverse expansion and forming a uniform resource pool. In the face of the increasing amount of data in a super-converged storage system, the storage resources used to store the data tend to be limited. Therefore, how to reasonably allocate the storage resources is very important for data storage in the super-fusion storage system.
Generally, when data in a super-fusion storage system is stored, storage resources are often randomly allocated to the data to be stored. However, in practical applications, the random allocation method cannot effectively achieve reasonable utilization of the storage resources. Meanwhile, since the storage resources of different attributes and locations will affect the access efficiency of the data stored therein, for example: the access speed of data in a storage resource of a type of Solid State Disk (SSD) is faster than that of a storage resource of a type of Hard Disk Drive (HDD), so that if the storage resource is randomly allocated, it is likely that data with a higher access rate is stored in a storage resource with a slower response speed, which is obviously not suitable. Therefore, in the prior art, when the storage resources are allocated to the data, the allocation mode is not reasonable enough, so that the allocation efficiency of the storage resources in the super-fusion storage system is low.
Disclosure of Invention
In view of the above problems, the present invention provides a resource allocation method, device and terminal based on a super-fusion storage system, and mainly aims to implement reasonable allocation of storage resources, thereby improving the allocation efficiency of storage resources.
In order to solve the above technical problem, in a first aspect, the present invention provides a resource allocation method based on a super-fusion storage system, where the method includes:
calculating the storage requirement of the data to be stored according to the storage information of the data to be stored;
acquiring a storage resource list, wherein the storage resource list is used for recording storage resources in the super-fusion storage system;
according to the storage demand, filtering the corresponding storage resources in the storage resource list by using a preset rule and the classification labels of the storage resources to obtain an available storage resource list, wherein the classification labels are used for classifying the storage resources according to different dimensions, and the classification labels at least comprise one or more of position labels and performance labels;
and allocating storage resources corresponding to the available storage resource list to the data to be stored.
Optionally, before the obtaining the storage resource list, the method further includes:
and adding a classification label corresponding to the resource attribute for the storage resource according to the resource attribute of the storage resource, wherein the resource attribute of the storage resource comprises a hardware attribute and a position attribute.
Optionally, the adding, according to the resource attribute of the storage resource, the classification label corresponding to the resource attribute to the storage resource includes:
adding a performance label corresponding to the hardware attribute for the storage resource according to the hardware attribute of the storage resource, wherein the performance label comprises an HDD label or an SSD label; and/or
And adding a position label corresponding to the position attribute for the storage resource according to the position information of the storage resource in the super-fusion storage system, wherein the position label comprises one or more of a rack label, a host label and an area label.
Optionally, the storage requirement of the data to be stored at least includes one or more of a performance requirement, a location requirement, and a load requirement;
the calculating the storage requirement of the data to be stored according to the storage information of the data to be stored comprises:
determining the performance requirement of the data to be stored according to the access requirement of the data to be stored; and/or
Determining the position requirement of the data to be stored according to the address information of the writing end of the data to be stored; and/or
And determining the load demand of the data to be stored according to the load state of the storage resource.
Optionally, the filtering, according to the storage requirement of the data to be stored, the storage resource corresponding to the storage resource list by using a preset rule and the classification label of the storage resource includes:
filtering the corresponding storage resources in the storage resource list according to the performance requirements of the data to be stored and the performance labels; and/or
Filtering the corresponding storage resources in the storage resource list according to the position requirement of the data to be stored and the position label; and/or
And filtering the corresponding storage resources in the storage resource list according to the load requirements of the data to be stored and the load states of the storage resources.
Optionally, the filtering, according to the performance requirement of the data to be stored and the performance label, the storage resource corresponding to the storage resource list includes:
extracting the performance requirement of the data to be stored, wherein the performance requirement is used for the appointed storage resource to be a Solid State Disk (SSD) or a mechanical hard disk (HDD);
when the performance requirement is that the specified storage resource is a Solid State Disk (SSD), the storage resource with an SSD label in the storage list is obtained;
and when the performance requirement is that the specified storage resource is a mechanical hard disk (HDD), acquiring the storage resource with the HDD label in the storage list.
Optionally, the filtering, according to the location requirement of the data to be stored and the location tag, the storage resource corresponding to the storage resource list includes:
extracting the position requirement of the data to be stored, wherein the position requirement is used for specifying one or more of storage resources at different rack positions, storage resources at different host positions and storage resources at different area positions;
when the position requirement specifies storage resources at different rack positions, filtering the corresponding storage resources in the storage resource list through rack labels so as to obtain the storage resources at different rack positions; and/or
When the position requirement specifies storage resources at different host positions, filtering the corresponding storage resources in the storage resource list through a host label so as to obtain the storage resources at different host positions; and/or
And when the position requirement specifies the storage resources at different area positions, filtering the corresponding storage resources in the storage resource list through the area labels so as to obtain the storage resources at different area positions.
Optionally, the filtering, according to the load demand of the data to be stored and the load state of the storage resource, the storage resource corresponding to the storage resource list includes:
extracting a load threshold value in the load demand;
judging whether storage resources with loads exceeding the load threshold exist or not;
and if so, acquiring the storage resources with the load smaller than the load threshold value in the storage resources.
Optionally, the allocating, to the data to be stored, a storage resource corresponding to the available storage resource list includes:
acquiring address information of a writing end of the data to be stored;
according to the address information of the write-in terminal, determining a storage resource which is closest to the address of the write-in terminal from the address information of the storage resource in the available storage resource list;
and saving the data to be stored to the closest storage resource.
Optionally, the storing the data to be stored in the closest storage resource includes:
and selecting the same number of storage resources from the available storage resource list according to the specified backup number of the data to be stored, and storing the data to be stored into the selected storage resources.
Optionally, the selecting, according to the backup number specified by the data to be stored, storage resources of the same number from the available storage resource list includes:
judging whether the backup quantity is larger than the quantity of the storage resources in the available storage resource list or not;
and if so, selecting the storage resources with the corresponding quantity in the storage resource list to store the redundant quantity of backups.
In a second aspect, the present invention further provides a resource allocation apparatus based on a super-fusion storage system, where the apparatus includes:
the computing unit is used for computing the storage requirement of the data to be stored according to the storage information of the data to be stored;
the system comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring a storage resource list which is used for recording storage resources in the super-fusion storage system;
the filtering unit is used for filtering the corresponding storage resources in the storage resource list acquired by the acquiring unit by using preset rules and the classification labels of the storage resources according to the storage requirements calculated by the calculating unit to obtain an available storage resource list, wherein the classification labels are used for classifying the storage resources according to different dimensions, and the classification labels at least comprise one or more of position labels and performance labels;
and the distribution unit is used for distributing the storage resources corresponding to the available storage resource list obtained after the filtering unit filters the data to be stored.
Optionally, the apparatus further comprises:
and the adding unit is used for adding a classification label corresponding to the resource attribute for the storage resource according to the resource attribute of the storage resource, wherein the resource attribute of the storage resource comprises a hardware attribute and a position attribute.
Optionally, the adding unit is specifically configured to add, according to the hardware attribute of the storage resource, a performance tag corresponding to the hardware attribute to the storage resource, where the performance tag includes an HDD tag or an SSD tag; and/or
The adding unit is specifically configured to add, to the storage resource, a location tag corresponding to the location attribute according to location information of the storage resource in the super-fusion storage system, where the location tag includes one or more of a rack tag, a host tag, and an area tag.
Optionally, the storage requirement of the data to be stored at least includes one or more of a performance requirement, a location requirement, and a load requirement;
the calculation unit includes:
the first determining module is used for determining the performance requirement of the data to be stored according to the access requirement of the data to be stored; and/or
The second determining module is used for determining the position requirement of the data to be stored according to the address information of the writing end of the data to be stored; and/or
And the third determining module is used for determining the load demand of the data to be stored according to the load state of the storage resource.
Optionally, the filtering unit includes:
the first filtering module is used for filtering the corresponding storage resources in the storage resource list according to the performance requirements of the data to be stored and the performance labels; and/or
The second filtering module is used for filtering the corresponding storage resources in the storage resource list according to the position requirements of the data to be stored and the position labels; and/or
And the third filtering module is used for filtering the corresponding storage resource in the storage resource list according to the load requirement of the data to be stored and the load state of the storage resource.
Optionally, the first filtering module includes:
the extraction submodule is used for extracting the performance requirement of the data to be stored, and the performance requirement is used for the appointed storage resource to be a Solid State Disk (SSD) or a mechanical hard disk (HDD);
the obtaining submodule is used for obtaining the storage resource with the SSD label in the storage list when the performance requirement extracted by the extracting submodule is that the specified storage resource is the SSD;
the obtaining submodule is further configured to obtain the storage resource with the HDD tag in the storage list when the performance requirement extracted by the extracting submodule is that the specified storage resource is a mechanical hard disk HDD.
Optionally, the second filtering module includes:
the extraction submodule is used for extracting the position requirement of the data to be stored, and the position requirement is used for appointing one or more of storage resources at different rack positions, storage resources at different host positions and storage resources at different area positions;
the first filtering submodule is used for filtering the corresponding storage resources in the storage resource list through the rack labels when the position requirements extracted by the extracting submodule specify the storage resources at different rack positions so as to obtain the storage resources at different rack positions; and/or
The second filtering submodule is used for filtering the corresponding storage resources in the storage resource list through the host tags when the position requirements extracted by the extracting submodule specify the storage resources at different host positions so as to obtain the storage resources at different host positions; and/or
And the third filtering submodule is used for filtering the corresponding storage resources in the storage resource list through the area tags when the position requirements extracted by the extracting submodule specify the storage resources at different area positions, so as to obtain the storage resources at different area positions.
Optionally, the third filtering module includes:
the extraction submodule is used for extracting a load threshold value in the load demand;
the judgment submodule is used for judging whether storage resources with loads exceeding the load threshold value extracted by the extraction submodule exist or not;
and the obtaining submodule is used for obtaining the storage resources with the load smaller than the load threshold value in the storage resources if the judging submodule judges that the storage resources with the load larger than the load threshold value extracted by the extracting submodule exist.
Optionally, the allocation unit includes:
the acquisition module is used for acquiring the address information of the writing end of the data to be stored;
a determining module, configured to determine, according to the address information of the write-in end acquired by the acquiring module, a storage resource closest to the address of the write-in end from the address information of the storage resource in the available storage resource list;
and the storage module is used for storing the data to be stored into the closest storage resource determined by the determination module.
Optionally, the saving module is specifically configured to select, according to the specified backup quantity of the data to be stored, the same quantity of storage resources from the available storage resource list, and save the data to be stored in the selected storage resources.
Optionally, the saving module includes:
the judging submodule is used for judging whether the backup quantity is larger than the quantity of the storage resources in the available storage resource list or not;
and the saving sub-module is used for selecting the storage resources with the corresponding quantity from the storage resource list to save the redundant quantity of the backups if the judging sub-module judges that the backup quantity is greater than the quantity of the storage resources in the available storage resource list.
In order to achieve the above object, according to a third aspect of the present invention, there is provided a resource allocation terminal based on a super-fusion storage system, the terminal including a processor and a storage medium, wherein the processor and the storage medium are capable of executing the above resource allocation method based on the super-fusion storage system when performing allocation of storage resources.
By means of the technical scheme, the resource allocation method, the resource allocation device and the resource allocation terminal based on the super-fusion storage system provided by the invention have the advantages that for the problem that the allocation mode is not reasonable enough when the storage resources are allocated for the data in the prior art, so that the allocation efficiency of the storage resources under the super-fusion storage system is low, the storage resources in the system are filtered by the preset rules and the classification labels of the storage resources according to the storage requirements, the effect of allocating the corresponding storage resources for the data to be stored according to different storage requirements is realized, and further, the function of allocating different storage resources for different data to be stored is realized. Compared with the prior art, the method and the device can filter the storage resources according to different classification labels when the data to be stored is received, and can achieve the effect of screening the storage resources. In addition, the storage requirement of the data to be stored is calculated according to the storage information of the data to be stored, and then the storage resources are filtered in a targeted manner by utilizing the classification tags according to the requirement, so that the effect of distributing different storage resources according to different storage requirements is realized, the condition that the storage resources are distributed at random when the data to be stored is to be stored in the prior art to cause unreasonable distribution is changed, and the distribution efficiency of the storage resources in the super-fusion storage system is improved.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flowchart illustrating a resource allocation method based on a hyper-converged storage system according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating another resource allocation method based on a super-converged storage system according to an embodiment of the present invention;
FIG. 3 is a block diagram illustrating a resource allocation apparatus based on a super-fusion storage system according to an embodiment of the present invention;
fig. 4 is a block diagram illustrating another resource allocation apparatus based on a super-fusion storage system according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
In order to implement reasonable allocation of storage resources and improve allocation efficiency of storage resources, an embodiment of the present invention provides a resource allocation method based on a super-fusion storage system, as shown in fig. 1, where the method includes:
101. and calculating the storage requirement of the data to be stored according to the storage information of the data to be stored.
Generally, the reason why the data is stored is that the data can be acquired at any time as needed. Therefore, different data requires different storage resources during storage according to the data acquisition times, the importance of the data, and other reasons, and users may have different requirements for the storage resources during storage. Therefore, the storage information of the data to be stored in this step includes information of different contents such as the attribute of the data itself and the request of the user at the time of storage.
Because the storage information of different data is different, the storage requirement of what kind of storage resource is needed when the corresponding data to be stored is stored can be calculated according to the specific content in the storage information of the data to be stored. The storage requirement in this step may be understood as a requirement for determining what kind of storage resource needs to be allocated when allocating storage resource for the data according to the storage information of the data when the data is stored.
Therefore, when data enters the super-fusion storage system, because different data contains different storage information, according to the method in the step, the storage information of the data to be stored is calculated, and the storage requirement of the data to be stored is determined according to the calculation result, specifically, the storage requirement of what form of storage needs to be performed when the data is stored, or the storage requirement of what storage resource needs to be performed when the data is stored.
It should be noted that, in this step, the storage information of the data to be stored may be determined according to the needs of the user when the data to be stored is subjected to a storage operation, may also be determined according to the attribute of the data itself, or includes the above two cases, and may specifically be determined according to a real-time situation, which is not limited herein.
For example, when the data is obtained by calculating the storage information of the data to be stored, and the data is common data that needs to be accessed multiple times, it can be determined that the data needs a faster access speed in the face of subsequent multiple accesses. Therefore, the data can be determined to need to be stored in the storage resource with the higher response speed, and the storage requirement of the data can be further determined to be the storage resource with the higher response speed.
102. And acquiring a storage resource list.
In general, for any storage system, in order to specify storage resources, the storage resources are usually integrated in a list so as to facilitate management of the storage resources therein, and similarly, for a super-fusion storage system, since a large number of storage resources are included therein, the storage resources also need to be integrated in a list to form a storage resource list so as to manage the storage resources in the system, where the storage resource list in this step may be understood as a list for recording the storage resources in the super-fusion storage system. In addition, the storage resource list may include one or more of a plurality of kinds of information such as a serial number, a category, and a location of the storage resource, in addition to the name of the storage resource.
Therefore, according to the method in this step, the storage resource list in the super-fusion system can be obtained so as to perform the operation in the subsequent step. Specifically, the manner of obtaining the storage resource list may be obtained through a system record in the super fusion system, or may be obtained through other manners, which is not limited herein and may be selected as needed.
With reference to the method in this step, when the storage resource list includes the name and the serial number of the storage resource and the hardware type of the storage resource, the obtained storage resource list may be as follows, specifically as shown in table 1:
TABLE 1
1 Storage resource A SSD
2 Storage resource B HDD
3 Storage resource C HDD
4 Storage resource D SSD
5 Storage resource E HDD
6 Storage resource F HDD
7 Storage resource G SSD
8 Storage resource H SSD
According to the storage resource list shown in table 1, numbers "1 to 8" are serial numbers of storage resources, and "storage resource a to storage resource H" are names of storage resources, and "SSD" and "HDD" represent hardware types of the storage resources, respectively.
103. And filtering the corresponding storage resources in the storage resource list by using a preset rule and the classification labels of the storage resources according to the storage requirements to obtain an available storage resource list.
Since there are a large number of storage resources in the super-fusion storage system, different types of storage resources are different, and the hardware attributes of the storage resources may include "SSD" and "HDD", and when the storage resources are distinguished by other dimensions, for example, when the storage resources are distinguished by using a location dimension, the storage resources may include other different areas or different hosts, in this step, the storage resources of different types need to be filtered by using preset rules and classification tags according to the storage requirements calculated in step 101, so as to obtain storage resources meeting different storage requirements.
It should be noted that, in this step, the classification label may be understood as a label that classifies the storage resource according to different dimensions, and the classification label may include at least one of different labels such as a location label and a performance label, which is not limited herein and is determined according to an actual situation.
For example, when the calculated storage requirement of the data to be stored is that the hardware type of the specified storage resource is SSD, according to the method in this step, the storage resources that are not the SSD type in the storage resource list need to be filtered, that is, the storage resources that are HDD type need to be filtered, the storage resources that are SSD type after filtering are obtained, and then the available storage resource list is obtained. Specifically, with reference to table 1 in step 102, according to the method in this step, an available storage resource list can be obtained, which is specifically shown in table 2:
TABLE 2
1 Storage resource A SSD
4 Storage resource D SSD
7 Storage resource G SSD
8 Storage resource H SSD
104. And allocating storage resources corresponding to the available storage resource list for the data to be stored.
After step 103, a list of available storage resources may be obtained, and therefore, in this step, the corresponding storage resources in the list may be allocated to the data to be stored, so that the data to be stored is stored.
Specifically, in combination with the example according to step 103, the storage resource a, the storage resource D, the storage resource G, and the storage resource H in table 2 may be allocated to the data to be stored, so as to meet the storage requirement that the data to be stored needs to be stored in the SSD type storage resource. The data to be stored may be stored in the storage resource selected from the four storage resources, or may be selected in other manners, which is not limited herein.
According to the resource allocation method based on the super-fusion storage system, provided by the embodiment of the invention, for the problem that the allocation mode is not reasonable enough when the storage resources are allocated to the data in the prior art, so that the allocation efficiency of the storage resources under the super-fusion storage system is low, the storage resources in the system are filtered by using the preset rules and the classification labels of the storage resources according to the storage requirements, the effect of allocating the corresponding storage resources to the data to be stored according to different storage requirements is realized, and further, the function of allocating different storage resources to different data to be stored is realized. Compared with the prior art, the method and the device can filter the storage resources according to different classification labels when the data to be stored is received, and can achieve the effect of screening the storage resources. In addition, the storage requirement of the data to be stored is calculated according to the storage information of the data to be stored, and then the storage resources are filtered in a targeted manner by utilizing the classification tags according to the requirement, so that the effect of distributing different storage resources according to different storage requirements is realized, the condition that the storage resources are distributed at random when the data to be stored is to be stored in the prior art to cause unreasonable distribution is changed, and the distribution efficiency of the storage resources in the super-fusion storage system is improved.
Further, as a refinement and an extension of the embodiment shown in fig. 1, an embodiment of the present invention further provides another resource allocation method based on the super-fusion storage system, as shown in fig. 2. Specifically, the implementation process can be as follows:
201. and adding a classification label corresponding to the resource attribute for the storage resource according to the resource attribute of the storage resource.
Because a large amount of storage resources exist in the super-fusion storage system, in order to manage the storage resources, after the storage resources are accessed to the system, the classification labels corresponding to the resource attributes need to be added according to the resource attributes of the storage resources. The resource attribute described in the embodiment of the present invention may be understood as a distinguishing attribute used for characterizing characteristics of the storage resource, and may include one or more of other attributes such as a hardware attribute and a location attribute.
Since the resource attributes are different, the corresponding added classification labels are also different. For example, when the tag is added according to the hardware attribute of the storage resource, a performance tag corresponding to the hardware attribute, such as HDD or SSD, is added; when the tag is added according to the location attribute of the resource, the tag is added to a location tag corresponding to the location attribute, such as one or more of a rack tag, a host tag, and an area tag. Meanwhile, before the classification label is added, the resource attribute of the storage resource can be acquired by testing the storage resource by using the prior art, or acquired according to the content in the storage resource detail table in the system log, and the acquisition mode is not limited here and can be selected as required.
When adding the tag, a performance tag and a location tag of the above tags may be added to the storage resource, or only one of the performance tag and the location tag may be added. In addition, when the classification tag is added to the storage resource, interface tags corresponding to different types of interfaces, such as PCI-E, NVME, and the like, may be further added according to the interface type of the storage resource, which is not limited herein, and the type and number of the classification tag may be selected according to actual needs.
According to the method in this step, specific application scenarios are combined, for example, as follows:
when a new storage resource A is accessed into the super-fusion storage system, testing and analyzing the storage resource A, acquiring the resource attribute of the storage resource A according to the analysis result, and adding a classification label according to the requirement, wherein a performance label 'HDD' corresponding to a mechanical hard disk is added to the storage resource A according to the hardware attribute of the storage resource A because the hardware attribute of the storage resource A is a conventional mechanical hard disk, and when a position label is also required to be added to the storage resource A, a position label of 'rack position 1-host position a-area position XI' corresponding to the position is added to the storage resource A according to the storage position because the storage resource A is stored in an area XI in a host a on the rack 1.
According to the method in the step, the corresponding classification labels are added to the storage resources in the super-fusion storage system according to the resource attributes of the storage resources, so that the storage resources can be classified, and a foundation is laid for reasonable distribution of the storage resources.
202. And calculating the storage requirement of the data to be stored according to the storage information of the data to be stored.
The storage requirement according to the embodiment of the present invention at least includes one or more of a performance requirement, a location requirement, and a load requirement. Specifically, the description of the storage information of the data to be stored and the storage requirement is completely the same as that in step 101 of the foregoing embodiment, and is not repeated herein.
Meanwhile, according to the method in this step, since the storage requirements include three types of performance requirements, location requirements, and load requirements, there are calculation methods corresponding to the three types of storage requirements, specifically, the method includes the following three aspects:
in a first aspect, the performance requirement of the data to be stored is determined according to the access requirement of the data to be stored. Because different data to be stored have different access requirements, different performance requirements of the data during storage need to be determined according to the different access requirements. For example, when the data to be stored is data that needs to be accessed for multiple times, since a large number of access times need to be faced after storage, in order to improve access efficiency, the data to be stored needs to be stored in a storage resource with a fast response speed, since different hardware attributes in the storage resource determine different performances, for example, the response speed of a conventional mechanical hard disk HDD is slower than the response speed of a solid state disk SSD, when the access demand according to the resource to be stored is a case with a fast access speed, the storage resource with the SSD type performance tag can be determined to be selected as the performance demand of the data to be stored during storage according to the hardware attributes of the storage resource.
In a second aspect, the location requirement of the data to be stored is determined according to the address information of the writing end of the data to be stored. Since the transmission speed of the data to be stored at different positions is different, the affinity of the data needs to be considered, wherein the affinity of the data can be understood as the closer the position where the data is stored is to the access end of the visitor, the shorter the data transmission path is, the faster the transmission speed is, and thus the affinity of the data is better, and in general, the write end of the data is the access end of the data. Thus, from this perspective, determining the write side address of the data actually determines the access side address of the data, so that the storage location of the data to be stored can be determined based on the address. Certainly, from another perspective, in the storage process of data, the storage resource closer to the address of the write-in end is the faster the stored data is, and in order to ensure that the data is not abnormal in the transmission process and further causes data storage abnormality, the data to be stored can be stored through the storage resource of the storage position closest to the write-in end, so that the security of the data is ensured. It should be noted that, the requirement for determining the position of the data to be stored according to the address of the write end of the data to be stored is only one of the actual determination manners, and may also be determined by other manners, which is not limited herein. Therefore, by analyzing the address of the writing end of the data to be stored, the storage resource of the position required by the data to be stored can be identified, and the position requirement of the data to be stored is determined according to the storage position required by the data to be stored.
And in a third aspect, determining the load requirement of the data to be stored according to the load state of the storage resource. In the super-fusion storage system, a large number of storage resources exist, and the load states of the storage resources are different, so that a load requirement needs to be determined to realize load balancing among the storage resources based on different load states. The load demand according to the embodiment of the present invention is actually a load threshold determined according to a load state of a storage resource, where the threshold may be a fixed value set manually or a floating value calculated each time the super-fusion storage system allocates a storage resource, and specifically, the setting of the load threshold may be selected according to actual needs, which is not limited herein, and the determination of the load demand may be determined by combining the load state of the storage resource in the current super-fusion storage system and a load balancing demand. Therefore, in the method described in this step, the load requirement can be determined by the load state of the storage resource, so as to achieve the purpose of balancing the load of the storage resource. In addition, the effect of providing storage resources within a certain load range for the data to be stored in the data storage process can be realized according to the determination of the load requirement.
According to the method in the step, for different storage information in the data to be stored, different storage requirements are determined by using different determination modes, the accuracy of the determination result of the storage requirements can be ensured, and then, the accuracy of the subsequent filtering result is guaranteed, so that the accuracy of the allocation of the storage resources is ensured.
203. And acquiring a storage resource list.
Specifically, the storage resource list and the obtaining method thereof according to the embodiment of the present invention are the same as those described in step 102 in the foregoing embodiment, and are not described herein again.
204. And filtering the corresponding storage resources in the storage resource list by using a preset rule and the classification labels of the storage resources according to the storage requirements to obtain an available storage resource list.
According to the calculation in step 202, it can be determined that the types of the storage requirements are different, and therefore, the filtering manner for the storage resources in this step is different according to different storage requirements.
In a first aspect, when the storage requirement is a performance requirement, filtering the corresponding storage resource in the storage resource list through a performance tag. Specifically, the performance requirement of the data to be stored may be extracted first, and then the performance tag of the storage resource is used to filter the storage resource in the storage resource table according to the performance requirement, so as to obtain the storage resource with the tag corresponding to the performance requirement. The performance requirement may be that the specified storage resource is any one of a Solid State Disk (SSD) or a mechanical hard disk (HDD).
Therefore, based on two situations in the performance requirements, when the performance requirement is that the specified storage resource is a Solid State Disk (SSD), filtering is performed in the storage resource list, and the storage resource with the SSD label is obtained; and when the storage resource with the performance requirement as the specified is the mechanical hard disk (HDD), filtering in the storage resource list and acquiring the storage resource with the HDD label.
In a second aspect, when the storage requirement is a location requirement, filtering the corresponding storage resource in the storage resource list through a location tag. Specifically, the location requirement of the data to be stored may be extracted first, and then the storage resource meeting the location requirement is queried by using the location tag according to the specific location requirement. The resource meeting the position requirement can be one or more of storage resources of appointed different rack positions, storage resources of different host positions and storage resources of different area positions. Specifically, several hierarchically positioned storage resources are required, which are determined according to the actual situation.
Therefore, based on various conditions of different position requirements, when the position requirements are that the data to be stored need to be stored in different rack positions, the corresponding storage resources in the storage resource list need to be filtered through the rack labels, so that the storage resources in different rack positions are obtained for selection of the data to be stored; when the position requirement is that the data to be stored needs to be stored in different host positions, filtering is carried out through host labels of the storage resources to obtain the storage resources in different host positions for selection of the resources to be stored; and when the position requirement is that the data to be stored needs to be stored in different area positions, filtering is performed through the area tags of the storage resources to obtain the storage resources in different host positions for selection of the resources to be stored. It should be noted that there may be a hierarchical relationship among the rack positions, the host positions, and the region positions in this step, that is, different hosts are disposed under the rack, and different regions are disposed under the host, so that the filtering method described in this aspect may also integrate filtering methods of different rack tags, host tags, and region tags, so as to meet the situation when the storage requirement of the data to be stored is different regions in different hosts in different racks.
In a third aspect, when the storage demand is a load demand, filtering the storage resource corresponding to the storage resource list according to a load state of the storage resource. The method of the invention specifically comprises the following steps: firstly, extracting a load threshold value from a load demand, then judging whether the storage resources in the storage resource list have the storage resources exceeding the load threshold value by using the load threshold value, and removing the part of the storage resources from the list and filtering out the storage resources with the load less than the load threshold value when the storage resources exceeding the load threshold value are determined to exist. The load threshold in this step may be determined according to any one of multiple manners, such as the current read-write data amount of the storage resource, the capacity fraction of the storage resource, and the current access number in the access queue of the storage resource, and may be selected as needed without limitation.
It should be noted that the methods of the three aspects described in this step may not exist in isolation in actual operation, and it is likely that the specific filtering methods described in this step are integrated and used together due to the existence of multiple requirements of storage requirements.
Therefore, based on the method in this step, taking a specific application scenario as an example, the following is specific:
when the data 1 to be stored needs to be stored in the storage resource of the HDD type, and needs to be stored in different areas under different hosts in different rack locations to ensure the security of data storage, and the data needs to be stored in a storage resource whose load status is that the storage resource capacity accounts for less than 20%, according to the method described in this step, the specific manner of filtering may be as follows:
when the acquired storage resource list according to the method in step 203 is shown in table 3:
TABLE 3
Serial number Name (R) Performance of Rack Main unit Region(s) Load(s)
1 Storage resource A HDD 1 a V 20%
2 Storage resource B SSD 1 b V 33%
3 Storage resource C SSD 2 a III 15%
4 Storage resource D HDD 2 c II 44%
5 Storage resource E SSD 3 f I 23%
6 Storage resource F HDD 4 g XI 34%
7 Storage resource G SSD 5 a VI 15%
8 Storage resource H SSD 5 c VII 8%
Table 3 includes a plurality of contents, such as the name, serial number, performance tag, rack tag, host tag, area tag, and load status of the storage resource, and corresponds to storage resource a to storage resource H, respectively.
According to the method in this step, firstly, table 3 is filtered through the performance tag, and since the performance requirement of the data to be stored is stored in the SSD type storage resource, the list obtained after filtering is shown in table 4, specifically as follows:
TABLE 4
Serial number Name (R) Performance of Rack Main unit Region(s) Load(s)
2 Storage resource B SSD 1 b V 33%
3 Storage resource C SSD 2 a III 15%
5 Storage resource E SSD 3 f I 23%
7 Storage resource G SSD 5 a VI 15%
8 Storage resource H SSD 5 c VII 8%
Then, the table 4 is filtered according to the position tags, and since the data to be stored requires to be stored in different areas in different hosts under different racks, the filtered list is shown in table 5, which is specifically as follows:
TABLE 5
Serial number Name (R) Performance of Rack Main unit Region(s) Load(s)
2 Storage resource B SSD 1 b V 33%
3 Storage resource C SSD 2 a III 15%
5 Storage resource E SSD 3 f I 23%
8 Storage resource H SSD 5 c VII 8%
Finally, table 5 is filtered according to the load status, and since the data to be stored needs to be stored in the storage resource with the capacity ratio smaller than 20%, it can be determined that the load threshold in the load demand is 20%, and the storage resource exceeding 20% will be filtered out, so the filtered list can be as shown in table 6, specifically as follows:
TABLE 6
Serial number Name (R) Performance of Rack Main unit Region(s) Load(s)
3 Storage resource C SSD 2 a III 15%
8 Storing assetsSource H SSD 5 c VII 8%
After the filtering, the obtained table 6 is the available storage resource list according to the embodiment of the present invention, where the storage resource C and the storage resource H in the table are storage resources that meet the storage requirement of the data to be stored 1.
It should be noted that, in the filtering manners in the specific application scenario, the order of different filtering manners based on the performance tag, the location tag, and the load state may be randomly selected, or may be determined by setting an order priority in the storage requirement, which is not limited herein, and may be determined as needed.
Therefore, by the method in the step, the storage resources in the storage resource list are screened and filtered in different ways by using the corresponding performance tags, position tags and load states according to different storage requirements such as performance requirements, position requirements and load requirements, and the effect of filtering different storage resources can be achieved. In addition, because different storage requirements are filtered by using different classification labels, the filtered storage resources can be more accurate, and thus, the accuracy of storage resource allocation in the embodiment of the invention is guaranteed.
205. And allocating storage resources corresponding to the available storage resource list for the data to be stored.
After the available storage resource list is obtained through filtering in step 204, since there may be a plurality of storage resources in the available storage resource list that meet the storage requirement of the data to be stored, it is considered in this step that the data is preferably stored in a storage resource closer to the data writing end when stored, so as to ensure the affinity of the data. Therefore, according to the method in this step, which of the storage resources in the available storage resource list is closer to the write-in end of the data to be stored can be determined by acquiring the address information of the write-in end of the data to be stored, and then the storage resource closest to the address of the write-in end is extracted, and the data to be stored is stored by using the storage resource.
Meanwhile, when the data to be stored is stored, backup is sometimes required according to a redundancy strategy, wherein the redundancy strategy is a strategy for storing a plurality of copies of data in other storage resources while storing the original data based on the requirement of data storage safety. Therefore, based on the redundancy strategy, when the data to be stored is stored, the backup number of the data needs to be counted, and then whether the storage resources in the current available storage resource list meet the backup number is judged according to the counted result. When the number of the storage resources in the available storage resource list is greater than or equal to the backup number of the data to be stored, a sufficient number of storage resources can be extracted from the available storage resource list for storing the data to be stored. When the number of the storage resources in the available resource class table is less than the backup number of the data to be stored, the part of the data to be stored, which exceeds the number, can be stored in the storage resources in the storage resource list in the embodiment of the present invention. When the storage resources are acquired from the storage resource list for storing the data to be stored, the acquisition mode may be random acquisition, or filtering may be performed again by setting corresponding steps, which may be specifically selected as needed, and is not limited again.
Specifically, taking the example "data to be stored 1" and "table 3-table 6" described in the foregoing step 204 as an example, the available storage resource list filtered in step 204 includes a storage resource C and a storage resource H, and the method described in this step analyzes the address information to determine that the storage resource H is a storage resource closest to the write end of the data to be stored 1, and then the storage resource H may be provided to the data to be stored 1 for storage; when the data 1 to be stored is stored, and it is determined that 3 parts of data need to be backed up to satisfy the redundancy policy, the storage resources included in the available storage resource list only include two storage resources, and the number of the storage resources is less than the backup number of the data to be stored, where the number exceeds 1 part. Therefore, storage resources need to be allocated from the storage resource list to meet the storage requirement of the excess part of the amount of the data to be stored, that is, one storage resource is selected from the storage resource a, the storage resource B, the storage resource D, the storage resource E, the storage resource F and the storage resource G for storing the data to be stored.
According to the method in the step, the address of the writing end of the data to be stored is obtained, the storage resource which is closest to the address can be obtained from the available storage resource list, and further the length of a transmission path in the storage process can be reduced, so that the time spent on storing the data is reduced, meanwhile, the data can be obtained from the nearby storage resource in the reading process, the data access efficiency is improved, and the reasonable distribution of the storage resource is realized. In addition, when the number of the storage resources in the available storage resource list cannot meet the backup number of the data to be stored, the storage resources with the same number as the excess number are acquired from the storage resource list for storing the data to be stored, so that the storage resources which best meet the storage requirement of the data to be stored can be provided for the data to be stored, and meanwhile, the requirement of the data to be stored on the storage number under a redundancy strategy can be met, the storage resources under the super-fusion storage system are reasonably distributed, and the distribution efficiency is improved.
Further, as an implementation of the method shown in fig. 1, an embodiment of the present invention further provides a resource allocation apparatus based on a super-fusion storage system, which is used for implementing the method shown in fig. 1. The embodiment of the apparatus corresponds to the embodiment of the method, and for convenience of reading, details in the embodiment of the apparatus are not repeated one by one, but it should be clear that the apparatus in the embodiment can correspondingly implement all the contents in the embodiment of the method. As shown in fig. 3, the apparatus includes: a calculation unit 31, an acquisition unit 32, a filtering unit 33, a distribution unit 34, wherein
The calculating unit 31 may be configured to calculate a storage requirement of the data to be stored according to storage information of the data to be stored.
The obtaining unit 32 may be configured to obtain a storage resource list, where the storage resource list is used to record storage resources in the super-fusion storage system.
The filtering unit 33 may be configured to filter, according to the storage requirement calculated by the calculating unit 31, the storage resource corresponding to the storage resource list acquired by the acquiring unit 32 by using a preset rule and the classification label of the storage resource, so as to obtain an available storage resource list, where the classification label is used to classify the storage resource according to different dimensions, and the classification label at least includes one or more of a location label and a performance label.
The allocating unit 34 may be configured to allocate, to the data to be stored, a storage resource corresponding to the available storage resource list obtained after filtering by the filtering unit 33.
Further, as an implementation of the method shown in fig. 2, an embodiment of the present invention further provides another resource allocation apparatus based on a super-fusion storage system, which is used to implement the method shown in fig. 2. The embodiment of the apparatus corresponds to the embodiment of the method, and for convenience of reading, details in the embodiment of the apparatus are not repeated one by one, but it should be clear that the apparatus in the embodiment can correspondingly implement all the contents in the embodiment of the method. As shown in fig. 4, the apparatus includes: a calculation unit 41, an acquisition unit 42, a filtering unit 43, a distribution unit 44, wherein
The calculating unit 41 may be configured to calculate a storage requirement of the data to be stored according to storage information of the data to be stored.
The obtaining unit 42 may be configured to obtain a storage resource list, where the storage resource list is used to record storage resources in the super-fusion storage system.
The filtering unit 43 may be configured to filter, according to the storage requirement calculated by the calculating unit 41, the storage resource corresponding to the storage resource list acquired by the acquiring unit 42 by using a preset rule and the classification label of the storage resource, so as to obtain an available storage resource list, where the classification label is used to classify the storage resource according to different dimensions, and the classification label at least includes one or more of a location label and a performance label.
The allocating unit 44 may be configured to allocate, to the data to be stored, a storage resource corresponding to the available storage resource list obtained after filtering by the filtering unit 43.
Further, the apparatus further comprises:
the adding unit 45 may be configured to add a classification tag corresponding to the resource attribute to the storage resource according to the resource attribute of the storage resource, so that the filtering unit 43 performs filtering according to the tag added by the adding unit 45, where the resource attribute of the storage resource includes a hardware attribute and a location attribute.
Further, the adding unit 45 may be specifically configured to add, according to the hardware attribute of the storage resource, a performance tag corresponding to the hardware attribute to the storage resource, where the performance tag is an HDD tag or an SSD tag. And/or
The adding unit 45 may be specifically configured to add, according to the location information of the storage resource in the super-fusion storage system, a location tag corresponding to the location attribute to the storage resource, where the location tag is one or more of a rack tag, a host tag, and an area tag.
Further, the storage requirements of the data to be stored at least comprise one or more of performance requirements, location requirements and load requirements;
the calculation unit 41 includes:
the first determining module 411 may be configured to determine a performance requirement of the data to be stored according to an access requirement of the data to be stored. And/or
The second determining module 412 may be configured to determine a location requirement of the data to be stored according to the address information of the write end of the data to be stored. And/or
The third determining module 413 may be configured to determine a load requirement of the data to be stored according to a load status of the storage resource.
Further, the filtering unit 43 includes:
the first filtering module 431 may be configured to filter the storage resource corresponding to the storage resource list according to the performance requirement of the data to be stored and the performance tag. And/or
The second filtering module 432 may be configured to filter the storage resource corresponding to the storage resource list according to the location requirement of the data to be stored and the location tag. And/or
The third filtering module 433 may be configured to filter the storage resource corresponding to the storage resource list according to a load demand of the data to be stored and a load state of the storage resource.
Further, the first filtering module 431 includes:
the extracting submodule 4311 may be configured to extract a performance requirement of the data to be stored, where the performance requirement is used for specifying that the storage resource is a solid state disk SSD or a mechanical hard disk HDD.
The obtaining sub-module 4312 may be configured to obtain the storage resource with the SSD tag in the storage list when the performance requirement extracted by the extracting sub-module 4311 is that the specified storage resource is the solid state disk SSD.
The obtaining sub-module 4312 may be further configured to obtain the storage resource with the HDD tag in the storage list when the performance requirement extracted by the extracting sub-module 4311 is that the specified storage resource is a mechanical hard disk HDD.
Further, the second filtering module 432 includes:
the extracting sub-module 4321 may be configured to extract a location requirement of the data to be stored, where the location requirement is used to specify one or more of storage resources at different rack locations, storage resources at different host locations, and storage resources at different area locations.
The first filtering sub-module 4322 may be configured to, when the location requirement extracted by the extracting sub-module 4321 specifies a storage resource at a different rack location, filter, by a rack tag, a corresponding storage resource in the storage resource list, so as to obtain a storage resource at a different rack location. And/or
The second filtering sub-module 4323 may be configured to, when the location requirement extracted by the extracting sub-module 4321 specifies a storage resource at a different host location, filter, by using a host tag, a corresponding storage resource in the storage resource list, so as to obtain a storage resource at a different host location. And/or
The third filtering sub-module 4324 may be configured to, when the location requirement extracted by the extracting sub-module 4321 specifies storage resources at different area locations, filter the corresponding storage resources in the storage resource list through the area tags, so as to obtain the storage resources at different area locations.
Further, the third filtering module 433 includes:
an extraction sub-module 4331 may be used to extract the load threshold in the load demand.
The determining sub-module 4332 may be configured to determine whether there is a storage resource with a load exceeding the load threshold extracted by the extracting sub-module 4331.
The obtaining sub-module 4333 may be configured to, if the determining sub-module 4332 determines that there is a storage resource whose load exceeds the load threshold extracted by the extracting sub-module, obtain a storage resource whose load is smaller than the load threshold in the storage resource.
Further, the distribution unit 44 includes:
the obtaining module 441 may be configured to obtain address information of a write end of the data to be stored.
The determining module 442 may be configured to determine, according to the address information of the writing end acquired by the acquiring module 441, a storage resource closest to the address of the writing end from the address information of the storage resources in the available storage resource list.
The saving module 443 may be configured to save the data to be stored in the closest storage resource determined by the determining module 442.
Further, the saving module 443 may be specifically configured to select storage resources of the same number from the available storage resource list according to the specified backup number of the data to be stored, and save the data to be stored in the selected storage resources.
Further, the saving module 443 includes:
a determining sub-module 4431, configured to determine whether the backup number is greater than the number of storage resources in the available storage resource list;
the saving sub-module 4432 may be configured to, if the determining sub-module 4431 determines that the number of the backups is greater than the number of the storage resources in the available storage resource list, select a corresponding number of storage resources from the storage resource list to save the redundant number of the backups.
Further, an embodiment of the present invention further provides a resource allocation terminal based on a super-fusion storage system, where the terminal includes all the devices in the resource allocation device based on the super-fusion storage system, and in practical applications, the terminal is an intelligent computing device, and in the embodiment of the present invention, the terminal is a server, and the server at least includes: the processor and the storage medium perform specific allocation operation of storage resources, and execute the resource allocation method based on the super-fusion storage system in the above embodiments.
By means of the scheme, the embodiment of the invention provides a resource allocation method, a device and a terminal based on a super-fusion storage system. For the problem that the allocation mode is not reasonable enough when the storage resources are allocated to the data in the prior art, which results in low allocation efficiency of the storage resources under the super-fusion storage system, the storage resources in the system are filtered by using the preset rules and the classification labels of the storage resources according to the storage requirements, so that the effect of allocating the corresponding storage resources to the data to be stored according to different storage requirements is realized, and further, the function of allocating different storage resources to different data to be stored is realized. Compared with the prior art, the method and the device can filter the storage resources according to different classification labels when the data to be stored is received, and can achieve the effect of screening the storage resources. In addition, the storage requirement of the data to be stored is calculated according to the storage information of the data to be stored, and then the storage resources are filtered in a targeted manner by utilizing the classification tags according to the requirement, so that the effect of distributing different storage resources according to different storage requirements is realized, the condition that the storage resources are distributed at random when the data to be stored is to be stored in the prior art to cause unreasonable distribution is changed, and the distribution efficiency of the storage resources in the super-fusion storage system is improved. Meanwhile, different storage requirements are determined by different determination modes through different storage information in the data to be stored, so that the accuracy of the determination result of the storage requirements can be ensured, and the accuracy of the filtering result can be ensured in the process of filtering the storage resources according to the storage requirements, thereby ensuring the accuracy of the distribution of the storage resources. Furthermore, by using classification labels corresponding to different storage requirements, such as performance labels and location labels, to filter the storage resources in the storage resource list in different ways, an effect of filtering different storage resources can be achieved. In addition, because different storage requirements are filtered by using the corresponding filtering modes, the filtered storage resources can be more accurate, and thus, the accuracy of storage resource allocation provided by the embodiment of the invention is guaranteed. In addition, by acquiring the address of the writing end of the data to be stored, the storage resource closest to the address can be obtained from the available storage resource list, so that the length of a transmission path during storage can be reduced, the time spent on storing the data is reduced, the data can be acquired from the nearby storage resource during reading, the data access efficiency is improved, and the reasonable distribution of the storage resource is realized. And when the quantity of the storage resources in the available storage resource list cannot meet the backup quantity of the data to be stored, the storage resources with the same quantity as the excess quantity are acquired from the storage resource list for storing the data to be stored, so that the storage resources which best meet the storage requirement of the data to be stored can be provided for the data to be stored, and meanwhile, the requirement of the data to be stored on the storage quantity under a redundancy strategy can be met, the effect of reasonably distributing the storage resources under the super-fusion storage system is achieved, and the distribution efficiency is improved. In addition, according to the resource attribute of the storage resources in the super-fusion storage system, the corresponding classification labels are added to the storage resources, so that the storage resources can be classified, and a foundation is laid for reasonable distribution of the storage resources.
The text processing device comprises a processor and a memory, wherein the calculating unit, the acquiring unit, the filtering unit, the distributing unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, the reasonable distribution of the storage resources is realized by adjusting the kernel parameters, and the distribution efficiency is improved.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein the processor executes the program and realizes the following steps: calculating the storage requirement of the data to be stored according to the storage information of the data to be stored; acquiring a storage resource list, wherein the storage resource list is used for recording storage resources in the super-fusion storage system; according to the storage demand, filtering the corresponding storage resources in the storage resource list by using a preset rule and the classification labels of the storage resources to obtain an available storage resource list, wherein the classification labels are used for classifying the storage resources according to different dimensions, and the classification labels at least comprise one or more of position labels and performance labels; and allocating storage resources corresponding to the available storage resource list to the data to be stored.
An embodiment of the present invention further provides a computer program product, which, when executed on a data processing apparatus, is adapted to execute a program that initializes the following method steps: calculating the storage requirement of the data to be stored according to the storage information of the data to be stored; acquiring a storage resource list, wherein the storage resource list is used for recording storage resources in the super-fusion storage system; according to the storage demand, filtering the corresponding storage resources in the storage resource list by using a preset rule and the classification labels of the storage resources to obtain an available storage resource list, wherein the classification labels are used for classifying the storage resources according to different dimensions, and the classification labels at least comprise one or more of position labels and performance labels; and allocating storage resources corresponding to the available storage resource list to the data to be stored.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of the storage medium of the computer include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM)
(DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (12)

1. A resource allocation method based on a super-fusion storage system is characterized by comprising the following steps:
calculating the storage requirement of the data to be stored according to the storage information of the data to be stored;
acquiring a storage resource list, wherein the storage resource list is used for recording storage resources in the super-fusion storage system;
according to the storage demand, filtering the corresponding storage resources in the storage resource list by using a preset rule and the classification labels of the storage resources to obtain an available storage resource list, wherein the classification labels are used for classifying the storage resources according to different dimensions, and the classification labels at least comprise one or more of position labels and performance labels;
allocating storage resources corresponding to the available storage resource list for the data to be stored;
the allocating storage resources corresponding to the available storage resource list to the data to be stored includes:
acquiring address information of a writing end of the data to be stored;
according to the address information of the write-in terminal, determining a storage resource which is closest to the address of the write-in terminal from the address information of the storage resource in the available storage resource list;
and saving the data to be stored to the closest storage resource.
2. The method of claim 1, wherein prior to said obtaining the list of storage resources, the method further comprises:
and adding a classification label corresponding to the resource attribute for the storage resource according to the resource attribute of the storage resource, wherein the resource attribute of the storage resource comprises a hardware attribute and a position attribute.
3. The method of claim 2, wherein the adding, according to the resource attribute of the storage resource, the classification label corresponding to the resource attribute for the storage resource comprises:
adding a performance label corresponding to the hardware attribute for the storage resource according to the hardware attribute of the storage resource, wherein the performance label comprises an HDD label or an SSD label; and/or
And adding a position label corresponding to the position attribute for the storage resource according to the position information of the storage resource in the super-fusion storage system, wherein the position label comprises one or more of a rack label, a host label and an area label.
4. The method of claim 1, wherein the storage requirements of the data to be stored at least comprise one or more of performance requirements, location requirements, and load requirements;
the calculating the storage requirement of the data to be stored according to the storage information of the data to be stored comprises:
determining the performance requirement of the data to be stored according to the access requirement of the data to be stored; and/or
Determining the position requirement of the data to be stored according to the address information of the writing end of the data to be stored; and/or
And determining the load demand of the data to be stored according to the load state of the storage resource.
5. The method according to claim 4, wherein the filtering the corresponding storage resource in the storage resource list according to the storage requirement by using a preset rule and the classification label of the storage resource comprises:
filtering the corresponding storage resources in the storage resource list according to the performance requirements of the data to be stored and the performance labels; and/or
Filtering the corresponding storage resources in the storage resource list according to the position requirement of the data to be stored and the position label; and/or
And filtering the corresponding storage resources in the storage resource list according to the load requirements of the data to be stored and the load states of the storage resources.
6. The method of claim 5, wherein the filtering the corresponding storage resources in the storage resource list according to the performance requirements of the data to be stored and the performance tags comprises:
extracting the performance requirement of the data to be stored, wherein the performance requirement is used for the appointed storage resource to be a Solid State Disk (SSD) or a mechanical hard disk (HDD);
when the performance requirement is that the specified storage resource is a Solid State Disk (SSD), the storage resource with the SSD label in the storage resource list is obtained;
and when the performance requirement is that the specified storage resource is a mechanical hard disk (HDD), acquiring the storage resource with the HDD label in the storage resource list.
7. The method according to claim 5, wherein the filtering the corresponding storage resource in the storage resource list according to the location requirement of the data to be stored and the location tag comprises:
extracting the position requirement of the data to be stored, wherein the position requirement is used for specifying one or more of storage resources at different rack positions, storage resources at different host positions and storage resources at different area positions;
when the position requirement specifies storage resources at different rack positions, filtering the corresponding storage resources in the storage resource list through rack labels so as to obtain the storage resources at different rack positions; and/or
When the position requirement specifies storage resources at different host positions, filtering the corresponding storage resources in the storage resource list through a host label so as to obtain the storage resources at different host positions; and/or
And when the position requirement specifies the storage resources at different area positions, filtering the corresponding storage resources in the storage resource list through the area labels so as to obtain the storage resources at different area positions.
8. The method according to claim 5, wherein the filtering the corresponding storage resources in the storage resource list according to the load demand of the data to be stored and the load status of the storage resources comprises:
extracting a load threshold value in the load demand;
judging whether storage resources with loads exceeding the load threshold exist or not;
and if so, acquiring the storage resources with the load smaller than the load threshold value in the storage resources.
9. The method of claim 1, wherein saving the data to be stored to the closest storage resource comprises:
and selecting the same number of storage resources from the available storage resource list according to the specified backup number of the data to be stored, and storing the data to be stored into the selected storage resources.
10. The method of claim 9, wherein selecting the same number of storage resources in the list of available storage resources according to the number of backups specified by the data to be stored comprises:
judging whether the backup quantity is larger than the quantity of the storage resources in the available storage resource list or not;
and if so, selecting the storage resources with the corresponding quantity in the storage resource list to store the redundant quantity of backups.
11. A resource allocation terminal based on a super-fusion storage system, wherein the terminal comprises a processor and a storage medium, and the processor and the storage medium can execute the resource allocation method based on the super-fusion storage system according to any one of claims 1 to 10 when allocating storage resources.
12. An apparatus for allocating resources based on a hyper-converged storage system, the apparatus comprising:
the computing unit is used for computing the storage requirement of the data to be stored according to the storage information of the data to be stored;
the system comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring a storage resource list which is used for recording storage resources in the super-fusion storage system;
the filtering unit is used for filtering the corresponding storage resources in the storage resource list acquired by the acquiring unit by using preset rules and the classification labels of the storage resources according to the storage requirements calculated by the calculating unit to obtain an available storage resource list, wherein the classification labels are used for classifying the storage resources according to different dimensions, and the classification labels at least comprise one or more of position labels and performance labels;
the distribution unit is used for distributing storage resources corresponding to the available storage resource list obtained after the filtering unit filters the data to be stored;
the distribution unit includes:
the acquisition module is used for acquiring the address information of the writing end of the data to be stored;
a determining module, configured to determine, according to the address information of the write-in end acquired by the acquiring module, a storage resource closest to the address of the write-in end from the address information of the storage resource in the available storage resource list;
and the storage module is used for storing the data to be stored into the closest storage resource determined by the determination module.
CN201711147145.4A 2017-11-17 2017-11-17 Resource allocation method, device and terminal based on super-fusion storage system Active CN107783734B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711147145.4A CN107783734B (en) 2017-11-17 2017-11-17 Resource allocation method, device and terminal based on super-fusion storage system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711147145.4A CN107783734B (en) 2017-11-17 2017-11-17 Resource allocation method, device and terminal based on super-fusion storage system

Publications (2)

Publication Number Publication Date
CN107783734A CN107783734A (en) 2018-03-09
CN107783734B true CN107783734B (en) 2021-03-16

Family

ID=61433302

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711147145.4A Active CN107783734B (en) 2017-11-17 2017-11-17 Resource allocation method, device and terminal based on super-fusion storage system

Country Status (1)

Country Link
CN (1) CN107783734B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108874316A (en) * 2018-06-06 2018-11-23 郑州云海信息技术有限公司 Method, apparatus, equipment and the readable storage medium storing program for executing of storage resource centralized management
CN108881942B (en) * 2018-06-06 2020-11-06 西安交通大学 Super-fusion normal state recorded broadcast system based on distributed object storage
KR102611566B1 (en) 2018-07-06 2023-12-07 삼성전자주식회사 Solid state drive and its memory allocation method
CN110858122B (en) * 2018-08-23 2023-10-20 杭州海康威视系统技术有限公司 Method and device for storing data
CN110928484B (en) * 2018-09-19 2023-12-22 上海仪电(集团)有限公司中央研究院 Hybrid cloud storage method based on software defined storage
CN109564567B (en) * 2018-10-17 2023-07-25 北京算能科技有限公司 Data storage method, device, electronic equipment and computer readable storage medium
CN111221644B (en) * 2018-11-27 2023-06-13 阿里巴巴集团控股有限公司 Resource scheduling method, device and equipment
CN112612660B (en) * 2020-12-16 2024-02-13 海光信息技术股份有限公司 Specification information database creation method, chip selection method, device and system
CN112905718A (en) * 2021-03-09 2021-06-04 东莞中国科学院云计算产业技术创新与育成中心 Data management method, system, electronic device and medium based on super-fusion architecture
CN117555493A (en) * 2024-01-11 2024-02-13 北京星辰天合科技股份有限公司 Data processing method, system, device, storage medium and electronic equipment

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1952941A (en) * 2005-10-21 2007-04-25 中国科学院计算技术研究所 A resource allocation method and system for shared memory
CN105446813A (en) * 2014-05-30 2016-03-30 华为技术有限公司 Resource distribution method and device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1952941A (en) * 2005-10-21 2007-04-25 中国科学院计算技术研究所 A resource allocation method and system for shared memory
CN105446813A (en) * 2014-05-30 2016-03-30 华为技术有限公司 Resource distribution method and device

Also Published As

Publication number Publication date
CN107783734A (en) 2018-03-09

Similar Documents

Publication Publication Date Title
CN107783734B (en) Resource allocation method, device and terminal based on super-fusion storage system
JP5932043B2 (en) Volatile memory representation of non-volatile storage set
CN104111897A (en) Data processing method, data processing device and computer system
US8190844B2 (en) Efficient alert mechanism for overutilization of storage resources
CN103324533A (en) distributed data processing method, device and system
CN110737717B (en) Database migration method and device
US20210011634A1 (en) Methods and systems for managing key-value solid state drives (kv ssds)
CN107479950B (en) Virtual machine scheduling method, device and system
US8443369B1 (en) Method and system for dynamically selecting a best resource from each resource collection based on resources dependencies, prior selections and statistics to implement an allocation policy
CN112463058B (en) Fragmented data sorting method and device and storage node
CN113110918A (en) Read-write rate control method and device, node equipment and storage medium
CN111984602A (en) Data stream management method and device
CN116244085A (en) Kubernetes cluster container group scheduling method, device and medium
EP3296878B1 (en) Electronic device and page merging method therefor
EP3264254A1 (en) System and method for a simulation of a block storage system on an object storage system
CN113254186A (en) Process scheduling method, scheduler and storage medium
US20160232166A1 (en) Method and Apparatus for Accessing File
CN113342274A (en) Data processing method and device
CN108733484B (en) Method and device for managing application program
CN109508140B (en) Storage resource management method and device, electronic equipment and system
CN112148486A (en) Memory page management method, device and equipment and readable storage medium
CN113553142B (en) Storage space arrangement method and configuration method of cloud platform and readable storage medium
CN111198649B (en) Disk selection method and device
US20230060835A1 (en) Method and apparatus for setting memory, and electronic device and storage medium
US20200356297A1 (en) Method of storage control based on log data types

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant