CN111970520B - Heterogeneous node streaming data distributed storage method - Google Patents
Heterogeneous node streaming data distributed storage method Download PDFInfo
- Publication number
- CN111970520B CN111970520B CN202010812005.XA CN202010812005A CN111970520B CN 111970520 B CN111970520 B CN 111970520B CN 202010812005 A CN202010812005 A CN 202010812005A CN 111970520 B CN111970520 B CN 111970520B
- Authority
- CN
- China
- Prior art keywords
- storage
- data
- data source
- node
- nodes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2181—Source of audio or video content, e.g. local disk arrays comprising remotely distributed storage units, e.g. when movies are replicated over a plurality of video servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Abstract
A heterogeneous node stream data distributed storage method utilizes the characteristics of stream data to match storage system resources with data traffic, thereby realizing better utilization of heterogeneous storage system resources; in the streaming storage application scene, the storage capacity is marked by P [ x ], the storage capacity is preferentially used, and the data distribution can be carried out as long as the residual storage capacity exists in the whole system, without balancing during the capacity expansion of the system; according to different scenes, different distribution modes are selected, so that higher storage safety of data, high system throughput, high utilization rate of system resources or the number of running systems is reduced, and power consumption is reduced.
Description
Technical Field
The invention relates to the field of data storage, in particular to a heterogeneous node streaming data distributed storage method.
Background
When a distributed storage scheme in the prior art is used for storing streaming data such as video storage and the like, a video file needs to be generated firstly, and then the formed video file is stored in each storage disk in a hash manner; the data generation rate in streaming storage schemes is relatively stable and regular; the control mode of the stream storage is superposed on the basis of the file storage, although equalization can be realized, the characteristics of the stream storage cannot be fully utilized, the distribution of data storage needs to be scheduled frequently, the equalization degree is prior, and the matching of data and storage capacity cannot be realized for systems with different storage capacities of storage nodes; when the system expands, the data stored in the original node cannot be balanced with the data stored in the new node, so that the primary balancing condition exists for a long time. In addition, such video data needs to be stored for a long time in most cases, but the data capacity of the data does not need to be maintained, which causes a waste of power consumption to a certain extent when the storage system stores the video data.
In the prior art, the control method for overlaying streaming storage on file storage can realize the equalization of storage resources, but still has the following disadvantages: the stored resources need to be scheduled and allocated frequently, which causes waste to a certain extent for bottom layer resources and computing resources; for systems with different storage nodes, matching between data and storage capacity cannot be realized, and unbalanced conditions which are difficult to eliminate can occur during system capacity expansion; in addition, when the video file is stored, the data still can be read at high speed, which causes power consumption of the storage system.
Disclosure of Invention
The invention aims to provide a heterogeneous node streaming data distributed storage method, so that the problems in the prior art are solved.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a heterogeneous node streaming data distributed storage method comprises the following steps:
s1, the heterogeneous storage Node is Node [0], Node [1 ]. Node [ N-1], the storage capacity of the xth Node is set as P [ x ]; the Stream data source is Stream [0], Stream [1 ]. cndot.stream [ S-1], and the data storage requirement of the yth data source is set as Ry;
s2, calculating the data storage requirement Ry of the streaming data source to the storage space of the storage node according to the code rate and the storage period of the streaming data source;
s3, acquiring a storage node set with storage capacity greater than the data storage requirement in the storage node list, namely acquiring the storage nodes meeting the requirement that P [ x ] is greater than or equal to R [ y ];
s4, according to the principle of an allocation algorithm, allocating the streaming data sources to the storage node sets obtained in the step S3 respectively according to the sequence that the storage capacity of the storage nodes is from large to small; judging whether the storage node set can finish the storage of the streaming data source, if the storage node set can meet the storage of the streaming data source, returning an allocation result to the storage node, and storing the data source in the storage node set; otherwise, a failure is returned, and the storage node is not sufficient to store the data source.
Preferably, the allocation relationship between the streaming data source Stream and the storage Node is as follows: when the storage node in the storage node list is added, the Stream data source Stream which is allocated for storage is not moved, and the newly added Stream data source is stored in the storage node of the newly added storage node list according to the steps S2 to S4; when the storage nodes in the storage node list are decreased, the streaming data sources stored in the decreased storage nodes are stored in the remaining storage nodes according to the steps S2 to S4.
Preferably, the storage of the data source is calculated by adopting an erasure operation mode: n is k + m, and n is k + m,
wherein k represents the number of original data blocks of the data source, m represents the number of data check blocks, and n represents the total number of data blocks of the data source after erasure operation.
Preferably, the calculation formula of the data storage requirement R [ y ] of the streaming data source to the storage space of the storage node is as follows:
R[y]＝T*S/k，
wherein, T is the storage period of the streaming data, S represents the data amount of a certain streaming data source within 1 second, and k is the original data block number of the data source used for erasure correction operation.
Preferably, the storage capacity P [ x ] of the storage node capable of storing the streaming data source is as follows:
P[x]≥R[y]/E，
wherein E is the storage capacity efficiency (%).
Preferably, the allocation algorithm principle includes a forced round hash, a forced reorder hash, a priority round hash, and a priority reorder hash.
Preferably, the forced round hashing principle requires: the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is more than or equal to R [ y ] must be more than the total number n of the data blocks of the same data source after erasure operation, otherwise, the system does not work and does not perform data distribution and storage; when the data source is distributed, distributing total data blocks of the same data source in n different storage nodes respectively; when the data block in the next data source is stored, according to the order from large to small of the storage capacity of the storage node, the last storage node stored in the previous data source is stored in the storage node acquired in step S3 end to end.
Preferably, the forced reordering hashing principle requires that: the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is more than or equal to R [ y ] must be more than the total number n of the data blocks of the same data source after erasure operation, otherwise, the system does not work and does not perform data distribution and storage; when the data source is distributed, distributing total data blocks of the same data source in n different storage nodes respectively; when the data blocks in each data source are stored, the data blocks are distributed from the first storage node in the storage node cluster according to the sequence of the residual storage capacity of the storage nodes from large to small.
Preferably, the priority round hashing principle requires: when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is greater than or equal to R [ y ] is greater than or equal to the total number n of the data blocks of the same data source after erasure correction operation, distributing the total data blocks of the same data source in n different storage nodes respectively; when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is greater than or equal to R [ y ] is smaller than the total number n of the data blocks of the same data source after erasure correction operation, different total data blocks of the same data source can be distributed in the same storage node; when the data block in the next data source is stored, according to the order from large to small of the storage capacity of the storage node, the last storage node stored in the previous data source is stored in the storage node acquired in step S3 end to end.
Preferably, the principle of prioritizing hashes requires: when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is greater than or equal to R [ y ] is greater than or equal to the total number n of the data blocks of the same data source after erasure correction operation, distributing the total data blocks of the same data source in n different storage nodes respectively; when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is greater than or equal to R [ y ] is smaller than the total number n of the data blocks of the same data source after erasure correction operation, different total data blocks of the same data source can be distributed in the same storage node; when the data blocks in each data source are stored, the data blocks are distributed from the first storage node in the storage node cluster according to the sequence of the residual storage capacity of the storage nodes from large to small.
The invention has the beneficial effects that: the invention discloses a heterogeneous node streaming data distributed storage method, which matches storage system resources with data flow by using the characteristics of streaming data, thereby realizing better utilization of the heterogeneous storage system resources; under the application scene of streaming storage, the data input quantity is relatively stable, so that the bandwidth waste and the calculation resource continuous balance at the bottom layer are not needed; the storage capacity is marked by P [ x ], the storage capacity is preferentially used, and data distribution can be carried out as long as the remaining storage capacity exists in the whole system, without balancing during system capacity expansion; when the priority rearrangement algorithm is selected, more data can be accumulated in individual nodes, and other unused nodes can be operated in a power-off and low-power-consumption mode in an operation and maintenance mode, so that the effect of the nodes is achieved. According to different scenes, different distribution modes are selected, so that higher storage safety of data, high system throughput, high utilization rate of system resources or the number of running systems is reduced, and power consumption is reduced.
Drawings
FIG. 1 is a flow diagram of a heterogeneous node streaming data distributed storage process;
FIG. 2 is a matching graph of data flow and storage nodes;
FIG. 3 is an initial allocation table;
FIG. 4 is an allocation table after adding nodes;
fig. 5 is an allocation table after node reduction.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
A method for storing streaming data in heterogeneous nodes in a distributed manner, as shown in fig. 1, includes the following steps:
s1, the heterogeneous storage Node is Node [0], Node [1 ]. Node [ N-1], the storage capacity of the xth Node is set as P [ x ]; the Stream data source is Stream [0], Stream [1 ]. cndot.stream [ S-1], and the data storage requirement of the yth data source is set as Ry;
s2, calculating the data storage requirement Ry of the streaming data source to the storage space of the storage node according to the code rate and the storage period of the streaming data source; the calculation formula of the data storage requirement R [ y ] of the streaming data source to the storage space of the storage node is as follows:
R[y]＝T*S/k，
wherein, T is the storage period (S) of the streaming data, S represents the data amount (bytes) of a certain streaming data source within 1S, and k is the original data block number of the data source used for erasure correction operation.
S3, acquiring a storage node set with storage capacity greater than the data storage requirement in the storage node list, namely acquiring the storage nodes meeting the requirement that P [ x ] is greater than or equal to R [ y ];
s4, according to the principle of an allocation algorithm, allocating the streaming data sources to the storage node sets obtained in the step S3 respectively according to the sequence that the storage capacity of the storage nodes is from large to small; judging whether the storage node set can finish the storage of the streaming data source, if the storage node set can meet the storage of the streaming data source, returning an allocation result to the storage node, and storing the data source in the storage node set; otherwise, a failure is returned, and the storage node is not sufficient to store the data source.
The storage capacity is P [ x ] and represents the residual storage capacity of the storage node, the storage capacity is a dynamic value, and the larger the value of P [ x ], the higher the storage capacity of the storage node is; the storage capacity P [ x ] of the storage node capable of storing the streaming data source is the requirement:
P[x]≥R[y]/E，
wherein E is the storage capacity efficiency (%).
In step S4, the allocation relationship between the streaming data source Stream and the storage Node is Map, and when any one of the streaming data source Stream and the storage Node changes, the allocation relationship reallocates: map1 ═ F (Nodes, Streams, Map0), where F is the allocation algorithm, Map0 indicates the previous allocation case, and Map1 indicates the new allocation result.
The stream data source is stored according to the stream data distributed storage method of the heterogeneous node; the allocation relationship between the streaming data source Stream and the storage Node is shown in fig. 2: when the storage node in the storage node list is added, the Stream data source Stream which is allocated for storage is not moved, and the newly added Stream data source is stored in the storage node of the newly added storage node set according to the above steps S2 to S4; when the storage nodes in the storage node list are decreased, the streaming data sources stored in the decreased storage nodes are stored in the remaining storage nodes according to the steps S2 to S4.
In step S4, the data source is stored in the storage node and calculated by erasure correction: n is k + m, where k denotes the number of original data blocks of the data source, m denotes the number of data check blocks, and n denotes the total number of data blocks of the data source after erasure correction operation;
if a fixed value of the size of the basic erasure correction block in the erasure correction operation is set to a, the principle of the erasure correction operation is as follows: when k data blocks with the size of A need to be stored in the storage nodes, generating n data blocks with the size of A after erasure operation, and needing to be written into the n storage nodes;
the distribution algorithm principle comprises forced circulation hashing, forced rearrangement hashing, priority circulation hashing and priority rearrangement hashing;
the forced round hashing principle requires: the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is more than or equal to R [ y ] must be more than the total number n of the data blocks of the same data source after erasure operation, otherwise, the system does not work and does not perform data distribution and storage; when the data source is distributed, distributing total data blocks of the same data source in n different storage nodes respectively; when the data block in the next data source is stored, according to the order from large to small of the storage capacity of the storage node, the last storage node stored in the previous data source is stored in the storage node acquired in step S3 end to end.
The forced reordering hashing principle requires that: the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is more than or equal to R [ y ] must be more than the total number n of the data blocks of the same data source after erasure operation, otherwise, the system does not work and does not perform data distribution and storage; when the data source is distributed, distributing total data blocks of the same data source in n different storage nodes respectively; when the data blocks in each data source are stored, the data blocks are distributed from the first storage node in the storage node cluster according to the sequence of the residual storage capacity of the storage nodes from large to small.
The priority round-robin hashing principle requires: when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is greater than or equal to R [ y ] is greater than or equal to the total number n of the data blocks of the same data source after erasure correction operation, distributing the total data blocks of the same data source in n different storage nodes respectively; when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is greater than or equal to R [ y ] is smaller than the total number n of the data blocks of the same data source after erasure correction operation, different total data blocks of the same data source can be distributed in the same storage node; when the data block in the next data source is stored, according to the order from large to small of the storage capacity of the storage node, the last storage node stored in the previous data source is stored in the storage node acquired in step S3 end to end.
The prioritized reordering hashing principle requires: when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is greater than or equal to R [ y ] is greater than or equal to the total number n of the data blocks of the same data source after erasure correction operation, distributing the total data blocks of the same data source in n different storage nodes respectively; when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is greater than or equal to R [ y ] is smaller than the total number n of the data blocks of the same data source after erasure correction operation, different total data blocks of the same data source can be distributed in the same storage node; when the data blocks in each data source are stored, the data blocks are distributed from the first storage node in the storage node cluster according to the sequence of the residual storage capacity of the storage nodes from large to small.
Examples
In this embodiment, the data block k in the streaming data source is 3, m is 1, and n is 1; and distributing the data blocks in the streaming data source into storage node clusters with the number of 10, 4 and 2 of storage nodes meeting the requirement that P [ x ] is not less than R [ y ].
Distributing the streaming data source according to four distribution algorithm principles, wherein the initial distribution result between the streaming data source and the storage node is shown in fig. 3;
on the basis of initial allocation, when a new storage node is added, the allocation result is shown in fig. 4;
when a new storage node is reduced based on the initial allocation, the allocation result is shown in fig. 5.
By adopting the technical scheme disclosed by the invention, the following beneficial effects are obtained:
the invention discloses a heterogeneous node streaming data distributed storage method, which matches storage system resources with data flow by using the characteristics of streaming data, thereby realizing better utilization of the heterogeneous storage system resources; under the application scene of streaming storage, the data input quantity is relatively stable, so that the bandwidth waste and the calculation resource continuous balance at the bottom layer are not needed; the storage capacity is marked by P [ x ], the storage capacity is preferentially used, and data distribution can be carried out as long as the remaining storage capacity exists in the whole system, without balancing during system capacity expansion; when the priority rearrangement algorithm is selected, more data can be accumulated in individual nodes, and other unused nodes can be operated in a power-off and low-power-consumption mode in an operation and maintenance mode, so that the effect of the nodes is achieved. According to different scenes, different distribution modes are selected, so that higher storage safety of data, high system throughput, high utilization rate of system resources or the number of running systems is reduced, and power consumption is reduced.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and improvements can be made without departing from the principle of the present invention, and such modifications and improvements should also be considered within the scope of the present invention.
Claims (10)
1. A heterogeneous node streaming data distributed storage method is characterized by comprising the following steps:
s1, the heterogeneous storage Node is Node [0], Node [1 ]. Node [ N-1], the storage capacity of the xth Node is set as P [ x ]; the Stream data source is Stream [0], Stream [1 ]. cndot.stream [ S-1], and the data storage requirement of the yth data source is set as Ry;
s2, calculating the data storage requirement Ry of the streaming data source to the storage space of the storage node according to the code rate and the storage period of the streaming data source;
s3, acquiring a storage node set with storage capacity greater than the data storage requirement in the storage node list, namely acquiring the storage nodes meeting the requirement that P [ x ] is greater than or equal to R [ y ];
s4, respectively allocating the streaming data sources to the storage node sets obtained in the step S3 according to the distribution algorithm principle according to the sequence of the storage capacities of the storage nodes from large to small; judging whether the storage node set can finish the storage of the streaming data source, if the storage node set can meet the storage of the streaming data source, returning an allocation result to the storage node, and storing the data source in the storage node set; otherwise, a failure is returned, and the storage node is not sufficient to store the data source.
2. The distributed storage method for streaming data of heterogeneous nodes according to claim 1, wherein the distribution relationship between the streaming data source Stream and the storage Node is: when the storage node in the storage node list is added, the Stream data source Stream which is allocated for storage is not moved, and the newly added Stream data source is stored in the storage node of the newly added storage node list according to the steps S2 to S4; when the storage nodes in the storage node list are decreased, the streaming data sources stored in the decreased storage nodes are stored in the remaining storage nodes according to the steps S2 to S4.
3. The distributed storage method of streaming data of heterogeneous nodes according to claim 1, wherein the storage of the data source is calculated by erasure correction operation: n is k + m, and n is k + m,
wherein k represents the number of original data blocks of the data source, m represents the number of data check blocks, and n represents the total number of data blocks of the data source after erasure operation.
4. The distributed storage method for streaming data of heterogeneous nodes according to claim 1, wherein the calculation formula of the data storage requirement R [ y ] of the streaming data source to the storage space of the storage node is as follows:
R[y]＝T*S/k，
wherein, T is the storage period of the streaming data, S represents the data amount of a certain streaming data source within 1 second, and k is the original data block number of the data source used for erasure correction operation.
5. The distributed storage method for streaming data of a heterogeneous node according to claim 1, wherein the storage capacity P [ x ] of the storage node capable of storing the streaming data source is defined as:
P[x]≥R[y]/E，
wherein E is the storage capacity efficiency (%).
6. The method of claim 1, wherein the distribution algorithm principles comprise forced round hashing, forced reorder hashing, prioritized round hashing, and prioritized reorder hashing.
7. The distributed storage method for streaming data of a heterogeneous node according to claim 6, wherein the forced round hashing principle requires: the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is more than or equal to R [ y ] must be more than the total number n of data blocks of the same data source after erasure operation, otherwise, the system does not work and does not distribute and store data; when the data source is distributed, distributing total data blocks of the same data source in n different storage nodes respectively; when the data block in the next data source is stored, according to the order from large to small of the storage capacity of the storage node, the last storage node stored in the previous data source is stored in the storage node acquired in step S3 end to end.
8. The distributed storage method for streaming data of a heterogeneous node according to claim 6, wherein the forced reordering hashing principle requires that: the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is more than or equal to R [ y ] must be more than the total number n of data blocks of the same data source after erasure operation, otherwise, the system does not work and does not distribute and store data; when the data source is distributed, distributing total data blocks of the same data source in n different storage nodes respectively; when the data blocks in each data source are stored, the data blocks are distributed from the first storage node in the storage node cluster according to the sequence of the residual storage capacity of the storage nodes from large to small.
9. The distributed storage method for streaming data of a heterogeneous node according to claim 6, wherein the priority round-robin hashing principle requires: when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is more than or equal to R [ y ] is more than or equal to the total number n of data blocks of the same data source after erasure correction operation, distributing the total data blocks of the same data source in n different storage nodes respectively; when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is greater than or equal to R [ y ] is smaller than the total number n of the data blocks of the same data source after erasure correction operation, different total data blocks of the same data source can be distributed in the same storage node; when the data block in the next data source is stored, according to the order from large to small of the storage capacity of the storage node, the last storage node stored in the previous data source is stored in the storage node acquired in step S3 end to end.
10. The distributed storage method for streaming data of a heterogeneous node according to claim 6, wherein the prioritizing of the reordered hash rule requires: when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is more than or equal to R [ y ] is more than or equal to the total number n of data blocks of the same data source after erasure correction operation, distributing the total data blocks of the same data source in n different storage nodes respectively; when the number of the storage nodes meeting the requirement that the number of the storage nodes P [ x ] is greater than or equal to R [ y ] is smaller than the total number n of the data blocks of the same data source after erasure correction operation, different total data blocks of the same data source can be distributed in the same storage node; when the data blocks in each data source are stored, the data blocks are distributed from the first storage node in the storage node cluster according to the sequence of the residual storage capacity of the storage nodes from large to small.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010812005.XA CN111970520B (en) | 2020-08-13 | 2020-08-13 | Heterogeneous node streaming data distributed storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010812005.XA CN111970520B (en) | 2020-08-13 | 2020-08-13 | Heterogeneous node streaming data distributed storage method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111970520A CN111970520A (en) | 2020-11-20 |
| CN111970520B true CN111970520B (en) | 2022-04-08 |
Family
ID=73365869
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010812005.XA Active CN111970520B (en) | 2020-08-13 | 2020-08-13 | Heterogeneous node streaming data distributed storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111970520B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107104992A (en) * | 2016-02-19 | 2017-08-29 | 杭州海康威视数字技术股份有限公司 | The storage resource distribution method and device of a kind of video cloud storage |
| CN107943867A (en) * | 2017-11-10 | 2018-04-20 | 中国电子科技集团公司第三十二研究所 | High-performance hierarchical storage system supporting heterogeneous storage |
| CN109428913A (en) * | 2017-08-25 | 2019-03-05 | 浙江宇视科技有限公司 | A kind of storage expansion method and device |
| CN109740038A (en) * | 2019-01-02 | 2019-05-10 | 安徽芃睿科技有限公司 | Network data distributed parallel computing environment and method |
| CN110569213A (en) * | 2018-05-18 | 2019-12-13 | 北京果仁宝软件技术有限责任公司 | File access method, device and equipment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104598316B (en) * | 2014-12-25 | 2018-06-05 | 华为技术有限公司 | A kind of storage resource distribution method and device |
| US10637923B2 (en) * | 2016-05-25 | 2020-04-28 | Cisco Technology, Inc. | Congestion aware load balancing for distributed storage |
| CN106406758B (en) * | 2016-09-05 | 2019-06-18 | 华为技术有限公司 | A kind of data processing method and storage equipment based on distributed memory system |
-
2020
- 2020-08-13 CN CN202010812005.XA patent/CN111970520B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107104992A (en) * | 2016-02-19 | 2017-08-29 | 杭州海康威视数字技术股份有限公司 | The storage resource distribution method and device of a kind of video cloud storage |
| CN109428913A (en) * | 2017-08-25 | 2019-03-05 | 浙江宇视科技有限公司 | A kind of storage expansion method and device |
| CN107943867A (en) * | 2017-11-10 | 2018-04-20 | 中国电子科技集团公司第三十二研究所 | High-performance hierarchical storage system supporting heterogeneous storage |
| CN110569213A (en) * | 2018-05-18 | 2019-12-13 | 北京果仁宝软件技术有限责任公司 | File access method, device and equipment |
| CN109740038A (en) * | 2019-01-02 | 2019-05-10 | 安徽芃睿科技有限公司 | Network data distributed parallel computing environment and method |
Non-Patent Citations (1)
| Title |
|---|
| 基于分布式环境的存储负载均衡算法研究;郝昱文；王继娜；李晓雪；赵喆；郑静晨；杨杨;《信息技术》;20160930;55-58 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111970520A (en) | 2020-11-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108268317B (en) | Resource allocation method and device | |
| CN101447939B (en) | Functional distribution method and load balancer | |
| CN108572873B (en) | Load balancing method and device for solving Spark data inclination problem | |
| US20120209943A1 (en) | Apparatus and method for controlling distributed memory cluster | |
| US10114866B2 (en) | Memory-constrained aggregation using intra-operator pipelining | |
| CN108462656B (en) | Container-based resource scheduling method and device for service convergence deployment | |
| CN102075425A (en) | Business allocation method and load balancer | |
| KR970012201A (en) | On-demand video system and video scheduling method of the system, physical resource scheduling method of on-demand customer service system, computer readable memory | |
| CN103297499A (en) | Scheduling method and system based on cloud platform | |
| CN110048882B (en) | Method and device for resource allocation based on edge calculation | |
| US20050268049A1 (en) | Dynamic memory management | |
| CN106775470B (en) | Data storage method and system | |
| CN114064588B (en) | Storage space scheduling method and system | |
| CN111290841A (en) | Task scheduling method and device, computing equipment and storage medium | |
| CN109492052B (en) | SCADA distributed historical data storage method and system | |
| WO1998054657A2 (en) | System for retrieving data in a video server | |
| CN107391039B (en) | Data object storage method and device | |
| CN115237580A (en) | Intelligent calculation-oriented flow parallel training self-adaptive adjustment system and method | |
| CN112817535A (en) | Method and device for distributing homing groups and distributed storage system | |
| CN114968566A (en) | Container scheduling method and device under shared GPU cluster | |
| CN111970520B (en) | Heterogeneous node streaming data distributed storage method | |
| US5940865A (en) | Apparatus and method for accessing plural storage devices in predetermined order by slot allocation | |
| CN113379587A (en) | Container sharing video memory method, device, equipment and system | |
| CN104778125A (en) | Method and system for memory management | |
| CN102970349A (en) | Distributed hash table (DHT) network storage load balancing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |