WO2017000094A1 - Procédé, dispositif et système de stockage de données - Google Patents

Procédé, dispositif et système de stockage de données Download PDF

Info

Publication number
WO2017000094A1
WO2017000094A1 PCT/CN2015/082581 CN2015082581W WO2017000094A1 WO 2017000094 A1 WO2017000094 A1 WO 2017000094A1 CN 2015082581 W CN2015082581 W CN 2015082581W WO 2017000094 A1 WO2017000094 A1 WO 2017000094A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage
storage nodes
node
data
nodes
Prior art date
Application number
PCT/CN2015/082581
Other languages
English (en)
Chinese (zh)
Inventor
董晓文
赵俊峰
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201580001239.3A priority Critical patent/CN107113323B/zh
Priority to PCT/CN2015/082581 priority patent/WO2017000094A1/fr
Publication of WO2017000094A1 publication Critical patent/WO2017000094A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40—Support for services or applications

Definitions

  • Embodiments of the present invention relate to the field of computers, and in particular, to a data storage method, apparatus, and system.
  • SSDs Solid State Drives
  • HDDs Hard Disk Drives
  • storage nodes can be deployed at different locations on the network. To fully utilize the speed of the distributed storage network, the data is divided into multiple data blocks and stored in different storage nodes. To achieve parallel access to multiple storage nodes.
  • data is generally divided into data blocks of the same size in a distributed storage network, and corresponding storage nodes are selected for random data blocks or based on a minimum delay policy, and then the data blocks are stored in different storage nodes.
  • corresponding storage nodes are selected for random data blocks or based on a minimum delay policy, and then the data blocks are stored in different storage nodes.
  • data blocks are equally divided, and storage nodes or minimum delay strategies are randomly selected for data blocks. It has been unable to meet the needs of practical applications, and further optimization of distributed storage methods is needed.
  • embodiments of the present invention provide a data storage method, apparatus, and system, which implement optimization of a distributed storage method.
  • an embodiment of the present invention provides a data storage method for storing a data distribution of a source node to be stored in a distributed storage system, including: the source node determining N from the distributed storage system.
  • Storage nodes where N is a positive integer greater than 1; the source node obtains Bandwidth information of each of the N storage nodes, the bandwidth information of each of the N storage nodes is the each of the N storage nodes to the The transmission bandwidth of the data transmission path of the source node; the source node is the one of the N storage nodes according to the data to be stored and the bandwidth information of each of the N storage nodes
  • Each storage node allocates corresponding data to be stored; the source node sends data to be stored allocated by each of the N storage nodes to the N storage nodes Each storage node is described.
  • the source node is configured according to the data to be stored and the bandwidth information of each of the N storage nodes.
  • Each of the N storage nodes is configured to allocate corresponding data to be stored, and the source node respectively allocates corresponding data to be stored to the J storage nodes of the N storage nodes, where
  • the size of the data to be stored allocated by the jth storage node of the J storage nodes is positively correlated with the transmission bandwidth of the jth storage node, and the J is a positive integer greater than 1 and less than or equal to N.
  • j is a positive integer increasing from 1 to J one by one.
  • the source node determines, from the distributed storage system, N storage nodes, including: Determining, by the source node, power consumption information of the M storage nodes in the distributed storage system, where the power consumption information is a transmission power consumption of each of the M storage nodes to the source node, where a positive integer greater than or equal to N; the source node selects the N storage nodes from the M storage nodes according to power consumption information of the M storage nodes.
  • the source node determines, from the distributed storage system, N storage nodes, including: Determining the delay information of the M storage nodes in the distributed storage system, where M is a positive integer greater than or equal to N; and the source node is based on the delay information of the M storage nodes from the M
  • the N storage nodes are selected among the storage nodes.
  • the source node according to the delay information of the M storage nodes, from the M Selecting the N storage nodes from the storage nodes, the source node selecting, from the M storage nodes, the N storage nodes that meet the first condition, wherein the delay meets the first condition:
  • the delay of the p+1th storage node is less than or equal to the sum of the sum of the data download times of the previous p storage nodes and the delay of the first storage node of the N storage nodes, where p is 1 is incremented by one to a positive integer of N-1. .
  • the source node selects, from the M storage nodes, a delay that satisfies the first condition Storage nodes, comprising: said source node as a first one of said N storage nodes as said first storage node of said M storage nodes; said source node being removed from said M storage nodes
  • the remaining N-1 storage nodes are sequentially determined in the order from 2 to N in the storage nodes other than the first storage node, wherein each of the remaining N-1 storage nodes The delay meets the first condition.
  • the source node, the first storage node of the M storage nodes is used as the a first storage node of the N storage nodes, comprising: the source node selecting a storage node from the M storage nodes according to the delay information, or the bandwidth information, or the power consumption information The first storage node.
  • the source node removes the first storage node from the M storage nodes When determining the xth storage node in the outer storage node, if the source node determines at least two candidate storage nodes that satisfy the first condition, the source node further transmits the bandwidth according to the delay, or the transmission bandwidth The power consumption selects one storage node from the at least two candidate storage nodes that satisfy the first condition as the xth storage node, where x is greater than or equal to 2 and less than or equal to N.
  • an embodiment of the present invention provides a computing device readable medium, including a computing device executing instructions, when the processor of the computing device executes the computing device to execute an instruction, the computing device performs the first aspect or the first Aspect of any of the possible implementations of the method.
  • an embodiment of the present invention provides a data storage device, including: a processor, a memory, a bus, and a communication interface; the memory is configured to store a computer to execute an instruction, and the processor and the memory pass the bus Connecting, when the data storage device is in operation, the processor executing the computer-executed instructions stored by the memory to cause the data storage device to perform the first aspect or any of the possible implementations of the first aspect The method described.
  • the embodiment of the present invention provides a data storage method, where the data of the source node to be stored is distributed to the distributed storage system, including: the source node acquiring the distributed storage system.
  • Power consumption information of M storage nodes the power consumption information is a transmission power consumption of each of the M storage nodes to the source node, where M is a positive integer greater than 1; Determining, by the source node, N storage nodes from the M storage nodes according to power consumption information of the M storage nodes, where N is a positive integer greater than 1 and less than or equal to M; Determining the stored data, assigning each of the N storage nodes a corresponding data to be stored; the source node assigns each of the N storage nodes to the storage node The stored data is sent to each of the N storage nodes.
  • the source node determines, according to the power consumption information of the M storage nodes, the N storage nodes from the M storage nodes, including: The source node acquires delay information of the M storage nodes; the source node selects, from the M storage nodes, an alternative storage node that meets a first condition, wherein the delay satisfies the first condition
  • the delay of the p+1th storage node is less than or equal to the sum of the sum of the data download times of the previous p storage nodes and the delay of the first storage node of the N storage nodes, the p A positive integer that is incremented from 1 to N-1; the source node selects the N storage nodes from the candidate storage nodes according to power consumption information of the candidate storage node.
  • the source node according to the power consumption information of the M storage nodes, from the M Determining N storage nodes among the storage nodes, comprising: the source node using a first storage node of the M storage nodes as a first storage node of the N storage nodes;
  • the power consumption information sequentially determines the remaining N-1 storage nodes from the candidate storage nodes in order from the 2nd to the Nth.
  • the source node, the first storage node of the M storage nodes is used as the a first storage node of the N storage nodes, the source node selects, as the first storage node, a storage node with the smallest transmission power consumption from the M storage nodes; or the source node Among the M storage nodes having the smallest delay among the M storage nodes, the storage node with the smallest transmission power consumption is selected as the first storage node.
  • the source node determines, according to the power consumption information, from the candidate storage node The xth storage node, where x is a positive integer greater than 1, the source node selects, from the candidate storage nodes satisfying the first condition, a storage node having the smallest transmission power consumption as the xth storage node; or And the source node selects, as the xth storage node, the storage node with the smallest transmission power consumption among the at least two storage nodes with the largest delay among the candidate storage nodes that satisfy the first condition.
  • the fourth possible implementation manner of the fourth aspect further comprising: the source node acquiring the each of the N storage nodes The bandwidth information of the storage node, the bandwidth information of each of the N storage nodes is a transmission bandwidth of the data transmission path of each of the N storage nodes to the source node
  • the source node allocates corresponding data to be stored to each of the N storage nodes according to the data to be stored, including: the source node according to the data to be stored and the The bandwidth information of each of the N storage nodes is allocated to the storage node to be stored for the storage node.
  • the source node is configured according to the data to be stored and the N storage nodes
  • the bandwidth information of each storage node is allocated to the storage node of the N storage nodes, where the data is to be stored, including: the source node is J of the N storage nodes.
  • the storage node allocates corresponding data to be stored, wherein the size of the data to be stored allocated by the jth storage node of the J storage nodes is positively correlated with the transmission bandwidth of the jth storage node.
  • J is a positive integer greater than 1 and less than or equal to N
  • j is a positive integer increasing from 1 to J one by one.
  • an embodiment of the present invention provides a computing device readable medium, including a computing device executing instructions, when the processor of the computing device executes the computing device to execute an instruction, the computing device performs the fourth aspect or the fourth Aspect of any of the possible implementations of the method.
  • an embodiment of the present invention provides a data storage device, including: a processor, a memory, a bus, and a communication interface; the memory is configured to store a computer to execute an instruction, and the processor and the memory pass the bus Connecting, when the data storage device is in operation, the processor executing the computer-executed instructions stored by the memory to cause the data storage device to perform any of the possible implementations of the fourth aspect or the fourth aspect The method described.
  • the embodiment of the present invention provides a data storage device, configured to store data to be stored in a distributed storage system, including: a determining unit, configured to determine N storages from the distributed storage system. a node, where N is a positive integer greater than 1; an obtaining unit, configured to acquire bandwidth information of each of the N storage nodes, and bandwidth information of each of the N storage nodes a transmission bandwidth of a data transmission path of the storage node to the source node of each of the N storage nodes; an allocation unit, configured to use, according to the data to be stored, and the N storage nodes Describe the bandwidth information of each storage node, for the N storages Each of the storage nodes in the node allocates corresponding data to be stored; the sending unit is configured to send, to the N, data to be stored allocated by each of the N storage nodes Each of the storage nodes in the storage node.
  • the allocating unit is configured to use, according to the data to be stored, and the bandwidth information of each of the N storage nodes, Allocating corresponding data to be stored to each of the N storage nodes
  • the: the allocation unit is configured to respectively allocate corresponding storage to be saved to the J storage nodes of the N storage nodes Data, wherein the size of the data to be stored allocated by the jth storage node of the J storage nodes is positively correlated with the transmission bandwidth of the jth storage node, and the J is greater than 1 and less than or equal to A positive integer of N, which is a positive integer that increases from 1 to J one by one.
  • the acquiring unit is further configured to acquire M storage nodes in the distributed storage system Power consumption information, which is a transmission power consumption of each of the M storage nodes to the source node, where M is a positive integer greater than or equal to N;
  • the determining unit is used to Determining the N storage nodes by the distributed storage system includes: the determining unit, configured to select the N storage nodes from the M storage nodes according to power consumption information of the M storage nodes.
  • the acquiring unit is further configured to acquire M storage nodes in the distributed storage system, in a third possible implementation manner of the seventh aspect, Delay information, M is a positive integer greater than or equal to N; the determining unit is configured to determine N storage nodes from the distributed storage system, and the determining unit is configured to use a delay according to the M storage nodes Information, selecting the N storage nodes from the M storage nodes.
  • the determining unit is configured to: according to the delay information of the M storage nodes, Selecting the N storage nodes from the M storage nodes, the determining unit is configured to select, from the M storage nodes, N storage nodes that satisfy a first condition, wherein the delay satisfies A condition is: the delay of the p+1th storage node is less than or equal to the sum of the sum of the data download time of the previous p storage nodes and the delay of the first storage node of the N storage nodes,
  • the p is a positive integer that increases from 1 to N-1. .
  • the determining unit is configured to select a delay from the M storage nodes to satisfy the first condition N storage nodes, including: the determining unit is configured to use the M storage nodes a first storage node as the first one of the N storage nodes; the determining unit is configured to remove the storage node other than the first storage node from the M storage nodes according to The sequence of N determines the remaining N-1 storage nodes in turn, wherein the delay of each of the remaining N-1 storage nodes satisfies the first condition.
  • the determining unit is configured to use the first storage node of the M storage nodes as The first storage node of the N storage nodes, the determining unit, configured to select, according to the delay information, or the bandwidth information, or the power consumption information, from the M storage nodes A storage node acts as the first storage node.
  • the storage node other than the first storage node is removed from the M storage nodes
  • the determining unit determines at least two candidate storage nodes that satisfy the first condition
  • the determining unit is further configured to use a delay, or a transmission bandwidth, or transmit
  • the power consumption selects one storage node from the at least two candidate storage nodes that satisfy the first condition as the xth storage node, where x is greater than or equal to 2 and less than or equal to N.
  • the embodiment of the present invention provides a data storage system, comprising the data storage device and the plurality of storage nodes according to any one of the seventh aspect or the seventh aspect, for using the data
  • the data distribution of the storage device is stored in N storage nodes of the plurality of storage nodes, where N is a positive integer greater than one.
  • an embodiment of the present invention provides a data storage device, configured to store data to be stored in a distributed storage system, including: an acquiring unit, configured to acquire M devices in the distributed storage system.
  • the power consumption information of the storage node wherein the power consumption information is a transmission power consumption of each of the M storage nodes to the source node, where M is a positive integer greater than 1; and the determining unit is configured to The power consumption information of the M storage nodes, determining N storage nodes from the M storage nodes, where N is a positive integer greater than 1 and less than or equal to M; and an allocation unit, configured to be used according to the to-be-stored Data, each of the N storage nodes is allocated corresponding data to be stored; and a sending unit is configured to allocate the data to be stored to each of the N storage nodes Sended to each of the N storage nodes.
  • the acquiring unit is further configured to acquire delay information of the M storage nodes
  • the determining unit is configured to use, according to the M Determining the power consumption information of the node, determining N storage nodes from the M storage nodes, including: the determining unit, configured to select, from the M storage nodes, an alternate storage node that meets a first condition with a delay, wherein The delay satisfies the first condition that the delay of the p+1th storage node is less than or equal to the sum of the data download times of the previous p storage nodes and the first storage node of the N storage nodes. a sum of delays, wherein p is a positive integer increasing from 1 to N-1; the determining unit is configured to select from the candidate storage nodes according to power consumption information of the candidate storage node N storage nodes are described.
  • the determining unit is configured to use, according to the power consumption information of the M storage nodes, Determining N storage nodes among the M storage nodes, comprising: the determining unit configured to use a first storage node of the M storage nodes as a first storage node of the N storage nodes; The determining unit is configured to sequentially determine the remaining N-1 storage nodes from the candidate storage nodes in order from the 2nd to the Nth according to the power consumption information.
  • the determining unit is configured to use the first storage node of the M storage nodes as The first storage node of the N storage nodes, the determining unit is configured to select, from the M storage nodes, a storage node with the smallest transmission power consumption as the first storage node; or Among the M storage nodes having the smallest delay among the M storage nodes, the storage node with the smallest transmission power consumption is selected as the first storage node.
  • the determining unit is configured to use the power storage information from the candidate storage node When the xth storage node is determined, wherein x is a positive integer greater than 1, the determining unit is configured to select, from the candidate storage nodes that satisfy the first condition, a storage node with the smallest transmission power consumption as the xth The storage node; or, from the at least two storage nodes having the largest delay among the candidate storage nodes satisfying the first condition, selecting the storage node with the smallest transmission power consumption as the xth storage node.
  • the obtaining unit is further configured to acquire each of the N storage nodes
  • the bandwidth information of the storage node, the bandwidth information of each of the N storage nodes is a transmission bandwidth of the data transmission path of each of the N storage nodes to the source node
  • the allocation unit is configured to give the N storages according to the data to be stored Each storage node in the node allocates corresponding data to be stored, including: the allocation unit is configured to use, according to the data to be stored, and bandwidth information of each of the N storage nodes, Each of the N storage nodes allocates corresponding data to be stored.
  • the allocating unit is configured to use the data to be stored and the N storage nodes All the storage nodes of the N storage nodes are allocated corresponding data to be stored, including: the allocation unit is used for the N storage nodes
  • the J storage nodes are allocated corresponding data to be stored, wherein the size of the data to be stored allocated by the jth storage node of the J storage nodes is positive with the transmission bandwidth of the jth storage node Correlation, the J is a positive integer greater than 1 and less than or equal to N, and the j is a positive integer increasing from 1 to J one by one.
  • the embodiment of the present invention provides a data storage system, comprising the data storage device and the plurality of storage nodes according to any of the possible implementations of the ninth aspect or the ninth aspect,
  • the data distribution of the storage device is stored in N storage nodes of the plurality of storage nodes, where N is a positive integer greater than one.
  • a data block size is allocated for each storage node according to a transmission bandwidth, and a large data block is allocated to a storage node with a large transmission bandwidth, thereby improving data transmission efficiency and shortening data access.
  • Time has realized the rational use of resources.
  • the storage power consumption of the storage node is taken into consideration, and the storage node that transmits the power consumption is preferentially selected, thereby reducing the power consumption of the data access.
  • FIG. 1 is a block diagram of an exemplary networked environment of a distributed data storage system
  • FIG. 2 is an exemplary flowchart of a data storage method according to an embodiment of the present invention.
  • FIG. 3 is an exemplary flowchart of a data storage method according to an embodiment of the invention.
  • FIG. 4 is a schematic diagram showing the logical structure of a data storage device according to an embodiment of the invention.
  • FIG. 5 is a schematic diagram showing the logical structure of a data storage device according to an embodiment of the invention.
  • FIG. 6 is a schematic diagram showing the hardware structure of a data storage device according to an embodiment of the invention.
  • FIG. 1 shows an exemplary networked environment block diagram of a distributed data storage system 100 that includes a source node 102 and a plurality of storage nodes 104, a source node 102 and a storage node 104, and a storage node 104 between the switching devices 106. Interconnected, the various components of the entire system 100 are connected to each other through a network, wherein the network may be the Internet, an intranet, a local area network (LANs), a wide area network (WLANs), a storage area network (SANs), or other connection means, or a combination of the above networks. .
  • LANs local area network
  • WLANs wide area network
  • SANs storage area network
  • source node 102 When source node 102 is to distribute data 108 across a plurality of storage nodes 104, source node 102 segments data 108 and sends the data blocks to corresponding storage nodes 104.
  • the data 108 is divided into five data blocks, but the embodiment of the present invention does not limit the data. In the specific implementation, the data 108 can be divided into more or less data blocks.
  • the source node 102 selects five storage nodes 104 from the plurality of storage nodes 104 connected thereto as the target storage node 104, and transmits the five data blocks to the five target storage nodes 104 for storage.
  • FIG. 1 is only for the participants of the distributed storage and their interrelationships for the purposes of the following discussion.
  • the depicted distributed data storage system 100 is greatly simplified, and the system 100 can have more or fewer components, which are not limited by the embodiments of the present invention.
  • a distributed storage system 100 in accordance with an embodiment of the present invention is shown in FIG.
  • the data 108 for distributing the source node 102 is distributed to N storage nodes 104 in the distributed storage system 100, where N is a positive integer greater than one.
  • the source node 102 determines N storage nodes 104 from the distributed storage system 100, where N is a positive integer greater than one.
  • the source node 102 determines N storage nodes 104 from the distributed storage system 100, including: the source node 102 acquires the distributed storage.
  • the power consumption information of the M storage nodes 104 in the storage system 100, the power consumption information is the transmission power consumption of each of the M storage nodes 104 to the source node 102, where M is greater than or equal to A positive integer of N; the source node 102 selects the N storage nodes 104 from the M storage nodes 104 according to power consumption information of the M storage nodes 104.
  • the transmission power consumption refers to the power consumption required for the transmission of the data block of the unit size from the corresponding storage node 104 to the source node 102.
  • the M storage nodes 104 are storage nodes 104 connected to the source node 102. It should be understood that the connection here means that the network topology implements the interconnection between the source node 102 and the M storage nodes 104, and the connection is not only Limited to direct connection, there may be one or more (including two) switching devices 106 between the source node 102 and the storage node 104.
  • the M storage nodes 104 may be all storage nodes 104 connected to the source node 102.
  • the M storage nodes 104 may also be portions of the storage nodes 104 of all of the storage nodes 104 connected to the source node 102.
  • the M storage nodes 104 are storage nodes 104 that satisfy the actual needs of the source node 102, such as load and delay, and can be actually used by the source node 102.
  • the transmission delay of the data transmission path to the source node 102 is greater than a certain threshold, or the load condition is greater than a certain threshold, or the transmission power consumption of the data transmission path to the source node 102 is greater than a certain threshold, or
  • the storage node 104 whose number of hops of the data transmission path of the source node 102 is greater than a certain threshold, can be considered as the storage node 104 that cannot be actually used by the source node 102.
  • the number of hops of the data transmission path reflects the power consumption of the data transmission path when the data transmission path is transmitted. The more hops, the greater the transmission power consumption.
  • the hop count of the path of the storage node 104 to the source node 102 may be used to reflect the transmission power consumption of the data transmission path of the storage node 104 to the source node 102.
  • the transmission power of the data transmission path is not only related to the hop count, but also related to the processing time of the data, the longer the processing time of the data, or the longer the waiting time of the data in the cache, consumes more energy.
  • the transmission delay of the data transmission path of the storage node 104 to the source node 102 may also be used to reflect the transmission power consumption of the corresponding storage node 104 to the source node 102.
  • the transmission power consumption can be expressed in various forms, and can be represented by the hop count of the data transmission path, or by the transmission delay of the data transmission path, or actual statistics can be performed.
  • the embodiments of the present invention are merely illustrative, and do not limit the expression of transmission power consumption, and allow other alternative expressions of transmission power consumption.
  • the source node 102 acquires power consumption information of the M storage nodes 104 in the distributed storage system 100, including: the source node 102 stores the M storages.
  • the node 104 respectively sends a query request; the source node 102 receives a receipt message from each of the M storage nodes 104, and each of the receipt messages includes power consumption information of the corresponding storage node 104.
  • the return message can be used to record the routing device that passes through the transmission process from the storage node 104 to the source node 102. number.
  • the source node 102 can send a query request to all of the storage nodes 104 connected to the source node 102, and only the M storage nodes 104 that can actually satisfy the storage requirements of the source node 102 send a return message to the source node 102. .
  • the storage node 104 that cannot satisfy the storage requirements of the source node 102 may not respond to the storage resource request of the source node 102 or send a reject message to the source node 102.
  • the transmission delay of the data transmission path to the source node 102 is greater than a certain threshold, or the load condition is greater than a certain threshold, or the transmission power consumption of the data transmission path to the source node 102 is greater than a certain threshold, or to the source
  • the storage node 104 whose hop count of the data transmission path of the node 102 is greater than a certain threshold may not respond to the storage resource request of the source node 102 or send a reject message.
  • the system 100 further includes a management server (not shown), and the management server is configured to maintain node information of the M storage nodes 104, where the node information includes work. Consumption information.
  • the source node 102 acquires power consumption information of the M storage nodes 104 in the distributed storage system 100, including: the source node 102 sends a query request to the management server; and the source node 102 receives the return message from the management server.
  • the receipt message includes power consumption information of each of the M storage nodes 104.
  • the management server may perform real-time statistics and update on the transmission power consumption of the data transmission path of the storage node 104 to the source node 102.
  • the management server may be an SDN controller, and the type of the management server may be in various implementation forms, which is not limited by the embodiment of the present invention.
  • the source node 102 includes a database.
  • the database stores node information of each of the M storage nodes 104, the node information including power consumption information.
  • the source node 102 acquires power consumption information of the M storage nodes 104 in the distributed storage system 100, including: the source node 102 acquires each of the M storage nodes 104 from the database. Power consumption information.
  • the node information of the destination node 104 connected to the source node 102 may be placed in the database in the source node 102 when the system is initialized.
  • the source node 102 determines N storage nodes 104 from the distributed storage system 100, including: the source node 102 acquires M in the distributed storage system 100.
  • the delay information of the storage node 104, M is a positive integer greater than or equal to N; the source node 102 selects the N from the M storage nodes 104 according to the delay information of the M storage nodes 104.
  • Storage node 104 storage node 104.
  • the delay information of each of the M storage nodes 104 is that the source node 102 sends a data access request time to each of the storage nodes 104 to the first data packet of each storage node 104.
  • the time between the time of sending to the source node 102 includes the access delay of each of the storage nodes 104 and the transmission delay of the data transmission path of each of the storage nodes 104 to the source node 102.
  • the access delay of the storage node 104 is the time between the source node 102 issuing an access request to the storage node 104 transmitting the first data packet of the data block.
  • the access delay may include a setup time of the link between the source node 102 and the storage node 104 and a preparation time before the storage node 104 sends the data block.
  • the delay is composed of multiple parts, and specifically may include a first delay that the source node 102 sends an access request to the storage node 104 to the storage node 104, and the storage node 104 reads the stored data.
  • the second latency of the fastest primary storage device is ready, and the third delay that the storage node 104 sends the first data packet to the source node 102 to receive the first data packet.
  • a part of the delay may be omitted, and a part of the delay represents the overall delay, which is not limited by the embodiment of the present invention, as long as the delay is to a certain extent.
  • the source node 102 may obtain the query in the distributed storage system 100 by sending a query request to the storage node 104, or by sending a query request to the management server, or storing the delay information in a local database. Delay information of the M storage nodes 104,
  • the specific implementation process is the same as the method for obtaining power consumption information, and details are not described herein again.
  • the setup time of the link of the source node 102 with the storage node 104, the preparation time before the storage node 104 sends the data block, and the transmission delay of the data transmission path of the storage node 104 to the source node 102 can be monitored and recorded by the management server.
  • the source node 102 sends a query request to the management server, and the delay message carried by the management server to the source node 102 carries the delay information.
  • a time record field may also be added to the communication message between the source node 102 and the storage node 104 for recording the time required to establish the link and the transmission time of the data transmission path of the storage node 104 to the source node 102. Delay.
  • the management server can detect the transmission delay of the data transmission path by using an Operation Administration and Maintenance (OAM) feature of the data transmission path. It is also possible for the management server to determine the transmission delay of the data transmission path by the empirical value of the recorded transmission delay.
  • OAM Operation Administration and Maintenance
  • the source node 102 further acquires bandwidth information of each of the N storage nodes 104, and the bandwidth information of each of the N storage nodes 104 is the N The transmission bandwidth of each of the storage nodes 104 to the data transmission path of the source node 102.
  • the source node 102 determines N storage nodes 104 from the distributed storage system 100, including: the source node 102 according to bandwidth information of the M storage nodes 104.
  • the N storage nodes 104 are selected from the M storage nodes 104.
  • the transmission bandwidth is a bandwidth reserved by the system for the data transmission path of each of the N storage nodes 104 to the source node 102, and the actual data transmission is performed by the data transmission path. Average transmission bandwidth.
  • the source node 102 may obtain the query request in the distributed storage system 100 by sending a query request to the storage node 104, or by sending a query request to the management server, or storing the bandwidth information in a local database.
  • the specific implementation process of the bandwidth information of the M storage nodes 104 is the same as the method for obtaining the power consumption information, and details are not described herein again.
  • the source node 102 determines the N storage nodes 104 from the M storage nodes 104 according to at least one of the transmission power consumption, the delay, and the transmission bandwidth of the storage node 104.
  • the source node 102 determines N storage nodes from the distributed storage system 100 104.
  • the source node 102 selects the N storage nodes 104 with the smallest delay from the M storage nodes 104, or selects the transmission function from the L storage nodes 104 with the smallest delay among the M storage nodes 104.
  • N storage nodes 104 that consume the least or the largest transmission bandwidth, where L is a positive integer greater than N.
  • the source node 102 determines N storage nodes 104 from the distributed storage system 100, including: the source node 102 selects, from the M storage nodes 104, N storage nodes 104 that have the lowest transmission power consumption, or N storage nodes 104 with the smallest delay or the largest transmission bandwidth are selected from the L storage nodes 104 with the least power consumption transmitted from the M storage nodes 104, where L is a positive integer greater than N.
  • the source node 102 determines the N storage nodes 104 from the distributed storage system 100, including: the source node 102 selects the N storage nodes 104 with the largest transmission bandwidth from the M storage nodes 104, or Among the L storage nodes 104 having the largest transmission bandwidth among the M storage nodes 104, N storage nodes 104 with the smallest transmission power consumption or the smallest delay are selected, where L is a positive integer greater than N.
  • the source node 102 selects the N storage nodes 104 from the M storage nodes 104 according to the delay information of the M storage nodes 104, including: The source node 102 selects, from the M storage nodes 104, the N storage nodes 104 that satisfy the first condition, wherein the delay satisfies the first condition: the delay of the p+1th storage node is less than Or equal to the sum of the sum of the data download times of the previous p storage nodes and the delay of the first storage node of the N storage nodes 104, where p is a positive integer that is incremented from 1 to N-1. .
  • the data download time of one storage node 104 is the ratio between the corresponding data size K to be stored allocated by the storage node 104 and the transmission bandwidth B of the data transmission path of the storage node 104 to the source node 102.
  • the source node 102 uses a first storage node of the M storage nodes 104 as a first storage node among the N storage nodes 104; the source node 102 from the M The storage node 104 removes the remaining N-1 storage nodes 104 in order from 2 to N in the storage node 104 other than the first storage node, wherein the remaining N-1 storages The delay of each of the storage nodes 104 in the node 104 satisfies the first condition.
  • one storage node 104 is selected as the p+1th storage node from the storage node 104 that satisfies the following conditions. :
  • L 1 is used to represent the time delay of the first storage node
  • K i is used to represent the data block size allocated for the i th storage node 104
  • B i is used to represent the i th storage node 104
  • the transmission bandwidth, L p+1 is used to represent the delay of the p+1th target node.
  • the source node 102 uses the first storage node of the M storage nodes 104 as the first storage node of the N storage nodes 104, including: The source node 102 selects one storage node 104 from the M storage nodes 104 as the first storage node according to the delay information, or the bandwidth information, or the power consumption information.
  • the source node 102 uses the first storage node of the M storage nodes 104 as the first storage node of the N storage nodes 104, and includes: the source node 102 from the plurality of storage nodes 104.
  • the storage node 104 with the smallest sum of the delay and the data block download time is selected as the first storage node, that is, the content is satisfied.
  • the storage node 104 having the smallest value serves as the first storage node.
  • L is the delay of the storage node 104
  • K is the data block size allocated to the storage node 104
  • B is the transmission bandwidth of the storage node 104.
  • the source node 102 uses the first storage node of the M storage nodes 104 as the first storage node of the N storage nodes 104, and includes: the source node 102 from the M storage nodes.
  • the storage node 104 having the smallest delay is selected, or the storage node 104 having the largest transmission bandwidth or the smallest transmission power consumption is selected as the first storage node from among the L storage nodes 104 having the smallest delay among the M storage nodes 104.
  • L is a positive integer greater than one.
  • the source node 102 uses the first storage node of the M storage nodes 104 as the first storage node of the N storage nodes 104, and includes: the source node 102 from the M storage nodes. 104 selects the storage node 104 with the smallest transmission power consumption, or selects the largest transmission bandwidth or the longest delay among the L storage nodes 104 that have the least power consumption from the M storage nodes 104.
  • the small storage node 104 acts as a first storage node, where L is a positive integer greater than one.
  • the source node 102 uses the first storage node of the M storage nodes 104 as the first storage node of the N storage nodes 104, and includes: the source node 102 from the M storage nodes. 104, the storage node 104 having the largest transmission bandwidth is selected, or the storage node 104 having the smallest delay or the smallest transmission power consumption is selected as the first storage node from among the L storage nodes 104 having the largest transmission bandwidth among the M storage nodes 104.
  • L is a positive integer greater than one.
  • the basis for the source node 102 to determine the first storage node from the M storage nodes 104 is various, and the embodiments of the present invention are merely examples and are not limited thereto.
  • the source node 102 when the source node 102 removes the xth storage node 104 from the storage node 104 except the first storage node, the The source node determines at least two candidate storage nodes 104 that satisfy the first condition, the source node further according to a delay, or a transmission bandwidth, or transmitting power consumption from at least two satisfying the first condition
  • One of the candidate storage nodes 104 is selected as the xth storage node 104, where x is greater than or equal to 2 and less than or equal to N.
  • the source node 102 selects the storage node 104 with the largest delay from the at least two candidate storage nodes 104 that satisfy the first condition, or from the at least two candidate storage nodes 104 that satisfy the first condition.
  • the storage node 104 having the largest transmission bandwidth or the smallest transmission power consumption is selected among the L largest storage nodes 104 as the xth storage node 104, where L is a positive integer greater than 1.
  • the source node 102 selects the storage node 104 having the largest transmission bandwidth from the at least two candidate storage nodes 104 that satisfy the first condition, or from at least two candidate storage nodes 104 that satisfy the first condition.
  • the storage node 104 having the largest transmission bandwidth is selected as the xth storage node 104, where L is a positive integer greater than 1.
  • the source node 102 selects the storage node 104 that has the lowest transmission power consumption from the at least two candidate storage nodes 104 that satisfy the first condition, or from the at least two candidate storage nodes 104 that satisfy the first condition.
  • the storage node 104 having the smallest transmission power consumption the storage node 104 having the largest delay or the largest transmission bandwidth is selected as the xth storage node 104, where L is a positive integer greater than one.
  • source node 102 determines that the basis of the xth storage node 104 is diverse, and the embodiments of the present invention are merely examples and are not limited thereto.
  • the source node 102 gives the N storage nodes 104 according to the data to be stored. Each storage node 104 allocates corresponding data to be stored.
  • the source node 102 allocates corresponding data to be stored to each of the N storage nodes 104 according to the data to be stored, including: The source node 102 divides the data into N equal-sized data blocks and allocates one data block for each storage node 104. It will be appreciated that the data may not be equally evenly divided into N equal-sized data blocks because of the minimum partitioning unit of the data, and embodiments of the present invention allow for adjustments to the data blocks on a uniform basis.
  • the source node 102 divides the data into N data blocks, and sorts the N data blocks according to the data, and the source node 102 allocates the data blocks to the corresponding storage nodes 104, thereby ensuring the data blocks. In order, when accessing data in the future, it can be accessed according to the order of the data blocks.
  • the source node 102 acquires a buffer size of each of the N storage nodes 104, and the buffer is a corresponding storage node 104 for receiving data.
  • the storage node 104 allocates corresponding data to be stored to each of the N storage nodes 104 according to the data to be stored, including: the source node 102 will store the node 104 A data block of an integer multiple of the buffer size is allocated to the corresponding storage node 104.
  • the source node 102 may allocate a data block of a buffer size of the corresponding storage node 104 to the corresponding storage node 104.
  • the source node 102 if the source node 102 allocates a data block of L size of the buffer size of the corresponding storage node 104 to the corresponding storage node 104, the source node 102 splits the data block into L sizes. A sub-block of buffer size, and respectively send L sub-blocks to a corresponding storage node 104, where L is a positive integer greater than one.
  • the source node 102 allocates corresponding data to be stored to each of the N storage nodes 104 according to the data to be stored, including:
  • the source node 102 is the each of the N storage nodes 104 according to the data to be stored and the bandwidth information of each of the storage nodes 104 in the N storage nodes 104. Assign the corresponding data to be stored.
  • the source node 102 is the each of the N storage nodes 104 according to the data to be stored and the bandwidth information of each of the storage nodes 104 in the N storage nodes 104.
  • Allocating corresponding data to be stored includes: the source node 102 respectively assigns corresponding data to be stored to the J storage nodes of the N storage nodes 104, wherein the The size of the data to be stored allocated by the jth storage node of the J storage nodes is positively correlated with the transmission bandwidth of the jth storage node, and the J is a positive integer greater than 1 and less than or equal to N, j is a positive integer that increments from 1 to J one by one.
  • the source node is the first N-1 according to the transmission bandwidth of the first N-1 storage nodes.
  • Each storage node in the storage node allocates a data block, and the size of the data to be stored allocated by each storage node is positively correlated with the transmission bandwidth of the storage node, and the remaining data is allocated to the Nth storage node.
  • the source node 102 allocates a data block to each of the N storage nodes 104 according to the transmission bandwidth of each of the N storage nodes 104, and the size of the data block. It is positively related to the transmission bandwidth.
  • each storage node 104 can be allocated a data block according to a transmission bandwidth ratio, and the allocation is as follows:
  • K i is the data block size allocated by the i th storage node 104
  • B i is the transmission bandwidth of the i th storage node 104
  • K is the size of the entire data.
  • the positive proportional relationship in Equation 1 is only a specific implementation of the positive correlation relationship. In the specific implementation process, the actual data block size can be adjusted on the basis of Equation 1. It should be understood that the positive proportional relationship of Equation 1 is not an absolute proportional relationship because of the limitation of the minimum division unit of data. Embodiments of the present invention allow adjustment of the data block size on the basis of Equation 1.
  • the source node 102 can also set the data block size K 0 of the unit bandwidth allocation, and the allocation basis can be as follows:
  • K i is the data block size allocated for the i th storage node 104
  • B i is the transmission bandwidth of the i th storage node 104
  • K 0 is the data block size allocated for the unit bandwidth.
  • the embodiment of the present invention allows the data block size of the storage node 104 to have other constraint relationship with the transmission bandwidth of the storage node 104.
  • the source node 102 determines the N storage nodes 104 one by one from the M storage nodes 104, and the source node 102 determines the jth storage. After storing the node, and before the source node 102 determines the j+1th storage node from the M storage nodes 104, allocate a data block for the jth storage node, where j is a positive integer greater than 0 and less than N .
  • the source node 102 is configured according to the data to be stored and the bandwidth information of each of the storage nodes 104 in the N storage nodes 104.
  • Each of the N storage nodes 104 allocates corresponding data to be stored, including: the source node 102 transmits the size and the transmission of the jth storage node before determining the j+1th storage node.
  • a data block whose bandwidth is positively correlated is allocated to the jth storage node, j is a positive integer increasing from 1 to N-1, and the j+1th storage node is a next node of the jth storage node;
  • the source node 102 assigns the remainder of the data to the Nth storage node 104.
  • the source node 102 determines the N storage nodes 104 from the M storage nodes 104, and assigns a corresponding correspondence to each of the N storage nodes 104 according to the data to be stored.
  • the data to be stored including:
  • Step A The source node 102 determines the first storage node from the M storage nodes 104, and allocates a data block to the first storage node according to the transmission bandwidth of the first storage node.
  • the source node 102 allocates a data block to the first storage node according to the transmission bandwidth of the first storage node, and includes: the source node 102, the data whose size is positively correlated with the transmission bandwidth of the first storage node.
  • a block is allocated to the first storage node.
  • the data block size, B 1 is the transmission bandwidth of the first storage node
  • K 0 is the data block size allocated by the unit bandwidth.
  • Step B The source node 102 determines the j+1th storage node from the M storage nodes 104 according to the transmission power consumption, and the delay of the j+1th storage node is not greater than the previous j storage nodes. The download time of the data block and the delay of the first storage node. Perform step C.
  • the initial value of j is 1.
  • a storage node 104 is selected as the j+1th storage node according to the transmission power consumption from the storage node 104 that satisfies the following conditions:
  • L 1 is used to represent the time delay of the first storage node
  • K i is used to represent the data block size allocated for the i th storage node 104
  • B i is used to represent the i th storage node 104
  • the transmission bandwidth, L j+1 is used to represent the delay of the j+1th target node.
  • the source node 102 satisfies the condition of the formula 5 from the M storage nodes 104, and the node in the storage node 104 that has not been selected is determined to have the smallest transmission power consumption as the j+1th storage node.
  • the source node 102 satisfies the condition of the formula 5 from the M storage nodes 104, and the node with the smallest transmission power is determined as the jth among the L storage nodes 104 having the largest delay among the storage nodes 104 that have not been selected. +1 storage node, where L is a positive integer greater than one.
  • the source node 102 satisfies the condition of the formula 5 from the M storage nodes 104, and the node with the largest delay is determined among the L storage nodes 104 that have the least power consumption in the storage node 104 that has not been selected.
  • the source node 102 satisfies the condition of the formula 5 from the M storage nodes 104, and the node that determines the transmission bandwidth is the largest among the L storage nodes 104 that have the least power consumption in the storage node 104 that has not been selected. +1 storage node, where L is a positive integer greater than one.
  • the source node 102 satisfies the condition of the formula 5 from the M storage nodes 104, and the node that determines the transmission power consumption is the least in the L storage nodes 104 that have the largest transmission bandwidth among the storage nodes 104 that have not been selected. +1 storage node, where L is a positive integer greater than one.
  • the source node 102 determines the basis of the j+1th storage node from the M storage nodes 104 according to the transmission power consumption.
  • the embodiments of the present invention are merely examples and are not limited thereto. .
  • Step C Determine whether the following conditions are met:
  • K is the size of the remainder of the data
  • B j+1 is the transmission bandwidth of the j+1th storage node
  • K 0 is the data block size allocated by the unit bandwidth.
  • the source node 102 transmits data to be stored to the storage node 104 of the N storage nodes 104 to the storage node 104 of the N storage nodes 104.
  • the maximum delay allowed by the storage node 104 is sequentially determined by predicting the data block access time. According to the delay, selecting the appropriate storage node 104 can effectively utilize the resources of the distributed network, and can comprehensively consider the delay of the storage node 104 on the basis of ensuring the continuous download of the data block, without having to select the delay each time.
  • the small storage node 104 implements load balancing of the distributed network.
  • the download time of the data block is continuous, and the download time of the final data access is:
  • an access request is sent to the storage node 104 storing the data block, where the access request instructs the first storage node to immediately send a data block, indicating the j+1th storage. Node waiting
  • L 1 is used to represent the time delay of the first storage node
  • K i is used to represent the data block size allocated for the i th storage node 104
  • B i is used to represent the i th storage node 104
  • the transmission bandwidth, L j+1 is used to represent the delay of the j+1th target node, and j is a positive integer greater than 1.
  • the node information further includes a size of a remaining storage space of the corresponding storage node 104, and the source node 102 allocates a data block to the corresponding storage node 104 within a range of the remaining storage space size, and the size of the allocated data block. It is not larger than the size of the remaining storage space of the corresponding storage node 104.
  • a data block size is allocated for each storage node according to a transmission bandwidth, and a large data block is allocated to a storage node with a large transmission bandwidth, thereby improving data transmission efficiency and shortening data access.
  • Time has realized the rational use of resources.
  • FIG. 2 is an exemplary flowchart of a data storage method 200 according to an embodiment of the present invention.
  • the method 200 is applied to a distributed storage system for storing data of a source node to be stored in a distributed storage system, as shown in FIG.
  • method 200 includes:
  • the source node determines N storage nodes from the distributed storage system, where N is a positive integer greater than 1.
  • the source node acquires bandwidth information of each storage node of the N storage nodes, where bandwidth information of each of the N storage nodes is the one of the N storage nodes.
  • the transmission bandwidth is a bandwidth reserved by the system for each of the N storage nodes to a data transmission path of the source node, and is an average transmission bandwidth of the data transmission path for actual data transmission. .
  • the source node determines N storage nodes from the distributed storage system, including: the source node, according to bandwidth information of the M storage nodes, from the M The N storage nodes are selected among the storage nodes.
  • the source node determines N storage nodes from the distributed storage system, including: the source node acquires M storage nodes in the distributed storage system. Power consumption information, the power consumption information is a transmission power consumption of each of the M storage nodes to the source node, where M is a positive integer greater than or equal to N; the source node is according to the M The power consumption information of the storage node is selected from the M storage nodes.
  • the source node determines N storage nodes from the distributed storage system, including: the source node acquires M storage nodes in the distributed storage system.
  • the delay information, M is a positive integer greater than or equal to N; the source node selects the N storage nodes from the M storage nodes according to the delay information of the M storage nodes.
  • the source node selects the N storage nodes from the M storage nodes according to the delay information of the M storage nodes, including: the source node. And selecting, by the M storage nodes, the N storage nodes that meet the first condition, wherein the delay satisfies the first condition: the delay of the p+1th storage node is less than or equal to the previous p storage nodes.
  • the sum of the sum of the data download times and the delay of the first one of the N storage nodes, the p being a positive integer increasing from 1 to N-1. .
  • the source node sequentially determines the p+1th storage node from the M storage nodes, selecting one storage node as the p+1th storage node from the storage nodes satisfying the following conditions:
  • L 1 is used to represent the time delay of the first storage node
  • K i is used to represent the data block size allocated for the i th storage node
  • B i is used to represent the transmission bandwidth of the i th storage node
  • L p+1 is used to represent the delay of the p+1th target node.
  • the source node selects, from the M storage nodes, the N storage nodes that meet the first condition, and the source node includes the M storage nodes. a first storage node as a first one of the N storage nodes; the source node removes from the M storage nodes a storage node other than the first storage node according to from 2 to N
  • the remaining N-1 storage nodes are sequentially determined in sequence, wherein the delay of each of the remaining N-1 storage nodes satisfies the first condition.
  • the source node uses the first storage node of the M storage nodes as the first storage node of the N storage nodes, and includes: the source node according to the delay information, or the bandwidth information. Or the power consumption information selects one storage node from the M storage nodes as the first storage node.
  • the source node uses the first storage node of the M storage nodes as the first storage node of the N storage nodes, and includes: the source node selects a delay from the plurality of storage nodes.
  • the storage node with the smallest sum of data block download times serves as the first storage node, that is, satisfies
  • the storage node with the smallest value is the first storage node.
  • L is the delay of the storage node
  • K is the data block size allocated to the storage node
  • B is the transmission bandwidth of the storage node.
  • the source node uses the first storage node of the M storage nodes as the first storage node of the N storage nodes, and the source node selects a delay from the M storage nodes.
  • the smallest storage node, or a storage node having the largest transmission bandwidth or the smallest transmission power consumption, is selected as the first storage node from among the L storage nodes having the smallest delay among the M storage nodes, where L is a positive integer greater than 1.
  • the source node uses the first storage node of the M storage nodes as the first storage node of the N storage nodes, and includes: the source node selects a transmission function from the M storage nodes.
  • the storage node with the smallest consumption, or the storage node with the smallest transmission bandwidth or the smallest delay among the L storage nodes with the smallest transmission power among the M storage nodes, is the first storage node, where L is a positive integer greater than 1. .
  • the source node uses the first storage node of the M storage nodes as the N
  • the first storage node of the storage nodes includes: the source node selects a storage node with the largest transmission bandwidth from the M storage nodes, or selects one of the L storage nodes with the largest transmission bandwidth among the M storage nodes.
  • a storage node having a minimum delay or a minimum transmission power consumption is used as the first storage node, where L is a positive integer greater than one.
  • the source node determines the xth storage node from the storage nodes other than the first storage node from the M storage nodes, if the source node determines at least two that satisfy the first condition An alternate storage node, wherein the source node further selects one storage node from the at least two candidate storage nodes that satisfy the first condition as the xth storage node according to a delay, or a transmission bandwidth, or a transmission power consumption, wherein , x is greater than or equal to 2 and less than or equal to N.
  • the source node selects the storage node with the largest delay from the at least two candidate storage nodes that satisfy the first condition, or the L with the largest delay from the at least two candidate storage nodes that satisfy the first condition.
  • a storage node having the largest transmission bandwidth or the smallest transmission power consumption is selected as the xth storage node, where L is a positive integer greater than 1.
  • the source node selects the storage node with the largest transmission bandwidth from the at least two candidate storage nodes that satisfy the first condition, or the transmission bandwidth from the at least two candidate storage nodes that satisfy the first condition.
  • the L storage nodes a storage node having the largest delay or the smallest transmission power consumption is selected as the xth storage node, where L is a positive integer greater than 1.
  • the source node selects a storage node with the smallest transmission power consumption from at least two candidate storage nodes that satisfy the first condition, or transmits power consumption from at least two candidate storage nodes that satisfy the first condition.
  • a storage node having the largest delay or the largest transmission bandwidth is selected as the xth storage node, where L is a positive integer greater than 1.
  • the source node determines the basis of the xth storage node to be various, and the embodiment of the present invention is merely an example and is not limited thereto.
  • the method for the source node to obtain the power consumption information, or the delay information, or the bandwidth information of each of the N storage nodes is described in the system embodiment of FIG. 1 , and details are not described herein again.
  • S206 The source node allocates, according to the data to be stored and the bandwidth information of each of the N storage nodes, the corresponding storage node to each of the N storage nodes. Stored data.
  • the source node is configured as the N storage nodes according to the data to be stored and the bandwidth information of each of the N storage nodes.
  • Each of the storage nodes is configured to allocate corresponding data to be stored, including: the source node respectively assigns corresponding data to be stored to the J storage nodes of the N storage nodes, where the J storages
  • the size of the data to be stored allocated by the jth storage node in the node is positively correlated with the transmission bandwidth of the jth storage node, and the J is a positive integer greater than 1 and less than or equal to N, and the j is a slave 1 increments one by one to a positive integer of J.
  • the source node allocates a data block to each of the N storage nodes according to a transmission bandwidth of each of the N storage nodes, and the size of the data block is positively related to the transmission bandwidth. relationship.
  • each storage node can be allocated a data block according to the bandwidth ratio, and the allocation basis is as follows:
  • K i is the data block size allocated by the i th storage node
  • B i is the transmission bandwidth of the i th storage node
  • K is the size of the entire data.
  • the proportional relationship in Equation 11 is only a specific implementation of the positive correlation. In the specific implementation process, the actual data block size can be adjusted on the basis of Equation 1. It should be understood that the positive proportional relationship of Equation 1 is not an absolute proportional relationship because of the limitation of the minimum division unit of data. Embodiments of the present invention allow adjustment of the data block size on the basis of Equation 1.
  • the source node can also set the data block size K 0 of the unit bandwidth allocation, and the allocation basis can be as follows:
  • K i is the data block size allocated for the i th storage node
  • B i is the transmission bandwidth of the i th storage node
  • K 0 is the data block size allocated for the unit bandwidth
  • the source node determines the N storage nodes one by one from the M storage nodes, and the source node is configured according to the data to be stored and the N The bandwidth information of each of the storage nodes is allocated to the storage node of the N storage nodes, and the data is to be stored, including: the source node is determined After the jth storage node, before determining the j+1th storage node, a data block whose size is positively correlated with the transmission bandwidth of the jth storage node is allocated to the jth storage node, j is a slave 1 is incremented one by one to a positive integer of N-1, the j+1th storage node is the next node of the jth storage node; the source node assigns the remaining portion of the data to the Nth storage node.
  • the source node determines N storage nodes from the distributed storage system, and according to the data to be stored and the bandwidth information of each of the N storage nodes, Each of the N storage nodes allocates corresponding data to be stored, including:
  • Step A The source node determines a first storage node from the M storage nodes, and allocates a data block to the first storage node according to a transmission bandwidth of the first storage node.
  • the allocating a data block to the first storage node according to the transmission bandwidth of the first storage node includes: the data node allocation that the source node has a positive correlation with the transmission bandwidth of the first storage node.
  • the data block size, B 1 is the transmission bandwidth of the first storage node
  • K 0 is the data block size allocated by the unit bandwidth.
  • Step B The source node determines the j+1th storage node from the M storage nodes, and the delay of the j+1th storage node is not greater than the download time of the data block of the first j storage nodes and the first The sum of the delays of the storage nodes. Perform step C.
  • the initial value of j is 1.
  • the source node determines the j+1th storage node
  • one of the storage nodes that satisfies the following condition is selected as the j+1th storage node:
  • L 1 is used to represent the time delay of the first storage node
  • K i is used to represent the data block size allocated for the i th storage node
  • B i is used to represent the transmission bandwidth of the i th storage node
  • L j+1 is used to represent the time delay of the j+1th target node.
  • the source node satisfies the condition of the formula 13 from the M storage nodes, and has not yet been selected.
  • the method for determining the j+1th storage node in the storage node has been described above, and details are not described herein again.
  • Step C Determine whether the following conditions are met:
  • K is the size of the remainder of the data
  • B j+1 is the transmission bandwidth of the j+1th storage node
  • K 0 is the data block size allocated by the unit bandwidth.
  • the source node sends, to the storage node, the data to be stored allocated by the storage node of the N storage nodes to the storage node.
  • a data block size is allocated for each storage node according to a transmission bandwidth, and a large data block is allocated to a storage node with a large transmission bandwidth, thereby improving data transmission efficiency and shortening data access.
  • Time has realized the rational use of resources.
  • FIG. 3 is an exemplary flowchart of a data storage method 300.
  • the method 300 is applied to a distributed storage system for storing data of a source node to be stored in a distributed storage system, as shown in FIG.
  • method 300 includes:
  • the source node acquires power consumption information of the M storage nodes in the distributed storage system, where the power consumption information is a transmission function of each of the M storage nodes to the source node. Consumption, where M is a positive integer greater than one.
  • the transmission power consumption refers to the power consumption required for the transmission of the data block of the unit size from the corresponding storage node to the source node.
  • the number of hops of the data transmission path reflects the power consumption of the data transmission path when the data is transmitted. The more the number, the greater the transmission power consumption.
  • the hop count of the path from the storage node to the source node may be used to reflect the transmission power consumption of the data transmission path from the storage node to the source node.
  • the transmission power of the data transmission path is not only related to the hop count, but also related to the processing time of the data, the longer the processing time of the data, or the longer the waiting time of the data in the cache, consumes more energy.
  • the transmission delay of the data transmission path from the storage node to the source node may also be used to reflect the transmission power consumption of the corresponding storage node to the source node.
  • the transmission power consumption can be expressed in various forms, and the hop count of the data transmission path can be used. It can also be represented by the transmission delay of the data transmission path, and the actual statistics can also be performed.
  • the embodiment of the present invention is merely an example, and does not limit the expression of the transmission power consumption, and allows other transmission power consumption. Alternative expression.
  • the method for the source node to obtain the power consumption information of the M storage nodes in the distributed storage system is described in the system embodiment of FIG. 1 and will not be further described herein.
  • the source node determines N storage nodes from the M storage nodes according to power consumption information of the M storage nodes, where N is a positive integer greater than 1 and less than or equal to M.
  • the source node determines N storage nodes from the M storage nodes according to power consumption information of the M storage nodes, including: the source node from the M Selecting N storage nodes with the smallest transmission power consumption among the storage nodes, or selecting the N storage nodes with the smallest delay or the largest transmission bandwidth among the L storage nodes with the smallest power consumption among the M storage nodes, wherein L Is a positive integer greater than N.
  • the source node determines, according to the power consumption information of the M storage nodes, the N storage nodes from the M storage nodes, where the source node obtains the delay of the M storage nodes.
  • the source node selects, from the M storage nodes, an alternate storage node that meets the first condition, wherein the delay satisfies the first condition: the delay of the p+1th storage node is less than or a sum equal to a sum of a sum of data download times of the first p storage nodes and a delay of the first one of the N storage nodes, where p is a positive integer that is incremented from 1 to N-1;
  • the source node selects the N storage nodes from the candidate storage nodes according to power consumption information of the candidate storage node.
  • the storage node that satisfies the following condition is used as the candidate storage node of the p+1th storage node:
  • L 1 is used to represent the time delay of the first storage node
  • K i is used to represent the data block size allocated for the i th storage node
  • B i is used to represent the transmission bandwidth of the i th storage node
  • L p+1 is used to represent the delay of the p+1th target node.
  • the source node determines N storage nodes from the M storage nodes according to power consumption information of the M storage nodes, including: the source node. a first storage node of the M storage nodes as a first storage node of the N storage nodes; the source node according to the power consumption information in order from 2nd to Nth The remaining N-1 storage nodes are determined in turn from the candidate storage nodes.
  • the method 300 further includes: the source node acquiring bandwidth information of each of the N storage nodes, where the N storage nodes are The bandwidth information of each storage node is the transmission bandwidth of the data transmission path of each of the N storage nodes to the source node.
  • the transmission bandwidth is a bandwidth reserved by the system for each of the N storage nodes to a data transmission path of the source node, and is an average transmission bandwidth of the data transmission path for actual data transmission. .
  • the source node uses the first storage node of the M storage nodes as the first storage node of the N storage nodes, and includes: the source node according to the delay information, or The bandwidth information, or the power consumption information, selects one storage node from the M storage nodes as the first storage node.
  • the source node uses the first storage node of the M storage nodes as the first storage node of the N storage nodes, and includes: the source node selects a delay from the plurality of storage nodes.
  • the storage node with the smallest sum of data block download times serves as the first storage node, that is, satisfies
  • the storage node with the smallest value is the first storage node.
  • L is the delay of the storage node
  • K is the data block size allocated to the storage node
  • B is the transmission bandwidth of the storage node.
  • the source node uses the first storage node of the M storage nodes as the first storage node of the N storage nodes, and the source node selects a delay from the M storage nodes.
  • the smallest storage node, or a storage node having the largest transmission bandwidth or the smallest transmission power consumption, is selected as the first storage node from among the L storage nodes having the smallest delay among the M storage nodes, where L is a positive integer greater than 1.
  • the source node uses the first storage node of the M storage nodes as the first storage node of the N storage nodes, and includes: the source node selects a transmission function from the M storage nodes.
  • the storage node with the smallest consumption, or the storage node with the smallest transmission bandwidth or the smallest delay among the L storage nodes with the smallest transmission power among the M storage nodes, is the first storage node, where L is a positive integer greater than 1. .
  • the source node uses the first storage node of the M storage nodes as the first storage node of the N storage nodes, and includes: the source node selects from the M storage nodes. a storage node having the largest transmission bandwidth, or a storage node having the smallest delay or the smallest transmission power consumption among the L storage nodes having the largest transmission bandwidth among the M storage nodes, wherein L is greater than 1 Integer.
  • the source node uses the first storage node of the M storage nodes as the first storage node of the N storage nodes, and the source node selects from the M storage nodes.
  • a storage node having the smallest transmission power consumption as the first storage node; or the source node selecting, as the storage node having the smallest transmission power consumption, from at least two storage nodes having the smallest delay among the M storage nodes The first storage node is described.
  • the source node determines the xth storage node from the candidate storage node according to the power consumption information, where x is a positive integer greater than 1, and the source node satisfies the first Selecting, from a candidate storage node of a condition, a storage node having the smallest transmission power consumption as the xth storage node; or, the source node is at least two storages having the largest delay from the candidate storage node satisfying the first condition In the node, the storage node with the smallest transmission power consumption is selected as the xth storage node.
  • the source node determines the xth storage node from the candidate storage node according to the power consumption information, where x is a positive integer greater than 1, and the source node satisfies the first Among the L storage nodes with the smallest transmission power consumption in the candidate storage node of one condition, the storage node with the largest delay or the largest transmission bandwidth is selected as the xth storage node, where L is a positive integer greater than 1.
  • S306 The source node allocates corresponding data to be stored to each of the N storage nodes according to the data to be stored.
  • the source node allocates corresponding data to be stored to each of the N storage nodes according to the data to be stored, including: the source node divides the data into N equal sizes. Data block and assign a block of data to each storage node. It will be appreciated that the data may not be equally evenly divided into N equal-sized data blocks because of the minimum partitioning unit of the data, and embodiments of the present invention allow for adjustments to the data blocks on a uniform basis.
  • the source node divides the data into N data blocks, and sorts the N data blocks according to the data, and the source node allocates the data blocks to the corresponding storage nodes, thereby ensuring the order of the data blocks, and the future. Access to data can be accessed in the order of the data blocks.
  • the source node acquires a buffer size of each of the N storage nodes, where the buffer is a corresponding storage node for receiving the number.
  • the source node allocates corresponding data to be stored to each of the N storage nodes according to the data to be stored, including: the source node will store the buffer size integer of the storage node.
  • a data block of multiple size is allocated to the corresponding storage node.
  • the source node may allocate a data block of a buffer size of the corresponding storage node to the corresponding storage node.
  • the source node if the source node allocates a data block of L size of the buffer size of the corresponding storage node to the corresponding storage node, the source node splits the data block into L sizes of the buffer size. Sub-blocks, and respectively send L sub-blocks to corresponding storage nodes, where L is a positive integer greater than one.
  • the source node allocates corresponding data to be stored to each of the N storage nodes according to the data to be stored, including: The source node allocates corresponding data to be stored to each of the N storage nodes according to the data to be stored and the bandwidth information of each of the N storage nodes. .
  • the source node is the N storages according to the data to be stored and the bandwidth information of each of the N storage nodes.
  • Each of the storage nodes in the node allocates corresponding data to be stored, including: the source node allocates corresponding data to be stored to the J storage nodes of the N storage nodes, where the J
  • the size of the data to be stored allocated by the jth storage node in the storage node is positively correlated with the transmission bandwidth of the jth storage node, and the J is a positive integer greater than 1 and less than or equal to N, and the j is A positive integer that increments from 1 to J.
  • the source node allocates corresponding storage to be stored to each of the N storage nodes according to the data to be stored and the bandwidth information of each of the N storage nodes.
  • the specific implementation process of the data is described in the system embodiment part of FIG. 1, and details are not described herein again.
  • the source node determines the N storage nodes one by one from the M storage nodes, after the source node determines the jth storage node, and in the The source node allocates a data block for the jth storage node before determining the j+1th storage node from the M storage nodes, where j is a positive integer greater than 0 and less than N.
  • the source node is the N storages according to the data to be stored and the bandwidth information of each of the N storage nodes.
  • Each of the storage nodes in the node allocates corresponding data to be stored, including: a source node Before determining the j+1th storage node, a data block whose size is positively correlated with the transmission bandwidth of the jth storage node is allocated to the jth storage node, and j is incremented from 1 to N-1 A positive integer, the j+1th storage node is the next node of the jth storage node; the source node allocates the remaining portion of the data to the Nth storage node.
  • the source node determines the N storage nodes from the M storage nodes according to the transmission power consumption, and allocates data blocks for each of the N storage nodes, including:
  • Step A The source node determines a first storage node from the M storage nodes according to the transmission power consumption, and allocates a data block to the first storage node.
  • the source node selecting, from the M storage nodes, the storage node with the smallest transmission power consumption as The first storage node.
  • the source node determines, by the source node, the first storage node from the M storage nodes according to the transmission power consumption, that: the source node selects, from the L storage nodes with the smallest delay of the M storage nodes.
  • the storage node with the smallest transmission power consumption is referred to as the first storage node, where L is a positive integer greater than one.
  • determining, by the source node, the first storage node from the M storage nodes according to the transmission power consumption where the source node is the L storage nodes with the largest transmission bandwidth from the M storage nodes.
  • the storage node with the smallest transmission power consumption is selected as the first storage node, where L is a positive integer greater than 1.
  • the determining, by the source node, the first storage node from the M storage nodes according to the transmission power consumption, where the source node has the least L storage node from the M storage nodes The storage node with the largest transmission bandwidth or the smallest delay is selected as the first storage node, where L is a positive integer greater than 1.
  • the source node determines the basis of the first storage node from the M storage nodes according to the transmission power consumption.
  • the embodiments of the present invention are merely examples and are not limited thereto.
  • the allocating the data block to the first storage node by the source node includes: the source node allocating the data block to the first storage node according to the transmission bandwidth of the first storage node.
  • the source node allocates a data block whose size is positively related to the transmission bandwidth of the first storage node to the first storage node.
  • the data block size, B 1 is the transmission bandwidth of the first storage node
  • K 0 is the data block size allocated by the unit bandwidth.
  • Step B The source node determines, according to the transmission power consumption, a j+1th storage node from the M storage nodes, where the delay of the j+1th storage node is not greater than that of the previous j storage nodes. The sum of the download time of the block and the delay of the first storage node. Perform step C.
  • the initial value of j is 1.
  • the candidate storage node of the j+1th storage node is determined according to the following conditions:
  • L 1 is used to represent the time delay of the first storage node
  • K i is used to represent the data block size allocated for the i th storage node
  • B i is used to represent the transmission bandwidth of the i th storage node
  • L j+1 is used to represent the time delay of the j+1th target node.
  • the source node satisfies the condition of Equation 17 from the M storage nodes, and the node that has determined that the transmission power consumption is the smallest among the storage nodes that have not been selected is the j+1th storage node.
  • the source node satisfies the condition of the formula 17 from the M storage nodes, and the node with the smallest transmission power is determined as the j+1th storage among the L storage nodes with the largest delay among the storage nodes that have not yet been selected.
  • L is a positive integer greater than one.
  • the source node satisfies the condition of the formula 17 from the M storage nodes, and the node with the smallest transmission power is determined as the j+1th storage among the L storage nodes with the largest transmission bandwidth among the storage nodes that have not yet been selected.
  • L is a positive integer greater than one.
  • the source node satisfies the condition of the formula 9 from the M storage nodes, and the node with the smallest delay or the largest transmission bandwidth is determined as the jth among the L storage nodes with the smallest transmission power among the storage nodes that have not yet been selected.
  • the source node determines the basis of the j+1th storage node from the M storage nodes according to the transmission power consumption.
  • the embodiments of the present invention are merely examples and are not limited thereto.
  • Step C Determine whether the following conditions are met:
  • the final value of j is N-1.
  • K is the size of the remainder of the data
  • B j+1 is the transmission bandwidth of the j+1th storage node
  • K 0 is the data block size allocated by the unit bandwidth
  • the source node sends, to the storage node, the data to be stored to which the storage node is allocated to each of the N storage nodes.
  • determining the storage node for storing data according to the transmission power consumption is prioritized by selecting a storage node with a smaller transmission power consumption, thereby reducing energy consumption when the source node accesses data.
  • FIG. 4 is a schematic diagram showing the logical structure of a data storage device 400 according to an embodiment of the present invention.
  • the device 400 is applied to a distributed storage system for storing data to be stored in the device 400 in a distributed storage system, such as As shown in FIG. 4, the apparatus 400 includes a determining unit 402, an obtaining unit 404, an allocating unit 406, and a transmitting unit 408.
  • a determining unit 402 configured to determine N storage nodes from the distributed storage system, where N is a positive integer greater than one;
  • An obtaining unit 404 configured to acquire bandwidth information of each of the N storage nodes, where bandwidth information of each of the N storage nodes is the one of the N storage nodes The transmission bandwidth of the data transmission path from each storage node to the source node;
  • the allocating unit 406 is configured to allocate a corresponding one of the N storage nodes according to the data to be stored and the bandwidth information of each of the N storage nodes. Data to be stored;
  • the sending unit 408 is configured to send, to the storage node, the data to be stored to which the storage node is allocated to each of the N storage nodes.
  • the allocating unit 406 is configured to use, according to the data to be stored, and the bandwidth information of each of the N storage nodes, the each of the N storage nodes.
  • the storage node allocates corresponding data to be stored, including: the allocating unit 406 is configured to respectively allocate corresponding data to be stored to the J storage nodes of the N storage nodes, where the J storage nodes are The size of the data to be stored allocated by the jth storage node and the jth The transmission bandwidth of the storage node is in a positive correlation, and the J is a positive integer greater than 1 and less than or equal to N, and the j is a positive integer increasing from 1 to J one by one.
  • the obtaining unit 404 is further configured to acquire power consumption information of the M storage nodes in the distributed storage system, where the power consumption information is each storage node of the M storage nodes.
  • the transmission power consumption of the source node, M is a positive integer greater than or equal to N;
  • the determining unit 402 is configured to determine N storage nodes from the distributed storage system, and the determining unit 402 is configured to use the M
  • the power consumption information of the storage node is selected from the M storage nodes.
  • the obtaining unit 404 is further configured to acquire delay information of M storage nodes in the distributed storage system, where M is a positive integer greater than or equal to N; the determining unit 402 is configured to use the distribution Determining, by the storage system, the N storage nodes, the determining unit 402, configured to select the N storage nodes from the M storage nodes according to the delay information of the M storage nodes.
  • the determining unit 402 is configured to select the N storage nodes from the M storage nodes according to the delay information of the M storage nodes, where the determining unit 402 is configured to use the The M storage nodes select the N storage nodes that satisfy the first condition, wherein the delay satisfies the first condition: the delay of the p+1th storage node is less than or equal to the data of the previous p storage nodes.
  • the sum of the sum of the download times and the delay of the first one of the N storage nodes, the p being a positive integer increasing from 1 to N-1. .
  • the determining unit 402 is configured to select, from the M storage nodes, the N storage nodes that satisfy the first condition, wherein the determining unit 402 is configured to use the first one of the M storage nodes. a storage node as the first storage node of the N storage nodes; the determining unit 402 is configured to remove the storage nodes other than the first storage node from the M storage nodes according to from 2 to N The sequence sequentially determines the remaining N-1 storage nodes, wherein the delay of each of the remaining N-1 storage nodes satisfies the first condition.
  • the determining unit 402 is configured to use the first storage node of the M storage nodes as the first storage node of the N storage nodes, and the determining unit 402 is configured to use the The delay information, or the bandwidth information, or the power consumption information, selects one storage node from the M storage nodes as the first storage node.
  • the determining unit 402 when determining the xth storage node from the storage nodes except the first storage node from the M storage nodes, if the determining unit 402 determines that at least two of the first conditions are met Alternative storage nodes, the determining unit 402 is further configured to transmit according to time delay or The bandwidth, or the transmission power consumption, selects one storage node from the at least two candidate storage nodes that satisfy the first condition as the xth storage node, where x is greater than or equal to 2 and less than or equal to N.
  • the embodiment of the present invention corresponds to the method embodiment of FIG. 2, and the feature description of the embodiment of FIG. 2 is applicable to the embodiment of the present invention, and details are not described herein again.
  • a data block size is allocated for each storage node according to a transmission bandwidth, and a large data block is allocated to a storage node with a large transmission bandwidth, thereby improving data transmission efficiency and shortening data access.
  • Time has realized the rational use of resources.
  • FIG. 5 is a schematic diagram of a logical structure of a data storage device 500.
  • the device 500 is applied to a distributed storage system for storing data to be stored in the device 500 in a distributed storage system.
  • the apparatus 500 includes an acquisition unit 502, a determination unit 504, an allocation unit 506, and a transmission unit 508.
  • the obtaining unit 502 is configured to acquire power consumption information of the M storage nodes in the distributed storage system, where the power consumption information is a transmission function of each of the M storage nodes to the source node Consumption, where M is a positive integer greater than one;
  • a determining unit 504 configured to determine N storage nodes from the M storage nodes according to power consumption information of the M storage nodes, where N is a positive integer greater than 1 and less than or equal to M;
  • the allocating unit 506 is configured to allocate corresponding data to be stored to each of the N storage nodes according to the data to be stored;
  • the sending unit 508 is configured to send, to the storage node, the data to be stored to which the storage node is allocated to each of the N storage nodes.
  • the obtaining unit 502 is further configured to acquire delay information of the M storage nodes, where the determining unit 504 is configured to use, according to the power consumption information of the M storage nodes, the M storage nodes.
  • Determining N storage nodes including: the determining unit 504 is configured to select, from the M storage nodes, an candidate storage node that meets a first condition with a delay, wherein the delay satisfies the first condition: p
  • the delay of the +1 storage node is less than or equal to the sum of the sum of the data download time of the previous p storage nodes and the delay of the first storage node of the N storage nodes, where p is one by one
  • the positive unit is incremented to N-1; the determining unit 504 is configured to select the N storage nodes from the candidate storage nodes according to the power consumption information of the candidate storage node.
  • the determining unit 504 is configured to determine N storage nodes from the M storage nodes according to the power consumption information of the M storage nodes, where the determining unit 504 is configured to use the M
  • the first storage node among the storage nodes is the first one of the N storage nodes a storage node; the determining unit 504 is configured to sequentially determine remaining N-1 storage nodes from the candidate storage nodes according to the power consumption information in order from 2nd to Nth.
  • the determining unit 504 is configured to use the first storage node of the M storage nodes as the first storage node of the N storage nodes, and the determining unit 504 is configured to use the Selecting, from the M storage nodes, a storage node having the smallest transmission power consumption as the first storage node; or selecting, from the at least two storage nodes having the smallest delay among the M storage nodes, the storage with the smallest transmission power consumption The node acts as the first storage node.
  • the determining unit 504 is configured to determine, according to the power consumption information, the xth storage node from the candidate storage node, where x is a positive integer greater than 1, and the determining unit 504 is used by Selecting, from the candidate storage nodes satisfying the first condition, a storage node having the smallest transmission power consumption as the xth storage node; or selecting at least two of the candidate storage nodes satisfying the first condition to have the largest delay Among the storage nodes, the storage node with the smallest transmission power consumption is selected as the xth storage node.
  • the obtaining unit 502 is further configured to acquire bandwidth information of each of the N storage nodes, where bandwidth information of each of the N storage nodes is a transmission bandwidth of the data transmission path of each of the N storage nodes to the source node; the allocation unit 506 is configured to: according to the data to be stored, to the N storage nodes Each storage node allocates corresponding data to be stored, including: the allocating unit 506 is configured to: according to the data to be stored and the bandwidth information of each of the N storage nodes, Each of the N storage nodes allocates corresponding data to be stored.
  • the allocating unit 506 is configured to use, according to the data to be stored, and the bandwidth information of each of the N storage nodes, the each of the N storage nodes.
  • the storage node allocates corresponding data to be stored, including: the allocating unit 506 is configured to allocate corresponding data to be stored to the J storage nodes of the N storage nodes, where the J storage nodes are The size of the data to be stored allocated by the jth storage node is positively correlated with the transmission bandwidth of the jth storage node, and the J is a positive integer greater than 1 and less than or equal to N, and the j is incremented from 1 to 1 A positive integer to J.
  • the embodiment of the present invention corresponds to the method embodiment of FIG. 3, and the feature description of the embodiment of FIG. 3 is applicable to the embodiment of the present invention, and details are not described herein again.
  • the storage node for storing data is determined according to the transmission power consumption, and the storage node with less transmission power consumption is selected, thereby reducing the source node access.
  • FIG. 6 is a schematic diagram showing the hardware structure of a data storage device 600 according to an embodiment of the invention.
  • data storage device 600 includes a processor 602, a memory 604, an input/output interface 606, a communication interface 608, and a bus 610.
  • the processor 602, the memory 604, the input/output interface 606, and the communication interface 608 implement a communication connection with each other through the bus 610.
  • the processor 602 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for executing related programs.
  • CPU central processing unit
  • ASIC application-specific integrated circuit
  • the memory 604 may be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).
  • Memory 604 can store operating systems and other applications.
  • the program code for implementing the technical solution provided by the embodiment of the present invention is stored in the memory 604 and executed by the processor 602.
  • the input/output interface 606 is for receiving input data and information, and outputting data such as operation results.
  • Communication interface 608 implements communication between data storage device 600 and other devices or communication networks using transceivers such as, but not limited to, transceivers.
  • Bus 610 can include a path for communicating information between various components of data storage device 600, such as processor 602, memory 604, input/output interface 606, and communication interface 608.
  • meter data storage device 600 shown in FIG. 6 only shows the processor 602, the memory 604, the input/output interface 606, the communication interface 608, and the bus 610, in a specific implementation process, those skilled in the art It should be understood that data storage device 600 also includes other devices necessary to achieve proper operation. Also, those skilled in the art will appreciate that data storage device 600 may also include hardware devices that implement other additional functions, depending on the particular needs. Moreover, those skilled in the art will appreciate that data storage device 600 may also only include the components necessary to implement embodiments of the present invention without necessarily including all of the devices shown in FIG.
  • FIG. 6 and the above description are applicable to various data storage devices and systems provided by the embodiments of the present invention, and are suitable for performing various data storage methods provided by the embodiments of the present invention.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the modules is only a logical function division, and may be implemented in another manner, for example, multiple modules or components may be combined or may be Integrate into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or otherwise.
  • the modules described as separate components may or may not be physically separated.
  • the components displayed as modules may or may not be physical modules, that is, may be located in one place, or may be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of hardware plus software function modules.
  • the above-described integrated modules implemented in the form of software function modules can be stored in a computer readable storage medium.
  • the software functional modules described above are stored in a storage medium and include instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform some of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a mobile hard disk, a read-only memory (English: Read-Only Memory, ROM for short), a random access memory (English: Random Access Memory, RAM for short), a magnetic disk or an optical disk, and the like. The medium of the code.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

La présente invention concerne un procédé, un dispositif et un système de stockage de données, qui sont utilisés pour stocker des données à stocker d'un nœud source dans un système de stockage distribué d'une manière distribuée. Le procédé consiste à : déterminer N nœuds de stockage à partir du système de stockage distribué (S202) ; acquérir des informations de bande passante concernant chaque nœud de stockage dans les N nœuds de stockage (S204), les informations de bande passante concernant chaque nœud de stockage étant une bande passante de transmission d'un chemin de transmission de données allant de chaque nœud de stockage au nœud source ; selon les données à stocker et les informations de bande passante concernant chaque nœud de stockage, attribuer des données correspondantes à stocker à chaque nœud de stockage (S206) ; et envoyer les données à stocker attribuées à chaque nœud de stockage à chaque nœud de stockage (S208). Selon la bande passante de transmission, une taille de bloc de données est attribuée à chaque nœud de stockage, et un bloc de données plus grand est attribué à un nœud de stockage ayant une grande bande passante de transmission, de façon à améliorer l'efficacité de transmission de données, et à raccourcir le temps d'accès de données, ce qui permet d'obtenir l'utilisation rationnelle de ressources.
PCT/CN2015/082581 2015-06-27 2015-06-27 Procédé, dispositif et système de stockage de données WO2017000094A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201580001239.3A CN107113323B (zh) 2015-06-27 2015-06-27 一种数据存储方法、装置和系统
PCT/CN2015/082581 WO2017000094A1 (fr) 2015-06-27 2015-06-27 Procédé, dispositif et système de stockage de données

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/082581 WO2017000094A1 (fr) 2015-06-27 2015-06-27 Procédé, dispositif et système de stockage de données

Publications (1)

Publication Number Publication Date
WO2017000094A1 true WO2017000094A1 (fr) 2017-01-05

Family

ID=57607489

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/082581 WO2017000094A1 (fr) 2015-06-27 2015-06-27 Procédé, dispositif et système de stockage de données

Country Status (2)

Country Link
CN (1) CN107113323B (fr)
WO (1) WO2017000094A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109768926A (zh) * 2019-01-10 2019-05-17 维沃移动通信有限公司 一种数据处理方法、终端设备及计算机可读存储介质
CN109831391A (zh) * 2018-12-29 2019-05-31 华为技术有限公司 分布式存储系统中流控的方法、存储设备和系统
CN111399748A (zh) * 2019-01-02 2020-07-10 中国移动通信有限公司研究院 一种数据放置方法、装置和计算机可读存储介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110569213A (zh) * 2018-05-18 2019-12-13 北京果仁宝软件技术有限责任公司 文件存取方法、装置和设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101645038A (zh) * 2009-05-20 2010-02-10 中国科学院声学研究所 基于彼特森的网络存储结构的数据存储方法
CN101674233A (zh) * 2008-09-12 2010-03-17 中国科学院声学研究所 基于彼得森图的存储网络结构及数据读写方法
US20110231490A1 (en) * 2010-03-16 2011-09-22 Brother Kogyo Kabushiki Kaisha Information communications system, node device, method of communicating contents, computer readable recording medium storing a program
CN102387179A (zh) * 2010-09-02 2012-03-21 联想(北京)有限公司 分布式文件系统及其结点、存储方法和存储控制方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101674233A (zh) * 2008-09-12 2010-03-17 中国科学院声学研究所 基于彼得森图的存储网络结构及数据读写方法
CN101645038A (zh) * 2009-05-20 2010-02-10 中国科学院声学研究所 基于彼特森的网络存储结构的数据存储方法
US20110231490A1 (en) * 2010-03-16 2011-09-22 Brother Kogyo Kabushiki Kaisha Information communications system, node device, method of communicating contents, computer readable recording medium storing a program
CN102387179A (zh) * 2010-09-02 2012-03-21 联想(北京)有限公司 分布式文件系统及其结点、存储方法和存储控制方法

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109831391A (zh) * 2018-12-29 2019-05-31 华为技术有限公司 分布式存储系统中流控的方法、存储设备和系统
CN109831391B (zh) * 2018-12-29 2022-05-13 华为云计算技术有限公司 分布式存储系统中流控的方法、存储设备和系统
CN111399748A (zh) * 2019-01-02 2020-07-10 中国移动通信有限公司研究院 一种数据放置方法、装置和计算机可读存储介质
CN111399748B (zh) * 2019-01-02 2023-09-05 中国移动通信有限公司研究院 一种数据放置方法、装置和计算机可读存储介质
CN109768926A (zh) * 2019-01-10 2019-05-17 维沃移动通信有限公司 一种数据处理方法、终端设备及计算机可读存储介质
CN109768926B (zh) * 2019-01-10 2021-07-20 维沃移动通信有限公司 一种数据处理方法、终端设备及计算机可读存储介质

Also Published As

Publication number Publication date
CN107113323A (zh) 2017-08-29
CN107113323B (zh) 2020-02-21

Similar Documents

Publication Publication Date Title
WO2017084393A1 (fr) Procédé de distribution de contenu, procédé de gestion de serveur virtuel, plateforme nuagique et système
EP3053055B1 (fr) Système et procédé destinés à l'envoi et à la mise en cache dynamiques conjoints dans le contexte des réseaux de diffusion de contenu
WO2017000094A1 (fr) Procédé, dispositif et système de stockage de données
WO2019072162A1 (fr) Procédé de mappage de réseau virtuel, dispositif et support de stockage
US8989037B2 (en) System for performing data cut-through
WO2014061481A1 (fr) Dispositif de transfert de données et système de transfert de données utilisant un algorithme de compression adaptative
KR20200017589A (ko) 무선 통신 시스템에서 모바일 노드의 태스크를 오프로딩하기 위한 클라우드 서버 및 그의 동작 방법
JPWO2018220708A1 (ja) 資源割当システム、管理装置、方法およびプログラム
JP6972714B2 (ja) データ取得プログラム、装置、及び方法
CN116360972A (zh) 资源管理方法、装置及资源管理平台
JP2016531372A (ja) メモリモジュールアクセス方法および装置
CN111866438B (zh) 用户体验驱动的多方视频会议的转码和组播路由方法及装置
WO2014101502A1 (fr) Procédé de traitement d'accès mémoire basé sur une interconnexion de puces mémoire, puce mémoire et système
CN111194090B (zh) 一种面向边缘计算的多策略信道分配方法
CN104219163A (zh) 一种基于动态副本法和虚拟节点法的节点动态前移的负载均衡方法
US11941445B2 (en) RLC channel management for low memory 5G devices
JP2004046372A (ja) 分散処理システム、リソース割当方法およびプログラムならびにリソース割当プログラムが記録された記録媒体
US10091116B2 (en) Service chain construction method and server apparatus
KR102610846B1 (ko) 고속 분산 저장 장치 및 방법
US10020979B1 (en) Allocating resources in multi-core computing environments
US10516596B2 (en) Information processing apparatus, method and non-transitory computer-readable storage medium
Alharbi et al. Optimizing jobs' completion time in cloud systems during virtual machine placement
Ma et al. Improved efficient queue resource reallocation strategy for traffic dynamics on scale-free networks
US20150263958A1 (en) Load balancing apparatus and method
KR102579474B1 (ko) 네트워크 부하 분산 방법 및 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15896626

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15896626

Country of ref document: EP

Kind code of ref document: A1