CN113676510A - Cloud storage method, system, equipment and storage medium - Google Patents

Cloud storage method, system, equipment and storage medium Download PDF

Info

Publication number
CN113676510A
CN113676510A CN202110789656.6A CN202110789656A CN113676510A CN 113676510 A CN113676510 A CN 113676510A CN 202110789656 A CN202110789656 A CN 202110789656A CN 113676510 A CN113676510 A CN 113676510A
Authority
CN
China
Prior art keywords
node
data
user data
storage
control center
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110789656.6A
Other languages
Chinese (zh)
Inventor
庄坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Innovation Co
Original Assignee
Alibaba Singapore Holdings Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Singapore Holdings Pte Ltd filed Critical Alibaba Singapore Holdings Pte Ltd
Priority to CN202110789656.6A priority Critical patent/CN113676510A/en
Publication of CN113676510A publication Critical patent/CN113676510A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1095Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Transfer Between Computers (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the application provides a cloud storage method, a cloud storage system, cloud storage equipment and a storage medium. In the embodiment of the application, the edge storage can be realized based on a plurality of edge nodes in the cloud network, that is, user data generated by an end user can be stored in the edge nodes, so that the end user can access the data in a shorter distance; in addition, a plurality of edge nodes can be communicated, and data synchronization is carried out on the user data among the edge nodes, so that multi-copy storage of the user data can be realized based on the edge nodes. Therefore, in the embodiment of the application, the multi-copy storage of the user data can be realized through the data synchronization of the nodes, so that the user data on the previous node can still be used after the terminal user is scheduled to other nodes, the response time of data access can be shortened, and the high availability of the user data can be effectively improved.

Description

Cloud storage method, system, equipment and storage medium
Technical Field
The present application relates to the field of cloud network technologies, and in particular, to a cloud storage method, system, device, and storage medium.
Background
Currently, a cloud storage manner of central storage is generally adopted in a cloud network. A large amount of user data is stored in the data center. For the end user, the user data may be used by accessing a data center.
However, the central storage method causes a serious problem of centralization of data storage, the storage distance of user data is long, and if the data center fails, the user data is directly unavailable.
Disclosure of Invention
Aspects of the present application provide a data processing method, system, device, and storage medium, which are used to implement high availability of user data during a cloud storage process.
The embodiment of the application provides a cloud storage system, which comprises a control center and a plurality of edge nodes;
the control center is used for receiving a data synchronization request sent by a first node in the plurality of edge nodes and determining at least one second node for the first node from other edge nodes; respectively sending data synchronization instructions to the first node and the second node;
the first node is used for receiving and storing user data generated by the terminal equipment; sending the data synchronization request for the user data to the control center; synchronizing the user data to the at least one second node according to a data synchronization instruction returned by the control center;
and the second node synchronizes the user data from the first node according to the data synchronization instruction so as to store the copy of the user data.
An embodiment of the present application further provides a cloud storage method, including:
receiving and storing user data generated by terminal equipment;
sending a data synchronization request aiming at the user data to a control center;
receiving a data synchronization instruction returned by the control center, wherein the data synchronization instruction comprises information of at least one synchronization node determined by the control center for the user data from other edge nodes contained in the cloud network;
and synchronizing the user data to the at least one synchronization node according to the data synchronization instruction.
An embodiment of the present application further provides a cloud storage method, including:
receiving a data synchronization request aiming at user data sent by a first node in a plurality of edge nodes in a cloud network;
determining at least one second node for the first node from the other edge nodes;
and respectively sending a data synchronization instruction to the first node and the at least one second node to control the data synchronization operation of the user data between the first node and the at least one second node.
An embodiment of the present application further provides a cloud storage method, including:
receiving a data synchronization instruction which is sent by a control center and aims at user data contained in a first edge node in a cloud network;
establishing a communication link with the first edge node;
synchronizing the user data from the first edge node to store a copy of the user data.
An embodiment of the present application further provides an edge node, including:
the storage module is used for receiving and storing user data generated by the terminal equipment;
the communication module is used for sending a data synchronization request aiming at the user data to a control center;
the coordination module is used for receiving a data synchronization instruction returned by the control center, wherein the data synchronization instruction comprises information of at least one synchronization node determined by the control center for the user data from other edge nodes contained in the cloud network;
and the synchronization module is used for synchronizing the user data to the at least one synchronization node according to the data synchronization instruction.
An embodiment of the present application further provides a control device, including:
the system comprises a receiving module, a synchronization module and a processing module, wherein the receiving module is used for receiving a data synchronization request aiming at user data, which is sent by a first node in a plurality of edge nodes in a cloud network;
a scheduling module for determining at least one second node for the first node from among other edge nodes;
a sending module, configured to send a data synchronization instruction to the first node and the at least one second node, respectively, so as to control data synchronization operation of user data between the first node and the at least one second node.
An embodiment of the present application further provides an edge node, including:
the coordination module is used for receiving a data synchronization instruction which is sent by the control center and aims at user data contained in a first edge node in the cloud network;
a communication module for establishing a communication link with the first edge node;
a synchronization module to synchronize the user data from the first edge node to store a copy of the user data.
Embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the aforementioned cloud storage method.
In the embodiment of the application, the edge storage can be realized based on a plurality of edge nodes in the cloud network, that is, user data generated by an end user can be stored in the edge nodes, so that the end user can access the data in a shorter distance; in addition, a plurality of edge nodes can be communicated, and data synchronization is carried out on the user data among the edge nodes, so that multi-copy storage of the user data can be realized based on the edge nodes. Therefore, in the embodiment of the application, the multi-copy storage of the user data can be realized through the data synchronization of the nodes, so that the user data on the previous node can still be used after the terminal user is scheduled to other nodes, the response time of data access can be shortened, and the high availability of the user data can be effectively improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a schematic structural diagram of a cloud storage system according to an exemplary embodiment of the present application;
FIG. 2 is a logical schematic diagram of an exemplary data synchronization scheme provided by an exemplary embodiment of the present application;
fig. 3 is a schematic flowchart of a cloud storage method according to another exemplary embodiment of the present application;
fig. 4 is a schematic flowchart of another cloud storage method according to another exemplary embodiment of the present application;
fig. 5 is a schematic flowchart of another cloud storage method according to another exemplary embodiment of the present application;
fig. 6 is a schematic structural diagram of an edge node according to another exemplary embodiment of the present application;
fig. 7 is a schematic structural diagram of a control device according to another exemplary embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
At present, a central storage mode can cause a serious centralization problem of data storage, the storage distance of user data is long, and in addition, if a data center fails, the user data can be directly unavailable. To this end, in some embodiments of the present application: the edge storage can be realized based on a plurality of edge nodes in the cloud network, that is, user data generated by an end user can be stored in the edge nodes, so that the end user can access the data in a closer distance; in addition, a plurality of edge nodes can be communicated, and data synchronization is carried out on the user data among the edge nodes, so that multi-copy storage of the user data can be realized based on the edge nodes. Therefore, in the embodiment of the application, the multi-copy storage of the user data can be realized through the data synchronization of the nodes, so that the user data on the previous node can still be used after the terminal user is scheduled to other nodes, the response time of data access can be shortened, and the high availability of the user data can be effectively improved.
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is a schematic structural diagram of a cloud storage system according to an exemplary embodiment of the present application. As shown in fig. 1, the system includes: the control center 10 and a plurality of edge nodes 20, the control center 10 can be respectively connected with the plurality of edge nodes 20 in a communication way, and communication channels can be opened among the plurality of edge nodes 20 as required, so that the communication connection is established.
In a cloud network, the edge node 20 is between an end user and a data center, and compared with a traditional data center, the edge node 20 has the characteristics of miniaturization, distribution and closer proximity to the user, the coverage of the edge node 20 is wide, and the city and even finer level can be achieved. Currently, edge nodes 20 may be used to support edge computing in a cloud network to optimize response latency, center load, and overall cost.
In this embodiment, a cloud storage system may be constructed in a cloud network, and the edge node 20 in the cloud network is introduced into the cloud storage system. For this reason, in this embodiment, a storage resource may be added to the edge node 20 in the cloud network in advance, and the storage resource may include a hardware resource, a software resource, and the like, so as to provide a storage capability to the edge node 20. In practical application, all the edge nodes 20 in the cloud network may be introduced into the cloud storage system, and this embodiment does not limit this, and only a specified part of the edge nodes 20 may be introduced.
In this embodiment, the control center 10 may also be configured in the cloud storage system. Since the control center 10 is not involved in data read-write operations, it is very lightweight, so that the control center 10 can be deployed in a variety of network locations in a cloud network. The control center 10 may be multi-center or single-center. For example: in the multi-center case, the plurality of centers included in the control center 10 may be respectively integrated into each original data center in the cloud network, wherein a single center may globally manage all the edge nodes 20 in the cloud storage system, and in this case, the data center may no longer need to undertake storage work as in the conventional central storage manner; in the case of a single center, the control center 10 may serve as a specific node in the cloud network, and may be integrated at other positions in the cloud network, in which case, the control center 10 may also globally manage all the edge nodes 20 in the cloud storage system. In addition, in this embodiment, the physical implementation form of the control center 10 is not limited, and the control center 10 may be implemented on a carrier such as a node, a server, or a data center in a cloud network.
Referring to fig. 1, in this embodiment, the control center may serve as a manager of the cloud storage system, and is configured to manage a plurality of edge nodes in the cloud storage system. The processing functions that may be provided by the control center 10 may include, but are not limited to, multiple copy management of user data, data synchronization service discovery, storage resource scheduling, data synchronization instruction management, storage status monitoring, flow control, and the like. The capabilities that the edge node 20 can provide include, but are not limited to, execution and management of data synchronization instructions, flow control, adapting storage types, adapting data transmission channels, monitoring and reporting storage status, registering with the control center 10 for data synchronization services, and the like. Processing logic in the cloud storage scheme of the control center 10 and the edge node 20 will be described below, respectively.
Referring to fig. 1, a first node of a plurality of edge nodes 20 may act as a sender, i.e., a type of edge node 20 that has a data synchronization requirement. For the first node, user data generated by the terminal device may be received and stored. Also, a data synchronization request may be initiated to the control center 10 for user data. The data synchronization request may include information such as the number of copies required.
For the control center 10, after receiving the data synchronization request sent by the first node, at least one second node may be determined for the first node from other nodes in the plurality of edge nodes 20, where the second node may be a receiver, that is, a type of edge node 20 for storing a copy of the user data.
Here, the control center 10 may determine the number of the second nodes according to the information of the number of copies required, which is carried in the data synchronization request sent by the first node. Furthermore, the control center 10 may adopt a plurality of scheduling policies to select the at least one second node, and an alternative scheduling policy may be: maintaining respective storage state information for a plurality of edge nodes 20; respectively determining storage capacity indexes of other nodes to the user data according to respective storage state information of other nodes except the first node in the plurality of edge nodes 20; at least one second node is determined for the first node from the other nodes according to the storage capacity index. The storage status information may include, but is not limited to, a geographical location, a bandwidth, a remaining storage capacity, a current load amount, a weight, a total amount of storable files, a storage cost, and the like. In this way, the control center 10 can evaluate the storage capacity index of each of the plurality of edge nodes 20 other than the first node for the user data, thereby selecting a specified number of excellent nodes. For example, appropriate weights may be configured for each of the storage status information, the respective storage capacity indexes of the other edge nodes 20 may be calculated by means of weighted summation, and a specified number of edge nodes 20 with the highest storage capacity index may be selected from the storage capacity indexes to serve as the second nodes. Of course, the scheduling scheme here is merely exemplary, and the scheduling scheme that the control center 10 can adopt in the present embodiment is not limited thereto.
On this basis, the control center 10 can send data synchronization instructions to the first node and the second node, respectively. The data synchronization instruction sent to the first node may carry information of at least one second node, and the data synchronization instruction sent to the second node may carry information of the first node, so that the first node and the at least one second node correctly start a subsequent data synchronization task.
For the first node, after the data synchronization instruction is obtained from the control center 10, the user data may be synchronized to at least one second node, and each second node synchronizes the user data from the first node according to the data synchronization instruction to store a copy of the user data. The first node and the second node can respectively start a data synchronization task after receiving the data synchronization instruction, so as to establish a data transmission channel and complete data synchronization. In this way, the first node and the at least one second node may be caused to have stored thereon user data or copies of user data.
In this embodiment, edge storage may be implemented based on a plurality of edge nodes in the cloud network, that is, user data generated by an end user may be stored in an edge node, which enables the end user to access data in a shorter distance; in addition, a plurality of edge nodes can be communicated, and data synchronization is carried out on the user data among the edge nodes, so that multi-copy storage of the user data can be realized based on the edge nodes. Therefore, in the embodiment of the application, the multi-copy storage of the user data can be realized through the data synchronization of the nodes, so that the user data on the previous node can still be used after the terminal user is scheduled to other nodes, the response time of data access can be shortened, and the high availability of the user data can be effectively improved.
In the above or below embodiments, a plurality of data synchronization methods may be used between the first node and the at least one second node. Taking a first node and one of the second nodes as an example, fig. 2 is a logic diagram of an exemplary data synchronization scheme provided in an exemplary embodiment of the present application. Referring to fig. 2, in this embodiment, a plurality of data transmission channels may be preconfigured between a first node and a second node, and the first node may select, in response to a data synchronization instruction, a target transmission channel adapted to user data from the plurality of data transmission channels, so that a communication connection between the first node and the second node may be established based on the target transmission channel; based on this, the first node can synchronize the user data to the second node through the target transmission channel. The first node may determine the adaptation degrees of the multiple data transmission channels and the user data according to channel information such as bandwidth and delay, and select a good target transmission channel from the multiple data transmission channels, for example, a channel with the smallest delay may be selected as the target transmission channel.
The first node may encapsulate the user data into specified format data; and sending the specified format data to the second node. Accordingly, for the second node, the specified format data may be decapsulated; and storing the unpacked data as a copy of the user data. When reading out the user data, the first node may perform data encapsulation on the user data according to a specified data format, which may also be understood as data format conversion. For example, the specified data format may be a binary format, etc., and is not limited herein. The second node can correctly decapsulate the received data with the specified format based on the transmission protocol agreed with the first node, and store the decapsulated data as a copy of the user data.
Referring to fig. 2, in this embodiment, multiple sets of storage resources may be respectively deployed on multiple edge nodes 20, and different storage types that can be supported by different sets of storage resources are different. The types of storage that may be supported by the edge nodes 20 may include, but are not limited to, block storage, object storage, network attached storage, NAS, and the like. Groups of storage resources may be scheduled internally to the edge node 20.
Based on this, the first node may carry the storage type of the user data to the specified format data in the process of encapsulating the user data, for example, the storage type of the user data may be configured in a header file of the specified format data. Correspondingly, after receiving the data in the specified format, the second node can analyze the storage type of the user data from the data in the specified format; and decapsulating the specified format data into a data format corresponding to the storage type of the user data, and storing the decapsulated data into a storage resource supporting the storage type of the user data. In this case, synchronization of user data of the same storage type may be supported between the first node and the second node.
Further, in this embodiment, the first node may also carry the storage type of the copy required by the user data to the specified format data in the process of encapsulating the user data, for example, the storage type of the copy required by the user data may be configured in a header file of the specified format data. Correspondingly, the second node can analyze the storage type of the user data and the storage type of the required copy from the specified format data after receiving the specified format data; decapsulating the data in the specified format to a data format corresponding to the storage type of the copy; and storing the unpackaged data into a storage resource of a storage type supporting the copy. In this case, the first node and the second node can support switching of the storage types during data synchronization, thereby realizing the mutual migration between different storage types.
For this reason, in this embodiment, storage interfaces may be respectively configured for multiple storage resources included in the edge node 20, and accordingly, the storage types supportable by different storage interfaces are different. In this way, for the first node, a storage interface corresponding to the storage type of the user data can be called, and the user data is packaged into the data with the specified format under the storage interface; for the second node, the corresponding target interface can be adapted in the plurality of storage interfaces according to the storage type of the copy required by the user data analyzed from the specified format data; and under the target interface, the operation of decapsulating the specified format data to the data format corresponding to the storage type of the copy and the operation of storing the decapsulated data to the storage resource supporting the storage type of the copy are executed. That is, under the target interface, the specified format data is decapsulated into a data format corresponding to the storage type of the copy required by the user data, and is stored in the storage resource supporting the storage type of the copy required by the user data.
For example, if the storage type of the user data in the first node is "block storage" and the storage type of the required copy is "NAS", the first node may read the user data from the "block storage" resource by calling the "block storage" interface, encapsulate the user data into data of a specified format, and send the data to the second node; the second node can convert the specified format data into the 'NAS' format by calling the 'NAS' interface so as to obtain a copy of the user data, and store the copy of the user data into the 'NAS' resource. Data encapsulation and decapsulation logic based on a specified format is pre-configured under each storage interface in the edge node 20, so as to ensure that user data can be smoothly synchronized between the edge nodes 20.
Accordingly, in this embodiment, the first node and the second node may support synchronization of user data of the same storage type, and may also support switching of storage types in a data synchronization process, so as to implement mutual migration between different storage types, which makes data synchronization in a cloud storage process more flexible, and user data copies of different storage types may adapt to different access scenarios of an end user, which may ensure higher data hit rate and faster speed of the end user, thereby further improving high availability of the user data.
In the above or below described embodiments, the control center 10 may also be used for flow control.
In this embodiment, the control center 10 may determine a target node that needs to perform flow control based on the storage state information of the at least one second node; and sending a data synchronization instruction to the first node and the target node, wherein the data synchronization instruction comprises waiting time so as to control the first node and the target node to start data synchronization operation on the user data after the waiting time elapses. For the storage status information, reference may be made to the description in the foregoing embodiments, which are not repeated herein.
The control node can respectively judge whether the storage state information of at least one second node meets the preset condition, and if the second node meeting the preset condition exists, the node is determined as a target node. For example, the preset condition may be that the current load exceeds a threshold, the current remaining amount of storage is lower than a specified remaining amount, and the like, in which case it may be determined that the second node needs to perform flow control.
For the first node, the data synchronization task with the target node may not be started for a while, but may be started after the latency in the data synchronization instruction is reached. Similarly, the target node may also initiate a data synchronization task with the first node after the latency has passed. Therefore, the flow control of the target node can be realized, and the storage burden on the target node is avoided.
In the above or below described embodiments, the control center 10 may support the orchestration of access requests for user data based on multiple copies of the user data stored.
In this embodiment, the control center 10 may receive an access request for user data initiated by a terminal device; determining candidate nodes storing user data or user data copies based on the storage relationship between the recorded data and the edge nodes 20; and selecting a response node from the candidate nodes to provide for the terminal equipment so that the terminal equipment can access the user data from the response node.
In this embodiment, the edge nodes 20 in the cloud network may report the information of the user data stored in the edge nodes 20 to the control center 10, and based on this, the control center 10 may globally record the storage relationship between the data and the edge nodes 20, that is, which edge nodes 20 store which user data. This may provide a global index for the scheduling of access requests by the control center 10, thereby ensuring the accuracy of the scheduling.
Accordingly, in the embodiment, the user data is stored on the plurality of edge nodes in the cloud network, and the storage relationship between the global data and the edge nodes can be maintained in the control center, so that accurate scheduling of the access request can be realized, and thus, when the original node has a problem, the access request of the terminal device can be scheduled to a new node, and the copy of the user data can be continuously used, thereby improving the high availability of the user data.
Fig. 3 is a flowchart illustrating a cloud storage method according to another exemplary embodiment of the present application, where the method may be performed by an edge node, and the edge node may be implemented as a combination of software and/or hardware. Referring to fig. 3, the method includes:
step 300, receiving and storing user data generated by the terminal equipment;
step 301, sending a data synchronization request aiming at user data to a control center;
step 302, receiving a data synchronization instruction returned by the control center, wherein the data synchronization instruction comprises information of at least one synchronization node determined by the control center for user data from other edge nodes contained in the cloud network;
step 303, synchronizing the user data to at least one synchronization node according to the data synchronization instruction.
In an alternative embodiment, the step of encapsulating the user data into data of a specified format comprises:
and sending the specified format data to the synchronous node so that the synchronous node unpacks the specified format data and stores the unpacked data as a copy of the user data.
In an optional embodiment, a plurality of sets of storage resources are respectively deployed on a plurality of edge nodes, and the storage types supported by different sets of storage resources are different, the method further includes:
and configuring the storage type of the user data and the storage type of the required copy in a header file of the specified format data, so that the synchronization node can specifically decapsulate the specified format data to a data format corresponding to the storage type of the copy and store the decapsulated data to a storage resource supporting the storage type of the copy.
In an optional embodiment, the step of synchronizing the user data to the synchronization node further includes:
selecting a target transmission channel adaptive to user data from a plurality of preset data transmission channels;
and synchronizing the user data to the synchronization node through the target transmission channel.
In an optional embodiment, the method further comprises:
and reporting the storage state information to a control center, wherein the storage state information comprises one or more of geographical position, bandwidth, residual storage capacity, current load capacity, weight, total amount of storable files and storage cost.
In an optional embodiment, the data synchronization instruction further includes information of a waiting time and a target node that needs to perform flow control among the synchronization nodes, and the method further includes:
and starting the operation of carrying out data synchronization on the user data after the waiting time.
It should be noted that, for the technical details in the embodiments of the cloud storage method, reference may be made to the related description about the first node in the foregoing system embodiment, and for the sake of brevity, no further description is provided herein, but this should not cause a loss of the protection scope of the present application.
Fig. 4 is a flowchart illustrating another cloud storage method according to another exemplary embodiment of the present application, where the method may be performed by a control center, and the control center may be implemented as a combination of software and/or hardware. Referring to fig. 4, the method includes:
step 400, receiving a data synchronization request aiming at user data, which is sent by a first node in a plurality of edge nodes in a cloud network;
step 401, determining at least one second node for the first node from other edge nodes;
step 402, sending a data synchronization instruction to the first node and the at least one second node, respectively, to control a data synchronization operation of user data between the first node and the at least one second node.
In an alternative embodiment, the step of determining at least one second node for the first node from among the other edge nodes comprises:
maintaining respective storage state information of a plurality of edge nodes;
respectively determining storage capacity indexes of other nodes to the user data according to respective storage state information of other nodes except the first node in the plurality of edge nodes;
at least one second node is determined for the first node from the other nodes according to the storage capacity index.
In an alternative embodiment, the storage status information includes one or more of a geographic location, a bandwidth, a remaining storage capacity, a current load amount, a weight, a total amount of storable files, and a storage cost.
In an optional embodiment, the method may further comprise:
determining a target node which needs to carry out flow control based on the storage state information of at least one second node;
and sending a data synchronization instruction to the first node and the target node, wherein the data synchronization instruction comprises waiting time so as to control the first node and the target node to start data synchronization operation on the user data after the waiting time elapses.
In an optional embodiment, the method may further comprise:
receiving an access request for user data initiated by terminal equipment;
determining candidate nodes storing user data or user data copies based on the storage relation between the recorded data and the edge nodes;
and selecting a response node from the candidate nodes to provide for the terminal equipment so that the terminal equipment can access the user data from the response node.
It should be noted that, for the technical details in the embodiments of the cloud storage method, reference may be made to the related description about the control center in the foregoing system embodiment, and for the sake of brevity, detailed description is not repeated herein, but this should not cause a loss of the protection scope of the present application.
Fig. 5 is a schematic flowchart of another cloud storage method according to another exemplary embodiment of the present application. The method may be performed by an edge node, which may be implemented as a combination of software and/or hardware. Referring to fig. 5, the method includes:
500, receiving a data synchronization instruction which is sent by a control center and aims at user data contained in a target edge node in a cloud network;
step 501, establishing a communication link with a target edge node;
step 502, synchronize user data from the target edge node to store a copy of the user data.
In an alternative embodiment, the step of synchronizing the user data from the target edge node may comprise:
receiving specified format data which is provided by a target edge node and obtained by carrying out data encapsulation on user data;
decapsulating the data in the specified format;
and storing the unpacked data as a copy of the user data.
In an optional embodiment, a plurality of groups of storage resources are respectively deployed on a plurality of edge nodes in a cloud network, the storage types supported by the storage resources of different groups are different, and a storage type of user data and a storage type of a required copy are configured in a header file of specified format data; the method specifically comprises the following steps:
decapsulating the data in the specified format to a data format corresponding to the storage type of the copy;
and storing the unpackaged data into a storage resource of a storage type supporting the copy.
In an optional embodiment, the method further comprises:
selecting a target interface which is adaptive to the storage type of the copy required by the user data from storage interfaces corresponding to the multiple groups of storage resources;
and under the target interface, performing an operation of decapsulating the specified format data to a data format corresponding to the storage type of the copy and an operation of storing the decapsulated data to a storage resource supporting the storage type of the copy.
In an optional embodiment, the method further comprises:
and reporting the storage state information to a control center, wherein the storage state information comprises one or more of geographical position, bandwidth, residual storage capacity, current load capacity, weight, total amount of storable files and storage cost.
In an optional embodiment, the method further comprises:
and if the data synchronization instruction sent by the control center contains waiting time, starting data synchronization operation on the user data after the waiting time.
It should be noted that, for the technical details in the embodiments of the cloud storage method, reference may be made to the related description about the second node in the foregoing system embodiment, and for the sake of brevity, detailed description is not repeated herein, but this should not cause a loss of the protection scope of the present application.
It should be noted that the execution subjects of the steps of the methods provided in the above embodiments may be the same device, or different devices may be used as the execution subjects of the methods. For example, the execution subjects of steps 300 to 303 may be device a; for another example, the execution subject of steps 301 and 302 may be device a, and the execution subject of step 303 may be device B; and so on.
In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of the order presented herein or in parallel, and the sequence numbers of the operations, such as 301, 302, etc., are merely used for distinguishing different operations, and the sequence numbers do not represent any execution order per se. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used to distinguish different nodes, etc., and do not represent the order of precedence, nor limit "first" and "second" to be different types.
Fig. 6 is a schematic structural diagram of an edge node according to another exemplary embodiment of the present application. As shown in fig. 6, the edge node may include:
a storage module 60, configured to receive and store user data generated by a terminal device;
a communication module 61, configured to send a data synchronization request for user data to a control center;
the coordination module 62 is configured to receive a data synchronization instruction returned by the control center, where the data synchronization instruction includes information of at least one synchronization node determined by the control center for the user data from other edge nodes included in the cloud network;
and a synchronization module 63, configured to synchronize the user data to at least one synchronization node according to the data synchronization instruction.
In an alternative embodiment, the synchronization module 63, in packaging the user data into the data with the specified format, is configured to:
and sending the specified format data to the synchronous node so that the synchronous node unpacks the specified format data and stores the unpacked data as a copy of the user data.
In an optional embodiment, a plurality of sets of storage resources are respectively deployed on the plurality of edge nodes, where the storage types supported by different sets of storage resources are different, and the synchronization module 63 is further configured to:
and configuring the storage type of the user data and the storage type of the required copy in a header file of the specified format data, so that the synchronization node can specifically decapsulate the specified format data to a data format corresponding to the storage type of the copy and store the decapsulated data to a storage resource supporting the storage type of the copy.
In an alternative embodiment, the synchronization module 63, in synchronizing the user data to the synchronization node, is further configured to:
selecting a target transmission channel adaptive to user data from a plurality of preset data transmission channels;
and synchronizing the user data to the synchronization node through the target transmission channel.
In an alternative embodiment, the coordination module 62 is further configured to:
and reporting the storage state information to a control center, wherein the storage state information comprises one or more of geographical position, bandwidth, residual storage capacity, current load capacity, weight, total amount of storable files and storage cost.
In an optional embodiment, the data synchronization instruction further includes information of a waiting time and a target node of the synchronization nodes that needs to perform flow control, and the synchronization module 63 is further configured to:
and starting the operation of carrying out data synchronization on the user data after the waiting time.
It should be noted that, for the technical details in the embodiments of the cloud storage method, reference may be made to the related description about the first node in the foregoing system embodiment, and for the sake of brevity, no further description is provided herein, but this should not cause a loss of the protection scope of the present application.
Based on the schematic structural diagram of the edge node in fig. 6, the edge node in another exemplary embodiment of the present application may also implement more processing logic.
In this exemplary embodiment:
the coordination module 62 is configured to receive a data synchronization instruction, which is sent by the control center and is for user data included in a first edge node in the cloud network;
a communication module 61, configured to establish a communication link with a first edge node;
a synchronization module 63 for synchronizing the user data from the first edge node to store a copy of the user data.
In an alternative embodiment, the synchronization module 63, in synchronizing the user data from the target edge node, is configured to:
receiving specified format data which is provided by a target edge node and obtained by carrying out data encapsulation on user data;
decapsulating the data in the specified format;
and storing the unpacked data as a copy of the user data.
In an optional embodiment, a plurality of groups of storage resources are respectively deployed on a plurality of edge nodes in a cloud network, the storage types supported by the storage resources of different groups are different, and a storage type of user data and a storage type of a required copy are configured in a header file of specified format data; the synchronization module 63 is specifically configured to:
decapsulating the data in the specified format to a data format corresponding to the storage type of the copy;
and storing the unpackaged data into a storage resource of a storage type supporting the copy.
In an alternative embodiment, the synchronization module 63 is further configured to:
selecting a target interface which is adaptive to the storage type of the copy required by the user data from storage interfaces corresponding to the multiple groups of storage resources;
and under the target interface, performing an operation of decapsulating the specified format data to a data format corresponding to the storage type of the copy and an operation of storing the decapsulated data to a storage resource supporting the storage type of the copy.
Only a part of the constituent elements is schematically shown in fig. 6, and it is not meant that the edge node includes only the constituent elements shown in fig. 6.
It should be noted that, for the technical details in the embodiments of the edge node, reference may be made to the related descriptions of the first node and the second node in the foregoing system embodiments, and for the sake of brevity, detailed descriptions are not provided herein, but this should not cause a loss of the scope of the present application.
Accordingly, the present application further provides a computer-readable storage medium storing a computer program, where the computer program is capable of implementing the steps that can be performed by the edge node in the foregoing method embodiments when executed.
Fig. 7 is a schematic structural diagram of a control device according to another exemplary embodiment of the present application. As shown in fig. 7, the control apparatus may include:
a receiving module 70, configured to receive a data synchronization request for user data sent by a first node in a plurality of edge nodes in a cloud network;
a scheduling module 71 for determining at least one second node for the first node from the other edge nodes;
a sending module 72, configured to send a data synchronization instruction to the first node and the at least one second node, respectively, so as to control a data synchronization operation of user data between the first node and the at least one second node.
In an alternative embodiment, the process of the scheduling module 71 determining at least one second node for the first node from the other edge nodes may be configured to:
maintaining respective storage state information of a plurality of edge nodes;
respectively determining storage capacity indexes of other nodes to the user data according to respective storage state information of other nodes except the first node in the plurality of edge nodes;
at least one second node is determined for the first node from the other nodes according to the storage capacity index.
In an alternative embodiment, the storage status information includes one or more of a geographic location, a bandwidth, a remaining storage capacity, a current load amount, a weight, a total amount of storable files, and a storage cost.
In an alternative embodiment, the scheduling module 71 may further be configured to:
determining a target node which needs to carry out flow control based on the storage state information of at least one second node;
and sending a data synchronization instruction to the first node and the target node, wherein the data synchronization instruction comprises waiting time so as to control the first node and the target node to start data synchronization operation on the user data after the waiting time elapses.
In an alternative embodiment, the scheduling module 71 may further be configured to:
receiving an access request for user data initiated by terminal equipment;
determining candidate nodes storing user data or user data copies based on the storage relation between the recorded data and the edge nodes;
and selecting a response node from the candidate nodes to provide for the terminal equipment so that the terminal equipment can access the user data from the response node.
Only some of the constituent elements are schematically shown in fig. 7, and it is not intended that the control apparatus includes only the constituent elements shown in fig. 7.
It should be noted that, for the sake of brevity, the technical details of the embodiments of the control device described above may refer to the related descriptions of the control center in the foregoing system embodiments, which should not be repeated herein, but should not cause a loss of the protection scope of the present application.
Accordingly, the present application further provides a computer-readable storage medium storing a computer program, where the computer program can implement the steps that can be performed by the control center in the above method embodiments when executed. The embodiments of the present application also provide a computing device, which includes a memory, a processor, a communication component, and the like, where the processor may execute computer instructions in the memory to perform the foregoing various cloud storage methods.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (14)

1. The cloud storage system is characterized by comprising a control center and a plurality of edge nodes;
the control center is used for receiving a data synchronization request sent by a first node in the plurality of edge nodes and determining at least one second node for the first node from other edge nodes; respectively sending data synchronization instructions to the first node and the second node;
the first node is used for receiving and storing user data generated by the terminal equipment; sending the data synchronization request for the user data to the control center; synchronizing the user data to the at least one second node according to a data synchronization instruction returned by the control center;
and the second node synchronizes the user data from the first node according to the data synchronization instruction so as to store the copy of the user data.
2. The system according to claim 1, wherein the first node, in synchronizing the user data to the at least one second node, is configured to: packaging the user data into specified format data; sending the specified format data to the second node;
the second node, in synchronizing the user data from the first node, is configured to: decapsulating the specified format data; and storing the unpackaged data as a copy of the user data.
3. The system according to claim 2, wherein a plurality of groups of storage resources are respectively deployed on the plurality of edge nodes, different storage types supported by different groups of storage resources are different, and the first node is further configured to: configuring the storage type of the user data and the storage type of the required copy in a header file of the specified format data;
the second node is specifically configured to: decapsulating the specified format data to a data format corresponding to the storage type of the copy; and storing the unpackaged data into a storage resource of a storage type supporting the copy.
4. The system of claim 3, wherein the second node is specifically configured to:
selecting a target interface which is adaptive to the storage type of the copy required by the user data from storage interfaces corresponding to multiple groups of storage resources;
and under the target interface, executing the operation of decapsulating the specified format data to the data format corresponding to the storage type of the copy and the operation of storing the decapsulated data to a storage resource supporting the storage type of the copy.
5. The system of claim 1, wherein the first node, in synchronizing the user data to the second node, is further configured to:
selecting a target transmission channel adaptive to the user data from a plurality of preset data transmission channels;
synchronizing the user data to the second node through the target transmission channel.
6. The system of claim 1, wherein the control center, in determining at least one second node for the first node from among the other edge nodes, is configured to:
maintaining respective storage state information of the plurality of edge nodes;
respectively determining storage capacity indexes of other nodes to the user data according to respective storage state information of other nodes except the first node in the plurality of edge nodes;
determining at least one second node for said first node from said other nodes according to said storage capacity index.
7. The system of claim 6, wherein the storage status information comprises one or more of a geographic location, a bandwidth, a remaining storage capacity, a current amount of load, a weight, a total amount of storable files, and a storage cost.
8. The system of claim 6, wherein the control center is further configured to:
determining a target node which needs to carry out flow control based on the storage state information of the at least one second node;
and sending a data synchronization instruction to the first node and the target node, wherein the data synchronization instruction comprises waiting time so as to control the first node and the target node to start data synchronization operation on the user data after the waiting time elapses.
9. The system of claim 1, wherein the control center is further configured to:
receiving an access request for the user data, which is initiated by terminal equipment;
determining a candidate node storing the user data or the user data copy based on a storage relation between the recorded data and the edge node;
and selecting a response node from the candidate nodes to provide for the terminal equipment so that the terminal equipment can access the user data from the response node.
10. A cloud storage method, comprising:
receiving and storing user data generated by terminal equipment;
sending a data synchronization request aiming at the user data to a control center;
receiving a data synchronization instruction returned by the control center, wherein the data synchronization instruction comprises information of at least one synchronization node determined by the control center for the user data from other edge nodes contained in the cloud network;
and synchronizing the user data to the at least one synchronization node according to the data synchronization instruction.
11. A cloud storage method, comprising:
receiving a data synchronization request aiming at user data sent by a first node in a plurality of edge nodes in a cloud network;
determining at least one second node for the first node from the other edge nodes;
and respectively sending a data synchronization instruction to the first node and the at least one second node to control the data synchronization operation of the user data between the first node and the at least one second node.
12. A cloud storage method, comprising:
receiving a data synchronization instruction which is sent by a control center and aims at user data contained in a first edge node in a cloud network;
establishing a communication link with the first edge node;
synchronizing the user data from the first edge node to store a copy of the user data.
13. An edge node, comprising:
the storage module is used for receiving and storing user data generated by the terminal equipment;
the communication module is used for sending a data synchronization request aiming at the user data to a control center;
the coordination module is used for receiving a data synchronization instruction returned by the control center, wherein the data synchronization instruction comprises information of at least one synchronization node determined by the control center for the user data from other edge nodes contained in the cloud network;
and the synchronization module is used for synchronizing the user data to the at least one synchronization node according to the data synchronization instruction.
14. A computer-readable storage medium storing computer instructions, which when executed by one or more processors, cause the one or more processors to perform the cloud storage method of claim 13.
CN202110789656.6A 2021-07-13 2021-07-13 Cloud storage method, system, equipment and storage medium Pending CN113676510A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110789656.6A CN113676510A (en) 2021-07-13 2021-07-13 Cloud storage method, system, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110789656.6A CN113676510A (en) 2021-07-13 2021-07-13 Cloud storage method, system, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113676510A true CN113676510A (en) 2021-11-19

Family

ID=78539068

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110789656.6A Pending CN113676510A (en) 2021-07-13 2021-07-13 Cloud storage method, system, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113676510A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114726871A (en) * 2022-03-22 2022-07-08 阿里巴巴(中国)有限公司 Data synchronization method, device and network system

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130318229A1 (en) * 2012-05-25 2013-11-28 Netapp, Inc. Method and system for name space propagation and file caching to remote nodes in a storage system
CN108121511A (en) * 2017-12-21 2018-06-05 北京黑螺技术有限公司 Data processing method, device and equipment in a kind of distributed edge storage system
CN110601992A (en) * 2019-09-20 2019-12-20 南方电网科学研究院有限责任公司 Data processing method and device of intelligent measurement terminal based on edge calculation
CN111444043A (en) * 2020-03-27 2020-07-24 中移雄安信息通信科技有限公司 Data backup method, device, node, system and medium
CN112532729A (en) * 2020-11-30 2021-03-19 北京百度网讯科技有限公司 Data synchronization method and device for edge device and cloud end
CN112565325A (en) * 2019-09-26 2021-03-26 华为技术有限公司 Mirror image file management method, device and system, computer equipment and storage medium
CN112839076A (en) * 2020-12-29 2021-05-25 杭州海康威视系统技术有限公司 Data storage method, data reading method, gateway, electronic equipment and storage medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130318229A1 (en) * 2012-05-25 2013-11-28 Netapp, Inc. Method and system for name space propagation and file caching to remote nodes in a storage system
CN108121511A (en) * 2017-12-21 2018-06-05 北京黑螺技术有限公司 Data processing method, device and equipment in a kind of distributed edge storage system
CN110601992A (en) * 2019-09-20 2019-12-20 南方电网科学研究院有限责任公司 Data processing method and device of intelligent measurement terminal based on edge calculation
CN112565325A (en) * 2019-09-26 2021-03-26 华为技术有限公司 Mirror image file management method, device and system, computer equipment and storage medium
CN111444043A (en) * 2020-03-27 2020-07-24 中移雄安信息通信科技有限公司 Data backup method, device, node, system and medium
CN112532729A (en) * 2020-11-30 2021-03-19 北京百度网讯科技有限公司 Data synchronization method and device for edge device and cloud end
CN112839076A (en) * 2020-12-29 2021-05-25 杭州海康威视系统技术有限公司 Data storage method, data reading method, gateway, electronic equipment and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114726871A (en) * 2022-03-22 2022-07-08 阿里巴巴(中国)有限公司 Data synchronization method, device and network system
CN114726871B (en) * 2022-03-22 2024-06-11 阿里巴巴(中国)有限公司 Data synchronization method, device and network system

Similar Documents

Publication Publication Date Title
CN110896355B (en) Network slice selection method and device
US9712402B2 (en) Method and apparatus for automated deployment of geographically distributed applications within a cloud
CN113973026B (en) Site interconnection method, central controller and route reflector
WO2020042612A1 (en) Method and device for storing and reading a message, server, and storage medium
US20220329495A1 (en) Network resource management method and system, network equipment and readable storage medium
CN113938520A (en) Service registration method, device and storage medium
CN113301079B (en) Data acquisition method, system, computing device and storage medium
CN113676510A (en) Cloud storage method, system, equipment and storage medium
CN113411363A (en) Uploading method of image file, related equipment and computer storage medium
CN112532665A (en) Server load balancing method and device and CDN node
US10951707B2 (en) Selection device, device selection method, and program
CN113965519A (en) Flow control method, cluster resource guarantee method, equipment and storage medium
US8601151B2 (en) Apparatus and method for receiving data
CN107426109B (en) Traffic scheduling method, VNF module and traffic scheduling server
CN108124021B (en) Method, device and system for obtaining Internet Protocol (IP) address and accessing website
WO2016000303A1 (en) Resource allocation method and system, and computer storage medium
CN114866553B (en) Data distribution method, device and storage medium
US20230188625A1 (en) Service request handling
CN114301826A (en) Message transmission method and device
CN117321973A (en) Applying automatic matching to predefined slice types in 5G networks
CN114727307A (en) Network slice deployment method and device
US11979824B2 (en) Service request handling
CN115712545B (en) Method, device, equipment and medium for realizing service topology awareness of cluster
CN110830531B (en) Resource request processing method and device, storage system, electronic device and storage device
US20230179653A1 (en) Service request handling

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40069104

Country of ref document: HK

TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20240308

Address after: # 03-06, Lai Zan Da Building 1, 51 Belarusian Road, Singapore

Applicant after: Alibaba Innovation Co.

Country or region after: Singapore

Address before: Room 01, 45th Floor, AXA Building, 8 Shanton Road, Singapore

Applicant before: Alibaba Singapore Holdings Ltd.

Country or region before: Singapore