WO2021057956A1 - 镜像文件管理方法、装置及系统、计算机设备、存储介质 - Google Patents

镜像文件管理方法、装置及系统、计算机设备、存储介质 Download PDF

Info

Publication number
WO2021057956A1
WO2021057956A1 PCT/CN2020/118047 CN2020118047W WO2021057956A1 WO 2021057956 A1 WO2021057956 A1 WO 2021057956A1 CN 2020118047 W CN2020118047 W CN 2020118047W WO 2021057956 A1 WO2021057956 A1 WO 2021057956A1
Authority
WO
WIPO (PCT)
Prior art keywords
edge cloud
image file
data block
mirror
cloud node
Prior art date
Application number
PCT/CN2020/118047
Other languages
English (en)
French (fr)
Inventor
罗锐
殷薇
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP20868383.9A priority Critical patent/EP4030724A4/en
Publication of WO2021057956A1 publication Critical patent/WO2021057956A1/zh
Priority to US17/704,730 priority patent/US20220215002A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1095Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/188Virtual file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/60Memory management
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/06Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]

Definitions

  • This application relates to the field of cloud computing technology, and in particular to a mirror file management method, device and system, computer equipment, and storage medium.
  • image files can be used to create instances such as virtual hosts or containers, and after creating instances such as virtual hosts or containers, new image files can be recreated based on the created virtual host or container instances. And store the image file for future use.
  • the size of the image file is usually large.
  • the size of the image file can usually reach the order of gigabytes (GB). Therefore, the effective management of the image file can improve the storage efficiency of the image file and the creation speed of the virtual host. It is of great significance.
  • the functions of the cloud service system are usually implemented by a central cloud node and multiple edge cloud nodes, and the central cloud node and multiple edge cloud nodes work together.
  • the mirror file in the cloud service system is managed by the mirror file management system.
  • the management method is: all image files are stored in the central cloud node.
  • the edge cloud node needs to use the image file, the edge cloud node downloads the required image file from the central cloud node.
  • This application provides a mirror file management method, device and system, computer equipment, and storage medium, which can solve the problems of large time overhead and slow download speed for edge cloud nodes to download mirror files from central cloud nodes.
  • the present application provides a mirror file management method.
  • the mirror file management method includes: receiving a write instruction for instructing to write a first mirror file; based on the write instruction, the mirror file management system includes multiple files.
  • the edge cloud nodes at least one first edge cloud node is determined; multiple data blocks included in the first image file are written to at least one first edge cloud node, so that each first edge cloud node stores the first image Part or all of multiple data blocks included in the file.
  • the edge cloud node stores the data blocks included in the first image file, so that when any edge cloud node in the image file management system needs to use the image file, it can obtain the corresponding data block from the edge cloud node that stores the data block in the image file.
  • To obtain the required image file there is no need to download the image file from the central cloud node. This reduces the probability that multiple edge cloud nodes share the available bandwidth of the same cloud node, and can make greater use of this task.
  • the available bandwidth of an edge cloud node, and the bandwidth between any edge cloud node and the edge cloud node that stores the data blocks required by any edge cloud node reduces the time cost of obtaining image files, and improves mirroring The download speed of the file.
  • the mirror file management method may further include: generating each data block included in the first mirror file The identification number; based on the identification number of each data block included in the first image file, the first identification number topology of the first image file is generated, and the first identification number topology records the data blocks included in the first image file; The identification number topology is stored in the edge cloud node of the image file management system.
  • the user can quickly determine the first data block included in the first image file according to the first identification number topology in the process of reading the first image file.
  • the first identification number topology may also carry information used to describe the content of the first mirror file, such as the version information of the first mirror file, the name of the first mirror file, and permission control information.
  • the description information of the first mirror file By adding the description information of the first mirror file to the topology of the first identification number, it is convenient to quickly determine the content of the first mirror file without reading the data blocks to determine the data block and the first mirror file included in the first mirror file. The content of the mirror file can reduce the read delay.
  • the mirror file management method may further include: The storage status of each data block included in the mirror file in at least one first edge cloud node, and establish each data block included in the first mirror file and the first edge cloud storing the corresponding data block in the first mirror file The storage correspondence of the nodes; the storage correspondence is stored in the edge cloud node of the image file management system.
  • multiple edge cloud nodes in the image file management system can obtain multiple first data blocks on one or more first edge cloud nodes according to the storage correspondence.
  • the method before writing the multiple data blocks included in the first image file to at least one first edge cloud node, the method further includes: determining that the first image file includes Multiple data blocks included in the first mirror file that are not stored in any edge cloud node among all data blocks.
  • the present application provides a mirror file management method, which is applied to a target edge cloud node, and the mirror file management method includes: receiving a read instruction for instructing to read a second mirror file; based on the read instruction , Among the multiple edge cloud nodes included in the image file management system, determine one or more second edge cloud nodes that store the data block of the second image file; read from one or more second edge cloud nodes that belong to The data block of the second mirror file to obtain the second mirror file.
  • each edge cloud node is configured with the function of storing image files, and the multiple edge cloud nodes are configured to be able to read the edge cloud node storage in the image file management system
  • any edge cloud node in the mirror file management system needs to use the mirror file, it can obtain the corresponding data block from the edge cloud node that stores the data block in the mirror file to obtain the mirror file.
  • There is no need to download the image file from the central cloud node which reduces the probability that multiple edge cloud nodes share the available bandwidth of the same cloud node, and can maximize the use of the available bandwidth of any edge cloud node and any edge.
  • the bandwidth between the cloud node and the edge cloud node storing the data block required by any edge cloud node reduces the time overhead for obtaining the image file and improves the download speed of the image file.
  • the implementation process of the edge cloud node includes: determining all data blocks included in the second mirror file based on the read instruction; determining one of all data blocks included in the second mirror file that is not stored in the target edge cloud node Or multiple data blocks; determine the second edge cloud node storing any one of the one or more data blocks.
  • the target edge The cloud node can obtain data blocks that are not stored by the target edge cloud node and belong to the second mirror file from the second edge cloud node, which can avoid obtaining duplicate data blocks, can reduce the total number of data blocks that need to be obtained, and reduce data The download amount of the block reduces the time overhead for obtaining the second image file.
  • the implementation process of determining all data blocks included in the second mirror file based on the read instruction includes: determining the second identification number of the second mirror file based on the read instruction Topology, the second identification number topology records data blocks included in the second mirror file; based on the second identification number topology, all data blocks included in the second mirror file are determined.
  • the implementation of determining all data blocks included in the second mirror file through the second identification number topology can reduce the determination of the data blocks belonging to the second mirror file by reading all the data blocks to determine the data blocks belonging to the second mirror file.
  • the amount of data processing in the data block process of the second mirror file can reduce the read delay of the second mirror file.
  • the implementation process of determining one or more data blocks that are not stored in the target edge cloud node includes: querying the first The storage correspondence between each data block in the second mirror file and the edge cloud node that stores the corresponding data block in the second mirror file; when the storage correspondence of a certain data block in the second mirror file does not record the second mirror file When the corresponding relationship between a certain data block and the target edge cloud node in, it is determined that a certain data block is a data block that is not stored in the target edge cloud node.
  • the implementation of determining data blocks that are not stored in the target edge cloud node through the storage correspondence is compared with comparing the content of all data blocks stored in the target edge cloud node with the content of each second data block,
  • the method of determining the second data block belonging to the second mirror file but not stored in the target edge cloud node can reduce the process of determining the second data block belonging to the second mirror file but not stored in the target edge cloud node The amount of data processing to further reduce the read delay of the second mirror file.
  • the implementation process of reading data blocks belonging to the second mirror file from one or more second edge cloud nodes includes: When a certain data block included in the second mirror file is stored, the download status parameter of each second edge node that all stores a certain data block is obtained; based on the data of each second edge node that all stores a certain data block Download status parameters, select the second edge cloud node to download a certain data block; download a certain data block from the selected second edge cloud node.
  • the second edge cloud node used as the download source By selecting the second edge cloud node used as the download source among the multiple second edge cloud nodes that all store a certain data block included in the second mirror file according to the second edge cloud set eh download status parameter, it can be guaranteed Download the download speed of the certain data block.
  • the process of downloading a certain data block from the selected second edge cloud node may include: continuing to obtain the selected data block in the process of downloading a certain data block from the selected second edge cloud node The download status parameter of the second edge cloud node; when the download status parameter of the selected second edge cloud node is less than the reference parameter threshold, the multiple second edge nodes that all store a certain data block re-select to download a certain data For the second edge cloud node of the block, the download state parameter of the reselected second edge cloud node is greater than the reference parameter threshold; re-download a certain data block from the reselected second edge cloud node.
  • the mirror file management method further includes: storing from the second edge cloud node.
  • the data block of the second mirror file read from the cloud node; the storage correspondence between each data block included in the second mirror file and the target edge cloud node is stored in the edge cloud node of the mirror file management system to indicate The target edge cloud node also stores the second data block, so that other edge cloud nodes in the image file management system can obtain storage information of the second data block.
  • the image file management method may further include: receiving an update instruction for instructing to update the information of the third image file, the third image file is the target edge cloud node based on the image file management Generated by the fourth mirror file stored in the system; generate the identification number of each data block included in the third mirror file; generate the third mirror file based on the identification number of each data block included in the third mirror file
  • the third identification number topology records the data blocks included in the third image file; the third identification number topology is stored in the edge cloud node of the image file management system.
  • edge cloud nodes in the image file management system can obtain the stored image file based on the information, so as to ensure the integrity of the image file stored in the image file management system. Data completeness.
  • Generating the third identification number topology of the third image file may include: in the fourth identification number topology, determining the first tree sub-topology, and the identification numbers of the same data blocks included in the third image file and the fourth image file are The leaf nodes in the first tree subtopology; based on the data blocks included in the third mirror file that are different from the data blocks included in the fourth mirror file, the second tree subtopology is generated, and the third mirror file is Among the included data blocks, the identification numbers of the data blocks that are different from the data blocks included in the fourth mirror file are leaf nodes in the second tree sub-topology; based on the first tree sub-topology and the second tree sub-topology, the first tree sub-topology is generated. Three identification number topology.
  • the implementation of generating the third identification number topology is essentially the process of obtaining the third identification number topology of the third image file by performing incremental updates on the basis of the fourth identification number topology, which can reduce the generation of the third identification number. No. topology calculations, and reduce the writing delay of updating the third image file information.
  • the third identification number topology also carries version information and update description information of the third mirror file, and the update description information is used to reflect the different content of the third mirror file relative to the fourth mirror file.
  • the update description information is used to reflect the different content of the third mirror file relative to the fourth mirror file.
  • version management on the mirror file in the process of writing data blocks to the mirror file management system, when determining whether the data block is stored in the mirror file management system, it can only be linked to a certain mirror file and based on the data block.
  • the data blocks of a version of the mirror file in the mirror file generated by the mirror file are compared to reduce the number of comparisons and the write delay.
  • the mirror file management method after receiving the update instruction for instructing to update the information of the third mirror file, the mirror file management method further includes: establishing each data block included in the third mirror file and The storage correspondence relationship of the target edge cloud node; the storage correspondence relationship between each data block included in the third image file and the target edge cloud node is stored in the edge cloud node of the image file management system.
  • the edge cloud node in the image file management system can obtain the storage status of the data blocks included in the third image file.
  • the present application provides a mirror file management device, the mirror file management device includes: a first receiving module, configured to receive a write instruction for instructing to write a first mirror file; a first determining module, using Based on the write instruction, at least one first edge cloud node is determined among the multiple edge cloud nodes included in the image file management system; the first write module is used to combine the multiple data blocks included in the first image file, Write at least one first edge cloud node, so that each first edge cloud node stores part or all of the multiple data blocks included in the first image file.
  • the image file management apparatus further includes: a first generation module, configured to generate an identification number of each data block included in the first image file; and the first generation module also uses Generating a first identification number topology of the first image file based on the identification number of each data block included in the first image file, the first identification number topology recording the data blocks included in the first image file; a first writing module, It is also used to store the first identification number topology in the edge cloud node of the image file management system.
  • a first generation module configured to generate an identification number of each data block included in the first image file
  • the first generation module also uses Generating a first identification number topology of the first image file based on the identification number of each data block included in the first image file, the first identification number topology recording the data blocks included in the first image file
  • a first writing module It is also used to store the first identification number topology in the edge cloud node of the image file management system.
  • the first identification number topology also carries version information of the first mirror file.
  • the image file management apparatus further includes: a first establishment module, configured to store each data block included in the first image file in at least one first edge cloud node Case, the storage correspondence between each data block included in the first mirror file and the first edge cloud node storing the corresponding data block in the first mirror file is established; the first writing module is also used to store the storage correspondence In the edge cloud node of the image file management system.
  • the first determining module is further configured to determine how many data blocks included in the first image file that is not stored in any edge cloud node among all the data blocks included in the first image file. Data blocks.
  • the present application provides a mirror file management device.
  • the mirror file management device includes: a second receiving module for receiving a read instruction for instructing to read the second mirror file; and a second determining module for Based on the read instruction, among the multiple edge cloud nodes included in the image file management system, determine one or more second edge cloud nodes that store the data block of the second image file; the read module is used to read from one or more The second edge cloud node reads the data block belonging to the second mirror file to obtain the second mirror file.
  • the second determining module is specifically configured to: determine all data blocks included in the second mirror file based on the read instruction; all data blocks included in the second mirror file , Determine one or more data blocks that are not stored in the target edge cloud node; determine the second edge cloud node that stores any one of the one or more data blocks.
  • the second determining module is specifically configured to determine the second identification number topology of the second mirror file based on the read instruction, and the second identification number topology record of the second mirror file includes Data blocks; based on the second identification number topology, determine all the data blocks included in the second mirror file.
  • the second determining module is specifically configured to query the storage correspondence of each data block in the second mirror file with the edge cloud node storing the corresponding data block in the second mirror file Relationship; When the storage correspondence of a certain data block in the second mirror file does not record the correspondence between a certain data block in the second mirror file and the target edge cloud node, it is determined that a certain data block is not stored at the target edge The data block in the cloud node.
  • the reading module is specifically used to: when a certain data block included in the second mirror file is stored in a plurality of second edge cloud nodes, obtain that a certain data block is stored in all the second edge cloud nodes.
  • the download status parameter of each second edge node of a data block based on the download status parameter of each second edge node that stores a certain data block, select the second edge cloud node that downloads a certain data block; select from Download a certain data block from the second edge cloud node.
  • the reading module is specifically used to: in the process of downloading a certain data block from the selected second edge cloud node, continue to obtain the selected second edge cloud node Download status parameter; when the download status parameter of the selected second edge cloud node is less than the reference parameter threshold, re-select the second edge cloud node that downloads a certain data block among multiple second edge nodes that all store a certain data block , The download state parameter of the re-selected second edge cloud node is greater than the reference parameter threshold; re-download a certain data block from the re-selected second edge cloud node.
  • the image file management apparatus further includes: a storage module for storing data blocks of the second image file read from the second edge cloud node; and a second writing module, It is used to store the storage correspondence between each data block included in the second image file and the target edge cloud node in the edge cloud node of the image file management system.
  • the second receiving module is further configured to: receive an update instruction for instructing to update the information of the third mirror file, the third mirror file is the target edge cloud node based on the mirror file management Generated by the fourth image file stored in the system;
  • the mirror file management device further includes: a second generation module, used to generate the identification number of each data block included in the third mirror file;
  • the identification number of each data block generates the third identification number topology of the third image file, and the third identification number topology records the data blocks included in the third image file;
  • the second writing module is used to store the third identification number topology In the edge cloud node of the image file management system.
  • the second generation module is specifically used for: in the fourth identification number topology , Determine the first tree sub-topology, the identification number of the same data block included in the third mirror file and the fourth mirror file is the leaf node in the first tree sub-topology; based on the data blocks included in the third mirror file and Data blocks in which the data blocks included in the fourth mirror file are not the same are generated, and a second tree-like sub-topology is generated. Among the data blocks included in the third mirror file, the data blocks in the fourth mirror file are different from those in the fourth mirror file.
  • the identification number is a leaf node in the second tree-shaped sub-topology; based on the first tree-shaped sub-topology and the second tree-shaped sub-topology, a third identification number topology is generated.
  • the third identification number topology also carries version information and update description information of the third mirror file, and the update description information is used to reflect that the third mirror file is relative to the fourth mirror file. Different content.
  • the image file management apparatus further includes: a second establishment module, configured to establish a storage correspondence between each data block included in the third image file and the target edge cloud node;
  • the second writing module is used to store the storage correspondence between each data block included in the third image file and the target edge cloud node in the edge cloud node of the image file management system.
  • this application provides a mirror file management system.
  • the mirror file management includes: a terminal and a plurality of edge cloud nodes.
  • the edge cloud node is used to store mirror files; wherein, the terminal is used to execute the mirror image provided in the first aspect.
  • the edge cloud node is used to execute any of the image file management methods of the second aspect; or, the terminal is used to send a write instruction for instructing to write the first image file to the edge cloud node, and the edge cloud node is used to Execute the mirror file management method provided by the first aspect or the second aspect.
  • this application provides a computer device that includes a processor and a memory; the memory stores a computer program; when the processor executes the computer program, the computer device implements the image file management method provided in the first aspect, or , To realize the mirror file management method provided by the second aspect.
  • the present application provides a storage medium that, when the instructions in the storage medium are executed by a processor, implements the mirror file management method provided in the first aspect, or implements the mirror file management method provided in the second aspect.
  • FIG. 1 is a schematic diagram of deployment of a central cloud and an edge cloud provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of a public cloud deployment method in a related technology provided by an embodiment of the present application
  • FIG. 3 is a schematic structural diagram of a mirror file management system provided by an embodiment of the present application.
  • Figure 4 is a flowchart of a mirror file management method provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a first identification number topology provided by an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a first directed acyclic graph obtained according to the hash tree shown in FIG. 5 according to an embodiment of the present application;
  • FIG. 7 is a flowchart of a method for updating information of a mirror file that already exists in a mirror file management system provided by an embodiment of the present application;
  • FIG. 8 is a flowchart of a method for incrementally updating a fourth identification number topology to obtain a third identification number topology according to an embodiment of the present application
  • FIG. 9 is a schematic diagram of a principle of incrementally updating a first identification number topology to obtain a third identification number topology according to an embodiment of the present application.
  • FIG. 10 is a flowchart of a method for reading a mirror file provided by an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a mirror file management apparatus provided by an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of another image file management apparatus provided by an embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of another image file management apparatus provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of another image file management apparatus provided by an embodiment of the present application.
  • Edge computing is a technology that deploys computing tasks between the cloud and the terminal. Edge computing can directly filter and analyze the data sent by the terminal, which can better support the real-time processing and execution of local services, saving energy, time and efficiency. The advantages.
  • edge cloud nodes used to implement edge computing are deployed in a distributed manner. The distributed deployment feature enables the terminal to use the edge cloud node that is logically closest to the terminal to provide service to the terminal when the terminal requests a service, which ensures the efficiency of serving the terminal.
  • Cloud service systems usually include central cloud and edge cloud.
  • the central cloud is a cloud computing platform based on traditional cloud infrastructure and is at the center of the network.
  • the functions of the central cloud are realized through the central cloud node.
  • Edge cloud is a cloud computing platform built on edge infrastructure.
  • the function of the edge cloud is realized through the edge cloud node.
  • the edge cloud can process and store part of the data under the control of the central cloud, and can integrate some of the services or capabilities of the central cloud (including but not limited to storage, computing, network, artificial intelligence, big data, security, etc.) Extending to the edge cloud enables the central cloud and edge cloud to work together, realizing functions such as center and edge collaboration, overall network computing power scheduling, and overall network unified management and control.
  • FIG. 1 is a schematic diagram of the deployment of a central cloud and an edge cloud provided by an embodiment of this application.
  • the central cloud 1 is used to provide central cloud services
  • the edge cloud 1, edge cloud 2 and edge cloud 3 are used to provide edge cloud services.
  • 1 and the central cloud 1, the edge cloud 2 and the central cloud 1, the edge cloud 3 and the central cloud 1 can all communicate, and the edge cloud 1, the edge cloud 2 and the edge cloud 3 can also communicate with each other.
  • the work of edge cloud 1, edge cloud 2 and edge cloud 3 depends on the services provided by central cloud 1, making the edge cloud an extension of the central cloud, realizing functions such as center and edge collaboration, computing power scheduling of the entire network, and unified management and control of the entire network .
  • the public cloud uses the powerful data processing capabilities of the central cloud node to manage image files.
  • the way to manage image files is as follows: According to the deployment area of the cloud service node in the public cloud, The public cloud is divided into different regions (regions), and each region independently deploys image management services and storage backends (such as hard disks), and each central cloud node is responsible for managing image files in a region.
  • Each zone includes multiple availability zones (availability zones), and multiple availability zones in the same zone share a mirroring application programming interface (API), mirroring the backend database and storage backend.
  • API application programming interface
  • a low-latency high-speed network connection such as infiniband (IB) network or gigabit ethernet is used between the central cloud node in the available zone and the storage backend to ensure the transmission bandwidth of the image file.
  • IB infiniband
  • gigabit ethernet is used between the central cloud node in the available zone and the storage backend to ensure the transmission bandwidth of the image file.
  • edge cloud node due to factors such as the capacity and scale of edge cloud nodes, the edge cloud node is not equipped with a storage backend, and edge cloud nodes can only store a small number of images in the form of cache. Therefore, when the edge cloud node needs to use the image file, if the image file needed by the edge cloud node is not cached in the cache of the edge cloud node, the edge cloud node needs to download the image file from the central cloud node it depends on first. Only the downloaded image file can be used.
  • edge cloud nodes that are dependent on a central cloud node, and the available bandwidth of the central cloud node is limited
  • the multiple edge cloud nodes Sharing the available bandwidth of the central cloud node causes the actual available bandwidth of each edge cloud node to download the image file from the central cloud node is small, the time overhead for downloading the image file is large, and the download speed is slow.
  • the distribution range of edge cloud nodes is generally wide.
  • the physical distance between different edge cloud nodes and central cloud nodes may be different.
  • the network conditions between different edge cloud nodes and central cloud nodes are quite different, resulting in different edge cloud nodes from the center.
  • the speed at which cloud nodes download mirror files varies greatly, which further increases the time overhead for some edge cloud nodes to download mirror files.
  • the embodiment of the present application provides a mirror file management system.
  • the mirror file management system includes a terminal and a plurality of edge cloud nodes.
  • the terminal is used to write the image file to the edge cloud node.
  • Each edge cloud node of the plurality of edge cloud nodes is used to store data blocks in the mirror file, and the plurality of edge cloud nodes are all configured to be able to read the data blocks stored by the edge cloud node in the mirror file management system. In this way, when any edge cloud node of the multiple edge cloud nodes needs to use the mirror file, the corresponding data block can be obtained from the edge cloud node that stores the data block in the mirror file to obtain the mirror file.
  • the bandwidth between the node and the edge cloud node storing the data block required by any edge cloud node reduces the time overhead for obtaining the image file and improves the download speed of the image file.
  • the mirror file management system is essentially a distributed mirror file management system for distributed management of mirror files.
  • each edge cloud node is responsible for storing a part of the mirror image.
  • File data, and responsible for routing information within a certain range, and the storage information of the data in the mirror file is shared among multiple edge cloud nodes through the network, so that when the mirror file is read from the mirror file management system, it can be shared according to the
  • the storage information of the mirror file is globally addressed in the mirror file management system, and the corresponding data block is read in the storage location indicated by the storage information, so as to realize the reading of the mirror file.
  • each edge cloud node may be configured with an application program interface and a storage backend.
  • the application program interface is used to provide an external access interface for responding to requests related to mirror files.
  • the application program interface of the edge cloud node is used for other edge cloud nodes or terminals to access the edge cloud node.
  • the application program interface can support multiple types of storage backends, and the storage backends are used to store data blocks in the image file.
  • SAN storage area network
  • NAS network attached storage
  • NAS local file system
  • NAS refers to a storage form in which storage devices are connected to a group of computers through a network topology (such as Ethernet) to achieve storage
  • SAN refers to a form of storage in which storage devices are connected to a group of computers through Fibre Channel to achieve storage.
  • the image file management system may include: a terminal and multiple servers.
  • Figure 3 is a schematic structural diagram of the image file management system when the image file management system includes three edge cloud nodes, and the edge cloud nodes are implemented by a server. As shown in FIG. 3, the mirror file management system includes: a terminal 10, a first server 11, a second server 12, and a third server 13.
  • a client may be installed in the terminal 10, and the user may interact with the edge cloud node through the client, or the user may also interact with the edge node through a web page in the terminal 10.
  • the terminal may be a computer, and the computer may be a machine with computing functions such as a general-purpose computer, a special-purpose computer, a personal computer, or a portable mobile terminal.
  • Each server can be a server, or a server cluster composed of several servers, or a cloud computing service center.
  • Each server and terminal can be connected via a wired network or a wireless network. Every two servers can be connected via a wired network or a wireless network. For example, every two servers can be connected through a wide area network.
  • the terminal 10 includes a terminal processor 101, a terminal communication interface 102, and a terminal memory 103.
  • the terminal processor 101, the terminal communication interface 102, and the terminal memory 103 are connected to each other through a terminal bus 104.
  • the first server 11 includes a first processor 111, a first communication interface 112 and a first memory 113.
  • the first processor 111, the first communication interface 112, and the first memory 113 are connected to each other through a first bus 114.
  • the second server 12 includes a second processor 121, a second communication interface 122 and a second memory 123.
  • the second processor 121, the second communication interface 122 and the second memory 123 are connected to each other through a second bus 124.
  • the third server 13 includes a third processor 131, a third communication interface 132 and a third memory 133.
  • the third processor 131, the third communication interface 132, and the third memory 133 are connected to each other through a third bus 134.
  • the terminal memory 103, the first memory 113, the second memory 123, and the third memory 133 are all used to store computer programs.
  • the computer program may be an application program, and when the processor calls the application program in the corresponding memory, the function of the application program can be realized.
  • the first processor 111 calls the application program in the first memory 113, it can realize the function of the edge cloud node in the image file management method provided in the embodiment of the present application.
  • any one of the terminal bus 104, the first bus 114, the second bus 124, and the third bus 134 can be divided into an address bus, a data bus, a control bus, and the like.
  • address bus a data bus
  • control bus a control bus
  • any one of the terminal processor 101, the first processor 111, the second processor 121, and the third processor 131 may be a hardware chip, and the hardware chip may be an application-specific integrated circuit (application-specific integrated circuit). integrated circuit, ASIC), programmable logic device (programmable logic device, PLD) or a combination thereof.
  • the above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.
  • CPLD complex programmable logic device
  • FPGA field-programmable gate array
  • GAL generic array logic
  • it may also be a general-purpose processor, for example, a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP.
  • any one of the terminal memory 103, the first memory 113, the second memory 123, and the third memory 133 may include a volatile memory (volatile memory), such as a random-access memory (random-access memory, RAM); it can also include non-volatile memory, such as flash memory, hard disk drive (HDD), or solid-state drive (SSD); it can also include A combination of the above types of memories.
  • volatile memory such as a random-access memory (random-access memory, RAM
  • non-volatile memory such as flash memory, hard disk drive (HDD), or solid-state drive (SSD); it can also include A combination of the above types of memories.
  • the embodiment of the present application provides a mirror file management method, and the mirror file management method is used to manage mirror files using a mirror file management system.
  • the mirror file management method mainly involves the initial storage, update and reading process of the mirror file.
  • Step 401 The terminal receives a write instruction for instructing to write the first image file.
  • a write instruction can be triggered by executing a specified operation on the terminal to instruct the first mirror file to be written into the mirror file management system through the write instruction.
  • the mirror file is made by multiple files using burning software or a mirror file making tool, and the mirror file can be regarded as a copy of the multiple files.
  • the multiple files need to be copied, it can be realized by copying the mirror files of the multiple files, which can save the time of the copy, migration, and import process.
  • an image file can be understood as a copy of data on a disk.
  • users need to copy multiple files on a disk to another disk, they can directly copy the mirror files of the multiple files.
  • Step 402 The terminal divides the first image file into multiple first data blocks.
  • the terminal can divide the first image file into multiple data blocks according to a specified size (for the convenience of description and easy distinction, the data blocks in the first image file are collectively referred to as the first data block in this article. ). For example, assuming that the size of the first image file is 1 GB, the terminal may divide the first image file into 1024 first data blocks according to the size of each first data block of 1 megabyte (MB).
  • MB megabyte
  • Step 403 The terminal generates an identification number corresponding to the first data block according to the content of each first data block, and generates a first identification number topology of the first mirror file based on the identification numbers of the multiple first data blocks.
  • the terminal can generate the identification number of the first data block according to the content of the first data block and the preset identification number generation algorithm.
  • the identification number of the first data block is used for mirroring
  • the file management system uniquely identifies the first data block.
  • the identification number of the first data block may be a hash value
  • the preset identification number generation algorithm may be: MD5 message digest algorithm (MD5 message digest algorithm) or secure hash algorithm (secure hash algorithm, SHA) and other encryption algorithm.
  • the identification numbers of the multiple first data blocks can be converted according to a preset conversion algorithm to obtain the identification number of the first image file and the first image file of the first image file.
  • An identification number topology The identification number of the mirror file topology records the relationship between the identification number of the data block in the mirror file and the identification number of the mirror file, that is, the identification number of the mirror file topology records which data blocks the mirror file includes.
  • the first identification number topology may be a hash tree with the hash values of the plurality of first data blocks as leaf nodes
  • the identification number of the first image file is the hash value located on the root node of the hash tree.
  • the hash tree shown in Figure 5 includes three layers (one layer in each dashed box A0). According to the top-to-bottom direction, the node at the bottom of the hash tree is the leaf node of the hash tree.
  • the top node of the hash tree is the root node of the hash tree.
  • the node located in the upper layer is the parent node of the node located in the lower layer.
  • the generation process of the hash tree is as follows: divide a plurality of first data blocks into N1 groups, and each group of first data blocks includes a first specified number of first data blocks. Then, in the unit of group, the encryption algorithm is executed on the hash values of the multiple first data blocks in each group to generate the hash values of the hash values of the multiple first data blocks in each group.
  • the hash value generated by the hash value of the first data block is the parent node of the hash value of each first data block in any group, and the hash value of any first data block in any group can be It is called the child node of the parent node.
  • a directed arrow pointing to the parent node from the hash value of each first data block in any group is added.
  • the N1 parent nodes are divided into N2 groups, and each group includes a second specified number of parent nodes.
  • the encryption algorithm used to generate the parent node in the hash tree may be the same or different from the encryption algorithm used when generating the identification number of the first data block, which is not specifically limited in the embodiment of the present application.
  • each parent node has 2 child nodes.
  • the root node constitutes a layer of the hash tree
  • the parent node with the hash value of hash 0 and the parent node with the hash value of hash 1 constitute the part of the hash tree
  • leaf nodes with a hash value of 0-0 leaf nodes with a hash value of 0-1, leaf nodes with a hash value of 1-0, and a hash value of hash
  • the leaf nodes of 1-1 form another layer of the hash tree
  • hash 0 is the hash value obtained by performing the encryption algorithm on hash 0-0 and hash 0-1
  • hash 1 is the hash value of hash 1- 0 and hash 1-1 are the hash values obtained by executing the encryption algorithm
  • the hash value of the root node is the hash value obtained by a hash value obtained by
  • Step 404 Based on the write instruction, the terminal determines at least one first edge cloud node among the multiple edge cloud nodes included in the image file management system, and writes the multiple first data blocks to the at least one first edge cloud node. This allows each first edge cloud node to store part or all of the multiple first data blocks.
  • the terminal may determine a first edge cloud node among the multiple edge cloud nodes included in the image file management system, and write all the multiple first data blocks to the first edge cloud node . After the terminal writes all the multiple first data blocks into the first edge cloud node, the first edge cloud node may store the multiple first data blocks to complete the writing of the multiple first data blocks process.
  • the first edge cloud node may be an edge cloud node logically closest to the terminal among multiple edge cloud nodes included in the image file management system. Since the edge cloud node that is logically closest to the terminal has a better communication link with the terminal, by storing multiple first data blocks on the first edge cloud node, it is possible to reduce the number of terminals writing into the first image. File write delay.
  • first data blocks when multiple first data blocks are all stored on an edge cloud node, if the edge cloud node needs to use the first image file, it can reduce the edge cloud node from obtaining the first image from other edge cloud nodes.
  • the number of first data blocks included in the file reduces the time overhead for the edge cloud node to obtain the first image file.
  • the terminal may determine multiple first edge cloud nodes among multiple edge cloud nodes included in the image file management system, and respectively assign part or all of the multiple first data blocks to the first edge cloud nodes.
  • One data block is correspondingly written into the plurality of first edge cloud nodes.
  • each first edge cloud node can write to the first edge cloud node of the first edge cloud node.
  • One data block is stored to complete the writing process of the plurality of first data blocks.
  • the multiple first edge cloud nodes may be determined by the terminal according to a specified policy. For example, the terminal may determine a plurality of first edge cloud nodes according to a specified mapping method according to the plurality of first data blocks, and the plurality of first edge cloud nodes respectively correspond to the plurality of first data blocks. At this time, when multiple first data blocks are written to multiple first edge cloud nodes, multiple first data blocks may be written to corresponding first edge cloud nodes respectively, so that the multiple first edge cloud nodes respectively Part or all of the first data blocks among the plurality of first data blocks are stored.
  • each first edge cloud node of the plurality of first edge cloud nodes stores at least part of the first data block included in the first image file, which can reduce the number of first edge cloud nodes from other edge cloud nodes.
  • the probability of obtaining the first data block included in the first mirror file can reduce the probability of the first edge cloud node having a delay problem.
  • each first data block is mapped to a segment of storage address, and the edge cloud node responsible for the segment of storage address is determined as the first edge cloud node for storing the corresponding first data block.
  • the first data blocks stored in different first edge cloud nodes may include the same first data block.
  • the first data blocks stored in different first edge cloud nodes may be completely different first data blocks. In this way, there is no need to store duplicate data blocks in the mirror file management system, and the storage utilization rate of the edge cloud nodes can be guaranteed.
  • the multiple first data blocks written by the terminal to the first edge cloud node may be multiple data blocks that are not stored in any edge cloud node among all the first data blocks obtained by dividing the first mirror file.
  • the first data block it is possible to determine multiple first data blocks that are not stored by any edge cloud node among all the first data blocks obtained by the division. , And then write multiple first data blocks that are not stored in any edge cloud node into at least one first edge cloud node to avoid writing duplicate data in the mirror file management system, thereby reducing bandwidth usage and reducing Write latency and improve the storage utilization of the mirror file management system.
  • the identification number of the data block can also be written into the mirror file management system. Therefore, when determining When any first data block has been stored in the mirror file management system, you can query whether the identification number of any first data block has been stored in the mirror file management system. When the mirror file management system does not store the When the identification number of any first data block is used, it is determined that the any first data block is not stored in the mirror file management system.
  • multiple first data blocks that are not stored in any edge cloud node can be determined first, and then one or more first data blocks that are not stored in any edge cloud node are determined. Multiple first edge cloud nodes, and then write multiple first data blocks that are not stored in any edge cloud node into the corresponding first edge cloud node, respectively.
  • one or more first edge cloud nodes can be determined based on all the first data blocks obtained by dividing, and then multiple first data blocks that are not stored in any edge cloud node can be determined, and then the first data blocks that are not stored in any edge cloud can be determined.
  • the multiple first data blocks in the node are written into one or more first edge cloud nodes, which is not specifically limited in the embodiment of the present application.
  • Step 405 The terminal establishes a storage correspondence between each first data block and the first edge cloud node storing the corresponding first data block according to the storage situation of each first data block in at least one first edge cloud node.
  • the storage correspondence is stored in the image file management system.
  • the first edge cloud node may send a response indicating that the storage is complete to the terminal, and the terminal may determine that the corresponding data block is stored in the first edge cloud node according to the response.
  • the first data block and the storage correspondence relationship is established, and then the storage correspondence relationship is written into the mirror file management system to store the storage correspondence relationship in the mirror file management system.
  • the storage correspondence is used to reflect the storage conditions of the multiple first data blocks in one or more first edge cloud nodes, that is, it is used to indicate which first edge cloud nodes any first data block is stored in.
  • the storage correspondence between any first data block and the first edge cloud node storing the any first data block may be: the hash value of the any first data block and the storage There is a correspondence between the physical address of the first edge cloud node of any first data block.
  • storing the storage correspondence in the image file management system can be implemented by storing the storage correspondence in an edge cloud node in the image file management system.
  • the implementation of step 405 may include: the terminal determines an edge cloud node among multiple edge cloud nodes, and writes the storage correspondence relationship to the edge cloud node, so that other edge cloud nodes in the file management system are mirrored Both can obtain the storage correspondence relationship by accessing the edge cloud node.
  • multiple edge cloud nodes in the image file management system can obtain multiple first data blocks on one or more first edge cloud nodes according to the storage correspondence.
  • Step 406 The terminal adds the description information of the first image file to the first identification number topology, and writes the first identification number topology to which the description information of the first image file is added into the image file management system.
  • the description information (also called commit information) of the first image file is used to describe the first image file, so that the user can understand the content of the first image file.
  • the description information of the first image file includes metadata information such as the name, version information, and permission control information of the first image file.
  • the version information records information such as the size and upload time of the first image file, and may also record information of the first data block included in the first image file.
  • the authority control information records the information of the user who has the authority to access or modify the first image file.
  • the first identification number topology added with the description information of the first image file can be written into the image file management system so that multiple edge cloud nodes can follow The first identification number topology with the description information added, the first data blocks included in the first mirror file are acquired, and the content of the first mirror file is learned according to the description information.
  • writing the first identification number topology to which the description information of the first image file is added to the image file management system can be implemented by writing it to an edge cloud node in the image file management system.
  • the edge cloud node storing the first identification number topology to which the description information of the first image file is added may be the same or different from the edge cloud node storing the storage correspondence relationship of the first image file. It is not specifically limited.
  • the implementation manner of adding the description information of the first image file to the first identification number topology may include: hashing the first image file
  • the hash value of the root node of the tree and the prompt information of the first mirror file are filled in the information unit, and the information unit filled with the information is connected in series to the root node of the hash tree of the first mirror file, and the information unit is used as the information unit.
  • the parent node of the root node of the hash tree obtains the first directed acyclic graph (DAG) of the first mirror file.
  • DAG directed acyclic graph
  • the information unit filled with information is an information set with the hash value of the root node of the hash tree of the first mirrored file and the prompt information of the first mirrored file as elements.
  • FIG. 6 is a schematic structural diagram of the first directed acyclic graph obtained according to the hash tree shown in FIG. 5.
  • the execution subject is the client as an example. However, its execution subject may not be limited to the terminal.
  • the execution subject of the foregoing steps 401 to 406 may also be an edge cloud node.
  • the write instruction received by the edge cloud node may be sent by the terminal, and the write instruction may be triggered after the terminal performs the upload operation of the first image file.
  • the first edge cloud node may be the edge cloud node itself used as the execution subject of step 404.
  • the manner of determining the multiple first edge cloud nodes may refer to the implementation manner of determining multiple first edge cloud nodes in step 404 accordingly.
  • the execution subject of step 402, step 403, step 405, and step 406 may be an edge cloud node
  • the implementation manner of the corresponding step can refer to the implementation manner of the corresponding step accordingly.
  • step 406 can be executed after step 403 is executed, and step 404 and step 405 are executed again.
  • step 406 can be executed after step 403 is executed, and step 404 and step 405 are executed again.
  • the image file management method stores the data blocks included in the first image file through the edge cloud node, so that when any edge cloud node in the image file management system needs to use the image file, it can Obtain the corresponding data block from the edge cloud node that stores the data block in the mirror file to obtain the required mirror file.
  • the probability of the available bandwidth of the same cloud node can maximize the use of the available bandwidth of any edge cloud node, and between any edge cloud node and the edge cloud node storing the data blocks required by any edge cloud node
  • the bandwidth reduces the time cost of obtaining the image file and improves the download speed of the image file.
  • the edge cloud node is stored in units of data blocks.
  • the writing granularity of mirror files is refined. This eliminates the need to write the stored data blocks in the mirror file management system to the mirror file management system again, avoids repeated writing of data blocks, reduces the bandwidth occupation of writing data, reduces write delays, and improves mirror file management The storage utilization of the system.
  • the user can quickly determine the first data block included in the first image file according to the first identification number topology during the process of reading the first image file, and further Yes, by adding the description information of the first mirror file to the topology of the first identification number, it is convenient to quickly determine the content of the first mirror file without reading the data blocks to determine the data blocks and data included in the first mirror file.
  • the content of the first image file can reduce the read delay.
  • the update process includes the following steps:
  • Step 701 The target edge cloud node receives an update instruction for instructing to update the information of the third image file, the third image file being generated by the target edge cloud node based on the fourth image file stored in the image file management system.
  • the third image file may be an image file obtained after the target edge cloud node performs a preset operation on the fourth image file.
  • the target edge cloud node can use the fourth image file to create an instance of a virtual machine or container in any edge cloud node of the image file management system, and after the instance of the virtual machine or container is created, It is also possible to re-create the image file according to the created instance of the virtual machine or container, and the re-made image file is the image file generated based on the fourth image file.
  • the fourth image file is the image file of the virtual machine
  • the virtual machine can be created in the target edge cloud node according to the fourth image file, and after the virtual machine is created, the virtual machine can be created in the newly created virtual machine.
  • the newly created virtual machine will have data that is different from the fourth image file (that is, new data).
  • the created image file Will include new data.
  • the newly produced image file since the newly created image file is produced in the target edge node, the newly produced image file has been stored in the image file management system.
  • the newly produced image file can be updated in the image file management system.
  • the information of the mirror file so that other edge cloud nodes can obtain the newly produced mirror file based on the information, and ensure the data integrity of the mirror file stored in the mirror file management system.
  • the user can select an image file from the fourth image file and the third image file according to application needs, and create a virtual machine according to the selected image file.
  • the fourth mirror file and the first mirror file may be the same mirror file or different mirror files, which are not specifically limited in the embodiment of the present application.
  • Step 702 The target edge cloud node divides the third image file into multiple third data blocks.
  • the implementation process of this step 702 can refer to the implementation process of step 402 accordingly.
  • the way of dividing the third mirror file may be the same as the way of dividing the fourth mirror file.
  • the structure of the third mirror file has a greater similarity with the structure of the fourth mirror file. Therefore, when the fourth mirror file is divided
  • the data blocks in the third mirror file are collectively referred to as the third data block in this article, and the data blocks in the fourth mirror file are collectively referred to as the fourth data block.
  • Step 703 The target edge cloud node generates an identification number corresponding to the third data block according to the content of each third data block, and generates a third identification number topology of the third mirror file based on the identification numbers of the multiple third data blocks .
  • the implementation process of generating the identification number of the third data block may refer to the implementation process of generating the identification number of the first data block accordingly.
  • the implementation manner of generating the third identification number topology can also refer to the implementation manner of generating the first identification number topology accordingly.
  • the fourth The identification number topology is incrementally updated to obtain the third identification number topology, so as to reduce the amount of calculation for generating the third identification number topology.
  • its implementation may include:
  • Step 7031 in the fourth identification number topology, determine the first tree subtopology, and the identification numbers of the same data blocks included in the third mirror file and the fourth mirror file are leaf nodes in the first tree subtopology.
  • the identification number of the data block is used to uniquely identify the data block, when the identification numbers of two data blocks are the same, it means that the contents of the two data blocks are the same. Therefore, the identification number of the third mirror file can be compared with the fourth
  • the data block with the same identification number in the mirror file is determined to be the same data block in the fourth mirror file and the third mirror file (for ease of description, hereinafter referred to as the same data block), and the identification number is compared with all the first data blocks in the fourth mirror file.
  • the third data block with different identification numbers of the four data blocks is determined to be the third data block that is different from the data blocks included in the fourth mirror file.
  • the topological part is determined to be the first tree-shaped sub-topology, and the identification number at the top of the first tree-shaped sub-topology can be regarded as the root node of the first tree-shaped sub-topology.
  • the hash value of the fourth data block L11 is 01a3b001a10c01 (indicated by hash 0-0 in FIG. 9), and the hash of the fourth data block L12
  • the value is 01a3b001a10c02 (represented by hash 0-1 in Figure 9)
  • the hash value of the fourth data block L13 is 01a3b001a10c03 (represented by hash 1-0 in Figure 9)
  • the hash of the fourth data block L14 The value is 01a3b001a10c04 (represented by hash 1-1 in FIG. 9).
  • the third image file includes third data blocks L21 to L24, the hash value of the third data block L21 is 01a3b001a10c01 (indicated by hash 0-0 in Figure 9), and the hash value of the third data block L22 is 01a3b001a10c02( It is represented by hash 0-1 in Figure 9), the hash value of the third data block L23 is 01a3b001a10c0d (represented by hash 2-0 in Figure 9), and the hash value of the third data block L24 is 01a3b001a10caf( In Figure 9 it is represented by hash 2-1), it can be determined that the fourth data block L11 and the third data block L21 are the same data block, the fourth data block L12 and the third data block L22 are the same data block, and the third data block L12 is the same data block as the third data block L22.
  • the data block L23 and the third data block L24 are different from all the data blocks in the fourth mirror file.
  • the topological part of the leaf node with the hash value of the fourth data block L11 and the hash value of the fourth data block L12 may be determined as the first tree subtopology. As shown in the first sub-hash tree in the dashed box A1 in FIG. 9, the first sub-hash tree uses the hash value of the fourth data block L11 to hash 0-0 and the hash value of the fourth data block L12.
  • Hash 0-1 is a leaf node, that is, the first sub-hash tree includes: the hash value hash 0-0 of the fourth data block L11, the hash value hash 0-1 of the fourth data block L12, and Use the hash value of the fourth data block L11 and the hash value of the fourth data block L12 as the hash value of the child node's parent node to hash 0, and since the parent node is at the top of the first child hash tree, The parent node can be regarded as the root node of the first child hash tree.
  • Step 7032 based on a third data block that is different from the data block included in the fourth mirror file, generate a second tree-shaped sub-topology.
  • the process of generating the second tree sub-topology is essentially based on the identification of the third data block that is different from the data blocks included in the fourth mirror file No., the process of generating the identification number topology of the image file composed of the third data block that is different from the data blocks included in the fourth image file. Therefore, for the implementation process of this step 7032, please refer to the implementation process of step 403 accordingly.
  • the second sub-hash tree generated according to the hash value of the third data block L23 and the hash value of the third data block L24 is shown in the dashed box A2 in Fig. 9 , And the hash value at the top of the second sub-hash tree (ie, hash 2) can be regarded as the root node of the second sub-hash tree.
  • Step 7033 Generate a third identification number topology based on the first tree-shaped sub-topology and the second tree-shaped sub-topology.
  • the encryption algorithm can be executed again on the root node of the first tree sub-topology and the root node of the second tree sub-topology, according to the root node of the first tree sub-topology and The identification number of the root node of the second tree sub-topology, which is the root node of the third identification number topology, and then the root node of the first tree sub-topology and the root node of the second tree sub-topology point to the The directed arrow of the root node of the third identification number topology obtains the third identification number topology.
  • Step 704 The target edge cloud node establishes a storage correspondence between each third data block and the target edge cloud node, and stores the storage correspondence in the image file management system.
  • the third image file Since the third image file is produced in the target edge cloud node, all third data blocks of the third image file have been stored in the target edge cloud node, that is, the image file management system has already stored At this time, the third mirror file needs to be stored in the mirror file management system for the storage correspondence of the third data block in the third mirror file, so that other edge cloud nodes can obtain the storage information of the third data block.
  • the implementation manner of storing the storage correspondence in the mirror file management system may include the following two situations:
  • a third data block is stored in the target edge cloud node, and the third data block is not stored in other edge cloud nodes of the image file management system, it can be added to the image file management system
  • the storage correspondence between the third data block and the target edge cloud node please refer to the corresponding storage correspondence between the multiple first data blocks and the first edge cloud node in step 405.
  • the realization process of storage in the mirror file management system For example, a record indicating the correspondence between the hash value of the third data block and the physical address of the target edge cloud node may be added to the mirror file management system.
  • Step 705 The target edge cloud node adds the description information of the third image file to the third identification number topology, and writes the third identification number topology to which the description information of the third image file is added into the image file management system.
  • step 705 After adding the description information of the third mirror file to the third identification number topology, the obtained second directed acyclic graph of the third mirror file is shown in FIG. 9.
  • the description information of the third mirror file may also include update description information, and the update description information is used to indicate that the third mirror file is relative to the first mirror file. Difference information of four image files.
  • version management of the mirror file in the process of writing data blocks into the mirror file management system, when determining whether the data block is stored in the mirror file management system, it can only be linked to a certain mirror file and based on the data block.
  • the data blocks of a version of the mirror file in the mirror file generated by the mirror file are compared to reduce the number of comparisons and the write delay.
  • step 704 and step 705 can be adjusted according to application requirements. Any person familiar with the technical field can easily think of a method of change within the technical scope disclosed in this application, which should be covered by the protection scope of this application, and therefore will not be repeated.
  • the mirror file management method updates the information of the mirror file stored in the mirror file management system, so that other edge cloud nodes in the mirror file management system can obtain the stored image according to the information.
  • Mirror file to ensure the data integrity of the mirror file stored in the mirror file management system.
  • the third identification number topology of the third image file is obtained by incrementally updating the fourth identification number topology of the fourth image file , Can reduce the calculation amount of generating the third identification number topology, and reduce the writing delay of updating the third image file information.
  • the update description information to the description information, on the one hand, it can realize the version management of the mirror file, so that the user can understand the difference between the third mirror file and the fourth mirror file according to the update description information, so that the user can quickly
  • the required mirror file is selected, on the other hand, the difference between the two can be determined without reading and comparing the contents of the data block, which can reduce the read delay of the mirror file system.
  • version management on the mirror file in the process of writing data blocks to the mirror file management system, when determining whether the data block is stored in the mirror file management system, it can only be linked to a certain mirror file and based on the data block. The data blocks of a version of the mirror file in the mirror file generated by the mirror file are compared to reduce the number of comparisons and the write delay.
  • the reading process of the mirror file includes the following steps:
  • Step 801 The target edge cloud node receives a read instruction for instructing to read the second image file.
  • the terminal may send a reading instruction carrying the identification number of the second mirror file to the target edge cloud node to instruct the target edge node to read the mirror file in the mirror file management system, and perform a specified operation according to the mirror file.
  • the terminal may instruct the target edge node to read the image file from the image file management system, and use the image file to create instances such as virtual machines and containers.
  • the terminal can first query whether the second image file required for creating the instance is stored in the image file management system, and when it is determined that the second image file is stored in the image file management system, the terminal can send a message to the target edge cloud node Send a read instruction carrying the identification number of the second image file to request the target edge cloud node to read the second image file from the image file management system, and use the second image file to create it in the target edge cloud node Instance.
  • the mirror file management system is equipped with an application program interface, and the terminal can query the mirror file management system for the information of the mirror file stored in the mirror file management system through the application program interface, and determine whether the mirror file management system is The second image file is stored.
  • Step 802 Based on the read instruction, the target edge cloud node determines one or more second edge cloud nodes that store the second data block of the second image file among the multiple edge cloud nodes included in the image file management system.
  • part of the second data block of the second mirror file may be stored in the target edge cloud node (for simplicity of description and ease of distinction, the data blocks in the second mirror file are collectively referred to as the first Two data blocks), therefore, in the process of reading the second image file, the second data block belonging to the second image file but not stored in the target edge cloud node can be read from the second edge cloud node to Obtain the second image file.
  • the implementation process of this step 802 may include: determining all the second data blocks included in the second mirror file according to the read instruction, and then determining that in all the second data blocks, it is determined that the target edge cloud node is not stored One or more second data blocks in the data block, and the edge cloud node storing any second data block that is not stored in the target edge cloud node is determined as the second edge cloud node.
  • the target edge cloud node can read the identification number of the second mirror file carried in the instruction , Determine the second identification number topology of the second image file, and determine all the second data blocks included in the second image file according to the second identification number topology.
  • the storage correspondence between the second data block and the edge cloud node used to store the second data block is stored in the mirror file management system, it can be determined that each second data block is stored according to the storage correspondence
  • the edge cloud node storing a certain second data block does not include the target edge cloud node, determine that the certain second data block belongs to the second image file but is not stored in the target edge cloud node The second data block.
  • the edge cloud node that stores the second data block belonging to the second image file but not stored in the target edge cloud node is the second edge cloud node.
  • the method can reduce the amount of data processing in the process of determining the second data block belonging to the second mirror file but not stored in the target edge cloud node, so as to further reduce the read delay of the second mirror file.
  • Step 803 When the same second data block set is stored in the multiple second edge cloud nodes, the target edge cloud node is based on the download state parameter of each second edge cloud node that all stores the second data block set. Select to download the second edge cloud node of the second data block in the second data block set, and download the second data block in the second data block set from the selected second edge cloud node.
  • the second data block set includes one or more second data blocks.
  • the second edge cloud node A1 stores the second data block B1
  • the second edge cloud node A2 also stores the second data block B1
  • the cloud node A2 all stores the same second data block set, and the second data block set includes the second data block B1.
  • the second edge cloud node A1 stores the second data block B1 and the second data block B2
  • the second edge cloud node A2 also stores the second data block B1 and the second data block B2
  • the second edge cloud node A1 and the second edge cloud node A2 both store the same second data block set, and the second data block set includes a second data block B1 and a second data block B2.
  • the target edge cloud node can obtain the download status parameter of each second edge cloud node that stores the second data block set, and based on the The download state parameters of each second edge cloud node in the second data block set are all stored, the multiple second edge cloud nodes that store the second data block set are filtered, and the second data block set is stored in the Among the plurality of second edge cloud nodes in the block set, the second edge cloud node with the best download performance indicated by the download status parameter is selected as the second edge cloud node used as the download source, and then download from the target with the best download performance.
  • the second edge cloud node downloads the second data block in the second data block set to ensure the download speed.
  • the download state parameter is used to reflect the download speed of the target edge cloud node downloading the second data block from the corresponding second edge cloud node.
  • the download state parameters of the second edge cloud node include: parameters such as network traffic and network delay between the second edge cloud node and the target edge cloud node, and load state parameters of the second edge cloud node.
  • the target edge cloud node may communicate with the second edge cloud node, and use a network detection technology to obtain the download state parameter of the second edge cloud node.
  • the target edge cloud node may send a detection data packet to the second edge cloud node, and obtain the difference between the target edge cloud node and the second edge cloud node according to the response of the second edge cloud node to receiving the detection data packet. Parameters such as network traffic and network delay.
  • the second data block in the data block can ensure the download speed of the second data block, reduce bandwidth consumption, and reduce download delay.
  • Step 804 In the process of downloading the second data block in the same second data block set from the selected second edge cloud node, the target edge cloud node continues to obtain the download status parameter of the selected second edge cloud node.
  • the target edge cloud node re-selects to download the second data block set among multiple second edge cloud nodes that all store the second data block set The second edge cloud node of the second data block, and downloads the second data block that was not successfully downloaded and belongs to the second data block set from the reselected second edge cloud node.
  • the download state parameter of the reselected second edge cloud node is greater than the reference parameter threshold.
  • the second data block that is not successfully downloaded may include: a second data block that is not completely downloaded, a second data block that is not downloaded at all, and a second data block that has an incorrect download content.
  • the implementation manner may include: generating a hash value according to the content of the downloaded second data block, And compare the generated hash value with the hash value of the second data block stored in the mirror file management system, when the generated hash value is compared with the hash value of the second data block stored in the mirror file management system When the value is the same, it is determined that the downloaded content of the downloaded second data block is correct.
  • the downloaded first data block is determined The second data block is the data block with the wrong download content.
  • the download status parameter of the second edge cloud node is less than the reference parameter threshold, it means that the download link between the target edge cloud node and the second edge cloud node is interrupted or the bandwidth is reduced, that is, the target edge cloud node starts from the The download speed of the download data block in the second edge cloud node cannot meet the download requirement.
  • the second edge cloud node used as the download source can be reselected to ensure the download speed.
  • the implementation process of obtaining the download status parameter of the second edge cloud node in step 804 please refer to the implementation process of obtaining the download status parameter of the second edge cloud node in step 803.
  • the second edge cloud node is stored in both
  • the implementation process of reselecting the second edge cloud node among the multiple second edge cloud nodes in the data block set reference may be made to the implementation process of selecting the second edge cloud node in step 803, which will not be repeated here.
  • the process of continuing to download the second data block in the same second data block set from the reselected second edge cloud node it may be repeatedly executed to obtain the download state parameters of the reselected second edge cloud node, and When the download status parameter is less than the reference parameter threshold, continue to reselect the second edge cloud node used as the download source, and continue to download the second data block from the reselected second edge cloud node, until the same second data block set is completed Download of the second data block in.
  • Step 805 After the target edge cloud node finishes downloading the second data block, store the second data block read from the second edge cloud node, and store the corresponding relationship between each second data block and the target edge cloud node. Stored in the mirror file management system.
  • the target edge cloud node After the target edge cloud node completes the download of the second data block, the target edge cloud node can store the downloaded second data block, and add the target edge cloud node's data to the storage correspondence of each second data block. Information to indicate that the second data block is also stored in the target edge cloud node, so that other edge cloud nodes in the image file management system can obtain storage information of the second data block. And the implementation process of step 805 may include: in the storage correspondence of each second data block, adding the correspondence between each second data block and the target edge cloud node.
  • the target edge cloud node may also use the second image file Create an instance in the target edge cloud node.
  • the sequence of steps in the mirror file management method can be appropriately adjusted, and the steps can also be increased or decreased according to the situation.
  • each edge cloud node is configured with a function of storing image files, and the multiple edge cloud nodes are all configured to store image files. It is configured to be able to read the file block data stored by the edge cloud node in the image file management system, so that when any edge cloud node in the image file management system needs to use the image file, it can read from the data block stored in the image file. The corresponding data block is obtained from the edge cloud node to obtain the image file. There is no need to download the image file from the central cloud node.
  • the edge cloud node can read the data block as a unit. Compared with the related technology that reads the mirror file in the unit of the mirror file, it refines the reading of the mirror file.
  • the granularity can be used to read the data blocks of the required mirror files from different edge cloud nodes.
  • the available bandwidth can be allocated according to the amount of data that needs to be read, which can maximize the use of the available bandwidth and reduce the read delay.
  • the second data block belonging to the second mirror file but not stored in the target edge cloud node the amount of data that needs to be read can be reduced, and the read delay can be further reduced.
  • the download status parameter of the second edge cloud node storing the second data block is obtained, and the second edge cloud node as the download source is determined according to the download status parameter, so as to avoid downloading from the fixed cloud
  • the download speed is limited by the network conditions between the edge cloud node and the fixed cloud node, which can reduce the time overhead of downloading the image file, ensure the download speed of the second data block, and reduce the read image File reading delay.
  • the embodiment of the present application also provides an image file management device, which can be deployed in a terminal or in an edge cloud node.
  • the image file management apparatus 110 may include:
  • the first receiving module 1101 is configured to receive a write instruction for instructing to write the first mirror file.
  • the first determining module 1102 is configured to determine at least one first edge cloud node among the multiple edge cloud nodes that may be included in the image file management system based on the write instruction.
  • the first writing module 1103 is configured to write multiple data blocks included in the first mirror file into at least one first edge cloud node, so that each first edge cloud node stores multiple data blocks included in the first mirror file. Part or all of the data block.
  • the image file management apparatus 110 may further include:
  • the first generating module 1104 is configured to generate the identification number of each data block included in the first image file.
  • the first generation module 1104 is further configured to generate a first identification number topology of the first image file based on the identification number of each data block included in the first image file, and the first identification number topology records what the first image file can include data block.
  • the first writing module 1103 is also used to store the first identification number topology in the edge cloud node of the image file management system.
  • the first identification number topology also carries version information of the first image file.
  • the image file management apparatus 110 may further include:
  • the first establishment module 1105 is configured to establish each data block included in the first mirror file with the first image file stored in the first edge cloud node according to the storage situation of each data block included in the first mirror file.
  • the first writing module 1103 is also used to store the storage correspondence in the edge cloud node of the image file management system.
  • the first determining module 1102 is further configured to determine multiple data blocks included in the first mirror file that are not stored in any edge cloud node among all the data blocks included in the first mirror file.
  • the image file management apparatus stores the data blocks included in the first image file through the edge cloud node, so that when any edge cloud node in the image file management system needs to use the image file, it can Obtain the corresponding data block from the edge cloud node that stores the data block in the mirror file to obtain the required mirror file.
  • the probability of the available bandwidth of the same cloud node can maximize the use of the available bandwidth of any edge cloud node, and between any edge cloud node and the edge cloud node storing the data blocks required by any edge cloud node
  • the bandwidth reduces the time cost of obtaining the image file and improves the download speed of the image file.
  • the edge cloud node is stored in units of data blocks.
  • the writing granularity of mirror files is refined. This eliminates the need to write the stored data blocks in the mirror file management system to the mirror file management system again, avoids repeated writing of data blocks, reduces the bandwidth occupation of writing data, reduces write delays, and improves mirror file management The storage utilization of the system.
  • the user can quickly determine the first data block included in the first image file according to the first identification number topology during the process of reading the first image file, and further Yes, by adding the description information of the first mirror file to the topology of the first identification number, it is convenient to quickly determine the content of the first mirror file without reading the data blocks to determine the data blocks and data included in the first mirror file.
  • the content of the first image file can reduce the read delay.
  • the embodiment of the present application also provides an image file management device, which can be deployed in an edge cloud node.
  • the image file management apparatus 130 may include:
  • the second receiving module 1301 is configured to receive a read instruction for instructing to read the second mirror file.
  • the second determining module 1302 is configured to determine one or more second edge cloud nodes that store data blocks of the second image file among the multiple edge cloud nodes that the image file management system may include based on the read instruction.
  • the reading module 1303 is configured to read data blocks belonging to the second mirror file from one or more second edge cloud nodes to obtain the second mirror file.
  • the second determining module 1302 is specifically configured to: determine all data blocks included in the second mirror file based on the read instruction; determine that all data blocks included in the second mirror file are not stored in the target edge One or more data blocks in the cloud node; determine the second edge cloud node that stores any data block in the one or more data blocks.
  • the second determining module 1302 determines the implementation process of all data blocks included in the second mirror file based on the read instruction, specifically including: determining the second identification number topology of the second mirror file based on the read instruction, The second identification number topology records the data blocks that can be included in the second image file; based on the second identification number topology, all data blocks included in the second image file are determined.
  • the second determining module 1302 determines the implementation process of one or more data blocks that are not stored in the target edge cloud node among all the data blocks included in the second mirror file, which specifically includes: querying the second mirror file The storage correspondence between each data block in the second mirror file and the edge cloud node storing the corresponding data block in the second mirror file; when the storage correspondence of a certain data block in the second mirror file does not record a certain data block in the second mirror file When determining the correspondence between the data block and the target edge cloud node, it is determined that a certain data block is a data block that is not stored in the target edge cloud node.
  • the reading module 1303 is specifically configured to: when a certain data block that can be included in the second mirror file is stored in a plurality of second edge cloud nodes, obtain each first data block that stores a certain data block. 2. Download status parameters of edge nodes; based on the download status parameters of each second edge node that stores a certain data block, select the second edge cloud node to download a certain data block; from the selected second edge cloud nodes Download a block of data.
  • the implementation process of the reading module 1303 downloading a certain data block from the selected second edge cloud node specifically includes: continuing to obtain a certain data block during the process of downloading a certain data block from the selected second edge cloud node The download status parameter of the selected second edge cloud node; when the download status parameter of the selected second edge cloud node is less than the reference parameter threshold, the multiple second edge nodes that all store a certain data block re-select to download a certain For the second edge cloud node of the data block, the download state parameter of the reselected second edge cloud node is greater than the reference parameter threshold; download a certain data block from the reselected second edge cloud node.
  • the image file management apparatus 130 may further include:
  • the storage module 1304 is configured to store the data blocks of the second mirror file read from the second edge cloud node.
  • the second writing module 1305 is configured to store the storage correspondence between each data block included in the second mirror file and the target edge cloud node in the edge cloud node of the mirror file management system.
  • each edge cloud node is configured with a function of storing image files
  • the multiple edge cloud nodes are all configured to be able to read
  • Take the file block data stored by the edge cloud node in the image file management system so that when any edge cloud node in the image file management system needs to use the image file, it can be obtained from the edge cloud node that stores the data block in the image file
  • Corresponding data block to obtain the image file there is no need to download the image file from the central cloud node, which reduces the probability that multiple edge cloud nodes share the available bandwidth of the same cloud node, and can make greater use of any edge
  • the available bandwidth of the cloud node, and the bandwidth between any edge cloud node and the edge cloud node that stores the data blocks required by any edge cloud node reduces the time cost of obtaining mirror files and improves the cost of mirror files. download speed.
  • the edge cloud node can read the data block as a unit. Compared with the related technology that reads the mirror file in the unit of the mirror file, it refines the reading of the mirror file.
  • the granularity can be used to read the data blocks of the required mirror files from different edge cloud nodes.
  • the available bandwidth can be allocated according to the amount of data that needs to be read, which can maximize the use of the available bandwidth and reduce the read delay.
  • the second data block belonging to the second mirror file but not stored in the target edge cloud node the amount of data that needs to be read can be reduced, and the read delay can be further reduced.
  • the download status parameter of the second edge cloud node storing the second data block is obtained, and the second edge cloud node as the download source is determined according to the download status parameter, so as to avoid downloading from the fixed cloud
  • the download speed is limited by the network conditions between the edge cloud node and the fixed cloud node, which can reduce the time overhead of downloading the image file, ensure the download speed of the second data block, and reduce the read image File reading delay.
  • the second receiving module 1301 is further configured to: receive an update instruction for instructing to update the information of the third mirror file, where the third mirror file is the fourth mirror image stored in the target edge cloud node based on the mirror file management system File generated.
  • the image file management apparatus 130 may further include:
  • the second generating module 1306 is used to generate the identification number of each data block included in the third image file.
  • the second generation module 1306 is further configured to generate a third identification number topology of the third image file based on the identification number of each data block included in the third image file, and the third identification number topology records what the third image file can include data block.
  • the second writing module 1305 is configured to store the third identification number topology in the edge cloud node of the image file management system.
  • the second generation module 1306 is specifically configured to: determine the first tree subtopology in the fourth identification number topology, The identification numbers of the same data blocks included in the third mirror file and the fourth mirror file are the leaf nodes in the first tree subtopology; based on the data blocks included in the third mirror file and the data included in the fourth mirror file For data blocks with different blocks, a second tree-shaped sub-topology is generated. Among the data blocks included in the third mirror file and the data blocks included in the fourth mirror file are different, the identification number of the data block is the second tree-shaped sub-topology. Based on the first tree-shaped sub-topology and the second tree-shaped sub-topology, a third identification number topology is generated.
  • the third identification number topology also carries version information and update description information of the third mirror file, and the update description information is used to reflect different contents of the third mirror file relative to the fourth mirror file.
  • the image file management apparatus 130 may further include:
  • the second establishment module 1307 is configured to establish a storage correspondence between each data block included in the third image file and the target edge cloud node.
  • the second writing module 1305 is configured to store the storage correspondence between each data block included in the third image file and the target edge cloud node in the edge cloud node of the image file management system.
  • the image file management device updates the information of the image file stored in the image file management system, so that other edge cloud nodes in the image file management system can obtain the stored image file according to the information.
  • Mirror file to ensure the data integrity of the mirror file stored in the mirror file management system.
  • the third identification number topology of the third image file is obtained by incrementally updating the fourth identification number topology of the fourth image file , Can reduce the calculation amount of generating the third identification number topology, and reduce the writing delay of updating the third image file information.
  • the update description information to the description information, on the one hand, it can realize the version management of the mirror file, so that the user can understand the difference between the third mirror file and the fourth mirror file according to the update description information, so that the user can quickly
  • the required mirror file is selected, on the other hand, the difference between the two can be determined without reading and comparing the contents of the data block, which can reduce the read delay of the mirror file system.
  • version management on the mirror file in the process of writing data blocks to the mirror file management system, when determining whether the data block is stored in the mirror file management system, it can only be linked to a certain mirror file and based on the data block. The data blocks of a version of the mirror file in the mirror file generated by the mirror file are compared to reduce the number of comparisons and the write delay.
  • the embodiment of the present application also provides a computer device, the computer device includes a processor and a memory; the memory stores a computer program; when the processor executes the computer program, the computer device implements the image file management provided by the embodiment of the present application method.
  • the computer device can be a server or a terminal.
  • the computer device is a server, please refer to the structure of the first server, the second server or the third server in FIG. 3 for the structure of the computer device.
  • the computer device is a terminal
  • For the structure of the computer device please refer to the structure of the terminal in FIG. 3, which will not be repeated here.
  • the embodiment of the present application also provides a storage medium, which is a non-volatile computer-readable storage medium, and when the instructions in the storage medium are executed by the processor, the image file management method provided in the embodiment of the present application is implemented.
  • the embodiment of the present application also provides a computer program product containing instructions.
  • the computer program product runs on a computer, the computer executes the image file management method provided by the embodiment of the present application.
  • the program can be stored in a computer-readable storage medium.
  • the storage medium mentioned can be a read-only memory, a magnetic disk or an optical disk, etc.
  • the terms “first”, “second” and “third” are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance.
  • the term “at least one” refers to one or more, and the term “plurality” refers to two or more, unless expressly defined otherwise.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

本申请公开了一种镜像文件管理方法,包括:接收用于指示写入第一镜像文件的写入指令;基于写入指令,在镜像文件管理系统包括的多个边缘云节点中,确定至少一个第一边缘云节点;将第一镜像文件所包括的多个数据块,写入至少一个第一边缘云节点,使得每个第一边缘云节点存储第一镜像文件所包括的多个数据块的部分或全部。本申请减小了下载镜像文件的时间开销,提高了镜像文件的下载速度。

Description

镜像文件管理方法、装置及系统、计算机设备、存储介质 技术领域
本申请涉及云计算技术领域,特别涉及一种镜像文件管理方法、装置及系统、计算机设备、存储介质。
背景技术
在云计算领域中,可以使用镜像文件创建虚拟主机或容器等实例,并且,在创建虚拟主机或容器等实例后,还可以根据已创建的虚拟主机或容器等实例,重新制作新的镜像文件,并存储该镜像文件以备使用。但是,镜像文件的大小通常较大,例如镜像文件的大小通常能够达到吉字节(GB)数量级,因此,对镜像文件进行有效地管理,对提高镜像文件的存储效率和虚拟主机的创建速度等具有重要的意义。
相关技术中,云服务系统的功能通常通过中心云节点和多个边缘云节点实现,中心云节点和多个边缘云节点协同工作。云服务系统中的镜像文件采用镜像文件管理系统进行管理。其管理方式为:将镜像文件均存储在中心云节点中,当边缘云节点需要使用镜像文件时,由边缘云节点从中心云节点中下载其所需的镜像文件。
但是,边缘云节点从中心云节点中下载镜像文件的时间开销较大,下载速度较慢。
发明内容
本申请提供了一种镜像文件管理方法、装置及系统、计算机设备、存储介质,可以解决边缘云节点从中心云节点中下载镜像文件的时间开销较大,下载速度较慢的问题。
第一方面,本申请提供了一种镜像文件管理方法,该镜像文件管理方法包括:接收用于指示写入第一镜像文件的写入指令;基于写入指令,在镜像文件管理系统包括的多个边缘云节点中,确定至少一个第一边缘云节点;将第一镜像文件所包括的多个数据块,写入至少一个第一边缘云节点,使得每个第一边缘云节点存储第一镜像文件所包括的多个数据块的部分或全部。
通过边缘云节点存储第一镜像文件所包括的数据块,使得该镜像文件管理系统中任一边缘云节点需要使用镜像文件时,可以从存储有该镜像文件中数据块的边缘云节点中获取对应的数据块,以得到其所需的镜像文件,无需再从中心云节点中下载该镜像文件,减小了多个边缘云节点共享同一云节点的可用带宽的几率,能够较大化利用该任一边缘云节点的可用带宽,及该任一边缘云节点与存储有该任一边缘云节点所需数据块的边缘云节点之间的带宽,减小了获取镜像文件的时间开销,提高了镜像文件的下载速度。
在第一方面的一种可能的实现方式中,在接收用于指示写入第一镜像文件的写入指令之后,镜像文件管理方法还可以包括:生成第一镜像文件所包括的每个数据块的标识号;基于第一镜像文件所包括的每个数据块的标识号,生成第一镜像文件的第一 标识号拓扑,第一标识号拓扑记录第一镜像文件包括的数据块;将第一标识号拓扑存储在镜像文件管理系统的边缘云节点中。
通过生成第一镜像文件的第一标识号拓扑,能够便于用户在读取第一镜像文件的过程中,根据该第一标识号拓扑快速确定第一镜像文件包括的第一数据块。
并且,第一标识号拓扑还可以携带有第一镜像文件的版本信息、第一镜像文件的名称和权限控制信息等用于描述第一镜像文件的内容的信息。通过在第一标识号拓扑中添加第一镜像文件的描述信息,能够便于快速确定该第一镜像文件的内容,而无需通过读取数据块的方式确定第一镜像文件包括的数据块和第一镜像文件的内容,能够减小读时延。
在第一方面的一种可能的实现方式中,在将第一镜像文件所包括的多个数据块,写入至少一个第一边缘云节点之后,该镜像文件管理方法还可以包括:根据第一镜像文件所包括的每个数据块在至少一个第一边缘云节点中的存储情况,建立第一镜像文件所包括的每个数据块与存储有第一镜像文件中对应数据块的第一边缘云节点的存储对应关系;将存储对应关系存储在镜像文件管理系统的边缘云节点中。
通过将该存储对应关系存储在镜像文件管理系统中,能够使镜像文件管理系统中的多个边缘云节点根据该存储对应关系,获取多个第一数据块在一个或多个第一边缘云节点中的存储情况。
在第一方面的一种可能的实现方式中,在将第一镜像文件所包括的多个数据块,写入至少一个第一边缘云节点之前,该方法还包括:确定第一镜像文件所包括的所有数据块中未存储在任一边缘云节点中的第一镜像文件所包括的多个数据块。
这样一来,在将第一镜像文件写镜像文件管理系统的过程中,可以只将第一镜像文件所包括的所有数据块中未存储在任一边缘云节点中的多个数据块,能够避免在该镜像文件管理系统中写入重复数据,从而减少带宽占用,降低写时延,并提高镜像文件管理系统的存储利用率。
第二方面,本申请提供了一种镜像文件管理方法,该方法应用于目标边缘云节点,该镜像文件管理方法包括:接收用于指示读取第二镜像文件的读取指令;基于读取指令,在镜像文件管理系统包括的多个边缘云节点中,确定存储有第二镜像文件的数据块的一个或多个第二边缘云节点;从一个或多个第二边缘云节点中读取属于第二镜像文件的数据块,以得到第二镜像文件。
由于镜像文件管理系统包括多个边缘云节点,每个边缘云节点均配置有存储镜像文件的功能,且该多个边缘云节点均被配置为能够读取该镜像文件管理系统中边缘云节点存储的文件块数据,使得该镜像文件管理系统中任一边缘云节点需要使用镜像文件时,可以从存储有该镜像文件中数据块的边缘云节点中获取对应的数据块,以得到该镜像文件,无需再从中心云节点中下载该镜像文件,减小了多个边缘云节点共享同一云节点的可用带宽的几率,能够较大化利用该任一边缘云节点的可用带宽,及该任一边缘云节点与存储有该任一边缘云节点所需数据块的边缘云节点之间的带宽,减小了获取镜像文件的时间开销,提高了镜像文件的下载速度。
在第二方面的一种可能的实现方式中,基于读取指令,在镜像文件管理系统包括的多个边缘云节点中,确定存储有属于第二镜像文件的数据块的一个或多个第二边缘 云节点的实现过程,包括:基于读取指令,确定第二镜像文件所包括的所有数据块;在第二镜像文件所包括的所有数据块中,确定未存储在目标边缘云节点中的一个或多个数据块;确定存储有一个或多个数据块中任一数据块的第二边缘云节点。
通过确定第二镜像文件所包括的所有数据块中且未存储在目标边缘云节点中一个或多个数据块,并确定存储有该一个或多个数据块的第二边缘云节点,使得目标边缘云节点能够从该第二边缘云节点中获取该目标边缘云节点未存储且属于第二镜像文件的数据块,能够避免获取重复数据块,可以减小需要获取的数据块的总数,减小数据块的下载量,减小获取第二镜像文件的时间开销。
在第二方面的一种可能的实现方式中,基于读取指令,确定第二镜像文件所包括的所有数据块的实现过程,包括:基于读取指令,确定第二镜像文件的第二标识号拓扑,第二标识号拓扑记录第二镜像文件包括的数据块;基于第二标识号拓扑,确定第二镜像文件所包括的所有数据块。
该通过第二标识号拓扑确定第二镜像文件所包括的所有数据块的实现方式,相较于通过读取所有数据块以确定属于第二镜像文件的数据块的实现的方式,能够减少确定属于第二镜像文件的数据块过程中的数据处理量,可以减小第二镜像文件的读时延。
在第二方面的一种可能的实现方式中,在第二镜像文件所包括的所有数据块中,确定未存储在目标边缘云节点中的一个或多个数据块的实现过程,包括:查询第二镜像文件中每个数据块与存储有第二镜像文件中对应数据块的边缘云节点的存储对应关系;当第二镜像文件中某一数据块的存储对应关系中,未记载第二镜像文件中某一数据块与目标边缘云节点的对应关系时,确定某一数据块为未存储在目标边缘云节点中的数据块。
该通过存储对应关系确定未存储在目标边缘云节点中的数据块的实现方式,相较于通过将目标边缘云节点中存储的所有数据块的内容与每个第二数据块的内容进行比较,以确定属于第二镜像文件但未存储在目标边缘云节点中的第二数据块的实现的方式,能够减少确定属于第二镜像文件但未存储在目标边缘云节点中的第二数据块过程中的数据处理量,以进一步减小第二镜像文件的读时延。
在第二方面的一种可能的实现方式中,从一个或多个第二边缘云节点中读取属于第二镜像文件的数据块的实现过程,包括:当多个第二边缘云节点中均存储有第二镜像文件包括的某一数据块时,获取均存储有某一数据块的每个第二边缘节点的下载状态参数;基于均存储有某一数据块的每个第二边缘节点的下载状态参数,选择下载某一数据块的第二边缘云节点;从选择的第二边缘云节点中下载某一数据块。
通过根据第二边缘云集诶按额下载状态参数,在均存储有第二镜像文件包括的某一数据块的多个第二边缘云节点中选择用作下载源的第二边缘云节点,能够保证下载该某一数据块的下载速度。
并且,为进一步保证下载速度,从选择的第二边缘云节点中下载某一数据块的过程,可以包括:在从选择的第二边缘云节点中下载某一数据块的过程中,继续获取选择的第二边缘云节点的下载状态参数;当选择的第二边缘云节点的下载状态参数小于参考参数阈值时,在均存储有某一数据块的多个第二边缘节点重新选择下载某一数据块的第二边缘云节点,重新选择的第二边缘云节点的下载状态参数大于参考参数阈值; 从重新选择的第二边缘云节点中,重新下载某一数据块。
在第二方面的一种可能的实现方式中,在从一个或多个第二边缘云节点中读取属于第二镜像文件的数据块之后,该镜像文件管理方法还包括:存储从第二边缘云节点中读取的第二镜像文件的数据块;将第二镜像文件所包括的每个数据块与目标边缘云节点的存储对应关系,存储在镜像文件管理系统的边缘云节点中,以指示该目标边缘云节点中也存储有该第二数据块,使得镜像文件管理系统中其他边缘云节点能够获取该第二数据块的存储信息。
在第二方面的一种可能的实现方式中,该镜像文件管理方法还可以包括:接收用于指示更新第三镜像文件的信息的更新指令,第三镜像文件是目标边缘云节点基于镜像文件管理系统中已存储的第四镜像文件生成的;生成第三镜像文件所包括的每个数据块的标识号;基于第三镜像文件所包括的每个数据块的标识号,生成第三镜像文件的第三标识号拓扑,第三标识号拓扑记录第三镜像文件包括的数据块;将第三标识号拓扑存储在镜像文件管理系统的边缘云节点中。
通过更新镜像文件管理系统中已存储的镜像文件的信息,使得镜像文件管理系统中的其他边缘云节点能够根据该信息获取该已存储的镜像文件,以保证镜像文件管理系统中存储的镜像文件的数据完备性。
在第二方面的一种可能的实现方式中,当镜像文件管理系统中还存储有第四镜像文件的第四标识号拓扑时,基于第三镜像文件所包括的每个数据块的标识号,生成第三镜像文件的第三标识号拓扑,可以包括:在第四标识号拓扑中,确定第一树状子拓扑,第三镜像文件与第四镜像文件所包括的相同的数据块的标识号为第一树状子拓扑中的叶子节点;基于第三镜像文件所包括的数据块中与第四镜像文件所包括的数据块均不相同的数据块,生成第二树状子拓扑,第三镜像文件所包括的数据块中与第四镜像文件所包括的数据块均不相同的数据块的标识号为第二树状子拓扑中的叶子节点;基于第一树状子拓扑和第二树状子拓扑,生成第三标识号拓扑。
该生成第三标识号拓扑的实现方式实质为,在为在第四标识号拓扑的基础上进行增量更新,得到第三镜像文件的第三标识号拓扑的过程,这样能够减少生成第三标识号拓扑的计算量,并减小更新第三镜像文件信息的写时延。
并且,该第三标识号拓扑还携带有第三镜像文件的版本信息和更新说明信息,更新说明信息用于反映第三镜像文件相对于第四镜像文件中不同的内容。通过在描述信息中添加更新说明信息,一方面,能够实现对镜像文件的版本管理,使得用户根据该更新说明信息了解该第三镜像文件相对于第四镜像文件的区别,以便于用户快速地选择需要的镜像文件,另一方面,使得无需通过读取并比较数据块的内容才能确定两者的区别,能够减小镜像文件系统的读时延。
进一步的,通过对镜像文件进行版本管理,在向镜像文件管理系统中写入数据块的过程中,在确定该镜像文件管理系统中是否存储有该数据块时,可以仅与某镜像文件和基于该镜像文件生成的镜像文件中一个版本的镜像文件的数据块进行比较,以减小比较的次数,减小写时延。
在第二方面的一种可能的实现方式中,在接收用于指示更新第三镜像文件的信息的更新指令之后,镜像文件管理方法还包括:建立第三镜像文件所包括的每个数据块 与目标边缘云节点的存储对应关系;将第三镜像文件所包括的每个数据块与目标边缘云节点的存储对应关系,存储在镜像文件管理系统的边缘云节点中。通过更新该存储对应关系,能够使镜像文件管理系统中的边缘云节点获取该第三镜像文件所包括的数据块的存储情况。
第三方面,本申请提供了一种镜像文件管理装置,该镜像文件管理装置包括:第一接收模块,用于接收用于指示写入第一镜像文件的写入指令;第一确定模块,用于基于写入指令,在镜像文件管理系统包括的多个边缘云节点中,确定至少一个第一边缘云节点;第一写入模块,用于将第一镜像文件所包括的多个数据块,写入至少一个第一边缘云节点,使得每个第一边缘云节点存储第一镜像文件所包括的多个数据块的部分或全部。
在第三方面的一种可能的实现方式中,镜像文件管理装置还包括:第一生成模块,用于生成第一镜像文件所包括的每个数据块的标识号;第一生成模块,还用于基于第一镜像文件所包括的每个数据块的标识号,生成第一镜像文件的第一标识号拓扑,第一标识号拓扑记录第一镜像文件包括的数据块;第一写入模块,还用于将第一标识号拓扑存储在镜像文件管理系统的边缘云节点中。
在第三方面的一种可能的实现方式中,第一标识号拓扑还携带有第一镜像文件的版本信息。
在第三方面的一种可能的实现方式中,镜像文件管理装置还包括:第一建立模块,用于根据第一镜像文件所包括的每个数据块在至少一个第一边缘云节点中的存储情况,建立第一镜像文件所包括的每个数据块与存储有第一镜像文件中对应数据块的第一边缘云节点的存储对应关系;第一写入模块,还用于将存储对应关系存储在镜像文件管理系统的边缘云节点中。
在第三方面的一种可能的实现方式中,第一确定模块,还用于确定第一镜像文件所包括的所有数据块中未存储在任一边缘云节点中的第一镜像文件所包括的多个数据块。
第四方面,本申请提供了一种镜像文件管理装置,镜像文件管理装置包括:第二接收模块,用于接收用于指示读取第二镜像文件的读取指令;第二确定模块,用于基于读取指令,在镜像文件管理系统包括的多个边缘云节点中,确定存储有第二镜像文件的数据块的一个或多个第二边缘云节点;读取模块,用于从一个或多个第二边缘云节点中读取属于第二镜像文件的数据块,以得到第二镜像文件。
在第四方面的一种可能的实现方式中,第二确定模块,具体用于:基于读取指令,确定第二镜像文件所包括的所有数据块;在第二镜像文件所包括的所有数据块中,确定未存储在目标边缘云节点中的一个或多个数据块;确定存储有一个或多个数据块中任一数据块的第二边缘云节点。
在第四方面的一种可能的实现方式中,第二确定模块,具体用于:基于读取指令,确定第二镜像文件的第二标识号拓扑,第二标识号拓扑记录第二镜像文件包括的数据块;基于第二标识号拓扑,确定第二镜像文件所包括的所有数据块。
在第四方面的一种可能的实现方式中,第二确定模块,具体用于:查询第二镜像文件中每个数据块与存储有第二镜像文件中对应数据块的边缘云节点的存储对应关系; 当第二镜像文件中某一数据块的存储对应关系中,未记载第二镜像文件中某一数据块与目标边缘云节点的对应关系时,确定某一数据块为未存储在目标边缘云节点中的数据块。
在第四方面的一种可能的实现方式中,读取模块,具体用于:当多个第二边缘云节点中均存储有第二镜像文件包括的某一数据块时,获取均存储有某一数据块的每个第二边缘节点的下载状态参数;基于均存储有某一数据块的每个第二边缘节点的下载状态参数,选择下载某一数据块的第二边缘云节点;从选择的第二边缘云节点中下载某一数据块。
在第四方面的一种可能的实现方式中,读取模块,具体用于:在从选择的第二边缘云节点中下载某一数据块的过程中,继续获取选择的第二边缘云节点的下载状态参数;当选择的第二边缘云节点的下载状态参数小于参考参数阈值时,在均存储有某一数据块的多个第二边缘节点重新选择下载某一数据块的第二边缘云节点,重新选择的第二边缘云节点的下载状态参数大于参考参数阈值;从重新选择的第二边缘云节点中,重新下载某一数据块。
在第四方面的一种可能的实现方式中,镜像文件管理装置还包括:存储模块,用于存储从第二边缘云节点中读取的第二镜像文件的数据块;第二写入模块,用于将第二镜像文件所包括的每个数据块与目标边缘云节点的存储对应关系,存储在镜像文件管理系统的边缘云节点中。
在第四方面的一种可能的实现方式中,第二接收模块,还用于:接收用于指示更新第三镜像文件的信息的更新指令,第三镜像文件是目标边缘云节点基于镜像文件管理系统中已存储的第四镜像文件生成的;
相应的,该镜像文件管理装置还包括:第二生成模块,用于生成第三镜像文件所包括的每个数据块的标识号;第二生成模块,还用于基于第三镜像文件所包括的每个数据块的标识号,生成第三镜像文件的第三标识号拓扑,第三标识号拓扑记录第三镜像文件包括的数据块;第二写入模块,用于将第三标识号拓扑存储在镜像文件管理系统的边缘云节点中。
在第四方面的一种可能的实现方式中,当镜像文件管理系统中还存储有第四镜像文件的第四标识号拓扑时,第二生成模块,具体用于:在第四标识号拓扑中,确定第一树状子拓扑,第三镜像文件与第四镜像文件所包括的相同的数据块的标识号为第一树状子拓扑中的叶子节点;基于第三镜像文件所包括的数据块中与第四镜像文件所包括的数据块均不相同的数据块,生成第二树状子拓扑,第三镜像文件所包括的数据块中与第四镜像文件所包括的数据块均不相同的数据块的标识号为第二树状子拓扑中的叶子节点;基于第一树状子拓扑和第二树状子拓扑,生成第三标识号拓扑。
在第四方面的一种可能的实现方式中,第三标识号拓扑还携带有第三镜像文件的版本信息和更新说明信息,更新说明信息用于反映第三镜像文件相对于第四镜像文件中不同的内容。
在第四方面的一种可能的实现方式中,该镜像文件管理装置还包括:第二建立模块,用于建立第三镜像文件所包括的每个数据块与目标边缘云节点的存储对应关系;第二写入模块,用于将第三镜像文件所包括的每个数据块与目标边缘云节点的存储对 应关系,存储在镜像文件管理系统的边缘云节点中。
第五方面,本申请提供了一种镜像文件管理系统,该镜像文件管理包括:终端和多个边缘云节点,边缘云节点用于存储镜像文件;其中,终端用于执行第一方面提供的镜像文件管理方法,边缘云节点用于执行第二方面任一的镜像文件管理方法;或者,终端用于向边缘云节点发送用于指示写入第一镜像文件的写入指令,边缘云节点用于执行第一方面或第二方面提供的镜像文件管理方法。
第六方面,本申请提供了一种计算机设备,该计算机设备包括处理器和存储器;存储器中存储有计算机程序;处理器执行计算机程序时,计算机设备实现第一方面提供的镜像文件管理方法,或者,实现第二方面提供的镜像文件管理方法。
第七方面,本申请提供了一种存储介质,当存储介质中的指令被处理器执行时,实现第一方面提供的镜像文件管理方法,或者,实现第二方面提供的镜像文件管理方法。
附图说明
图1是本申请实施例提供的一种中心云和边缘云的部署示意图;
图2是本申请实施例提供的一种相关技术中公有云的部署方式示意图;
图3是本申请实施例提供的一种镜像文件管理系统的结构示意图;
图4是本申请实施例提供的一种镜像文件管理方法的流程图;
图5是本申请实施例提供的一种第一标识号拓扑的示意图;
图6是本申请实施例提供的一种根据图5所示的哈希树得到的第一有向无环图的结构示意图;
图7是本申请实施例提供的一种更新镜像文件管理系统中已存在的镜像文件的信息的方法流程图;
图8是本申请实施例提供的一种对第四标识号拓扑进行增量更新,得到第三标识号拓扑的方法的流程图;
图9是本申请实施例提供的一种对第一标识号拓扑进行增量更新,得到第三标识号拓扑的原理示意图;
图10是本申请实施例提供的一种读取镜像文件的方法流程图;
图11是本申请实施例提供的一种镜像文件管理装置的结构示意图;
图12是本申请实施例提供的另一种镜像文件管理装置的结构示意图;
图13是本申请实施例提供的又一种镜像文件管理装置的结构示意图;
图14是本申请实施例提供的再一种镜像文件管理装置的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请实施方式作进一步地详细描述。
边缘计算是将计算任务部署在云端和终端之间的技术,边缘计算能够直接对终端发送的数据进行过滤和分析,能够更好地支撑本地业务的实时处理与执行,具有节能省时和效率高的优势。并且,用于实现边缘计算的边缘云节点采用分布式的方式部署。 该分布式部署的特性,使得在终端请求服务时,能够采用在逻辑上距离终端最近的边缘云节点为终端提供服务,保证了为终端服务的效率。
云服务系统通常包括中心云和边缘云。中心云是基于传统云基础设施构筑的云计算平台,在网络中处于中心位置。中心云的功能通过中心云节点实现。边缘云是以边缘基础设施构筑的云计算平台。边缘云的功能通过边缘云节点实现。边缘云作为中心云的延伸,可以在中心云的控制下处理和存储部分数据,能够将中心云的部分服务或者能力(包括但不限于存储、计算、网络、人工智能、大数据和安全等)扩展到边缘云上,使得中心云和边缘云能够协同工作,实现中心与边缘协同、全网算力调度和全网统一管控等功能。
图1是本申请实施例提供的一种中心云和边缘云的部署示意图,中心云1用于提供中心云服务,边缘云1、边缘云2和边缘云3用于提供边缘云服务,边缘云1与中心云1、边缘云2与中心云1、边缘云3与中心云1之间均能够通信,且边缘云1、边缘云2和边缘云3中两两之间也可以通信。边缘云1、边缘云2和边缘云3的工作依赖于中心云1提供的服务,使得边缘云作为中心云的延伸,实现了中心与边缘协同、全网算力调度和全网统一管控等功能。
相关技术中,公有云利用中心云节点强大的数据处理能力对镜像文件进行管理,如图2所示,其对镜像文件进行管理的实现方式为:按照公有云中云服务节点的部署地域,将公有云划分为不同的区域(region),各个区域独立部署镜像管理服务和存储后端(例如硬盘),且每个中心云节点负责管理一个区域中的镜像文件。每个区域包括多个可用区(availability zone),同一区域内的多个可用区共用镜像应用程序接口(application programming interface,API),镜像后台数据库和存储后端。并且,可用区内的中心云节点与存储后端之间采用无限带宽(infiniband,IB)网络或千兆以太网(gigabit ethernet)等低时延的高速网络连接,以保证镜像文件的传输带宽。其中,图2为将公有云划分为区域1和区域2,区域1划分为可用区1和可用区2,区域2划分为可用区3的示意图。
并且,在该对镜像文件进行管理的方案中,由于边缘云节点的容量规模等因素限制,边缘云节点中未配置有存储后端,边缘云节点只能采用缓存(cache)的方式存储少量镜像文件,因此,在边缘云节点需要使用镜像文件时,若该边缘云节点的缓存中未缓存有其所需的镜像文件,该边缘云节点需要先从其依赖的中心云节点中下载镜像文件,才能使用下载到的镜像文件。
但是,由于与一个中心云节点存在依赖关系的边缘云节点通常较多,且中心云节点的可用带宽有限,当多个边缘云节点均从中心云节点下载镜像文件时,该多个边缘云节点共享中心云节点的可用带宽,导致每个边缘云节点从中心云节点中下载镜像文件的实际可用带宽较小,下载镜像文件的时间开销较大,下载速度较慢。并且,边缘云节点的分布范围通常较广,不同边缘云节点到中心云节点的物理距离可能不同,不同边缘云节点与中心云节点之间的网络条件差异较大,导致不同边缘云节点从中心云节点下载镜像文件的速度差异较大,进一步增加了部分边缘云节点下载镜像文件的时间开销。
本申请实施例提供了一种镜像文件管理系统,该镜像文件管理系统包括:终端和多个边缘云节点。终端用于向边缘云节点写入镜像文件。该多个边缘云节点中的每个边缘云节点用于存储镜像文件中数据块,且该多个边缘云节点均被配置为能够读取该镜像文件管理系统中边缘云节点存储的数据块。这样一来,该多个边缘云节点中任一边缘云节点需要使用镜像文件时,可以从存储有该镜像文件中数据块的边缘云节点中获取对应的数据块,以得到该镜像文件,无需再从中心云节点中下载该镜像文件,减小了多个边缘云节点共享同一云节点的可用带宽的几率,能够较大化利用该任一边缘云节点的可用带宽,及该任一边缘云节点与存储有该任一边缘云节点所需数据块的边缘云节点之间的带宽,减小了获取镜像文件的时间开销,提高了镜像文件的下载速度。
由上可知,本申请实施例提供的镜像文件管理系统实质为对镜像文件进行分布式管理的分布式镜像文件管理系统,在该分布式镜像文件管理系统中,每个边缘云节点负责存储一部分镜像文件的数据,并负责一定范围内的路由信息,且镜像文件中数据的存储信息通过网络在多个边缘云节点中实现共享,使得从该镜像文件管理系统中读取镜像文件时,能够根据共享的存储信息在该镜像文件管理系统中进行全局寻址,并在存储信息所指示的存储位置中读取对应的数据块,从而实现镜像文件的读取。
其中,为使边缘云节点具有存储镜像文件的功能,每个边缘云节点可以配置有应用程序接口和存储后端。应用程序接口用于对外提供访问接口,用于响应与镜像文件相关的请求。例如,边缘云节点的应用程序接口用于供其他边缘云节点或终端访问该边缘云节点。该应用程序接口可以支持多种类型的存储后端,该存储后端用于存储镜像文件中的数据块。
并且,在本申请实施例中,镜像文件在边缘云节点中的存储形式可以有多种。例如,可以采用存储区域网络(storage area network,SAN)、网络连接存储(network attached storage,NAS)、或者本地文件系统存储镜像文件,本申请实施例对其不做具体限定。其中,NAS是指将存储设备通过网络拓扑结构(例如以太网)连接到一群计算机上实现存储的存储形式。SAN是指将存储设备通过光纤通道连接到一群计算机上实现存储的存储形式。
当边缘云节点由服务器实现时,本申请实施例提供的镜像文件管理系统可以包括:终端和多个服务器。图3为镜像文件管理系统中包括三个边缘云节点,且边缘云节点由服务器实现时,该镜像文件管理系统的结构示意图。如图3所示,该镜像文件管理系统包括:终端10、第一服务器11、第二服务器12和第三服务器13。
其中,终端10中可以安装有客户端,用户可以通过该客户端与边缘云节点交互,或者,用户也可以通过终端10中的网页与边缘节点交互。终端可以为计算机,该计算机可以为通用计算机(general-purpose computer)、专用计算机(special-purpose computer)、个人电脑或便携式移动终端等具有运算功能的机器。每个服务器均可以是一台服务器,或者由若干台服务器组成的服务器集群,或者是一个云计算服务中心。每个服务器和终端之间可以通过有线网络或无线网络连接。每两个服务器之间可以通过有线网络或无线网络连接。例如,每两个服务器之间可以通过广域网连接。
并且,如图3所示,终端10包括终端处理器101,终端通信接口102和终端存储器103。终端处理器101、终端通信接口102和终端存储器103之间通过终端总线104 相互连接。第一服务器11包括第一处理器111,第一通信接口112和第一存储器113。第一处理器111、第一通信接口112和第一存储器113之间通过第一总线114相互连接。第二服务器12包括第二处理器121,第二通信接口122和第二存储器123。第二处理器121、第二通信接口122和第二存储器123之间通过第二总线124相互连接,第三服务器13包括第三处理器131,第三通信接口132和第三存储器133。第三处理器131、第三通信接口132和第三存储器133之间通过第三总线134相互连接。
终端存储器103、第一存储器113、第二存储器123和第三存储器133均用于存储计算机程序。且该计算机程序可以是应用程序,处理器调用对应存储器中的应用程序时,能够实现该应用程序的功能。例如,第一处理器111调用第一存储器113中的应用程序时,能够实现本申请实施例提供的镜像文件管理方法中边缘云节点的功能。
在图3中,终端总线104、第一总线114、第二总线124和第三总线134中的任一总线可以分为地址总线、数据总线、控制总线等。为便于表示,图3中仅用一条粗线表示,但并不表示仅有一根总线或一种类型的总线。
在图3中,终端处理器101、第一处理器111、第二处理器121和第三处理器131中的任一处理器可以是硬件芯片,该硬件芯片可以是专用集成电路(application-specific integrated circuit,ASIC),可编程逻辑器件(programmable logic device,PLD)或其组合。上述PLD可以是复杂可编程逻辑器件(complex programmable logic device,CPLD),现场可编程逻辑门阵列(field-programmable gate array,FPGA),通用阵列逻辑(generic array logic,GAL)或其任意组合。或者,也可以是通用处理器,例如,中央处理器(central processing unit,CPU),网络处理器(network processor,NP),或者,CPU和NP的组合。
在图3中,终端存储器103、第一存储器113、第二存储器123和第三存储器133中的任一存储器可以包括易失性存储器(volatile memory),例如随机存取存储器(random-access memory,RAM);也可以包括非易失性存储器(non-volatile memory),例如快闪存储器(flash memory),硬盘(hard disk drive,HDD)或固态硬盘(solid-state drive,SSD);还可以包括上述种类的存储器的组合。
本申请实施例提供了一种镜像文件管理方法,该镜像文件管理方法用于采用镜像文件管理系统对镜像文件进行管理。该镜像文件管理方法主要涉及镜像文件的初始化存储、更新和读取过程。下面先对镜像文件的初始化存储过程进行说明。如图4所示,镜像文件的初始化存储包括以下步骤:
步骤401、终端接收用于指示写入第一镜像文件的写入指令。
用户在需要将镜像文件上传至镜像文件管理系统中时,可以在终端上通过执行指定的操作触发写入指令,以通过该写入指令指示将第一镜像文件写入镜像文件管理系统。
镜像文件是由多个文件通过刻录软件或者镜像文件制作工具制作而成的,镜像文件可视为该多个文件的副本。在需要拷贝该多个文件时,可以通过拷贝该多个文件的镜像文件实现,这样能够节省拷贝、迁移和导入过程的时间。例如,镜像文件可以理解为一个磁盘上的数据的副本。当用户需要将磁盘上的多个文件拷贝到另一个磁盘上 时,可以直接拷贝该多个文件的镜像文件。
步骤402、终端将该第一镜像文件划分为多个第一数据块。
在一种可实现方式中,终端可以按照指定大小将第一镜像文件划分为多个数据块(为描述简便和便于区分,在本文中将第一镜像文件中的数据块统称为第一数据块)。例如,假设第一镜像文件的大小为1GB,终端可以按照每个第一数据块为1兆字节(MB)的大小,将该第一镜像文件划分为1024个第一数据块。
步骤403、终端根据每个第一数据块的内容生成对应第一数据块的标识号,并基于多个第一数据块的标识号,生成第一镜像文件的第一标识号拓扑。
对于任一第一数据块,终端可以根据该第一数据块的内容,按照预设的标识号生成算法,生成该第一数据块的标识号,该第一数据块的标识号用于在镜像文件管理系统中唯一地标识该第一数据块。示例的,第一数据块的标识号可以为哈希值,该预设的标识号生成算法可以为:MD5消息摘要算法(MD5message digest algorithm)或安全散列算法(secure hash algorithm,SHA)等加密算法。
在生成每个第一数据块的标识号后,可以按照预设换算算法对该多个第一数据块的标识号进行换算,得到该第一镜像文件的标识号和该第一镜像文件的第一标识号拓扑。镜像文件的标识号拓扑记录该镜像文件中数据块的标识号与该镜像文件的标识号之间的关系,也即是,该镜像文件的标识号拓扑记录该镜像文件包括哪些数据块。
示例的,如图5所示,当第一数据块的标识号为哈希值时,该第一标识号拓扑可以为以该多个第一数据块的哈希值为叶子节点的哈希树,此时,该第一镜像文件的标识号为位于该哈希树根节点上的哈希值。其中,图5所示的哈希树包括三层(每个虚线框A0中为一层),按照由上至下的方向,位于哈希树最底层的节点为哈希树的叶子节点,位于哈希树最顶层的节点为哈希树的根节点,在位于相邻两层且相互连接的节点中,位于上层的节点为位于下层的节点的父节点。
其中,哈希树的生成过程为:将多个第一数据块分为N1组,每组第一数据块包括第一指定数量个第一数据块。然后以组为单位,对每组中多个第一数据块的哈希值执行加密算法,生成每组中多个第一数据块的哈希值的哈希值,根据任一组中多个第一数据块的哈希值所生成的哈希值为该任一组中每个第一数据块的哈希值的父节点,该任一组中任一第一数据块的哈希值可称为该父节点的子节点。在获取父节点的哈希值后,添加由该任一组中每个第一数据块的哈希值指向该父节点的有向箭头。然后,再将N1个父节点分为N2组,每组包括第二指定数量个父节点。然后以组为单位,根据每组中多个父节点的哈希值再生成哈希值,并将根据任一组中多个父节点的哈希值生成的哈希值作为该任一组中每个父节点的父节点。再添加由任一组中每个父节点的哈希值指向根据该任一组多个父节点的哈希值所生成的父节点的有向箭头,如此循环,直到得到该哈希树的根节点。并且,生成哈希树中父节点的加密算法,与生成第一数据块的标识号时使用的加密算法可以相同或者不同,本申请实施例对其不做具体限定。
需要说明的是,父节点所具有的子节点的数量可以根据实际需要进行调整,例如,在图5中,每个父节点具有2个子节点。相应的,该图5所示的哈希树中,根节点构成哈希树的一层,哈希值为哈希0的父节点与哈希值为哈希1的父节点构成哈希树的另一层,哈希值为哈希0-0的叶子节点、哈希值为哈希0-1的叶子节点、哈希值为哈 希1-0的叶子节点、及哈希值为哈希1-1的叶子节点构成哈希树的又一层,且哈希0为对哈希0-0和哈希0-1执行加密算法得到的哈希值,哈希1为对哈希1-0和哈希1-1执行加密算法得到的哈希值,根节点的哈希值为对哈希0和哈希1执行加密算法得到的哈希值。
步骤404、终端基于写入指令,在镜像文件管理系统包括的多个边缘云节点中确定至少一个第一边缘云节点,并将多个第一数据块写入该至少一个第一边缘云节点,使得每个第一边缘云节点存储多个第一数据块的部分或全部。
在步骤404的一种实现方式中,终端可以在镜像文件管理系统包括的多个边缘云节点中确定一个第一边缘云节点,并将多个第一数据块全部写入该第一边缘云节点。在终端将多个第一数据块全部写入该第一边缘云节点后,该第一边缘云节点可以对该多个第一数据块进行存储,以完成该多个第一数据块的写入过程。
可选的,该第一边缘云节点可以为镜像文件管理系统包括的多个边缘云节点中,在逻辑上距离该终端最近的一个边缘云节点。由于在逻辑上距离终端最近的边缘云节点与该终端通信的链路质量较好,通过将多个第一数据块存储在该第一边缘云节点上,能够减小终端写入该第一镜像文件的写时延。
并且,当将多个第一数据块均存储在一个边缘云节点上时,若该边缘云节点需要使用该第一镜像文件,能够减少该边缘云节点从其他边缘云节点上获取该第一镜像文件所包括的第一数据块的数量,减小了该边缘云节点获取第一镜像文件的时间开销。
在步骤404的另一种实现方式中,终端可以在镜像文件管理系统包括的多个边缘云节点中确定多个第一边缘云节点,并分别将多个第一数据块中的部分或全部第一数据块对应写入该多个第一边缘云节点。在终端分别将多个第一数据块中的部分或全部第一数据块对应写入多个第一边缘云节点后,每个第一边缘云节点可以将写入该第一边缘云节点的第一数据块进行存储,以完成该多个第一数据块的写入过程。
该多个第一边缘云节点可以为终端根据指定策略确定的。例如,终端可以根据该多个第一数据块,按照指定的映射方法确定多个第一边缘云节点,且该多个第一边缘云节点分别与该多个第一数据块对应。此时,在将多个第一数据块写入多个第一边缘云节点时,可以将多个第一数据块分别写入对应的第一边缘云节点,使得多个第一边缘云节点分别存储多个第一数据块中的部分或全部第一数据块。这样一来,多个第一边缘云节点中的每个第一边缘云节点中至少存储有第一镜像文件所包括的部分第一数据块,可以减小第一边缘云节点从其他边缘云节点上获取该第一镜像文件所包括的第一数据块的几率,能够减小第一边缘云节点出现时延问题的几率。
示例的,在镜像文件管理系统中,由于多个边缘云节点分别负责一段存储地址,在确定多个第一边缘云节点时,终端可以按照一致性哈希算法,根据每个第一数据块的内容,将每个第一数据块映射至一段存储地址,并将负责该段存储地址的边缘云节点确定为用于存储对应第一数据块的第一边缘云节点。
需要说明的是,在该步骤404的另一种可实现方式中,不同第一边缘云节点中存储的第一数据块中可以包括相同的第一数据块。或者,不同第一边缘云节点中存储的第一数据块可以为完全不同的第一数据块,这样一来,镜像文件管理系统中无需存储重复数据块,能够保证边缘云节点的存储利用率。
并且,在该步骤404中,终端向第一边缘云节点写入的多个第一数据块,可以为第一镜像文件划分得到的所有第一数据块中未存储在任一边缘云节点中的多个第一数据块。此时,在执行将第一数据块写入第一边缘云节点的操作之前,可以先在划分得到的所有第一数据块中,确定任一边缘云节点均未存储的多个第一数据块,再将该任一边缘云节点均未存储的多个第一数据块写入至少一个第一边缘云节点中,以避免在该镜像文件管理系统中写入重复数据,从而减少带宽占用,降低写时延,并提高镜像文件管理系统的存储利用率。
其中,在将数据块写入镜像文件管理系统时,为了便于在该镜像文件管理系统中查询该数据块,可以将该数据块的标识号也写入该镜像文件管理系统中,因此,在确定镜像文件管理系统中是否已存储有任一第一数据块时,可以查询镜像文件管理系统中是否已存储有该任一第一数据块的标识号,当该镜像文件管理系统中未存储有该任一第一数据块的标识号时,确定该镜像文件管理系统中未存储有该任一第一数据块。
需要说明的是,在该步骤404中,可以先确定未存储在任一边缘云节点中的多个第一数据块,然后根据未存储在任一边缘云节点中的多个第一数据块确定一个或多个第一边缘云节点,然后将未存储在任一边缘云节点中的多个第一数据块分别写入对应的第一边缘云节点中。或者,也可以根据划分得到的所有第一数据块确定一个或多个第一边缘云节点,然后确定未存储在任一边缘云节点中的多个第一数据块,再将未存储在任一边缘云节点中的多个第一数据块写入一个或多个第一边缘云节点中,本申请实施例对此不作具体限定。
步骤405、终端根据每个第一数据块在至少一个第一边缘云节点中的存储情况,建立每个第一数据块与存储有对应第一数据块的第一边缘云节点的存储对应关系,将该存储对应关系存储在镜像文件管理系统中。
在第一边缘云节点存储终端向其写入的第一数据块后,第一边缘云节点可以向终端发送指示存储完成的响应,终端可以根据该响应确定第一边缘云节点中存储有对应的第一数据块,并建立该存储对应关系,然后将该存储对应关系写入镜像文件管理系统中,以将该存储对应关系存储在该镜像文件管理系统中。
该存储对应关系用于反映多个第一数据块在一个或多个第一边缘云节点中的存储情况,即用于指示任一第一数据块存储在哪些第一边缘云节点中。在一种可实现方式中,任一第一数据块与存储有该任一第一数据块的第一边缘云节点的存储对应关系可以为:该任一第一数据块的哈希值与存储有该任一第一数据块的第一边缘云节点的物理地址的对应关系。
其中,将存储对应关系存储在镜像文件管理系统中,可以通过将该存储对应关系存储在该镜像文件管理系统中的一个边缘云节点实现。此时,该步骤405的实现方式可以包括:终端在多个边缘云节点中确定一个边缘云节点,并将该存储对应关系写入该边缘云节点,使得镜像文件管理系统中的其他边缘云节点均能够通过访问该边缘云节点获取该存储对应关系。
通过将该存储对应关系存储在镜像文件管理系统中,能够使镜像文件管理系统中的多个边缘云节点根据该存储对应关系,获取多个第一数据块在一个或多个第一边缘云节点中的存储情况。
步骤406、终端在第一标识号拓扑中添加第一镜像文件的描述信息,并将添加有第一镜像文件的描述信息的第一标识号拓扑写入镜像文件管理系统。
第一镜像文件的描述信息(也称提示(commit)信息)用于描述该第一镜像文件,以便于用户了解该第一镜像文件的内容。第一镜像文件的描述信息包括:第一镜像文件的名称,版本信息,权限控制信息等元数据(metadata)信息。其中,版本信息中记载有该第一镜像文件的大小和上传时间等信息,还可以记载有第一镜像文件包括的第一数据块的信息。权限控制信息中记载有对该第一镜像文件具有访问或修改的权限的用户的信息。
在第一标识号拓扑中添加第一镜像文件的描述信息后,可以将添加有该第一镜像文件的描述信息的第一标识号拓扑写入镜像文件管理系统,以便于多个边缘云节点根据该添加有描述信息的第一标识号拓扑,获取第一镜像文件包括哪些第一数据块,以及根据该描述信息了解该第一镜像文件的内容。
并且,将添加有该第一镜像文件的描述信息的第一标识号拓扑写入镜像文件管理系统,可以通过将其写入镜像文件管理系统中的一个边缘云节点实现。其中,存储该添加有该第一镜像文件的描述信息的第一标识号拓扑的边缘云节点,与存储该第一镜像文件的存储对应关系的边缘云节点可以相同或不同,本申请实施例对其不做具体限定。
在一种可实现方式中,当该第一标识号拓扑为哈希树时,在第一标识号拓扑中添加第一镜像文件的描述信息的实现方式可以包括:将第一镜像文件的哈希树根节点的哈希值和第一镜像文件的提示信息填充在信息单元中,并将填充信息后的信息单元串联在第一镜像文件的哈希树的根节点上,将该信息单元作为该哈希树根节点的父节点,得到该第一镜像文件的第一有向无环图(directed acyclic graph,DAG)。其中,填充信息后的信息单元为以第一镜像文件的哈希树根节点的哈希值和第一镜像文件的提示信息等为元素的信息集合。图6为根据图5所示的哈希树得到的第一有向无环图的结构示意图。
需要说明的是,在上述步骤401至步骤406的描述中,均是以执行主体为客户端为例进行说明的。但是,其执行主体可以不限于终端。例如,上述步骤401至步骤406的执行主体还可以为边缘云节点。此时,在步骤401中,边缘云节点接收的写入指令可以为终端发送的,且该写入指令可以为终端通过执行第一镜像文件的上传操作后触发的。在步骤404中,当将多个第一数据块写入一个第一边缘云节点时,该第一边缘云节点可以为用作该步骤404的执行主体的边缘云节点本身,当将多个第一数据块写入多个第一边缘云节点时,该多个第一边缘云节点的确定方式可以相应参考步骤404中确定多个第一边缘云节点的实现方式。相应的,当步骤402、步骤403、步骤405和步骤406执行主体可以为边缘云节点时,对应步骤的实现方式可以相应参考对应步骤的实现方式。
并且,该镜像文件管理方法的步骤先后顺序可以进行适当调整,步骤也可以根据情况进行相应增减,例如,可以根据应用需求,在执行完步骤403后执行步骤406,再执行步骤404和步骤405。任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化的方法,都应涵盖在本申请的保护范围之内,因此不再赘述。
综上所述,本申请实施例提供的镜像文件管理方法,通过边缘云节点存储第一镜像文件所包括的数据块,使得该镜像文件管理系统中任一边缘云节点需要使用镜像文件时,可以从存储有该镜像文件中数据块的边缘云节点中获取对应的数据块,以得到其所需的镜像文件,无需再从中心云节点中下载该镜像文件,减小了多个边缘云节点共享同一云节点的可用带宽的几率,能够较大化利用该任一边缘云节点的可用带宽,及该任一边缘云节点与存储有该任一边缘云节点所需数据块的边缘云节点之间的带宽,减小了获取镜像文件的时间开销,提高了镜像文件的下载速度。
并且,通过对镜像文件进行分块,使得边缘云节点以数据块为单位进行存储,相较于以镜像文件为单元对镜像文件进行写入的相关技术,细化了镜像文件的写入粒度,使得无需将镜像文件管理系统中已存储的数据块再次写入镜像文件管理系统,避免了数据块的重复写入,减少了写数据的带宽占用,降低了写时延,并提高了镜像文件管理系统的存储利用率。
同时,通过生成第一镜像文件的第一标识号拓扑,能够便于用户在读取第一镜像文件的过程中,根据该第一标识号拓扑快速确定第一镜像文件包括的第一数据块,进一步的,通过在第一标识号拓扑中添加第一镜像文件的描述信息,能够便于快速确定该第一镜像文件的内容,而无需通过读取数据块的方式确定第一镜像文件包括的数据块和第一镜像文件的内容,能够减小读时延。
下面更新镜像文件管理系统中已存在的镜像文件的信息的过程进行说明。如图7所示,更新过程包括以下步骤:
步骤701、目标边缘云节点接收用于指示更新第三镜像文件的信息的更新指令,该第三镜像文件是目标边缘云节点基于镜像文件管理系统中已存储的第四镜像文件生成的。
第三镜像文件可以为目标边缘云节点对第四镜像文件执行预设操作后得到的镜像文件。示例的,在云计算领域中,目标边缘云节点可以使用第四镜像文件在镜像文件管理系统的任一边缘云节点中创建虚拟机或容器等实例,且在创建虚拟机或容器等实例后,还可以根据已创建的虚拟机或容器等实例,重新制作镜像文件,该重新制作的镜像文件即为基于该第四镜像文件生成的镜像文件。例如,当第四镜像文件为虚拟机的镜像文件时,可以根据该第四镜像文件,在目标边缘云节点中创建虚拟机,且在创建该虚拟机后,可以在该新创建的虚拟机中安装新的应用,此时,该新创建的虚拟机中会存在不同于第四镜像文件的数据(即新的数据),当根据该新创建的虚拟机制作镜像文件时,该制作的镜像文件会包括新的数据。并且,由于该新制作的镜像文件是在目标边缘节点中制作的,因此,该新制作的镜像文件已存储在镜像文件管理系统中,此时,可以在镜像文件管理系统中更新该新制作的镜像文件的信息,以便于其他边缘云节点能够根据该信息获取该新制作的镜像文件,保证镜像文件管理系统中存储的镜像文件的数据完备性。并且,通过在镜像文件管理系统中保存不同版本的镜像文件的信息,可以便于用户根据需要选择要使用的镜像文件。例如,用户可以根据应用需要在第四镜像文件和第三镜像文件中选择镜像文件,并根据选择的镜像文件创建虚拟机。其中,第四镜像文件与第一镜像文件可以为相同的镜像文件,也可以为不同的镜像文 件,本申请实施例对其不做具体限定。
步骤702、目标边缘云节点将该第三镜像文件划分为多个第三数据块。
该步骤702的实现过程可以相应参考步骤402的实现过程。并且,该第三镜像文件的划分方式可以与第四镜像文件的划分方式相同。并且,由于第三镜像文件是在第四镜像文件的基础上生成的,该第三镜像文件的构成方式与第四镜像文件的构成方式具有较大相似度,因此,当第四镜像文件的划分方式与第三镜像文件的划分方式相同时,划分得到的多个第三数据块中存在与第四镜像文件所包括的第四数据块相同的数据块的几率更高。其中,为描述简便和便于区分,在本文中将第三镜像文件中的数据块统称为第三数据块,将第四镜像文件中的数据块统称为第四数据块。
步骤703、目标边缘云节点根据每个第三数据块的内容生成对应第三数据块的标识号,并基于多个第三数据块的标识号,生成该第三镜像文件的第三标识号拓扑。
其中,生成第三数据块的标识号的实现过程可以相应参考生成第一数据块的标识号的实现过程。并且,该生成第三标识号拓扑的实现方式也可以相应参考生成第一标识号拓扑的实现方式。
或者,在生成第三标识号拓扑的另一种实现方式中,可以根据与第四镜像文件所包括的数据块均不同的第三数据块,在第四标识号拓扑的基础上,对第四标识号拓扑进行增量更新,得到第三标识号拓扑,以减少生成第三标识号拓扑的计算量。例如,如图8所示,其实现方式可以包括:
步骤7031、在第四标识号拓扑中,确定第一树状子拓扑,第三镜像文件与第四镜像文件所包括的相同的数据块的标识号为第一树状子拓扑中的叶子节点。
由于数据块的标识号用于唯一地标识号数据块,当两个数据块的标识号相同时,说明该两个数据块的内容相同,因此,可以将第三镜像文件的标识号与第四镜像文件中标识号相同的数据块,确定为第四镜像文件和第三镜像文件中的相同数据块(为便于描述,下面简称为相同数据块),将标识号与第四镜像文件中所有第四数据块的标识号均不同的第三数据块,确定为与第四镜像文件所包括的数据块均不同的第三数据块。
在第四标识号拓扑中确定第一树状子拓扑时,可以先确定第三镜像文件和第四镜像文件中所有相同数据块的标识号,将以该所有相同数据块的标识号为叶子节点的拓扑部分,确定为第一树状子拓扑,且位于该第一树状子拓扑顶端的标识号可视为该第一树状子拓扑的根节点。
示例的,假设第四镜像文件包括第四数据块L11至L14,第四数据块L11的哈希值为01a3b001a10c01(在图9中以哈希0-0表示),第四数据块L12的哈希值为01a3b001a10c02(在图9中以哈希0-1表示),第四数据块L13的哈希值为01a3b001a10c03(在图9中以哈希1-0表示),第四数据块L14的哈希值为01a3b001a10c04(在图9中以哈希1-1表示)。第三镜像文件包括第三数据块L21至L24,第三数据块L21的哈希值为01a3b001a10c01(在图9中以哈希0-0表示),第三数据块L22的哈希值为01a3b001a10c02(在图9中以哈希0-1表示),第三数据块L23的哈希值为01a3b001a10c0d(在图9中以哈希2-0表示),第三数据块L24的哈希值为01a3b001a10caf(在图9中以哈希2-1表示),则可以确定第四数据块L11与 第三数据块L21为相同数据块,第四数据块L12与第三数据块L22为相同数据块,第三数据块L23和第三数据块L24与第四镜像文件中的所有数据块均不相同。此时,在该步骤7031中,可以将以第四数据块L11的哈希值和第四数据块L12的哈希值为叶子节点的拓扑部分确定为第一树状子拓扑。如图9中虚线框A1中的第一子哈希树所示,该第一子哈希树以第四数据块L11的哈希值哈希0-0和第四数据块L12的哈希值哈希0-1为叶子节点,即该第一子哈希树包括:第四数据块L11的哈希值哈希0-0、第四数据块L12的哈希值哈希0-1,及以第四数据块L11的哈希值和第四数据块L12的哈希值为子节点的父节点的哈希值哈希0,且由于该父节点位于该第一子哈希树的顶端,可视为该父节点为该第一子哈希树的根节点。
步骤7032、基于与第四镜像文件所包括的数据块均不同的第三数据块,生成第二树状子拓扑。
基于与第四镜像文件所包括的数据块均不同的第三数据块,生成第二树状子拓扑的过程,实质为根据与第四镜像文件所包括的数据块均不同的第三数据块的标识号,生成与第四镜像文件所包括的数据块均不同的第三数据块组成的镜像文件的标识号拓扑的过程,因此,该步骤7032的实现过程请相应参考步骤403的实现过程。示例的,同样以步骤7031中的例子为例,根据第三数据块L23的哈希值和第三数据块L24的哈希值生成的第二子哈希树如图9中虚线框A2所示,且位于该第二子哈希树顶端的哈希值(即哈希2)可视为该第二子哈希树的根节点。
步骤7033、基于第一树状子拓扑和第二树状子拓扑,生成第三标识号拓扑。
在获取第一树状子拓扑和第二树状子拓扑后,可以对第一树状子拓扑的根节点和第二树状子拓扑的根节点再次执行加密算法,根据该第一树状子拓扑的根节点和第二树状子拓扑的根节点的标识号,该标识号即为该第三标识号拓扑的根节点,然后添加分别由第一树状子拓扑的根节点和第二树状子拓扑的根节点指向该第三标识号拓扑的根节点的有向箭头,得到该第三标识号拓扑。
示例的,如图9所示,根据图9中虚线框A1中所示的第一子哈希树和图9中虚线框A2中所示的第二子哈希树,生成的第二哈希树如图9所示。
步骤704、目标边缘云节点建立每个第三数据块与目标边缘云节点的存储对应关系,并将该存储对应关系存储在镜像文件管理系统中。
由于该第三镜像文件是在目标边缘云节点中制作得到的,因此,该第三镜像文件的所有第三数据块已存储在该目标边缘云节点中,即该镜像文件管理系统中已存储有该第三镜像文件,此时,需要在该镜像文件管理系统中存储该第三镜像文件中第三数据块的存储对应关系,以便于其他边缘云节点能够获取第三数据块的存储信息。且该在镜像文件管理系统中存储该存储对应关系的实现方式可以包括以下两种情况:
在第一种情况中,若某一第三数据块存储在目标边缘云节点中,且镜像文件管理系统的其他边缘云节点中均未存储该第三数据块,可以在镜像文件管理系统中添加该第三数据块与目标边缘云节点的存储对应关系。该在镜像文件管理系统中添加该第三数据块与目标边缘云节点的存储对应关系的实现过程,请相应参考步骤405中终端将多个第一数据块与第一边缘云节点的存储对应关系存储在镜像文件管理系统的实现过程。示例的,可以在镜像文件管理系统中添加用于指示该第三数据块的哈希值与该目 标边缘云节点的物理地址的对应关系的记录。
在第二种情况中,若某一第三数据块不仅存储在目标边缘云节点中,还存储在镜像文件管理系统的其他边缘云节点中,此时,可以在第三数据块的存储对应关系中,添加该第三数据块与该目标边缘云节点的对应关系。
步骤705、目标边缘云节点在第三标识号拓扑中添加第三镜像文件的描述信息,并将添加有第三镜像文件的描述信息的第三标识号拓扑写入镜像文件管理系统。
该步骤705的实现过程请相应参考步骤406的实现过程。在第三标识号拓扑中添加第三镜像文件的描述信息后,得到的第三镜像文件的第二有向无环图如图9所示。
并且,由于该第三镜像文件是基于第四镜像文件生成的,在该第三镜像文件的描述信息中还可以包括有更新说明信息,该更新说明信息用于指示该第三镜像文件相对于第四镜像文件的差异信息。通过在描述信息中添加更新说明信息,一方面,能够实现对镜像文件的版本管理,使得用户根据该更新说明信息了解该第三镜像文件相对于第四镜像文件的区别,以便于用户快速地选择需要的镜像文件,另一方面,使得无需通过读取并比较数据块的内容才能确定两者的区别,能够减小镜像文件系统的读时延。同时,通过对镜像文件进行版本管理,在向镜像文件管理系统中写入数据块的过程中,在确定该镜像文件管理系统中是否存储有该数据块时,可以仅与某镜像文件和基于该镜像文件生成的镜像文件中一个版本的镜像文件的数据块进行比较,以减小比较的次数,减小写时延。
需要说明的是,该镜像文件管理方法的步骤先后顺序可以进行适当调整,步骤也可以根据情况进行相应增减,例如,可以根据应用需求调整步骤704和步骤705的执行顺序。任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化的方法,都应涵盖在本申请的保护范围之内,因此不再赘述。
综上所述,本申请实施例提供的镜像文件管理方法,通过更新镜像文件管理系统中已存储的镜像文件的信息,使得镜像文件管理系统中的其他边缘云节点能够根据该信息获取该已存储的镜像文件,以保证镜像文件管理系统中存储的镜像文件的数据完备性。
并且,当第三镜像文件为在第四镜像文件的基础上生成的镜像文件时,通过对第四镜像文件的第四标识号拓扑进行增量更新,得到第三镜像文件的第三标识号拓扑,能够减少生成第三标识号拓扑的计算量,并减小更新第三镜像文件信息的写时延。
同时,通过在描述信息中添加更新说明信息,一方面,能够实现对镜像文件的版本管理,使得用户根据该更新说明信息了解该第三镜像文件相对于第四镜像文件的区别,以便于用户快速地选择需要的镜像文件,另一方面,使得无需通过读取并比较数据块的内容才能确定两者的区别,能够减小镜像文件系统的读时延。进一步的,通过对镜像文件进行版本管理,在向镜像文件管理系统中写入数据块的过程中,在确定该镜像文件管理系统中是否存储有该数据块时,可以仅与某镜像文件和基于该镜像文件生成的镜像文件中一个版本的镜像文件的数据块进行比较,以减小比较的次数,减小写时延。
下面对镜像文件的读取过程进行说明,如图10所示,该镜像文件的读取过程包括 以下步骤:
步骤801、目标边缘云节点接收用于指示读取第二镜像文件的读取指令。
终端可以向目标边缘云节点发送携带有第二镜像文件的标识号的读取指令,以指示目标边缘节点在镜像文件管理系统中读取镜像文件,并根据该镜像文件执行指定的操作。例如,终端可以指示目标边缘节点从镜像文件管理系统中读取镜像文件,并使用该镜像文件创建虚拟机和容器等实例。并且,在创建实例之前,终端可以先查询镜像文件管理系统中是否存储有用于创建实例所需的第二镜像文件,在确定镜像文件管理系统中存储有第二镜像文件时,向目标边缘云节点发送携带有第二镜像文件的标识号的读取指令,以请求目标边缘云节点从镜像文件管理系统中读取该第二镜像文件,并使用该第二镜像文件在该目标边缘云节点中创建实例。其中,镜像文件管理系统配置有应用程序接口,终端可以通过该应用程序接口,在镜像文件管理系统中查询该镜像文件管理系统中所存储的镜像文件的信息,并确定该镜像文件管理系统中是否存储有该第二镜像文件。
步骤802、目标边缘云节点基于读取指令,在镜像文件管理系统包括的多个边缘云节点中,确定存储有第二镜像文件的第二数据块的一个或多个第二边缘云节点。
在一种可实现方式中,由于目标边缘云节点中可能存储有第二镜像文件的部分第二数据块(为描述简便和便于区分,在本文中将第二镜像文件中的数据块统称为第二数据块),因此,在读取第二镜像文件的过程中,可以从第二边缘云节点中,读取属于第二镜像文件但未存储在目标边缘云节点中的第二数据块,以得到该第二镜像文件。相应的,该步骤802的实现过程可以包括:根据读取指令,确定第二镜像文件所包括的所有第二数据块,然后在在该所有第二数据块中,确定未存储在目标边缘云节点中的一个或多个第二数据块,并将存储有任一未存储在目标边缘云节点中的第二数据块的边缘云节点确定为第二边缘云节点。
由于镜像文件的标识号拓扑记录了镜像文件所包括的数据块的标识号与该镜像文件的标识号之间关系,因此,目标边缘云节点可以根据读取指令携带的第二镜像文件的标识号,确定该第二镜像文件的第二标识号拓扑,并根据第二标识号拓扑确定该第二镜像文件所包括的所有第二数据块。
并且,由于镜像文件管理系统中存储有第二数据块与用于存储该第二数据块的边缘云节点的存储对应关系,因此,可以根据该存储对应关系,确定存储有每个第二数据块的边缘云节点,当存储某个第二数据块的边缘云节点不包括该目标边缘云节点时,确定该某个第二数据块为属于第二镜像文件但未存储在目标边缘云节点中的第二数据块。相应的,存储有属于第二镜像文件但未存储在目标边缘云节点中的第二数据块的边缘云节点即为第二边缘云节点。
通过标识号拓扑确定镜像文件包括的数据块,并根据数据块与边缘云节点的存储对应关系确定属于第二镜像文件但未存储在目标边缘云节点中的第二数据块的实现方式,相较于通过将目标边缘云节点中存储的所有数据块的内容与每个第二数据块的内容进行比较,以确定属于第二镜像文件但未存储在目标边缘云节点中的第二数据块的实现的方式,能够减少确定属于第二镜像文件但未存储在目标边缘云节点中的第二数据块过程中的数据处理量,以进一步减小第二镜像文件的读时延。
步骤803、当多个第二边缘云节点中均存储有同一第二数据块集合时,目标边缘云节点基于均存储有该第二数据块集合的每个第二边缘云节点的下载状态参数,选择下载该第二数据块集合中第二数据块的第二边缘云节点,从选择的第二边缘云节点中下载该第二数据块集合中的第二数据块。
该第二数据块集合包括一个或多个第二数据块。例如,当第二边缘云节点A1中存储有第二数据块B1,且第二边缘云节点A2中也存储有第二数据块B1时,可视为该第二边缘云节点A1和第二边缘云节点A2中均存储有同一第二数据块集合,且该第二数据块集合包括第二数据块B1。当第二边缘云节点A1中存储有第二数据块B1和第二数据块B2,且第二边缘云节点A2中也存储有第二数据块B1和第二数据块B2时,可视为该第二边缘云节点A1和第二边缘云节点A2中均存储有同一第二数据块集合,且该第二数据块集合包括第二数据块B1和第二数据块B2。
当多个第二边缘云节点中存储有同一第二数据块集合时,目标边缘云节点可以获取均存储有该第二数据块集合的每个第二边缘云节点的下载状态参数,并根据该均存储有该第二数据块集合的每个第二边缘云节点的下载状态参数,对存储有该第二数据块集合的多个第二边缘云节点进行筛选,并在存储有该第二数据块集合的多个第二边缘云节点中,将下载状态参数指示的下载性能最优的第二边缘云节点选择为用作下载源的第二边缘云节点,然后从该下载性能最优的目第二边缘云节点中下载第二数据块集合中的第二数据块,以保证下载速度。
其中,该下载状态参数用于反映目标边缘云节点从对应第二边缘云节点中下载第二数据块的下载速度。可选的,第二边缘云节点的下载状态参数包括:该第二边缘云节点与目标边缘云节点之间的网络流量和网络延迟等参数,以及该第二边缘云节点的负载状态参数等。在一种可实现方式中,目标边缘云节点可以与该第二边缘云节点通信,并使用网络探测技术获取该第二边缘云节点的下载状态参数。例如,目标边缘云节点可以向该第二边缘云节点发送探测数据包,并根据该第二边缘云节点对接收该探测数据包的响应,获取该目标边缘云节点与该第二边缘云节点之间的网络流量和网络延迟等参数。
通过在均存储有第二数据块集合的多个第二边缘云节点中选择出下载性能最优的第二边缘云节点,并从该选择得到的第二边缘云节点中下载第二数据块集合中的第二数据块,能够保证第二数据块的下载速度,减小带宽消耗,并减小下载时延。
步骤804、目标边缘云节点在从选择的第二边缘云节点中下载该同一第二数据块集合中第二数据块的过程中,继续获取该选择的第二边缘云节点的下载状态参数,当该选择的第二边缘云节点的下载状态参数小于参考参数阈值时,目标边缘云节点在均存储有第二数据块集合的多个第二边缘云节点中,重新选择下载第二数据块集合中第二数据块的第二边缘云节点,并从该重新选择的第二边缘云节点中下载未成功下载且属于第二数据块集合的第二数据块。
其中,该重新选择的第二边缘云节点的下载状态参数大于参考参数阈值。该未成功下载的第二数据块可以包括:未完全下载的第二数据块、完全未下载的第二数据块、及下载内容错误的第二数据块。可选的,可以通过哈希校验确定下载完成的第二数据块是否为下载内容错误的第二数据块,其实现方式可以包括:根据已下载的第二数据 块的内容生成哈希值,并将该生成的哈希值与镜像文件管理系统中存储的该第二数据块的哈希值进行比较,当生成的哈希值与镜像文件管理系统中存储的该第二数据块的哈希值相同时,确定该已下载的第二数据块的下载内容正确,当生成的哈希值与镜像文件管理系统中存储的该第二数据块的哈希值不同时,确定该已下载的第二数据块为下载内容错误的数据块。
当第二边缘云节点的下载状态参数小于参考参数阈值时,说明目标边缘云节点与该第二边缘云节点之间的下载链接,出现了中断或者带宽下降等情况,即目标边缘云节点从该第二边缘云节点中下载数据块的下载速度无法满足下载要求,此时,可以重新选择用作下载源的第二边缘云节点,以保证下载速度。其中,该步骤804中获取第二边缘云节点的下载状态参数的实现过程,可以相应参考步骤803中获取第二边缘云节点的下载状态参数的实现过程,该步骤804中在均存储有第二数据块集合的多个第二边缘云节点中重新选择第二边缘云节点的实现过程,可以相应参考步骤803中选择第二边缘云节点的实现过程,此处不再赘述。
并且,在从重新选择的第二边缘云节点中继续下载同一第二数据块集合中的第二数据块的过程中,可以重复执行获取重新选择的第二边缘云节点的下载状态参数,并在下载状态参数小于参考参数阈值时,继续重新选择用作下载源的第二边缘云节点,并从重新选择的第二边缘云节点中继续下载第二数据块,直至完成该同一第二数据块集合中的第二数据块的下载。
步骤805、在目标边缘云节点完成第二数据块的下载后,存储从第二边缘云节点中读取的第二数据块,将每个第二数据块与目标边缘云节点的存储对应关系,存储在镜像文件管理系统中。
在目标边缘云节点完成第二数据块的下载后,目标边缘云节点可以将下载到的第二数据块进行存储,并在每个第二数据块的存储对应关系中添加该目标边缘云节点的信息,以指示该目标边缘云节点中也存储有该第二数据块,使得镜像文件管理系统中其他边缘云节点能够获取该第二数据块的存储信息。且该步骤805的实现过程可以包括:在每个第二数据块的存储对应关系中,添加每个第二数据块与该目标边缘云节点的对应关系。
需要说明的是,当该第二镜像文件为终端指示目标边缘节点用于创建实例的镜像文件时,在目标边缘云节点获取第二镜像文件后,目标边缘云节点还可以根据该第二镜像文件在该目标边缘云节点中创建实例。
并且,该镜像文件管理方法的步骤先后顺序可以进行适当调整,步骤也可以根据情况进行相应增减,例如,可以根据应用需求选择是否执行步骤804。任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化的方法,都应涵盖在本申请的保护范围之内,因此不再赘述。
综上所述,本申请实施例提供的镜像文件管理方法,由于镜像文件管理系统包括多个边缘云节点,每个边缘云节点均配置有存储镜像文件的功能,且该多个边缘云节点均被配置为能够读取该镜像文件管理系统中边缘云节点存储的文件块数据,使得该镜像文件管理系统中任一边缘云节点需要使用镜像文件时,可以从存储有该镜像文件中数据块的边缘云节点中获取对应的数据块,以得到该镜像文件,无需再从中心云节 点中下载该镜像文件,减小了多个边缘云节点共享同一云节点的可用带宽的几率,能够较大化利用该任一边缘云节点的可用带宽,及该任一边缘云节点与存储有该任一边缘云节点所需数据块的边缘云节点之间的带宽,减小了获取镜像文件的时间开销,提高了镜像文件的下载速度。
并且,通过对镜像文件进行分块,使得边缘云节点可以以数据块为单位进行读取操作,相较于以镜像文件为单元对镜像文件进行读取的相关技术,细化了镜像文件的读取粒度,使得能够分别从不同边缘云节点中读取所需镜像文件的数据块,一方面能够根据需要读取的数据量分配可用带宽,能够较大化利用可用带宽,减小读时延,另一方面,通过读取属于第二镜像文件但未存储在目标边缘云节点中的第二数据块,能够减小需要读取的数据量,进一步减小读时延。
同时,在下载第二数据块时,通过获取存储有第二数据块的第二边缘云节点的下载状态参数,并根据该下载状态参数确定作为下载源的第二边缘云节点,避免从固定云节点下载镜像文件时下载速度受限于边缘云节点与该固定云节点之间的网络条件的限制,能够减小下载镜像文件的时间开销,保证第二数据块的下载速度,减小读取镜像文件的读时延。
本申请实施例还提供了一种镜像文件管理装置,该镜像文件管理装置可以部署在终端中或部署在边缘云节点中。如图11所示,该镜像文件管理装置110可以包括:
第一接收模块1101,用于接收用于指示写入第一镜像文件的写入指令。
第一确定模块1102,用于基于写入指令,在镜像文件管理系统可以包括的多个边缘云节点中,确定至少一个第一边缘云节点。
第一写入模块1103,用于将第一镜像文件所包括的多个数据块,写入至少一个第一边缘云节点,使得每个第一边缘云节点存储第一镜像文件所包括的多个数据块的部分或全部。
可选的,如图12所示,镜像文件管理装置110还可以包括:
第一生成模块1104,用于生成第一镜像文件所包括的每个数据块的标识号。
第一生成模块1104,还用于基于第一镜像文件所包括的每个数据块的标识号,生成第一镜像文件的第一标识号拓扑,第一标识号拓扑记录第一镜像文件可以包括的数据块。
第一写入模块1103,还用于将第一标识号拓扑存储在镜像文件管理系统的边缘云节点中。
可选的,第一标识号拓扑还携带有第一镜像文件的版本信息。
可选的,如图12所示,镜像文件管理装置110还可以包括:
第一建立模块1105,用于根据第一镜像文件所包括的每个数据块在至少一个第一边缘云节点中的存储情况,建立第一镜像文件所包括的每个数据块与存储有第一镜像文件中对应数据块的第一边缘云节点的存储对应关系。
第一写入模块1103,还用于将存储对应关系存储在镜像文件管理系统的边缘云节点中。
可选的,第一确定模块1102,还用于确定第一镜像文件所包括的所有数据块中未 存储在任一边缘云节点中的第一镜像文件所包括的多个数据块。
综上所述,本申请实施例提供的镜像文件管理装置,通过边缘云节点存储第一镜像文件所包括的数据块,使得该镜像文件管理系统中任一边缘云节点需要使用镜像文件时,可以从存储有该镜像文件中数据块的边缘云节点中获取对应的数据块,以得到其所需的镜像文件,无需再从中心云节点中下载该镜像文件,减小了多个边缘云节点共享同一云节点的可用带宽的几率,能够较大化利用该任一边缘云节点的可用带宽,及该任一边缘云节点与存储有该任一边缘云节点所需数据块的边缘云节点之间的带宽,减小了获取镜像文件的时间开销,提高了镜像文件的下载速度。
并且,通过对镜像文件进行分块,使得边缘云节点以数据块为单位进行存储,相较于以镜像文件为单元对镜像文件进行写入的相关技术,细化了镜像文件的写入粒度,使得无需将镜像文件管理系统中已存储的数据块再次写入镜像文件管理系统,避免了数据块的重复写入,减少了写数据的带宽占用,降低了写时延,并提高了镜像文件管理系统的存储利用率。
同时,通过生成第一镜像文件的第一标识号拓扑,能够便于用户在读取第一镜像文件的过程中,根据该第一标识号拓扑快速确定第一镜像文件包括的第一数据块,进一步的,通过在第一标识号拓扑中添加第一镜像文件的描述信息,能够便于快速确定该第一镜像文件的内容,而无需通过读取数据块的方式确定第一镜像文件包括的数据块和第一镜像文件的内容,能够减小读时延。
本申请实施例还提供了一种镜像文件管理装置,该镜像文件管理装置可以部署在边缘云节点中。如图13所示,该镜像文件管理装置130可以包括:
第二接收模块1301,用于接收用于指示读取第二镜像文件的读取指令。
第二确定模块1302,用于基于读取指令,在镜像文件管理系统可以包括的多个边缘云节点中,确定存储有第二镜像文件的数据块的一个或多个第二边缘云节点。
读取模块1303,用于从一个或多个第二边缘云节点中读取属于第二镜像文件的数据块,以得到第二镜像文件。
可选的,第二确定模块1302,具体用于:基于读取指令,确定第二镜像文件所包括的所有数据块;在第二镜像文件所包括的所有数据块中,确定未存储在目标边缘云节点中的一个或多个数据块;确定存储有一个或多个数据块中任一数据块的第二边缘云节点。
可选的,第二确定模块1302基于读取指令,确定第二镜像文件所包括的所有数据块的实现过程,具体包括:基于读取指令,确定第二镜像文件的第二标识号拓扑,第二标识号拓扑记录第二镜像文件可以包括的数据块;基于第二标识号拓扑,确定第二镜像文件所包括的所有数据块。
可选的,第二确定模块1302在第二镜像文件所包括的所有数据块中,确定未存储在目标边缘云节点中的一个或多个数据块的实现过程,具体包括:查询第二镜像文件中每个数据块与存储有第二镜像文件中对应数据块的边缘云节点的存储对应关系;当第二镜像文件中某一数据块的存储对应关系中,未记载第二镜像文件中某一数据块与目标边缘云节点的对应关系时,确定某一数据块为未存储在目标边缘云节点中的数据 块。
可选的,读取模块1303,具体用于:当多个第二边缘云节点中均存储有第二镜像文件可以包括的某一数据块时,获取均存储有某一数据块的每个第二边缘节点的下载状态参数;基于均存储有某一数据块的每个第二边缘节点的下载状态参数,选择下载某一数据块的第二边缘云节点;从选择的第二边缘云节点中下载某一数据块。
可选的,读取模块1303从选择的第二边缘云节点中下载某一数据块的实现过程,具体包括:在从选择的第二边缘云节点中下载某一数据块的过程中,继续获取选择的第二边缘云节点的下载状态参数;当选择的第二边缘云节点的下载状态参数小于参考参数阈值时,在均存储有某一数据块的多个第二边缘节点重新选择下载某一数据块的第二边缘云节点,重新选择的第二边缘云节点的下载状态参数大于参考参数阈值;从重新选择的第二边缘云节点中,重新下载某一数据块。
可选的,如图14所示,该镜像文件管理装置130还可以包括:
存储模块1304,用于存储从第二边缘云节点中读取的第二镜像文件的数据块。
第二写入模块1305,用于将第二镜像文件所包括的每个数据块与目标边缘云节点的存储对应关系,存储在镜像文件管理系统的边缘云节点中。
本申请实施例提供的镜像文件管理装置,由于镜像文件管理系统包括多个边缘云节点,每个边缘云节点均配置有存储镜像文件的功能,且该多个边缘云节点均被配置为能够读取该镜像文件管理系统中边缘云节点存储的文件块数据,使得该镜像文件管理系统中任一边缘云节点需要使用镜像文件时,可以从存储有该镜像文件中数据块的边缘云节点中获取对应的数据块,以得到该镜像文件,无需再从中心云节点中下载该镜像文件,减小了多个边缘云节点共享同一云节点的可用带宽的几率,能够较大化利用该任一边缘云节点的可用带宽,及该任一边缘云节点与存储有该任一边缘云节点所需数据块的边缘云节点之间的带宽,减小了获取镜像文件的时间开销,提高了镜像文件的下载速度。
并且,通过对镜像文件进行分块,使得边缘云节点可以以数据块为单位进行读取操作,相较于以镜像文件为单元对镜像文件进行读取的相关技术,细化了镜像文件的读取粒度,使得能够分别从不同边缘云节点中读取所需镜像文件的数据块,一方面能够根据需要读取的数据量分配可用带宽,能够较大化利用可用带宽,减小读时延,另一方面,通过读取属于第二镜像文件但未存储在目标边缘云节点中的第二数据块,能够减小需要读取的数据量,进一步减小读时延。
同时,在下载第二数据块时,通过获取存储有第二数据块的第二边缘云节点的下载状态参数,并根据该下载状态参数确定作为下载源的第二边缘云节点,避免从固定云节点下载镜像文件时下载速度受限于边缘云节点与该固定云节点之间的网络条件的限制,能够减小下载镜像文件的时间开销,保证第二数据块的下载速度,减小读取镜像文件的读时延。
可选的,第二接收模块1301,还用于:接收用于指示更新第三镜像文件的信息的更新指令,第三镜像文件是目标边缘云节点基于镜像文件管理系统中已存储的第四镜像文件生成的。
可选的,如图14所示,镜像文件管理装置130还可以包括:
第二生成模块1306,用于生成第三镜像文件所包括的每个数据块的标识号。
第二生成模块1306,还用于基于第三镜像文件所包括的每个数据块的标识号,生成第三镜像文件的第三标识号拓扑,第三标识号拓扑记录第三镜像文件可以包括的数据块。
第二写入模块1305,用于将第三标识号拓扑存储在镜像文件管理系统的边缘云节点中。
可选的,当镜像文件管理系统中还存储有第四镜像文件的第四标识号拓扑时,第二生成模块1306,具体用于:在第四标识号拓扑中,确定第一树状子拓扑,第三镜像文件与第四镜像文件所包括的相同的数据块的标识号为第一树状子拓扑中的叶子节点;基于第三镜像文件所包括的数据块中与第四镜像文件所包括的数据块均不相同的数据块,生成第二树状子拓扑,第三镜像文件所包括的数据块中与第四镜像文件所包括的数据块均不相同的数据块的标识号为第二树状子拓扑中的叶子节点;基于第一树状子拓扑和第二树状子拓扑,生成第三标识号拓扑。
可选的,第三标识号拓扑还携带有第三镜像文件的版本信息和更新说明信息,更新说明信息用于反映第三镜像文件相对于第四镜像文件中不同的内容。
可选的,如图14所示,镜像文件管理装置130还可以包括:
第二建立模块1307,用于建立第三镜像文件所包括的每个数据块与目标边缘云节点的存储对应关系。
第二写入模块1305,用于将第三镜像文件所包括的每个数据块与目标边缘云节点的存储对应关系,存储在镜像文件管理系统的边缘云节点中。
综上所述,本申请实施例提供的镜像文件管理装置,通过更新镜像文件管理系统中已存储的镜像文件的信息,使得镜像文件管理系统中的其他边缘云节点能够根据该信息获取该已存储的镜像文件,以保证镜像文件管理系统中存储的镜像文件的数据完备性。
并且,当第三镜像文件为在第四镜像文件的基础上生成的镜像文件时,通过对第四镜像文件的第四标识号拓扑进行增量更新,得到第三镜像文件的第三标识号拓扑,能够减少生成第三标识号拓扑的计算量,并减小更新第三镜像文件信息的写时延。
同时,通过在描述信息中添加更新说明信息,一方面,能够实现对镜像文件的版本管理,使得用户根据该更新说明信息了解该第三镜像文件相对于第四镜像文件的区别,以便于用户快速地选择需要的镜像文件,另一方面,使得无需通过读取并比较数据块的内容才能确定两者的区别,能够减小镜像文件系统的读时延。进一步的,通过对镜像文件进行版本管理,在向镜像文件管理系统中写入数据块的过程中,在确定该镜像文件管理系统中是否存储有该数据块时,可以仅与某镜像文件和基于该镜像文件生成的镜像文件中一个版本的镜像文件的数据块进行比较,以减小比较的次数,减小写时延。
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的装置和模块的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。
本申请实施例还提供了一种计算机设备,该计算机设备包括处理器和存储器;该存储器中存储有计算机程序;该处理器执行计算机程序时,该计算机设备实现本申请实施例提供的镜像文件管理方法。该计算机设备可以为服务器或终端,当该计算机设备为服务器时,该计算机设备的结构请相应参考图3中第一服务器、第二服务器或第三服务器的结构,当该计算机设备为终端时,该计算机设备的结构请参考图3中终端的结构,此处不再赘述。
本申请实施例还提供了一种存储介质,该存储介质为非易失性计算机可读存储介质,当存储介质中的指令被处理器执行时,实现本申请实施例提供的镜像文件管理方法。
本申请实施例还提供了一种包含指令的计算机程序产品,当计算机程序产品在计算机上运行时,使得计算机执行本申请实施例提供的镜像文件管理方法。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成,也可以通过程序来指令相关的硬件完成,所述的程序可以存储于一种计算机可读存储介质中,上述提到的存储介质可以是只读存储器,磁盘或光盘等。
在本申请实施例中,术语“第一”、“第二”和“第三”仅用于描述目的,而不能理解为指示或暗示相对重要性。术语“至少一个”是指一个或多个,术语“多个”指两个或两个以上,除非另有明确的限定。
本申请中术语“和/或”,仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。另外,本文中字符“/”,一般表示前后关联对象是一种“或”的关系。
以上所述仅为本申请的可选实施例,并不用以限制本申请,凡在本申请的构思和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请的保护范围之内。

Claims (25)

  1. 一种镜像文件管理方法,其特征在于,所述镜像文件管理方法包括:
    接收用于指示写入第一镜像文件的写入指令;
    基于所述写入指令,在镜像文件管理系统包括的多个边缘云节点中,确定至少一个第一边缘云节点;
    将所述第一镜像文件所包括的多个数据块,写入所述至少一个第一边缘云节点,使得每个第一边缘云节点存储所述第一镜像文件所包括的多个数据块的部分或全部。
  2. 根据权利要求1所述的镜像文件管理方法,其特征在于,在所述接收用于指示写入第一镜像文件的写入指令之后,所述镜像文件管理方法还包括:
    生成所述第一镜像文件所包括的每个数据块的标识号;
    基于所述第一镜像文件所包括的每个数据块的标识号,生成所述第一镜像文件的第一标识号拓扑,所述第一标识号拓扑记录所述第一镜像文件包括的数据块;
    将所述第一标识号拓扑存储在所述镜像文件管理系统的边缘云节点中。
  3. 根据权利要求2所述的镜像文件管理方法,其特征在于,所述第一标识号拓扑还携带有所述第一镜像文件的版本信息。
  4. 根据权利要求1至3任一所述的镜像文件管理方法,其特征在于,在所述将所述第一镜像文件所包括的多个数据块,写入所述至少一个第一边缘云节点之后,所述镜像文件管理方法还包括:
    根据所述第一镜像文件所包括的每个数据块在所述至少一个第一边缘云节点中的存储情况,建立所述第一镜像文件所包括的每个数据块与存储有所述第一镜像文件中对应数据块的第一边缘云节点的存储对应关系;
    将所述存储对应关系存储在所述镜像文件管理系统的边缘云节点中。
  5. 根据权利要求1至4任一所述的镜像文件管理方法,其特征在于,在所述将所述第一镜像文件所包括的多个数据块,写入所述至少一个第一边缘云节点之前,所述方法还包括:
    确定所述第一镜像文件所包括的所有数据块中未存储在任一边缘云节点中的所述第一镜像文件所包括的多个数据块。
  6. 一种镜像文件管理方法,其特征在于,所述方法应用于目标边缘云节点,所述镜像文件管理方法包括:
    接收用于指示读取第二镜像文件的读取指令;
    基于所述读取指令,在镜像文件管理系统包括的多个边缘云节点中,确定存储有所述第二镜像文件的数据块的一个或多个第二边缘云节点;
    从所述一个或多个第二边缘云节点中读取属于所述第二镜像文件的数据块,以得到所述第二镜像文件。
  7. 根据权利要求6所述的镜像文件管理方法,其特征在于,所述基于所述读取指令,在镜像文件管理系统包括的多个边缘云节点中,确定存储有属于所述第二镜像文件的数据块的一个或多个第二边缘云节点,包括:
    基于所述读取指令,确定所述第二镜像文件所包括的所有数据块;
    在所述第二镜像文件所包括的所有数据块中,确定未存储在所述目标边缘云节点 中的一个或多个数据块;
    确定存储有所述一个或多个数据块中任一数据块的所述第二边缘云节点。
  8. 根据权利要求7所述的镜像文件管理方法,其特征在于,所述基于所述读取指令,确定所述第二镜像文件所包括的所有数据块,包括:
    基于所述读取指令,确定所述第二镜像文件的第二标识号拓扑,所述第二标识号拓扑记录所述第二镜像文件包括的数据块;
    基于所述第二标识号拓扑,确定所述第二镜像文件所包括的所有数据块。
  9. 根据权利要求6至8任一所述的镜像文件管理方法,其特征在于,所述从所述一个或多个第二边缘云节点中读取属于所述第二镜像文件的数据块,包括:
    当多个第二边缘云节点中均存储有所述第二镜像文件包括的某一数据块时,获取均存储有所述某一数据块的每个第二边缘节点的下载状态参数;
    基于均存储有所述某一数据块的每个第二边缘节点的下载状态参数,选择下载所述某一数据块的第二边缘云节点;
    从选择的第二边缘云节点中下载所述某一数据块。
  10. 根据权利要求6至9任一所述的镜像文件管理方法,其特征在于,所述镜像文件管理方法还包括:
    接收用于指示更新第三镜像文件的信息的更新指令,所述第三镜像文件是目标边缘云节点基于所述镜像文件管理系统中已存储的第四镜像文件生成的;
    生成所述第三镜像文件所包括的每个数据块的标识号;
    基于所述第三镜像文件所包括的每个数据块的标识号,生成所述第三镜像文件的第三标识号拓扑,所述第三标识号拓扑记录所述第三镜像文件包括的数据块;
    将所述第三标识号拓扑存储在所述镜像文件管理系统的边缘云节点中。
  11. 一种镜像文件管理装置,其特征在于,所述镜像文件管理装置包括:
    第一接收模块,用于接收用于指示写入第一镜像文件的写入指令;
    第一确定模块,用于基于所述写入指令,在镜像文件管理系统包括的多个边缘云节点中,确定至少一个第一边缘云节点;
    第一写入模块,用于将所述第一镜像文件所包括的多个数据块,写入所述至少一个第一边缘云节点,使得每个第一边缘云节点存储所述第一镜像文件所包括的多个数据块的部分或全部。
  12. 根据权利要求11所述的镜像文件管理装置,其特征在于,所述镜像文件管理装置还包括:
    第一生成模块,用于生成所述第一镜像文件所包括的每个数据块的标识号;
    所述第一生成模块,还用于基于所述第一镜像文件所包括的每个数据块的标识号,生成所述第一镜像文件的第一标识号拓扑,所述第一标识号拓扑记录所述第一镜像文件包括的数据块;
    所述第一写入模块,还用于将所述第一标识号拓扑存储在所述镜像文件管理系统的边缘云节点中。
  13. 根据权利要求12所述的镜像文件管理装置,其特征在于,所述第一标识号拓扑还携带有所述第一镜像文件的版本信息。
  14. 根据权利要求11至13任一所述的镜像文件管理装置,其特征在于,所述镜像文件管理装置还包括:
    第一建立模块,用于根据所述第一镜像文件所包括的每个数据块在所述至少一个第一边缘云节点中的存储情况,建立所述第一镜像文件所包括的每个数据块与存储有所述第一镜像文件中对应数据块的第一边缘云节点的存储对应关系;
    所述第一写入模块,还用于将所述存储对应关系存储在所述镜像文件管理系统的边缘云节点中。
  15. 根据权利要求11至14任一所述的镜像文件管理装置,其特征在于,所述第一确定模块,还用于确定所述第一镜像文件所包括的所有数据块中未存储在任一边缘云节点中的所述第一镜像文件所包括的多个数据块。
  16. 一种镜像文件管理装置,其特征在于,所述镜像文件管理装置包括:
    第二接收模块,用于接收用于指示读取第二镜像文件的读取指令;
    第二确定模块,用于基于所述读取指令,在镜像文件管理系统包括的多个边缘云节点中,确定存储有所述第二镜像文件的数据块的一个或多个第二边缘云节点;
    读取模块,用于从所述一个或多个第二边缘云节点中读取属于所述第二镜像文件的数据块,以得到所述第二镜像文件。
  17. 根据权利要求16所述的镜像文件管理装置,其特征在于,所述第二确定模块,具体用于:
    基于所述读取指令,确定所述第二镜像文件所包括的所有数据块;
    在所述第二镜像文件所包括的所有数据块中,确定未存储在所述目标边缘云节点中的一个或多个数据块;
    确定存储有所述一个或多个数据块中任一数据块的所述第二边缘云节点。
  18. 根据权利要求17所述的镜像文件管理装置,其特征在于,所述第二确定模块,具体用于:
    基于所述读取指令,确定所述第二镜像文件的第二标识号拓扑,所述第二标识号拓扑记录所述第二镜像文件包括的数据块;
    基于所述第二标识号拓扑,确定所述第二镜像文件所包括的所有数据块。
  19. 根据权利要求16至18任一所述的镜像文件管理装置,其特征在于,所述读取模块,具体用于:
    当多个第二边缘云节点中均存储有所述第二镜像文件包括的某一数据块时,获取均存储有所述某一数据块的每个第二边缘节点的下载状态参数;
    基于均存储有所述某一数据块的每个第二边缘节点的下载状态参数,选择下载所述某一数据块的第二边缘云节点;
    从选择的第二边缘云节点中下载所述某一数据块。
  20. 根据权利要求16至19任一所述的镜像文件管理装置,其特征在于,所述第二接收模块,还用于:接收用于指示更新第三镜像文件的信息的更新指令,所述第三镜像文件是目标边缘云节点基于所述镜像文件管理系统中已存储的第四镜像文件生成的;
    所述镜像文件管理装置还包括:
    第二生成模块,用于生成所述第三镜像文件所包括的每个数据块的标识号;
    所述第二生成模块,还用于基于所述第三镜像文件所包括的每个数据块的标识号,生成所述第三镜像文件的第三标识号拓扑,所述第三标识号拓扑记录所述第三镜像文件包括的数据块;
    第二写入模块,用于将所述第三标识号拓扑存储在所述镜像文件管理系统的边缘云节点中。
  21. 一种镜像文件管理系统,其特征在于,所述镜像文件管理包括:终端和多个边缘云节点,所述边缘云节点用于存储镜像文件;
    其中,所述终端用于执行权利要求1至5任一所述的镜像文件管理方法,所述边缘云节点用于执行权利要求6至10任一所述的镜像文件管理方法;
    或者,所述终端用于向所述边缘云节点发送用于指示写入第一镜像文件的写入指令,所述边缘云节点用于执行权利要求1至5任一、或权利要求6至10任一所述的镜像文件管理方法。
  22. 一种计算机设备,其特征在于,所述计算机设备包括处理器和存储器;
    所述存储器中存储有计算机程序;
    所述处理器执行所述计算机程序时,所述计算机设备实现权利要求1至5中任一所述的镜像文件管理方法。
  23. 一种计算机设备,其特征在于,所述计算机设备包括处理器和存储器;
    所述存储器中存储有计算机程序;
    所述处理器执行所述计算机程序时,所述计算机设备实现权利要求6至10中任一所述的镜像文件管理方法。
  24. 一种存储介质,其特征在于,当所述存储介质中的指令被处理器执行时,实现权利要求1至5中任一所述的镜像文件管理方法。
  25. 一种存储介质,其特征在于,当所述存储介质中的指令被处理器执行时,实现权利要求6至10中任一所述的镜像文件管理方法。
PCT/CN2020/118047 2019-09-26 2020-09-27 镜像文件管理方法、装置及系统、计算机设备、存储介质 WO2021057956A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20868383.9A EP4030724A4 (en) 2019-09-26 2020-09-27 METHOD, DEVICE AND SYSTEM FOR MANAGEMENT OF A MIRROR IMAGE FILE, COMPUTER DEVICE AND STORAGE MEDIUM
US17/704,730 US20220215002A1 (en) 2019-09-26 2022-03-25 Image File Management Method, Apparatus, and System, Computer Device, and Storage Medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910918335.4A CN112565325B (zh) 2019-09-26 2019-09-26 镜像文件管理方法、装置及系统、计算机设备、存储介质
CN201910918335.4 2019-09-26

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/704,730 Continuation US20220215002A1 (en) 2019-09-26 2022-03-25 Image File Management Method, Apparatus, and System, Computer Device, and Storage Medium

Publications (1)

Publication Number Publication Date
WO2021057956A1 true WO2021057956A1 (zh) 2021-04-01

Family

ID=75029935

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/118047 WO2021057956A1 (zh) 2019-09-26 2020-09-27 镜像文件管理方法、装置及系统、计算机设备、存储介质

Country Status (4)

Country Link
US (1) US20220215002A1 (zh)
EP (1) EP4030724A4 (zh)
CN (1) CN112565325B (zh)
WO (1) WO2021057956A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115277728A (zh) * 2022-07-01 2022-11-01 阳光电源股份有限公司 一种模型文件的下发方法、系统及云服务器

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11463250B2 (en) * 2020-12-14 2022-10-04 Kyndryl, Inc. Sharing data among different service providers at edge level through collaboration channels
CN115208872A (zh) * 2021-04-09 2022-10-18 顺丰科技有限公司 边缘云镜像数据处理方法、装置、计算机设备和存储介质
CN113676510A (zh) * 2021-07-13 2021-11-19 阿里巴巴新加坡控股有限公司 一种云存储方法、系统、设备及存储介质
CN114844879B (zh) * 2022-03-25 2024-04-12 网宿科技股份有限公司 镜像管理方法、电子设备和存储介质
CN117061503B (zh) * 2023-10-11 2024-01-26 腾讯科技(深圳)有限公司 镜像文件管理方法、装置、计算机可读介质及电子设备
CN117295199B (zh) * 2023-11-14 2024-06-25 深圳市铭灏天智能照明设备有限公司 运用边缘侧计算的物联网车库照明系统
CN117459526A (zh) * 2023-12-25 2024-01-26 中建三局集团有限公司 无固定公网基于云边端的模型协同传输方法、系统及介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102082832A (zh) * 2011-01-24 2011-06-01 中国人民解放军国防科学技术大学 一种分布式文件系统中的数据存储方法和装置
US8352941B1 (en) * 2009-06-29 2013-01-08 Emc Corporation Scalable and secure high-level storage access for cloud computing platforms
CN105335513A (zh) * 2015-10-30 2016-02-17 迈普通信技术股份有限公司 一种分布式文件系统及文件存储方法
CN105516367A (zh) * 2016-02-02 2016-04-20 北京百度网讯科技有限公司 分布式数据存储系统、方法和装置
CN107528924A (zh) * 2017-10-09 2017-12-29 郑州云海信息技术有限公司 一种分布式集群元数据服务部署方法及系统
US20180357017A1 (en) * 2017-06-12 2018-12-13 Pure Storage, Inc. Accessible fast durable storage integrated into a bulk storage device
CN109104464A (zh) * 2018-07-23 2018-12-28 东南大学 一种边缘计算环境下面向协同存储的分布式数据更新方法

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130254248A1 (en) * 2012-03-23 2013-09-26 Alcatel-Lucent Usa Inc. Method And Apparatus For A Distributed File System In A Cloud Network
CN103078957B (zh) * 2013-02-01 2016-03-02 北京航空航天大学 支持跨idc域功能的数据中心镜像分发系统
CN105205143B (zh) * 2015-09-18 2019-10-25 杭州华为数字技术有限公司 一种文件存储及处理方法、设备和系统
US11030187B1 (en) * 2016-05-18 2021-06-08 Salesforce.Com, Inc. Distributed database systems and structures
CN107426258B (zh) * 2016-05-23 2020-04-28 华为技术有限公司 一种镜像文件的上传和下载方法及装置
CN107483573A (zh) * 2017-08-08 2017-12-15 郑州云海信息技术有限公司 云平台中镜像文件的传输方法和装置
CN108173953A (zh) * 2017-12-29 2018-06-15 北京奇虎科技有限公司 一种基于内容分发网络cdn的文件分发方法和装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8352941B1 (en) * 2009-06-29 2013-01-08 Emc Corporation Scalable and secure high-level storage access for cloud computing platforms
CN102082832A (zh) * 2011-01-24 2011-06-01 中国人民解放军国防科学技术大学 一种分布式文件系统中的数据存储方法和装置
CN105335513A (zh) * 2015-10-30 2016-02-17 迈普通信技术股份有限公司 一种分布式文件系统及文件存储方法
CN105516367A (zh) * 2016-02-02 2016-04-20 北京百度网讯科技有限公司 分布式数据存储系统、方法和装置
US20180357017A1 (en) * 2017-06-12 2018-12-13 Pure Storage, Inc. Accessible fast durable storage integrated into a bulk storage device
CN107528924A (zh) * 2017-10-09 2017-12-29 郑州云海信息技术有限公司 一种分布式集群元数据服务部署方法及系统
CN109104464A (zh) * 2018-07-23 2018-12-28 东南大学 一种边缘计算环境下面向协同存储的分布式数据更新方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4030724A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115277728A (zh) * 2022-07-01 2022-11-01 阳光电源股份有限公司 一种模型文件的下发方法、系统及云服务器

Also Published As

Publication number Publication date
CN112565325A (zh) 2021-03-26
EP4030724A4 (en) 2022-11-16
CN112565325B (zh) 2022-09-23
EP4030724A1 (en) 2022-07-20
US20220215002A1 (en) 2022-07-07

Similar Documents

Publication Publication Date Title
WO2021057956A1 (zh) 镜像文件管理方法、装置及系统、计算机设备、存储介质
US10896102B2 (en) Implementing secure communication in a distributed computing system
US9489443B1 (en) Scheduling of splits and moves of database partitions
US9053167B1 (en) Storage device selection for database partition replicas
US6782389B1 (en) Distributing files across multiple, permissibly heterogeneous, storage devices
US11080253B1 (en) Dynamic splitting of contentious index data pages
US8930364B1 (en) Intelligent data integration
TW201935243A (zh) 固態驅動器、分散式資料儲存系統和利用鍵值儲存的方法
KR20210075845A (ko) 네이티브 키-밸류 분산 스토리지 시스템
JP2014525082A (ja) 規模変更可能なデータ記憶サービスを実装するためのシステムおよび方法
US9983823B1 (en) Pre-forking replicas for efficient scaling of a distribued data storage system
US11741144B2 (en) Direct storage loading for adding data to a database
CN107153512B (zh) 一种数据迁移方法和装置
US10831714B2 (en) Consistent hashing configurations supporting multi-site replication
US9223517B1 (en) Scalable index store
WO2023179784A1 (zh) 数据处理方法以及装置
EP4016312B1 (en) Data operations using a cache table in a file system
JP7398567B2 (ja) 動的適応型パーティション分割
US12026177B2 (en) Dynamic adaptive partition splitting
US11755538B2 (en) Distributed management of file modification-time field
WO2021110176A1 (zh) 一种边缘系统及数据操作请求的处理方法
JP2005234919A (ja) クラスタメモリファイルシステム
Babu Concurrency Control In Deduplication Enabled Cloud Storage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20868383

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020868383

Country of ref document: EP

Effective date: 20220412