CN109976667B - Mirror image management method, device and system - Google Patents

Mirror image management method, device and system Download PDF

Info

Publication number
CN109976667B
CN109976667B CN201910145316.2A CN201910145316A CN109976667B CN 109976667 B CN109976667 B CN 109976667B CN 201910145316 A CN201910145316 A CN 201910145316A CN 109976667 B CN109976667 B CN 109976667B
Authority
CN
China
Prior art keywords
node
mirror image
metadata
nodes
warehouse server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910145316.2A
Other languages
Chinese (zh)
Other versions
CN109976667A (en
Inventor
线超博
刘基
罗浩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Cloud Computing Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201910145316.2A priority Critical patent/CN109976667B/en
Publication of CN109976667A publication Critical patent/CN109976667A/en
Application granted granted Critical
Publication of CN109976667B publication Critical patent/CN109976667B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45562—Creating, deleting, cloning virtual machine instances
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45587—Isolation or security of virtual machine instances

Abstract

The embodiment of the invention provides a mirror image management method, a device and a system, relates to the technical field of computers, and aims to reduce the transmission times of mirror image data and improve the utilization rate of a storage space, thereby reducing the overhead of a docker system. The system comprises: the system comprises a warehouse server and N nodes connected with the warehouse server, wherein storage modules of the N nodes form a distributed storage system, and mirror image data of mirror images in the N nodes are shared in the distributed storage system; wherein the first node is configured to: generating a first image, wherein the first image comprises first metadata and first image data; saving the first mirror image data to the distributed storage system; sending an upload request to the warehouse server, wherein the upload request carries the first metadata, and the first node is any one of the N nodes; the warehouse server is configured to: receiving the uploading request sent by the first node; and saving the first metadata in the upload request.

Description

Mirror image management method, device and system
Technical Field
The invention relates to the technical field of computers, in particular to a mirror image management method, device and system.
Background
The docker is an application hosting framework (application hosting framework) that can deploy and manage applications through a container similar to a virtual machine.
In the current docker system, each host is connected to a docker registry (i.e., a warehouse server) at a remote end, and when performing docker instantiation operation, each host needs to download (pull) a required image (i.e., image) from the warehouse server and store the image in a local warehouse; correspondingly, if the host modifies the mirror image, and a new mirror image is obtained, the new mirror image needs to be uploaded (push) to the warehouse server.
Specifically, the mirror may be hierarchically stored in the repository server in a tree form, and each node may form a mirror by branching to the root node, such as a binary tree shown in fig. 1, which includes: the image processing method includes a plurality of images, namely an image 1 formed by a node 1 and a node 2, an image 2 formed by the node 1, the node 2 and a node 4, and an image 3 formed by the node 1 and a node 3, wherein each image can be specifically described by image data and metadata, wherein the image data includes data of a complete Linux operating system, and the metadata is generally described by a JSON (JavaScript Object notification, a lightweight data interchange format) file and is used for representing attribute characteristics of the image, such as information of the name of the image, the ID of a leaf node, and the like.
In the docker system, the number of the images may be very large, and therefore, when a plurality of hosts download the images, not only image data of the images but also metadata of the images are downloaded, which increases the overhead of the entire docker system, and different hosts may use the same image.
Disclosure of Invention
Embodiments of the present invention provide a mirror image management method, apparatus, and system, which can reduce the transmission times of mirror image data and improve the utilization rate of a storage space, thereby reducing the overhead of a docker system.
In order to achieve the above purpose, the embodiment of the invention adopts the following technical scheme:
in a first aspect, an embodiment of the present invention provides a docker system, including: the system comprises a warehouse server and N nodes connected with the warehouse server, wherein storage modules of the N nodes form a distributed storage system, mirror image data of mirror images in the N nodes are shared in the distributed storage system, and N is an integer greater than or equal to 2; wherein the first node is configured to: generating a first image, wherein the first image comprises first metadata and first image data; saving the first mirror image data to the distributed storage system; sending an upload request to the warehouse server, wherein the upload request carries the first metadata, and the first node is any one of the N nodes; the warehouse server is configured to: receiving the uploading request sent by the first node; and saving the first metadata in the upload request.
It can be seen that, since the mirror image data of the mirror images in the N nodes in the docker system is shared in the distributed storage system, in the mirror image uploading process, the first node only needs to upload the generated metadata of the new mirror image to the warehouse server, and does not need to upload the mirror image data of the new mirror image, thereby reducing the transmission times of the mirror image data in the docker system, improving the utilization rate of the storage space, and reducing the overhead of the whole docker system.
In one possible design, the warehouse server includes a warehouse driver; the first node is further configured to send the first mirror image data to the repository server; the warehouse server is further configured to: receiving the first mirror image data sent by the first node; and invoking the warehouse driver to save the first mirror image data to the distributed storage system.
In one possible design, the first node includes a mirror management module, where the first node is further configured to: sending a downloading request to the warehouse server through the mirror image management module, wherein the downloading request carries an identifier of the first metadata; receiving the first metadata sent by the warehouse server through the mirror image management module, and searching whether first mirror image data corresponding to the first metadata are stored in the distributed storage system; if the first mirror image data is stored in the distributed storage system, the first mirror image data is acquired from the distributed storage system through the mirror image management module; the warehouse server is further configured to: receiving the downloading request sent by the first node; searching the first metadata according to the identifier of the first metadata in the downloading request; and sending the first metadata to the first node.
Therefore, mirror image data of mirror images in the N nodes in the docker system are shared in the distributed storage system, and therefore only metadata of the mirror images need to be transmitted between the first node and the warehouse server in the process of downloading the mirror images, mirror image data of the mirror images do not need to be transmitted, transmission times of the mirror image data can be reduced, the utilization rate of storage space is improved, and therefore the expense of the docker system is reduced.
In one possible design, the first node is further configured to: saving the first metadata to a local repository within the first node; and performs an add 1 operation on a reference counter preset for the first mirror.
In one possible design, the first node is further configured to: deleting the first metadata in the first node; a subtract 1 operation is performed on the reference counter.
In one possible design, the first node is further configured to send a delete request to the repository server, where the delete request is used to instruct deletion of the first image in the repository server; the warehouse server is further configured to: receiving the deletion request sent by the first node; deleting the first metadata in the repository server according to the deletion request; and if the first mirror image is not used by each node, deleting the first mirror image data stored in the distributed storage system.
Therefore, in the docker system provided in the embodiment of the present invention, mirror image data and metadata of a mirror image can be managed respectively, and mirror image data of a mirror image in each node is stored in a distributed storage system in a unified manner, that is, each node can share mirror image data in the distributed storage system, so that not only can the problem of storage space waste caused by storing the same mirror image in a large number of nodes in the prior art be avoided, but also each node and a warehouse server only need to manage metadata of a mirror image in the node, and thus transmission times of mirror image data can be reduced, and overhead of the docker system is reduced.
In a second aspect, an embodiment of the present invention provides an image management method, where the method is applied to any docker system in the first aspect, and the method includes: a first node generates a first mirror image, wherein the first mirror image comprises first metadata and first mirror image data, and the first node is any one of N nodes in the docker system; the first node sends an uploading request to a warehouse server, wherein the uploading request carries the first metadata, so that the warehouse server stores the first metadata; the first node saves the first mirrored data to the distributed storage system.
In one possible design, the first node saving the first mirrored data to the distributed storage system includes: the first node sends the first mirror image data to the warehouse server, so that the warehouse server saves the first mirror image data to the distributed storage system.
In one possible design, the method further includes: the first node sends a downloading request to the warehouse server, wherein the downloading request carries an identifier of the first metadata; the first node receives first metadata which is sent by the warehouse server and corresponds to the identifier of the first metadata; the first node searches whether first mirror image data corresponding to the first metadata are stored in the distributed storage system or not according to the first metadata; if the first mirror image data is stored in the distributed storage system, the first node acquires the first mirror image data from the distributed storage system.
In a possible design, after the first node searches whether the first mirror data corresponding to the first metadata is stored in the distributed storage system according to the first metadata sent by the repository server, the method further includes: and if the first mirror image data is not stored in the distributed storage system, the first node acquires the first mirror image data from the warehouse server.
In one possible design, after the first node receives the first metadata sent by the repository server, the method further includes: the first node saves the first metadata to a local store within the first node.
In one possible design, after the first node saves the first metadata to a local repository within the first node, the method further includes: the first node performs an add 1 operation on a reference counter preset for the first mirror.
In one possible design, if the first node determines to delete the first image in the first node, the method further includes: the first node deleting the first metadata in the first node; the first node performs a subtract 1 operation on the reference counter.
In a third aspect, an embodiment of the present invention provides an image management method, where the method is applied to any docker system in the first aspect, and the method includes: the method comprises the steps that a warehouse server receives a deleting request sent by a first node, wherein the deleting request is used for indicating that a first mirror image in the warehouse server is deleted, the first mirror image comprises first metadata and first mirror image data, and the first node is any one of N nodes in the docker system; the warehouse server deletes the first metadata in the warehouse server according to the deletion request; and if the first mirror image is not used by each node, deleting the first mirror image data stored in the distributed storage system by the warehouse server.
In one possible design, if none of the nodes uses the first image, the repository server deleting the first image data stored in the distributed storage system, including: the warehouse server judges whether the reference counter of the first mirror image is 0; if the reference counter of the first image is 0, the warehouse server deletes the first image data stored in the distributed storage system.
In a fourth aspect, an embodiment of the present invention provides a node, where the node is applied to any docker system in the first aspect, and the node includes: a mirror image generation unit configured to generate a first mirror image, the first mirror image including first metadata and first mirror image data; a sending unit, configured to send an upload request to a warehouse server, where the upload request carries the first metadata, so that the warehouse server stores the first metadata; and the storage unit is used for storing the first mirror image data into the distributed storage system.
In a possible design, the sending unit is further configured to send the first mirrored data to the repository server, so that the repository server stores the first mirrored data in the distributed storage system.
In a possible design, the node further includes an obtaining unit and a searching unit, where the sending unit is further configured to send a download request to the warehouse server, where the download request carries an identifier of the first metadata; the acquisition unit is used for receiving first metadata which is sent by the warehouse server and corresponds to the identifier of the first metadata; if the first mirror image data is stored in the distributed storage system, the first mirror image data is obtained from the distributed storage system; the searching unit is used for searching whether the distributed storage system stores first mirror image data corresponding to the first metadata or not according to the first metadata.
In a possible design, the obtaining unit is further configured to obtain the first mirror data from the repository server if the first mirror data is not stored in the distributed storage system.
In one possible design, the saving unit is further configured to save the first metadata to a local repository in the first node.
In one possible design, the node further includes: and the counting unit is used for executing 1 adding operation on a reference counter preset for the first mirror image.
In one possible design, the node further includes a deletion unit, where the deletion unit is configured to delete the first metadata in the first node; the counting unit is further configured to perform a 1 subtraction operation on the reference counter.
In a fifth aspect, an embodiment of the present invention provides a warehouse server applied to any docker system in the first aspect, the warehouse server including: a receiving unit, configured to receive a deletion request sent by a first node, where the deletion request is used to instruct to delete a first mirror image in the repository server, the first mirror image includes first metadata and first mirror image data, and the first node is any one of N nodes in the docker system; a deleting unit, configured to delete the first metadata in the repository server according to the deletion request; and if the first mirror image is not used by each node, deleting the first mirror image data stored in the distributed storage system.
In a possible design, the warehouse server further includes a determining unit, where the determining unit is configured to determine whether the reference counter of the first mirror is 0; the deleting unit is specifically configured to delete the first mirror image data stored in the distributed storage system if the reference counter of the first mirror image is 0.
In a sixth aspect, an embodiment of the present invention provides a node, including: a processor, a memory, a bus, and a communication interface; the memory is used for storing computer execution instructions, the processor is connected with the memory through the bus, and when the node runs, the processor executes the computer execution instructions stored by the memory, so that the node executes the image management method according to any one of the second aspect.
In a seventh aspect, an embodiment of the present invention provides a repository server, including: a processor, a memory, a bus, and a communication interface; the storage is used for storing computer execution instructions, the processor is connected with the storage through the bus, and when the warehouse server runs, the processor executes the computer execution instructions stored by the storage, so that the warehouse server executes the image management method according to any one of the third aspect.
In an eighth aspect, an embodiment of the present invention provides a computer storage medium for storing computer software instructions for the node, which includes a program designed for executing the above aspect for the node.
In a ninth aspect, an embodiment of the present invention provides a computer storage medium for storing computer software instructions for the warehouse server, which includes a program designed for executing the above aspect for the warehouse server.
In the present invention, the names of the nodes and the repository server do not limit the devices themselves, and in actual implementation, the devices may appear by other names. Provided that the respective devices function similarly to the present invention, are within the scope of the claims of the present invention and their equivalents.
In addition, the technical effects brought by any one of the possible design manners in the second aspect to the ninth aspect can be referred to the technical effects brought by the different design manners in the first aspect, and are not described herein again.
These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a schematic diagram of a prior art mirror image;
fig. 2 is a first schematic diagram of an architecture of a docker system according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a structure of a docker system according to an embodiment of the present invention;
fig. 4 is a third schematic diagram of an architecture of a docker system according to an embodiment of the present invention;
FIG. 5 is an architecture diagram of a node in a docker system in the prior art;
fig. 6 is an architecture diagram of a node in a docker system according to an embodiment of the present invention;
fig. 7 is a flowchart illustrating an image downloading method according to an embodiment of the present invention;
fig. 8 is a schematic flowchart of a mirror image uploading method according to an embodiment of the present invention;
fig. 9 is a schematic flowchart of a mirror deletion method according to an embodiment of the present invention;
fig. 10 is a first schematic structural diagram of a node according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a node according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of a node according to an embodiment of the present invention;
fig. 13 is a schematic structural diagram of a node according to a fourth embodiment of the present invention;
fig. 14 is a first schematic structural diagram of a warehouse server according to an embodiment of the present invention;
fig. 15 is a schematic structural diagram of a warehouse server according to an embodiment of the present invention;
fig. 16 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
In addition, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless otherwise specified.
The mirror image management method provided by the embodiment of the present invention is applied to a docker system shown in fig. 2, where the docker system includes a warehouse server 01 and N (N is an integer greater than or equal to 2) nodes 02 connected to the warehouse server 01, where the warehouse server 01 is a warehouse storing mirror images, and the nodes 02 may obtain required mirror images from the warehouse server 01, where each mirror image specifically includes mirror image data and metadata. The mirrored data of the mirror images in the nodes 02 can be uniformly stored in the distributed storage system 03, that is, the mirrored data stored in the distributed storage system 03 is shared by the nodes 02, and the metadata of the mirror images in each node 02 is stored and maintained by each node 02.
The image data includes data of a complete Linux operating system, and the metadata is usually described by a JSON (JavaScript Object notification, a lightweight data exchange format) file, and is used to characterize attribute characteristics of the image, such as information of the name of the image, the ID of the leaf node, and the like.
It should be noted that, in the present embodiment, any node 02 in the distributed storage system may be implemented by one physical device, such as a computer, a portable mobile terminal, or a server, or may be implemented by multiple physical devices together. It is understood that any functional node 02 in the distributed storage system may be a logical functional module in an entity device, or may be a logical functional module composed of a plurality of entity devices. Therefore, in the following embodiments of the present invention, one entity device may perform each step in the method provided in the embodiments of the present invention, and a plurality of entity devices may cooperate to perform each step in the method provided in the embodiments of the present invention, which is not limited to this invention.
In addition, the specific structure of the node 02 in the docker system will be described in detail in the following embodiments, and thus will not be described herein again.
For example, as shown in fig. 3, a first node stores metadata 1 of a mirror image 1, a second node stores metadata 1 of the mirror image 1 and metadata 2 of the mirror image 2, and the mirror image data 1 of the mirror image 1 and the mirror image data 2 of the mirror image 2 are both stored in a distributed storage system 03 in a unified manner, so that when the first node 1 needs to download (pull) the mirror image data of the mirror image 2, the first node only needs to obtain the metadata 2 of the mirror image 2 from a warehouse server 01 and store the metadata to the local of the first node, and then reads the mirror image data 2 of the mirror image 2 from the distributed storage system 03 according to the metadata 2 of the mirror image 2.
It can be seen that, based on the above docker system, when downloading a required mirror image, the node 02 only needs to obtain metadata of the mirror image from the warehouse server 01, and does not need to obtain metadata and mirror image data of the mirror image from the warehouse server 01 at the same time as in the prior art, thereby reducing the overhead of the docker system.
Meanwhile, since the mirror image data of the mirror images in each node 02 are uniformly stored in the distributed storage system 03, the mirror image data of each mirror image is only used for storing one mirror image in the distributed storage system 03, thereby avoiding the problem of storage space waste caused by the fact that the same mirror image is stored in a large number of nodes in the prior art.
Of course, similar to the prior art, the partial node 02 in the docker system shown in fig. 2 may also store the mirrored image data in the repository server 01, and at this time, the partial node 02 may still upload the mirrored image data and the metadata to the repository server 01 and download the mirrored image data and the metadata from the repository server 01 according to the method in the prior art, which is not limited in this respect.
Further, the storage unit for storing the mirror image data in the warehouse server 01 may also be disposed in the distributed storage system 03, that is, as shown in fig. 4, the warehouse driver 04 may be disposed to store the mirror image data stored in the warehouse server 01 in the distributed storage system 03, so that when the node 02 generates a certain mirror image, since the mirror image data of the mirror image may be directly generated in the distributed storage system 03, when the node 02 uploads (push) the mirror image to the warehouse server 01, only the metadata of the mirror image needs to be sent to the warehouse server 01, thereby avoiding that the node 02 uploads the mirror image data to the warehouse server 01, and further reducing the transmission frequency of the mirror image data, thereby reducing the overhead of the docker system.
Further, as shown in fig. 5, it is an architecture diagram of a node 02 in the docker system. Wherein, node 02 in the docker system specifically includes: a client 11 (i.e., Docker client), a server 12, a driver 13 (i.e., Docker driver), a container management library 14 (i.e., libcontaner), a container 15 (i.e., Docker container), and an underlying module 16 (i.e., rootfs) supporting the above architecture.
Among them, the client 11 (i.e., docker client) is a client in the docker architecture, which is used by a user to establish communication with the server 12. After the user sends a container management request through the client 11 by using the docker command line, the server 12 receives and processes the container management request.
The server 12 (i.e., docker daemon) may specifically include a running engine in the docker architecture, i.e., docker engine, and a docker server specifically serving the client 11.
The docker server may receive and schedule various requests sent by the distribution client 11, and specifically includes an HTTP server, a Router (Router), and a processing module (Handler).
And the docker engine is responsible for the image and the life cycle management of the container, such as the downloading (pull) and uploading (push) operations of the image, the instantiation operation of the container and the like. In addition, docker engine may communicate with warehouse server 01 in fig. 2 or fig. 4.
The driver 13 (i.e., docker driver) may implement setting of the docker container execution environment. In the lifecycle of the Docker operation, not all the operations of the user are directed to the management of the Docker container, and in addition, the acquisition of information about the Docker operation, and the like, therefore, in order to distinguish the management of the Docker container from the business logic inside the Docker engine, the driver 13 is provided to manage the management requests of the non-Docker containers. Among them, in the implementation of docker driver, it can be classified into the following three types of driving: graph driver, network driver, and executive driver.
The container management library 14 (i.e. libcontaner) is a library implemented in the docker architecture by using Go language design, and by calling libcontaner, the namespace, cgroups, appramor, network device, firewall rules, etc. of the container can be manipulated.
The container 15 (i.e., Docker container) is a final embodiment of service delivery in the Docker framework, and may customize a corresponding Docker container according to a requirement and an instruction of a user.
The underlying module 16, i.e., a rootfs (root file system), contains standard directories and files in a typical Linux system, which may also be referred to as a root file system.
Based on the docker architecture diagram shown in fig. 5, as shown in fig. 6, the node 02 provided in the embodiment of the present invention is further provided with an image management module 100 (i.e., an Images Manager), and the image management module 100 may communicate with the server 12 and the warehouse server 01 respectively to serve as an agent for the client 11 to access the warehouse server 01.
In addition, the mirror image management module 100 is further configured to maintain metadata of the mirror image on the node 02, maintain the number of times of reference of the mirror image in the distributed storage system 03, determine whether mirror image data needed by the node 02 exists in the distributed storage system 03, and the like.
Hereinafter, a mirror image management method provided by the embodiment of the present invention will be described in detail with reference to the docker system provided in fig. 2 to fig. 6, where the mirror image management method provided by the embodiment of the present invention may specifically relate to a mirror image downloading (pull) method, a mirror image uploading (push) method, and a mirror image deleting (delete) method, and the following examples are described in detail.
As shown in fig. 7, an embodiment of the present invention provides an image downloading method, including:
101. and the first node sends a downloading request to the warehouse server, wherein the downloading request carries the identifier of the first metadata.
102. The warehouse server searches the first metadata corresponding to the identifier of the first metadata according to the downloading request.
103. The first node receives the first metadata sent by the warehouse server,
104. the first node saves the first metadata to a local repository within the first node.
105. The first node performs an add 1 operation on the reference counter preset for the first mirror.
106. And the first node searches whether the distributed storage system stores first mirror image data corresponding to the first metadata or not according to the first metadata.
107. And if the distributed storage system stores the first mirror image data, the first node acquires the first mirror image data from the distributed storage system.
In step 101, when a first node (the first node is any node in the docker system) needs to use a first image (the first image includes first image data and first metadata), and the first node does not store the first metadata of the first image, the first node sends a download request to the warehouse server, where the download request carries an identifier of the first metadata, for example, an ID of the first metadata.
For example, as shown in fig. 3, the node 1 stores metadata of the image 1, the node 2 stores metadata of the image 1 and the image 2, and the image data of the image 1 and the image 2 are all stored in the distributed storage system 03 in a unified manner, so that when the node 1 needs the image data of the image 2, if it is determined that the node itself does not store the metadata of the image 2, a download request is sent to the repository server, where the download request carries an identifier of the metadata of the image 2.
Specifically, as shown in fig. 6, the client 11 in the first node generates a download request and sends the download request to the server 12, and the image management module 100 proxies the server 12 to communicate with the repository server and sends the download request to the repository server.
In step 102, after receiving the download request sent by the first node, the warehouse server searches for the first metadata corresponding to the identifier of the first metadata according to the identifier of the first metadata carried in the download request. The warehouse server stores metadata of all images in the docker system.
Further, the warehouse server sends the found first metadata to the mirror management module 100 of the first node.
In step 103, the image management module 100 receives the first metadata sent by the repository server.
Furthermore, in step 104, the first node stores the first metadata to a local repository in the first node, and further obtains the first mirror image data by performing step 105 and step 106, so that when the first node needs the first mirror image data again in the following, the first mirror image data can be obtained from the distributed storage system by searching the locally stored first metadata directly according to attributes such as the name or leaf ID of the first mirror image in the first metadata.
In step 105, if the image management module 100 receives the first metadata sent by the repository server, the first node may further perform an increment operation of 1 on the reference counter of the first image.
Specifically, in the docker system, a reference counter may be preset for each image to record the number of times of use of the image, for the reference counter of a certain image, the reference counter may be set in the distributed storage system, N nodes in the docker system may share the reference counter, when a node performs an image downloading operation, the image management module 100 adds 1 to the reference counter of the corresponding image, because the same image may be used by multiple nodes at the same time, the reference counter of the image may be used to indicate the number of times of use of the image, and when the counter is 0, it indicates that no node uses the image.
It should be noted that the steps 104-105 are optional steps, and the execution order of the steps 104 and 105 is not limited.
In step 106, the mirror image management module 100 may establish communication with the server 12 according to the first metadata sent by the repository server, and further search whether the distributed storage system stores the first mirror image data corresponding to the first metadata.
Since the two docker systems shown in fig. 2 and fig. 4 are provided in the embodiment of the present invention, based on the docker system shown in fig. 2, since the storage unit in the warehouse server for storing the mirror image data is not set in the distributed storage system, the first mirror image data of the first mirror image required by the first node may be stored in the distributed storage system, and may also be stored in the warehouse server, at this time, after the first node receives the first metadata, because the first metadata carries the attribute features such as the name and the label of the first mirror image data, the first node may search whether the first mirror image data corresponding to the first metadata is stored in the distributed storage system according to the attribute features of the first mirror image data.
Further, if the distributed storage system stores the first mirror image data, in step 107, the first node obtains the first mirror image data from the distributed storage system, and if the distributed storage system does not store the first mirror image data, the first node may obtain the first mirror image data corresponding to the first metadata from the repository server.
In addition, based on the docker system shown in fig. 4, since the storage unit for storing the mirror image data in the warehouse server may also be disposed in the distributed storage system, that is, the mirror image data stored in the warehouse server and the mirror image data in each node are both located in the distributed storage system, after receiving the first metadata, the first node may directly obtain the first mirror image data from the distributed storage system according to the attribute feature of the first mirror image data carried in the first metadata.
To this end, based on the docker system, an embodiment of the present invention provides a mirror image downloading method, where a first node sends a downloading request to a warehouse server, where the downloading request carries an identifier of first metadata, so as to search, according to the downloading request, the warehouse server for the first metadata corresponding to the identifier of the first metadata; furthermore, if the first metadata sent by the warehouse server is received, the first node acquires the first mirror image data corresponding to the first metadata from the distributed storage system according to the first metadata, and therefore in the process of downloading the mirror image, since the mirror image data of the mirror image in each node in the docker system are stored in the distributed storage system, only the metadata of the mirror image needs to be transmitted between the first node and the warehouse server, and the mirror image data of the mirror image does not need to be transmitted, the transmission times of the mirror image data can be reduced, the utilization rate of the storage space is improved, and the overhead of the docker system is reduced.
In addition, based on step 101-107 in the above embodiment, the embodiment of the present invention further provides a mirror deletion method, where the method is used to delete a mirror in any node in the docker system.
Illustratively, taking the example of deleting the first image in the first node in the above embodiment as an example, the first image includes the first metadata and the first image data.
Specifically, the client 11 in the first node sends a deletion request for deleting the first mirror image to the server 12, the server 12 forwards the deletion request to the mirror image management module 100, the mirror image management module 100 deletes the first metadata stored in the first node according to the deletion request, and further, the mirror image management module 100 performs a subtraction operation of 1 on a reference counter of the first mirror image in the distributed storage system to indicate that the number of times of use of the first mirror image is subtracted by 1.
It can be seen that, because the first mirror image can be used by multiple nodes in the distributed storage system at the same time, the first node does not need to delete the first mirror image data of the first mirror image, and only needs to delete the first metadata stored in the first node, and subtract 1 from the reference counter of the first mirror image.
Further, as shown in fig. 8, taking the example that the first node uploads the first image to the repository server, an embodiment of the present invention provides an image uploading method, including:
201. the first node generates a first image, which includes first image data and first metadata.
202. The first node sends an uploading request to the warehouse server, wherein the uploading request carries the first metadata.
203. The repository server stores the first metadata.
204. And (optionally) the repository server sending a first upload success message to the first node.
205. (optional) the first node sends the first mirrored data to the repository server.
206. The (optional) repository server saves the first mirrored data to the distributed storage system.
207. And (optionally) the repository server sending a second upload success message to the first node.
In step 201, a first node (the first node is any node in the docker system) may generate a new mirror in an operating process, where the new mirror is the first mirror described in step 201, and the first mirror specifically includes first mirror data and first metadata.
In step 202, after the first node generates the first image, the first node sends an upload request to the repository server to request to store the first image in the repository server, so that when other nodes need to use the first image, the first image can be obtained by sending a download request to the repository server, where the upload request sent by the first node carries the first metadata of the first image.
It should be noted that the step of generating the first image may be executed by the server 12 in the first node, and then, when the server 12 of the first node is disposed in the distributed storage system shown in fig. 2 or fig. 4, the first image data included in the first image generated by the server 12 is directly stored in the distributed storage system, at this time, in step 203, after the repository server receives the first metadata sent by the first node, the repository server directly stores the first metadata.
Optionally, after the repository server stores the first metadata, in step 204, the repository server may further send a first upload success message to the first node to notify the first node that the first metadata is uploaded successfully.
Therefore, under the condition, the first node only needs to upload the generated metadata of the new mirror image to the warehouse server, and mirror image data of the new mirror image does not need to be uploaded, so that the transmission times of the mirror image data in the docker system can be reduced, and the overhead of the whole docker system is reduced.
Still alternatively, when the server 12 is not disposed in the distributed storage system shown in fig. 4, the first node further needs to perform step 205 and step 207 in order to store the first mirrored data in the distributed storage system.
Wherein, in step 205, the first node sends the first mirrored data to the repository server. Further, in step 206, the repository server may call the repository driver 04 shown in fig. 4 to save the first mirrored data to the distributed storage system. Optionally, the warehouse server may further send a second upload success message to the first node to notify the first node that the upload of the first mirrored data is successful.
In addition, based on step 101-.
Illustratively, as shown in fig. 9, taking deletion of a first image (the first image includes first image data and first metadata) in a repository server as an example, an embodiment of the present invention provides an image deletion method, including:
301. the warehouse server receives a deletion request sent by the first node, wherein the deletion request is used for indicating that the first mirror in the warehouse server is deleted.
302. The repository server deletes the first metadata within the repository server according to the deletion request.
303. The repository server determines whether the reference counter of the first image is 0.
304. And if the reference counter of the first mirror image is 0, deleting the first mirror image data stored in the distributed storage system by the warehouse server.
In step 301, a server 12, for example, an HTTP server, in a first node (the first node is any node in the docker system) may send a deletion request to a repository server, where the deletion request is used to instruct to delete a first image in the repository server, for example, the deletion request carries an identifier of first metadata of the first image.
Further, in step 302, the repository server first deletes the first metadata in the repository server in response to the received deletion request.
Further, since the mirror image data of the mirror image in each node in the docker system is uniformly stored in the distributed storage system, even the mirror image data in the warehouse server may also be stored in the distributed storage system, when deleting the first mirror image data of the first mirror image, the warehouse server may delete the first mirror image data only after determining that each node does not use the first mirror image.
Of course, since the deletion request is sent by the first node to the repository server, the repository server may default that the first node does not use the first image, and further only needs to determine that other nodes except the first node do not use the first image.
For example, as mentioned above, in the docker system, a reference counter may be preset for each image to record the number of times the image is used, so that, in step 303, the warehouse server may determine whether the first image is used by one or more nodes by determining whether the reference counter of the first image is 0.
Specifically, the warehouse server may call the warehouse driver 04 in fig. 4 to search for a reference counter of the first mirror image in the distributed storage system, and if the reference counter of the first mirror image is 0, it indicates that no node uses the first mirror image in the current docker system, at this time, in step 304, the warehouse server deletes the first mirror image data stored in the distributed storage system, thereby completing the deletion operation of the first mirror image.
Therefore, the mirror image management method provided by the embodiment of the present invention may specifically include a mirror image downloading method, a mirror image uploading method, and a mirror image deleting method, wherein in the docker system provided by the embodiment of the present invention, mirror image data and metadata of a mirror image may be respectively managed, and mirror image data of a mirror image in each node is uniformly stored in a distributed storage system, that is, each node may share mirror image data in a distributed storage system, so that not only can the problem of storage space waste caused by storing the same mirror image in a large number of nodes in the prior art be avoided, but also each node and a warehouse server only need to manage metadata of a mirror image in the node and the warehouse server only, thereby reducing transmission times of the mirror image data and reducing overhead of the docker system.
Fig. 10 is a schematic structural diagram of a node 02 in a docker system according to an embodiment of the present invention, where the node 02 according to the embodiment of the present invention may be used to implement the method implemented by the embodiments of the present invention shown in fig. 2 to fig. 9, and for convenience of description, only the part related to the embodiment of the present invention is shown, and details of the specific technology are not disclosed, please refer to the embodiments of the present invention shown in fig. 2 to fig. 9.
Specifically, as shown in fig. 10, the node 02 includes: a mirror image generating unit 21, a transmitting unit 22, and a storing unit 23.
The image generating unit 21 is configured to generate a first image, where the first image includes first metadata and first image data;
a sending unit 22, configured to send an upload request to a warehouse server, where the upload request carries the first metadata, so that the warehouse server stores the first metadata;
a saving unit 23, configured to save the first mirrored data to the distributed storage system.
Further, the sending unit 22 is further configured to send the first mirrored data to the repository server, so that the repository server saves the first mirrored data in the distributed storage system.
Further, as shown in fig. 11, the node 02 further includes an obtaining unit 24 and a searching unit 25, wherein,
the sending unit 22 is further configured to send a download request to the warehouse server, where the download request carries an identifier of the first metadata;
the obtaining unit 24 is configured to receive first metadata sent by the warehouse server and corresponding to the identifier of the first metadata; if the first mirror image data are stored in the distributed storage system, the first mirror image data are obtained from the distributed storage system;
the searching unit 25 is configured to search, according to the first metadata, whether first mirror image data corresponding to the first metadata is stored in the distributed storage system.
Further, the obtaining unit 24 is further configured to obtain the first mirror image data from the warehouse server if the first mirror image data is not stored in the distributed storage system.
Further, the saving unit 23 is further configured to save the first metadata to a local repository in the first node.
Further, as shown in fig. 12, the node 02 further includes:
a counting unit 26 for performing an add-1 operation on a reference counter preset for the first mirror.
Further, as shown in fig. 13, the node 02 further includes a deletion unit 27, wherein,
the deleting unit 27 is configured to delete the first metadata in the first node;
the counting unit 26 is further configured to perform a decrement operation on the reference counter by 1.
The specific functions of the image generating unit 21, the sending unit 22, the saving unit 23, the sending unit 24, the obtaining unit 24, the searching unit 25, and the counting unit 26 can be realized by the image management module 100 shown in fig. 6 interacting with the client 11, the server 12, the driver 13, the container management library 14, and the container 15 shown in fig. 6.
In addition, fig. 14 is a schematic structural diagram of a warehouse server 01 in a docker system according to an embodiment of the present invention, and the warehouse server 01 according to the embodiment of the present invention may be used to implement the method implemented by the embodiments of the present invention shown in fig. 2 to fig. 9, for convenience of description, only the part related to the embodiments of the present invention is shown, and details of the technology are not disclosed, please refer to the embodiments of the present invention shown in fig. 2 to fig. 9.
Specifically, as shown in fig. 14, the repository server 01 includes: a receiving unit 31 and a deleting unit 32.
The receiving unit 31 is configured to receive a deletion request sent by a first node, where the deletion request is used to instruct to delete a first mirror image in the repository server, the first mirror image includes first metadata and first mirror image data, and the first node is any one of N nodes in the docker system;
a deleting unit 32, configured to delete the first metadata in the repository server according to the deletion request; and if the first mirror image is not used by each node, deleting the first mirror image data stored in the distributed storage system.
Further, as shown in fig. 15, the repository server 01 further includes a determining unit 33, where the determining unit 33 is configured to determine whether the reference counter of the first image is 0; the deleting unit 32 is specifically configured to delete the first mirror image data stored in the distributed storage system if the reference counter of the first mirror image is 0.
As shown in fig. 16, the node 02 shown in fig. 10 to 13 or the warehouse server 01 shown in fig. 14 to 15 may be implemented in the form of a computer device (or system) in fig. 16.
Fig. 16 is a schematic diagram of a computer device according to an embodiment of the present invention. The computer device 100 includes at least one processor 41, a communication bus 42, a memory 43, and at least one communication interface 44.
For example, the specific functions of the receiving unit 31, the deleting unit 32, and the judging unit 33 can be implemented by the processor 41 in the repository server 01 calling the computer instructions in the memory 43.
Specifically, the processor 41 may be a general purpose Central Processing Unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs according to the present invention.
The communication bus 42 may include a path that conveys information between the aforementioned components. The communication interface 44 may be implemented using any transceiver or the like for communicating with other devices or communication Networks, such as ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.
The Memory 43 may be a Read-Only Memory (ROM) or other type of static storage device that can store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that can store information and instructions, an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc Read-Only Memory (CD-ROM) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these. The memory 43 may be a separate memory, connected to the processor by a bus. The memory 43 may also be integrated with the processor.
The memory 43 is used for storing application program codes for executing the scheme of the invention, and is controlled by the processor 41 to execute. The processor 41 is configured to execute application program code stored in the memory 43.
In particular implementations, processor 41 may include one or more CPUs such as CPU0 and CPU1 in fig. 16, for example, as one embodiment.
In particular implementations, computer device 100 may include multiple processors, such as processor 41 and processor 48 in FIG. 16, as one embodiment. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).
In particular implementations, computer device 100 may also include an output device 45 and an input device 46, as one embodiment. The output device 45 is in communication with the processor 41 and may display information in a variety of ways. For example, the output device 45 may be a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display device, a Cathode Ray Tube (CRT) display device, a projector (projector), or the like. The input device 46 is in communication with the processor 41 and may accept user input in a variety of ways. For example, the input device 46 may be a mouse, a keyboard, a touch screen device or a sensing device, and the like.
The computer device 100 described above may be a general purpose computer device or a special purpose computer device. In a specific implementation, the computer device 100 may be a desktop computer, a laptop computer, a web server, a Personal Digital Assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, or a device with a similar structure as in fig. 16. Embodiments of the present invention are not limited by the type of computer device 100.
It should be noted that, in the example of the present invention, any one of the functional nodes in the docker system, for example, the warehouse server 01 and the node 02, may be implemented by one entity device, or may be implemented by a plurality of entity devices together, and each functional node in the docker system may be implemented by different entity devices, or may be implemented by the same entity device. It is understood that any functional node in the docker system may be a logical functional module in the physical device, or may be a logical functional module composed of a plurality of physical devices.
Therefore, the mirror image management device provided by the embodiment of the present invention specifically includes a warehouse server and each node, wherein in the docker system provided by the embodiment of the present invention, mirror image data and metadata of mirror images can be managed respectively, and the mirror image data of the mirror images in each node are uniformly stored in the distributed storage system, that is, each node can share the mirror image data in the distributed storage system, so that not only can the problem of storage space waste caused by storing the same mirror images in a large number of nodes in the prior art be avoided, but also each node and the warehouse server only need to manage the metadata of the mirror images in the node, and thus the transmission frequency of the mirror image data can be reduced, and the overhead of the docker system can be reduced.
It will be clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to perform all or part of the above described functions. For the specific working processes of the system, the apparatus and the unit described above, reference may be made to the corresponding processes in the foregoing method embodiments, and details are not described here again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

Claims (19)

1. The mirror image management method is applied to a container system, the container system comprises a warehouse server and N nodes connected with the warehouse server, each node comprises a storage module, and N is an integer greater than or equal to 2; the method comprises the following steps:
a first node sends a downloading request to the warehouse server, wherein the downloading request carries an identifier of first metadata of a first mirror image, and the first node is any one of the N nodes;
the first node receives the first metadata sent by the warehouse server;
and the first node acquires the first mirror image data of the first mirror image from the storage modules of the N nodes according to the first metadata.
2. The method of claim 1, wherein before the first node obtains the first mirrored data from the storage modules of the N nodes according to the first metadata, the method further comprises:
and the first node determines that first mirror image data corresponding to the first metadata are stored in the storage modules of the N nodes according to the first metadata.
3. The method of any of claims 1-2, wherein after the first node receives the first metadata sent by the warehouse server, the method further comprises:
the first node saves the first metadata to a local repository within the first node.
4. The method of claim 3, wherein after the first node saves the first metadata to a local repository within the first node, the method further comprises:
the first node performs an add 1 operation on a reference counter of the first image.
5. The method according to any one of claims 1, 2, 4, further comprising:
the first node generates a second mirror image, wherein the second mirror image comprises second metadata and second mirror image data;
the first node sends an uploading request to the warehouse server, wherein the uploading request carries the second metadata, so that the warehouse server stores the first metadata;
and the first node stores the second mirror image data to the storage modules of the N nodes.
6. The method of claim 5, wherein the first node saving the second mirrored data to the storage modules of the N nodes comprises:
and the first node sends the second mirror image data to the warehouse server so that the warehouse server stores the second mirror image data to the storage modules of the N nodes.
7. The method of claim 6, further comprising:
the first node determines to delete a third mirror image in the first node, wherein the third mirror image comprises third metadata;
the first node deleting the third metadata in the first node;
the first node performs a subtract 1 operation on the reference counter of the first image.
8. The mirror image management method is applied to a container system, the container system comprises a warehouse server and N nodes connected with the warehouse server, each node comprises a storage module, and N is an integer greater than or equal to 2; the method comprises the following steps:
the method comprises the steps that a deleting request sent by a first node is received by the warehouse server, the deleting request is used for indicating that a first mirror image in the warehouse server is deleted, the first mirror image comprises first metadata and first mirror image data, and the first node is any one of N nodes in the container system;
the warehouse server deletes the first metadata in the warehouse server according to the deletion request;
and if the N nodes do not use the first mirror image, deleting the first mirror image data stored in the storage modules of the N nodes by the warehouse server.
9. The method of claim 8, wherein deleting, by the repository server, first mirrored data stored in storage modules of the N nodes if none of the N nodes use the first mirror comprises:
the warehouse server judges whether a reference counter of the first mirror image is 0;
and if the reference counter of the first mirror image is 0, deleting the first mirror image data stored in the storage modules of the N nodes by the warehouse server.
10. A container system, characterized in that the system comprises: the system comprises a warehouse server and N nodes connected with the warehouse server, wherein each node comprises a storage module, and N is an integer greater than or equal to 2; wherein the content of the first and second substances,
a first node configured to: sending a downloading request to the warehouse server, wherein the downloading request carries an identifier of first metadata of a first mirror image, and the first node is any one of the N nodes; receiving the first metadata sent by the warehouse server; acquiring first mirror image data of the first mirror image from storage modules of the N nodes according to the first metadata;
the warehouse server is configured to: receiving the downloading request sent by the first node; searching the first metadata according to the identifier of the first metadata in the downloading request; and sending the first metadata to the first node.
11. The system of claim 10, wherein prior to obtaining the first mirrored data from the storage modules of the N nodes based on the first metadata, the first node is further configured to:
and determining that first mirror image data corresponding to the first metadata are stored in the storage modules of the N nodes according to the first metadata.
12. The system according to claim 10 or 11, wherein after receiving the first metadata sent by the repository server, the first node is further configured to:
saving the first metadata to a local repository within the first node.
13. The system of claim 12, wherein after saving the first metadata to a local repository within the first node, the first node is further configured to:
performing an add 1 operation on a reference counter of the first image.
14. The system of any one of claims 10, 11, or 13,
the first node is further configured to: generating a second mirror image, wherein the second mirror image comprises second metadata and second mirror image data; storing the second mirror image data to a storage module of the N nodes; sending an upload request to the warehouse server, wherein the upload request carries the second metadata;
the warehouse server is configured to: receiving the uploading request sent by the first node; and storing the second metadata in the upload request.
15. The system of claim 14, wherein the warehouse server comprises a warehouse driver; wherein the content of the first and second substances,
storing the second mirror image data to the storage modules of the N nodes, specifically including: the first node is configured to: sending the second mirrored data to the repository server;
the warehouse server is further configured to: receiving the second mirror image data sent by the first node; and calling the warehouse driver to store the second mirror image data to the storage modules of the N nodes.
16. The system of claim 15,
the first node is further configured to: determining to delete a third image within the first node, the third image comprising a third metadata; deleting the third metadata within the first node; deleting the first metadata within the first node; a subtract 1 operation is performed on the reference counter of the first image.
17. The system of claim 16,
the first node is further configured to send a deletion request to the repository server, where the deletion request is used to instruct to delete the third mirror image in the repository server;
the warehouse server is further configured to: receiving the deletion request sent by the first node; deleting the third metadata in the warehouse server according to the deletion request; and if the N nodes do not use the third mirror image, deleting the mirror image data of the third mirror image stored in the storage modules of the N nodes.
18. A node adapted for use in a container system comprising a warehouse server and N of said nodes; the node includes an interface for communicating with other nodes of the N nodes and the repository server, and a processor for performing the method of claims 1-7, N being an integer greater than or equal to 2.
19. A warehouse server, characterized in that the warehouse server belongs to a container system, the container system further comprising N nodes, the warehouse server comprising an interface for communicating with the N nodes and a processor for performing the method as claimed in claims 8-9, N being an integer greater than or equal to 2.
CN201910145316.2A 2016-01-26 2016-01-26 Mirror image management method, device and system Active CN109976667B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910145316.2A CN109976667B (en) 2016-01-26 2016-01-26 Mirror image management method, device and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610051837.8A CN105740048B (en) 2016-01-26 2016-01-26 A kind of mirror image management method, apparatus and system
CN201910145316.2A CN109976667B (en) 2016-01-26 2016-01-26 Mirror image management method, device and system

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN201610051837.8A Division CN105740048B (en) 2016-01-26 2016-01-26 A kind of mirror image management method, apparatus and system

Publications (2)

Publication Number Publication Date
CN109976667A CN109976667A (en) 2019-07-05
CN109976667B true CN109976667B (en) 2021-09-14

Family

ID=56247362

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201610051837.8A Active CN105740048B (en) 2016-01-26 2016-01-26 A kind of mirror image management method, apparatus and system
CN201910145316.2A Active CN109976667B (en) 2016-01-26 2016-01-26 Mirror image management method, device and system

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN201610051837.8A Active CN105740048B (en) 2016-01-26 2016-01-26 A kind of mirror image management method, apparatus and system

Country Status (1)

Country Link
CN (2) CN105740048B (en)

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107797806A (en) * 2016-08-29 2018-03-13 北京雪球信息科技有限公司 A kind of dispositions method of program
CN106506587B (en) * 2016-09-23 2021-08-06 中国人民解放军国防科学技术大学 Docker mirror image downloading method based on distributed storage
CN106708599B (en) * 2016-12-01 2020-08-25 携程旅游信息技术（上海）有限公司 Virtual machine mirror image issuing system and method
CN108205481B (en) * 2016-12-19 2022-03-08 华为技术有限公司 Application container instance creation method and device
CN106997392B (en) * 2017-04-10 2020-03-03 北京搜狐新媒体信息技术有限公司 Processing method and device and reading method and device of Docker mirror image cache information
CN107105054B (en) * 2017-05-17 2019-12-24 苏州浪潮智能科技有限公司 Mirror image garbage cleaning system and method for docker mirror image warehouse
CN107515776B (en) * 2017-07-18 2021-04-09 深信服科技股份有限公司 Method for upgrading service continuously, node to be upgraded and readable storage medium
CN107450961B (en) * 2017-09-22 2020-10-16 济南浚达信息技术有限公司 Distributed deep learning system based on Docker container and construction method and working method thereof
CN109669808A (en) * 2017-10-17 2019-04-23 中移(苏州)软件技术有限公司 A kind of method and apparatus of system backup
CN109962843B (en) * 2017-12-22 2021-04-27 上海威派格智慧水务股份有限公司 Method for realizing edge information processing of gateway of Internet of things
CN108156246B (en) * 2017-12-27 2020-12-04 北京微网通联股份有限公司 File distribution method, system, server and client
CN110099076A (en) * 2018-01-29 2019-08-06 中兴通讯股份有限公司 A kind of method and its system that mirror image pulls
CN110198330B (en) * 2018-03-27 2022-03-22 腾讯科技（深圳）有限公司 Mirror image downloading method and device
US11132293B2 (en) 2018-04-10 2021-09-28 Red Hat Israel, Ltd. Intelligent garbage collector for containers
CN108647248B (en) * 2018-04-16 2021-03-09 新华三技术有限公司成都分公司 WORM state monitoring transfer method and device
CN110908671A (en) * 2018-09-18 2020-03-24 北京京东尚科信息技术有限公司 Method and device for constructing docker mirror image and computer readable storage medium
CN109936571B (en) * 2019-02-22 2020-05-29 全球能源互联网研究院有限公司 Mass data sharing method, open sharing platform and electronic equipment
CN109951551B (en) * 2019-03-21 2021-03-19 联想(北京)有限公司 Container mirror image management system and method
CN110119251B (en) * 2019-05-14 2023-01-03 北京杰思安全科技有限公司 Buffer control method and system based on disk large flow
CN110347535B (en) * 2019-07-16 2020-05-26 北京谷数科技股份有限公司 Method for improving writing performance of double controllers
CN110704162B (en) * 2019-09-27 2022-09-20 北京百度网讯科技有限公司 Method, device and equipment for sharing container mirror image by physical machine and storage medium
CN111221623B (en) * 2019-12-26 2024-01-19 中科曙光国际信息产业有限公司 Mirror image library management system
CN113872999A (en) * 2020-06-30 2021-12-31 中兴通讯股份有限公司 Mirror image distribution method, electronic device and storage medium
CN112748879B (en) * 2020-12-30 2023-03-10 中科曙光国际信息产业有限公司 Data acquisition method, system, device, computer equipment and storage medium
CN117667298A (en) * 2022-08-31 2024-03-08 成都华为技术有限公司 Method and device for starting container, computing node and shared storage equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103761275A (en) * 2014-01-09 2014-04-30 浪潮电子信息产业股份有限公司 Management method for metadata in distributed file system
CN104391802A (en) * 2014-11-24 2015-03-04 浪潮电子信息产业股份有限公司 Streamline pool metadata node refreshing consistency protecting method
CN104516967A (en) * 2014-12-25 2015-04-15 国家电网公司 Electric power system mass data management system and use method thereof
WO2015127083A2 (en) * 2014-02-21 2015-08-27 Solidfire, Inc. Data syncing in a distributed system
CN104885076A (en) * 2012-05-10 2015-09-02 以太存储股份有限公司 Systems and methods for distributed storage

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102008037094B4 (en) * 2008-05-08 2015-07-30 Siemens Aktiengesellschaft Storing and providing medical image data in a computer-based distributed system
US8495356B2 (en) * 2010-12-31 2013-07-23 International Business Machines Corporation System for securing virtual machine disks on a remote shared storage subsystem
CN102932415B (en) * 2012-09-26 2015-04-15 东软集团股份有限公司 Method and device for storing mirror image document
CN103544045A (en) * 2013-10-16 2014-01-29 南京大学镇江高新技术研究院 HDFS-based virtual machine image storage system and construction method thereof
US20150149410A1 (en) * 2013-11-27 2015-05-28 Microsoft Corporation Publishing to a content delivery system
CN104731516B (en) * 2013-12-18 2019-03-01 腾讯科技（深圳）有限公司 A kind of method, apparatus and distributed memory system of accessing file
CN104951308B (en) * 2015-06-30 2019-05-21 北京奇虎科技有限公司 The management optimization mode and device of Docker Registry
CN105045656B (en) * 2015-06-30 2018-11-30 深圳清华大学研究院 Big data storage and management method based on virtual container
CN105187500A (en) * 2015-08-07 2015-12-23 浪潮(北京)电子信息产业有限公司 Container-based distributed storage system deployment method
CN105094924B (en) * 2015-08-21 2018-06-26 华为技术有限公司 The introduction method and device of image file

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104885076A (en) * 2012-05-10 2015-09-02 以太存储股份有限公司 Systems and methods for distributed storage
CN103761275A (en) * 2014-01-09 2014-04-30 浪潮电子信息产业股份有限公司 Management method for metadata in distributed file system
WO2015127083A2 (en) * 2014-02-21 2015-08-27 Solidfire, Inc. Data syncing in a distributed system
CN104391802A (en) * 2014-11-24 2015-03-04 浪潮电子信息产业股份有限公司 Streamline pool metadata node refreshing consistency protecting method
CN104516967A (en) * 2014-12-25 2015-04-15 国家电网公司 Electric power system mass data management system and use method thereof

Also Published As

Publication number Publication date
CN105740048B (en) 2019-03-08
CN105740048A (en) 2016-07-06
CN109976667A (en) 2019-07-05

Similar Documents

Publication Publication Date Title
CN109976667B (en) Mirror image management method, device and system
RU2673403C2 (en) Website access method, device and website system
US10515058B2 (en) Unified file and object data storage
US10185626B2 (en) Automatic application error detection and rollback
Paternò et al. A logical framework for multi-device user interfaces
US9590859B2 (en) Discovering resources of a distributed computing environment
JP2020096357A (en) Dynamic routing using container orchestration service
US20170161511A1 (en) Database public interface
CA2901042C (en) Non-precise garbage collection in non-cooperative systems
EP2951710A1 (en) Providing access to information across multiple computing devices
US20130007184A1 (en) Message oriented middleware with integrated rules engine
JP7366664B2 (en) Offline briefcase sync
CN109828830B (en) Method and apparatus for managing containers
CN114443076A (en) Mirror image construction method, device, equipment and storage medium
CN109729121B (en) Cloud storage system and method for realizing custom data processing in cloud storage system
CN113127430B (en) Mirror image information processing method, mirror image information processing device, computer readable medium and electronic equipment
US20220229858A1 (en) Multi-cloud object store access
US10114864B1 (en) List element query support and processing
US20140059538A1 (en) Virtual machine state tracking using object based storage
US20180341717A1 (en) Providing instant preview of cloud based file
CN110851192A (en) Method and device for responding to configuration of degraded switch
CN110209914B (en) Information query method and related device
EP3872630A2 (en) Request processing method and apparatus, electronic device, and computer storage medium
Lin et al. Implementation and design of a middleware platform based on WCF
CN113326108A (en) Method and device for managing application service

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20220218

Address after: 550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province

Patentee after: Huawei Cloud Computing Technology Co.,Ltd.

Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen

Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd.