CN113760453A - Container mirror image distribution system and container mirror image pushing, pulling and deleting method - Google Patents

Container mirror image distribution system and container mirror image pushing, pulling and deleting method Download PDF

Info

Publication number
CN113760453A
CN113760453A CN202110894323.XA CN202110894323A CN113760453A CN 113760453 A CN113760453 A CN 113760453A CN 202110894323 A CN202110894323 A CN 202110894323A CN 113760453 A CN113760453 A CN 113760453A
Authority
CN
China
Prior art keywords
mirror image
node
docker
warehouse
pulling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110894323.XA
Other languages
Chinese (zh)
Other versions
CN113760453B (en
Inventor
黄开天
杨祎巍
洪超
匡晓云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China South Power Grid International Co ltd
China Southern Power Grid Co Ltd
Original Assignee
China South Power Grid International Co ltd
China Southern Power Grid Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China South Power Grid International Co ltd, China Southern Power Grid Co Ltd filed Critical China South Power Grid International Co ltd
Priority to CN202110894323.XA priority Critical patent/CN113760453B/en
Publication of CN113760453A publication Critical patent/CN113760453A/en
Application granted granted Critical
Publication of CN113760453B publication Critical patent/CN113760453B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45562—Creating, deleting, cloning virtual machine instances
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45575—Starting, stopping, suspending or resuming virtual machine instances
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45595—Network integration; Enabling network access in virtual machine instances

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a container mirror image distribution system based on a P2P technology and a container mirror image pushing, pulling and deleting method; the container mirror image distribution system comprises a container node cluster with a plurality of working nodes and mirror image warehouse nodes; the working node is used for receiving and forwarding the mirror image event request of the client in real time and executing the mirror image event request according to the mirror image request event response; and the mirror image warehouse node is used for operating a mirror image warehouse program and the P2P centralized retrieval server, responding to the mirror image event request of any working node, matching the corresponding working node to complete the mirror image event request, and maintaining the mirror image change of the mirror image warehouse node through the P2P centralized retrieval server. The container mirror image distribution system and the container mirror image pushing, pulling and deleting method effectively solve the problem that a mirror image warehouse limits the container starting speed in a large-scale container cluster in mirror image distribution application, reduce the number of seed files in the cluster, improve the mirror image distribution stability and reduce unnecessary network bandwidth loss.

Description

Container mirror image distribution system and container mirror image pushing, pulling and deleting method
Technical Field
The invention relates to the technical field of cloud computing, in particular to an intelligent container mirror image distribution system and method based on a P2P technology, computer equipment and a storage medium.
Background
The emergence of virtualization technology has promoted the birth and development of cloud computing field, and human society becomes more and more informationized and intelligent. With the advent of a lightweight virtualization technology based on a resource isolation and control technology, which is typical of Docker, stronger power is injected for the development of the cloud computing field. In the existing Docker mirror image distribution system, a container mirror image needs to be stored in a working node of a starting container before starting, and the container mirror image is usually pulled from a unique Docker mirror image warehouse of a cluster. The basic unit of mirror image pulling is a mirror image level, namely a file change set generated by an instruction is constructed, and a Docker mirror image warehouse is easy to become a container cluster performance bottleneck by adopting an end-to-end C/S model single-point pulling mode.
The existing technical scheme for solving the performance bottleneck problem of the container cluster without changing the mirror image format and the mirror image storage mode mainly adopts a P2P (Peer-to-Peer) transmission technology to realize point-to-point mirror image distribution work among container working node clusters. Although the P2P transmission technology is well suited for being applied to mirror transmission in a container work node cluster due to the content invariance of the Docker container mirror level, research shows that the mirror level mostly belongs to small files with small volume and low compression rate, for such files, the cost of transmission using P2P is higher than that of direct transmission, and the mirror warehouse and the P2P retrieval server both need to use extra storage space and network transmission bandwidth for maintaining the small mirror level. Therefore, the default way of transmitting P2P will increase the operation cost of the Docker image distribution system.
Therefore, it is desirable to provide a Docker image distribution system and method for effectively solving the performance bottleneck problem that the image warehouse limits the container start speed in the large-scale container cluster and ensuring that the operation cost of the image distribution system is controllable.
Disclosure of Invention
The invention aims to provide a container mirror image distribution system and a container mirror image pushing, pulling and deleting method, which realize the container mirror image pushing and distribution functions based on a P2P technology and a time sequence prediction method, solve the performance bottleneck problem that a mirror warehouse limits the container starting speed in a large-scale container cluster, reduce the number of seed files in the cluster, improve the mirror image distribution stability and reduce unnecessary network bandwidth loss.
In order to achieve the above object, it is necessary to provide a container image distribution system and a container image push, pull and delete method in response to the above technical problems.
In a first aspect, an embodiment of the present invention provides a container mirror distribution system, where the system includes a Docker container node cluster and Docker mirror warehouse nodes, and is characterized in that:
the Docker container node cluster comprises a plurality of working nodes; the working node is used for receiving a mirror image event request of a Docker client in real time, generating a node mirror image event request according to the mirror image event request, sending the node mirror image event request to the Docker mirror image warehouse node, and executing the mirror image event request according to a mirror image request event response returned by the Docker mirror image warehouse node;
the Docker mirror image warehouse node is used for operating a mirror image warehouse program and a P2P centralized retrieval server, responding to the node mirror image event request of any one working node, matching the corresponding working node to complete the mirror image event request, and maintaining the mirror image change of the Docker mirror image warehouse node through the P2P centralized retrieval server.
Further, the working nodes comprise a Docker container engine and a node processing module which is deployed simultaneously with the Docker container engine;
the Docker container engine is used for receiving the mirror event request and forwarding the mirror event request to the node processing module; the mirror image event request comprises a mirror image pushing request, a mirror image pulling request and a mirror image deleting request;
the node processing module is used for receiving and analyzing the mirror image pushing request, sending mirror pixel information to be pushed of the mirror image pushing request to the Docker mirror image warehouse node, making a corresponding seed file according to a mirror image level list to be pushed returned by the Docker mirror image warehouse node, sending the stored mirror image level list to be pushed and the corresponding seed file to the Docker mirror image warehouse node, and sending a registration application of a P2P mirror image resource provider to the Docker mirror image warehouse node in response to a successful mirror image pushing result of the Docker mirror image warehouse node; or the like, or, alternatively,
the system comprises a Docker mirror image warehouse node, a mirror image pull request receiving and analyzing module, a mirror image information list sending module, a mirror image pull request receiving and analyzing module, a mirror image pull request sending module, a mirror image information list sending module, a mirror image pull request sending module and a mirror image pull request sending module, wherein the mirror image pull request sending module is used for sending mirror image information to be pulled of the mirror image pull request to the Docker mirror image warehouse node, comparing the mirror image information list to be pulled returned by the Docker mirror image warehouse node with a mirror image storage directory of the working node, determining a mirror image level to be pulled, sending the mirror image pull request to be pulled, sending the mirror image pull request to the mirror image level to be pulled to the Docker mirror image warehouse node, sending to be pulled to the Docker mirror image warehouse node to be pulled to be; or the like, or, alternatively,
the system is used for receiving and analyzing the mirror image deleting request, sequentially deleting the seed file of the mirror image hierarchy to be deleted and the mirror image hierarchy to be deleted according to the mirror image hierarchy to be deleted of the mirror image deleting request, and sending a logout application of a P2P mirror image resource provider to the Docker mirror image warehouse node.
Further, the Docker mirror image warehouse node comprises a Docker mirror image warehouse, a push processing module, a pull processing module and an index processing module which are deployed simultaneously with the Docker mirror image warehouse;
the Docker mirror image warehouse is used for storing mirror images and seed files required by the operation of the Docker container node cluster;
the pushing processing module is used for receiving the mirror image pushing request of any working node, comparing the mirror image information to be pushed of the mirror image pushing request with the existing mirror image level of the Docker mirror image warehouse, returning to the corresponding mirror image level list to be pushed, receiving and storing the mirror image level list to be pushed and the corresponding seed file sent by the working node, creating a corresponding seed index according to the seed file, and registering the Docker mirror image warehouse node as a P2P mirror image resource provider;
the pull processing module is used for receiving the mirror image pull request of any one working node, returning a mirror image information list to be pulled of the Docker mirror image warehouse according to the mirror image pull request, inquiring a seed file corresponding to the working node according to the mirror image level to be pulled of the working node responding to the mirror image information list to be pulled, determining a mirror image pull mode according to the working load of the Docker mirror image warehouse node, sending mirror image pull response data generated according to the seed file and the mirror image pull mode to the working node, and responding to the single-point pull action of the working node to obtain a mirror image level file to be pulled for transmission;
the index processing module is used for connecting the P2P centralized retrieval server and realizing registration and logout of the P2P mirror image resource provider according to the mirror image event request.
Further, the Docker mirror warehouse node further comprises a load processing module deployed simultaneously with the Docker mirror warehouse;
the load processing module is used for recording the workload of the Docker mirror image warehouse node at preset time intervals, establishing a corresponding load prediction model according to the time sequence data of the workload, and predicting the workload of the Docker mirror image warehouse node at the next time node in real time according to the load prediction model.
In a second aspect, an embodiment of the present invention provides a method for pushing a work node image of a container image distribution system, where the method includes the following steps:
receiving and analyzing a mirror image pushing request corresponding to a Docker client to obtain mirror image information to be pushed, and sending the mirror image information to be pushed to a Docker mirror image warehouse node; the working node is any node in the Docker container node cluster;
receiving a mirror image hierarchical list to be pushed, manufacturing a corresponding seed file according to the mirror image hierarchical list to be pushed, and pushing the mirror image hierarchical list to be pushed and the corresponding seed file to the Docker mirror image warehouse node after index storage of the mirror image hierarchical list to be pushed and the corresponding seed file; the mirror image level list to be pushed is determined by the Docker mirror image warehouse node according to the comparison result between the mirror image information to be pushed and the existing mirror image level of the Docker mirror image warehouse;
responding to a successful mirror image pushing result, and sending a registration application of a P2P mirror image resource provider to the Docker mirror image warehouse node; the successful mirror image pushing result is received by the Docker mirror image warehouse node, the to-be-pushed mirror image hierarchical list and the corresponding seed file are stored, the corresponding seed index is created, and the Docker mirror image warehouse node is registered as a P2P mirror image resource provider and then generated.
In a third aspect, an embodiment of the present invention further provides a method for pulling a work node mirror image of a container mirror image distribution system, where the method includes the following steps:
receiving and analyzing a mirror image pulling request corresponding to a Docker client to obtain mirror image information to be pulled, and sending the mirror image information to be pulled to a Docker mirror image warehouse node;
receiving a mirror image information list to be pulled, comparing the mirror image information list to be pulled with a local mirror image storage directory, determining a mirror image level to be pulled, and sending the mirror image level to be pulled to the Docker mirror image warehouse node; the mirror image level meta-information list is obtained by the Docker mirror image warehouse node inquiring a Docker mirror image warehouse of the Docker mirror image warehouse node according to the mirror image information to be pulled;
receiving and analyzing mirror image pulling response data, acquiring a mirror image pulling mode in the mirror image pulling response data, executing mirror image pulling operation according to the mirror image pulling mode, and sending a P2P mirror image resource provider registration application to the Docker mirror image warehouse node after the pulling operation is completed; and the mirror image pulling response data is inquired by the Docker mirror image warehouse node according to the mirror image level to be pulled to obtain a corresponding seed file, and after a mirror image pulling mode is determined according to the workload of the Docker mirror image warehouse node, the mirror image pulling response data is generated according to the seed file and the mirror image pulling mode.
Further, the step of receiving and analyzing the mirror image pull response data to obtain a mirror image pull mode in the mirror image pull response data, and the step of executing the mirror image pull operation according to the mirror image pull mode includes:
analyzing the mirror image pulling response data to obtain the mirror image pulling mode;
judging whether the mirror image pulling mode is P2P mirror image pulling;
if the mirror image pulling mode is the P2P mirror image pulling, acquiring a corresponding P2P mirror image pulling resource table, selecting any P2P resource provider to be pulled on the P2P mirror image pulling resource table to establish a chain, executing the mirror image pulling operation, and otherwise, sending a single-point mirror image pulling operation to the Docker mirror image warehouse node.
In a fourth aspect, an embodiment of the present invention further provides a method for pulling a node mirror image of a Docker mirror image warehouse of a container mirror image distribution system, where the method includes the following steps:
receiving to-be-pulled mirror pixel information sent by a working node, inquiring a Docker mirror image warehouse of a Docker mirror image warehouse node according to the to-be-pulled mirror pixel information to obtain a to-be-pulled mirror pixel information list, and sending the to-be-pulled mirror pixel information list to the working node; the working node is any node in the Docker container node cluster; the mirror image element information to be pulled is obtained by receiving and analyzing a mirror image pulling request corresponding to the Docker client by the working node;
receiving a mirror image level to be pulled sent by the working node, inquiring to obtain a corresponding seed file according to the mirror image level to be pulled, determining a mirror image pulling mode according to the working load of the Docker mirror image warehouse node, generating mirror image pulling response data according to the seed file and the mirror image pulling mode, and sending the mirror image pulling response data to the working node; the mirror image level to be pulled is determined by the working node according to the comparison result between the mirror image information list to be pulled and a local mirror image storage directory;
and responding to the single-point mirror image pulling operation of the working node, reading the mirror image hierarchy to be pulled and the seed file in the Docker mirror image warehouse, and sending the mirror image hierarchy to be pulled and the seed file to the working node.
Further, the step of determining the mirror image pulling mode according to the workload of the node itself includes:
after the Docker mirror image warehouse node is started, continuously recording the workload of the Docker mirror image warehouse node according to a preset time interval;
training to obtain a corresponding load prediction model according to the time sequence data of the working load;
obtaining a working load prediction value of a next time node of the Docker mirror image warehouse node according to the load prediction model;
and judging whether the working load predicted value is greater than a preset load threshold value, if so, determining that the mirror image pulling mode is P2P mirror image pulling, and inquiring the P2P centralized retrieval server to obtain a corresponding P2P mirror image pulling resource table, otherwise, determining that the mirror image pulling mode is single-point mirror image pulling.
In a fifth aspect, an embodiment of the present invention further provides a method for pulling a node mirror image of a Docker mirror image warehouse of a container mirror image distribution system, where the method includes the following steps:
receiving and analyzing a mirror image deleting request corresponding to the Docker client to obtain a mirror image hierarchy to be deleted;
searching a seed file corresponding to the mirror image level to be deleted according to the mirror image level to be deleted;
and sequentially executing the seed file corresponding to the mirror image hierarchy to be deleted and the deletion operation of the mirror image hierarchy to be deleted, and sending a logout application of the P2P mirror image resource provider to the Docker mirror image warehouse node.
The application provides a container mirror image distribution system, and a corresponding work node mirror image pushing method, a work node mirror image pulling method, a Docker mirror image warehouse node mirror image pulling method and a work node mirror image deleting method. Through the design of a container mirror image distribution system which simultaneously deploys a Docker container engine and a node processing module on a working node of a Docker container node cluster and simultaneously deploys a Docker mirror image warehouse, a P2P centralized retrieval server, a pushing processing module, a pulling processing module, an index processing module and a load processing module on a Docker mirror image warehouse node, the method realizes that after a seed file is manufactured by the node processing module of the working node according to a mirror image level to be pushed and is stored in a local mirror image directory, and after the mirror image level to be pushed and a corresponding seed file are pushed to the mirror image warehouse through interaction with the pushing processing module of the Docker mirror image warehouse node, the working node and the Docker mirror image warehouse node are both registered as a pushing method of a P2P mirror image resource provider; after the node processing module of the working node determines the mirror image level to be pulled, the interaction with the pulling processing module and the index processing module of the Docker mirror image warehouse node and the interaction between the pulling processing module and the load processing module are realized, a mirror image pulling mode is appointed to the working node by the Docker mirror image warehouse node based on the current load of the Docker mirror image warehouse node on the basis of the P2P technology and a time sequence prediction method, and the working node adopts single-point mirror image pulling or P2P mirror image pulling to obtain the mirror image level to be pulled and then registers as a pulling method of a P2P mirror image resource provider; the technical scheme of container mirror image distribution of the deletion method for dynamically deleting the content mirror images is realized by the interaction of the node processing module of the working node, the index processing module of the Docker mirror image warehouse node and the P2P centralized retrieval server. Compared with the prior art, the container mirror image distribution system and the corresponding pushing, pulling and deleting method solve the performance bottleneck problem that a mirror image warehouse limits the container starting speed in a large-scale container cluster, reduce the number of seed files in the cluster, improve the mirror image distribution stability and reduce unnecessary network bandwidth loss.
Drawings
FIG. 1 is a schematic diagram of an architecture of a container mirror distribution system in an embodiment of the invention;
FIG. 2 is a schematic diagram of a container mirror image dispensing system according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of a work node image pushing method of a container image distribution system according to an embodiment of the present invention;
fig. 4 is a schematic flowchart of a method for pulling a work node image of a container image distribution system according to an embodiment of the present invention;
fig. 5 is a schematic flowchart of a node mirror image pull method of a Docker mirror image warehouse of a container mirror image distribution system according to an embodiment of the present invention;
fig. 6 is a schematic flowchart of the Docker mirror image warehouse node in step S32 in fig. 5 determining a mirror image pulling manner according to a workload of its own node;
fig. 7 is a schematic flowchart of a method for deleting a work node mirror image in a container mirror image distribution system according to an embodiment of the present invention;
fig. 8 is an internal structural diagram of a computer device in the embodiment of the present invention.
Detailed Description
In order to make the purpose, technical solution and advantages of the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments, and it is obvious that the embodiments described below are part of the embodiments of the present invention, and are used for illustrating the present invention only, but not for limiting the scope of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a container mirror image distribution system and a corresponding pushing, pulling and deleting method, based on the fact that a mirror image warehouse in the existing container mirror image distribution system is used as a key component in the mirror image distribution process, the response time delay of the mirror image warehouse is in positive correlation with the container creation time delay, the consideration of the problem of limiting the starting speed of containers in a large-scale container cluster is combined with the advantages and disadvantages of the existing P2P mirror image transmission and single-point mirror image transmission, the container mirror image distribution system with 1 mirror image warehouse corresponding to a plurality of working nodes is designed, a more flexible mirror image distribution mode based on the working load decision mirror image distribution mode of the mirror image warehouse nodes is correspondingly provided, the P2P mirror image transmission among the working nodes is supported, the performance bottleneck problem of the mirror image warehouse is effectively solved, the number of seed files in the cluster is reduced, and the mirror image distribution stability is improved, unnecessary network bandwidth loss is reduced.
In one embodiment, as shown in fig. 2, a container mirror distribution system is provided, the system comprising a cluster of Docker container nodes 1 and a Docker mirror warehouse node 2;
the Docker container node cluster 1 comprises a plurality of working nodes 11; the working node 11 is configured to receive a mirror event request of a Docker client in real time, generate a node mirror event request according to the mirror event request, send the node mirror event request to the Docker mirror warehouse node 2, and execute the mirror event request according to a mirror request event response returned by the Docker mirror warehouse node 2;
the working node 11 comprises a Docker container engine 111 and a node processing module 112 deployed simultaneously with the Docker container engine;
the Docker container engine 111 is configured to receive the mirror event request, and forward the mirror event request to the node processing module; the mirror image event request comprises a mirror image pushing request, a mirror image pulling request and a mirror image deleting request;
the node processing module 112 is configured to receive and analyze the mirror image pushing request, send mirror pixel information to be pushed of the mirror image pushing request to the Docker mirror image warehouse node 2, make a corresponding seed file according to a mirror image hierarchy list to be pushed returned by the Docker mirror image warehouse node 2, send the stored mirror image hierarchy list to be pushed and the corresponding seed file to the Docker mirror image warehouse node 2, and send a registration application of a P2P mirror image resource provider to the Docker mirror image warehouse node 2 in response to a successful mirror image pushing result of the Docker mirror image warehouse node 2; or the like, or, alternatively,
the system comprises a Docker mirror image warehouse node 2, a Docker mirror image resource provider registration request receiving and analyzing module, a mirror image pull request sending module, a mirror image information list to be pulled, a mirror image storage directory of a working node, a mirror image level to be pulled, a Docker mirror image warehouse node 2, a mirror image pull operation executing module, and a P2P mirror image resource provider registration request sending module, wherein the mirror image resource provider registration request sending module is used for receiving and analyzing the mirror image pull request, comparing the mirror image information list to be pulled, returned by the Docker mirror image warehouse node 2, with the mirror image storage directory of the working node, determining the mirror image level to be pulled, sending the mirror image level to be pulled to the Docker mirror image warehouse node 2, executing the mirror image pull operation according to the mirror image pull response data returned by the Docker mirror image warehouse node 2, and sending the P2P mirror image resource provider registration request to the Docker mirror image warehouse node 2 after the pull operation is completed; or the like, or, alternatively,
and the node is used for receiving and analyzing the mirror image deletion request, sequentially deleting the seed file of the mirror image hierarchy to be deleted and the mirror image hierarchy to be deleted according to the mirror image hierarchy to be deleted of the mirror image deletion request, and sending a logout application of the P2P mirror image resource provider to the Docker mirror image warehouse node 2.
It should be noted that the mirror image push request, the mirror image pull request, and the mirror image delete request are all requests received by the Docker container engine and sent to the Docker client, and are finally fed back to the mirror image warehouse node for processing, and after reaching the working node, the corresponding mirror image event request is intercepted by the node processing module of the working node in a manner of corresponding function call, and the node processing module performs corresponding preprocessing according to the specific mirror image event request content obtained through analysis, and then forwards the processed mirror image event request to the mirror image warehouse node.
The Docker image warehouse node 2 is configured to run an image warehouse program and the P2P centralized retrieval server 22, respond to the node image event request of any one of the working nodes, complete the image event request in cooperation with the corresponding working node, and maintain the image change of the Docker image warehouse node 2 through the P2P centralized retrieval server 22.
The Docker mirror image warehouse node 2 comprises a Docker mirror image warehouse 21, a push processing module 23, a pull processing module 24 and an index processing module 25, wherein the push processing module 23, the pull processing module 24 and the index processing module are deployed simultaneously with the Docker mirror image warehouse 21;
the Docker mirror image warehouse 21 is configured to store a mirror image required by the Docker container node cluster 1 to operate;
the pushing processing module 23 is configured to receive the mirror image pushing request of any one of the working nodes 11, compare the to-be-pushed mirror image information of the mirror image pushing request with the existing mirror image hierarchy of the Docker mirror image warehouse 21, return to a corresponding to-be-pushed mirror image hierarchy list, receive and store the to-be-pushed mirror image hierarchy list and a corresponding seed file sent by the working node 1, create a corresponding seed index according to the seed file, and register the Docker mirror image warehouse node 2 as a P2P mirror image resource provider;
the pulling processing module 24 is configured to receive the mirror image pulling request of any one of the working nodes 1, return to a to-be-pulled mirror image information list of the Docker mirror image warehouse 21 according to the mirror image pulling request, query a seed file corresponding to the to-be-pulled mirror image information list according to a to-be-pulled mirror image hierarchy of the working node in response to the to-be-pulled mirror image information list, determine a mirror image pulling manner according to a working load of the Docker mirror image warehouse node 2, send mirror image pulling response data generated according to the seed file and the mirror image pulling manner to the working node 1, and obtain a to-be-pulled mirror image hierarchy file for transmission in response to a single-point pulling action of the working node 1;
the index processing module 25 is configured to connect to the P2P centralized retrieval server 22, and implement registration and deregistration of a P2P mirroring resource provider according to the mirroring event request.
The P2P mirror resource provider centralizes nodes in the search server 22 for P2P that can support P2P mirror pull, including the worker node 11 and the Docker mirror repository node 2. The node processing module 112 sends a registration request of the P2P mirror image resource provider or a logout request of the P2P mirror image resource provider to the Docker mirror image warehouse node 2, and the pushing processing module 23 registers the Docker mirror image warehouse node 2 as the P2P mirror image resource provider, which are all implemented by establishing a connection between the index processing module 25 and the P2P centralized retrieval server 22.
In addition, in order to ensure the distribution efficiency of the mirror image distribution system, the Docker mirror warehouse node 2 further includes a load processing module 26 deployed simultaneously with the Docker mirror warehouse;
the load processing module 26 is configured to record the workload of the Docker mirror warehouse node 2 at preset time intervals, establish a corresponding load prediction model according to time sequence data of the workload, and predict the workload of the Docker mirror warehouse node 2 at the next time node in real time according to the load prediction model.
The preset time interval can be flexibly adjusted before the system runs according to application requirements, specific values of the preset time interval are not specifically limited, and the preset time interval cannot be changed in the whole running process of the system after the preset time interval is set before the system runs. After the Docker mirror image warehouse node 2 starts working, the load processing module 26 starts to record the workload at the corresponding moment at preset time intervals, and stores the workload in the load database for subsequent mirror image pull mode decision.
The load prediction model is obtained by training according to the existing workload data in the load database, and may be a time sequence model, or may be another model capable of realizing the prediction function, when the pull processing module 23 processes the mirror image pull request of the node processing module 112, the load prediction model is used when the load processing module 26 is called, the workload of the mirror image warehouse node 2 at the next time point is predicted, and a decision is made as to which mirror image pull mode the working node 11 uses to complete the mirror image pull task according to the magnitude relation between the predicted workload value and the preset load threshold value.
In the embodiment, a Docker container engine and a node processing module are deployed on a working node of a Docker container node cluster, the node processing module preprocesses a mirror image event request message received by the Docker container engine from a client, and then forwards the mirror image event request message to a mirror image warehouse node, a push processing module, a pull processing module and an index processing module deployed by the mirror image warehouse node cooperate to complete a corresponding mirror image event request, a load processing module is designed to count the working load of the mirror image warehouse node according to a fixed time frequency, and a mirror image pull mode which is reasonable for distributing the mirror image pull request of the working node according to whether the working load prediction value of the mirror image warehouse node is greater than a preset load threshold value during each corresponding mirror image pull request processing is fed back to the pull processing module in time, so that the performance bottleneck problem that the mirror image warehouse of the existing content mirror image distribution system limits the starting speed of containers is effectively solved, the number of seed files in the cluster is reduced, the distribution stability of the mirror image is improved, and unnecessary network bandwidth loss is reduced.
It should be noted that the Docker mirror warehouse node 2 further includes a mirror warehouse operation module corresponding to a mirror warehouse program for ensuring normal operation of the node, and the function of operating and managing the Docker mirror warehouse 21 may be referred to in the prior art, which is not described herein again. In addition, each module in the working node and the Docker image warehouse node in the content image distribution system may be wholly or partially implemented by software, hardware and a combination thereof, may be embedded in a hardware form or independent from a processor in a computer device, or may be stored in a memory in the computer device in a software form, so that the processor may invoke and execute operations corresponding to each module, and functional descriptions between each module that need to cooperate with each other to complete a task of requesting an image event may be referred to each other.
In one embodiment, as shown in fig. 3, there is provided a working node image pushing method of a container image distribution system, the method comprising the following steps;
s11, receiving and analyzing a mirror image pushing request corresponding to the Docker client, obtaining mirror image information to be pushed, and sending the mirror image information to be pushed to the Docker mirror image warehouse node; the working node is any node in the Docker container node cluster;
the mirror image information comprises a mirror image library name, a mirror image label, mirror image level information and the like, and as the function of the container mirror image distribution system is known, after a Docker container engine of a working node receives a mirror image pushing request from a client, a corresponding node processing module analyzes the mirror image pushing request to obtain corresponding mirror image information to be pushed, the mirror image information to be pushed is sent to a mirror image warehouse node, and a mirror image level list which really needs to be pushed is fed back by the mirror image processing module to complete the mirror image pushing task.
S12, receiving a mirror image hierarchical list to be pushed, making a corresponding seed file according to the mirror image hierarchical list to be pushed, and pushing the mirror image hierarchical list to be pushed and the corresponding seed file to the Docker mirror image warehouse node after index storage of the mirror image hierarchical list to be pushed and the corresponding seed file; the mirror image level list to be pushed is determined by the Docker mirror image warehouse node according to the comparison result between the mirror image information to be pushed and the existing mirror image level of the Docker mirror image warehouse;
the mirror image information to be pushed sent by the working node is compared with the existing mirror image levels of the Docker mirror image warehouse by the pushing processing module of the Docker mirror image warehouse node to find out the mirror image levels which are contained in the mirror image information to be pushed and are not contained in the mirror image warehouse, so that the real meaningful mirror image pushing is ensured, and meanwhile, unnecessary network resource waste is avoided. It should be noted that, there may be a case where the mirror hierarchy list to be pushed is empty, and if the working node applies for pushing all the mirror information and stores the mirror information in the mirror warehouse, it is not necessary for the working node to push any mirror at this time.
S13, responding to the mirror image pushing success result, and sending a P2P mirror image resource provider registration application to the Docker mirror image warehouse node; the successful mirror image pushing result is received by the Docker mirror image warehouse node, the to-be-pushed mirror image hierarchical list and the corresponding seed file are stored, the corresponding seed index is created, and the Docker mirror image warehouse node is registered as a P2P mirror image resource provider and then generated.
The working node and the mirror image warehouse node application registration P2P centralized retrieval server can be realized by establishing communication connection with the P2P centralized retrieval server through the index processing module of the mirror image warehouse node in the container mirror image distribution system.
It should be noted that, the container mirror image distribution system implements a function of pushing a mirror image from a client to a mirror image warehouse, and needs a work node and a Docker mirror image warehouse node to cooperate together, where an implementation scheme of the Docker mirror image warehouse node may refer to a manner of obtaining data information that must be relied on in a work node pushing process flow, and details are not described here.
In order to improve the efficiency of image pushing, after a working node obtains a to-be-pushed image hierarchical list, a corresponding seed file is manufactured in advance according to the to-be-pushed image hierarchical list and is persistently stored in a local image seed file directory, the to-be-pushed image hierarchical list and the corresponding seed file are pushed to an image warehouse for storage, after a pushing task is completed, the working node and the image warehouse node are registered as image resource providers corresponding to the pushed image resource on a P2P centralized retrieval server, more choices are provided for subsequent P2P image pulling, the number of seed files in a cluster is effectively reduced, and meanwhile, the stability of an image distribution system is improved.
In one embodiment, as shown in fig. 4, there is provided a method for pulling a work node image of a container image distribution system, the method comprising the steps of;
s21, receiving and analyzing a mirror image pulling request corresponding to the Docker client, obtaining mirror image element information to be pulled, and sending the mirror image element information to be pulled to the Docker mirror image warehouse node;
the mirror image pulling request of the Docker client is also acquired by each working node through a Docker container engine of the working node, the node processing module in the working node analyzes the original mirror image pulling request to obtain corresponding mirror image element information to be pulled, the mirror image element information to be pulled is sent to the Docker mirror image warehouse node, and the pulling processing module in the Docker client is responsible for receiving and processing the mirror image element information.
S22, receiving a mirror image information list to be pulled, comparing the mirror image information list to be pulled with a local mirror image storage directory, determining a mirror image level to be pulled, and sending the mirror image level to be pulled to the Docker mirror image warehouse node; the mirror image level meta-information list is obtained by the Docker mirror image warehouse node inquiring a Docker mirror image warehouse of the Docker mirror image warehouse node according to the mirror image information to be pulled;
after receiving mirror pixel information to be pulled corresponding to a certain working node, the pulling processing module of the Docker mirror warehouse node feeds back all the inquired mirror image level information of the Docker mirror warehouse in the node as a mirror pixel information list to be pulled to the node processing module of the working node, determines the mirror image level to be pulled according to the mirror image storage condition of the working node, feeds back the mirror image level to be pulled to the pulling processing module of the Docker mirror warehouse node as the mirror image level to be pulled, and is used for completing subsequent real mirror image pulling tasks.
S23, receiving and analyzing mirror image pull response data, acquiring a mirror image pull mode in the mirror image pull response data, executing mirror image pull operation according to the mirror image pull mode, and sending a P2P mirror image resource provider registration application to the Docker mirror image warehouse node after the pull operation is completed; and the mirror image pulling response data is inquired by the Docker mirror image warehouse node according to the mirror image level to be pulled to obtain a corresponding seed file, and after a mirror image pulling mode is determined according to the workload of the Docker mirror image warehouse node, the mirror image pulling response data is generated according to the seed file and the mirror image pulling mode.
After receiving the mirror level to be pulled fed back by the working node, the pulling processing module of the Docker mirror warehouse node firstly queries a seed file of the Docker mirror warehouse corresponding to the mirror level to be pulled for subsequent feedback to the working node. The mirror image pulling manner provided by the load processing module is one of single-point mirror image pulling and P2P mirror image pulling, and how to decide the mirror image pulling manner of the current working node refers to the related content in the content mirror image distribution system, which is not described herein again. It should be noted that, after being called by the pull processing module for the first time, the load processing module will utilize the stored workload sequence to perform training to generate a prediction model, mirror the first workload prediction, and all subsequent workload predictions are obtained based on the load prediction model generated by the first training.
The step of receiving and analyzing the mirror image pull response data, obtaining a mirror image pull mode in the mirror image pull response data, and executing mirror image pull operation according to the mirror image pull mode comprises the following steps: analyzing the mirror image pulling response data to obtain the mirror image pulling mode; judging whether the mirror image pulling mode is P2P mirror image pulling; if the mirror image pulling mode is the P2P mirror image pulling, acquiring a corresponding P2P mirror image pulling resource table, selecting any P2P resource provider to be pulled on the P2P mirror image pulling resource table to establish a chain, executing the mirror image pulling operation, and otherwise, sending a single-point mirror image pulling operation to the Docker mirror image warehouse node.
The P2P mirror image pull resource table is obtained by the working node through querying from the P2P centralized retrieval server through the index processing module of the Docker mirror image warehouse node according to the seed text in the mirror image pull response data. That is, when the currently allocated mirror image pull mode of the working node is P2P mirror image pull, the node processing module firstly queries the P2P mirror image pull resource table from the P2P centralized retrieval server according to the seed file in the mirror image pull response data, then selects any P2P resource provider to be pulled from the obtained P2P mirror image pull resource table, performs parallel handshake with the selected P2P resource provider to be pulled, obtains Bitfield, request data block, receive data block and check data block, completes mirror image download, stores in the mirror image level storage directory in the local Docker working directory, and notifies the Docker container engine until the mirror image pull operation is completed. In addition, if the currently-allocated mirror image pulling mode of the working node is single-point mirror image pulling, the node processing module firstly informs the Docker container engine to complete single-point mirror image transmission action with the Docker mirror image warehouse, and then the node processing module downloads the seed file corresponding to the mirror image level. It should be noted that no matter how the working node performs mirror image pulling, after the mirror image level pulling is completed, a registration request is initiated to the index processing module of the Docker mirror image warehouse node, and the working node where the node processing module is located is registered as the P2P mirror image resource provider of the corresponding mirror image level, so as to participate in subsequent P2P mirror image transmission.
In this embodiment, based on the P2P technology, the work nodes storing the target mirror image can perform high-speed fine-grained mirror image distribution for the mirror image hierarchy among each other, so as to reduce the load of the mirror image warehouse, increase the mirror image pull speed of a large-scale container cluster, reduce the container scheduling delay, and increase the container deployment speed. Meanwhile, a time sequence prediction technology is fully utilized, future loads are predicted based on the existing load sequences, the influence of load wave peaks on a container cluster is effectively avoided, meanwhile, intelligent decision-making is carried out between P2P transmission and single-point transmission according to needs, distribution flexibility is improved, and image distribution stability is improved on the premise that image distribution speed is guaranteed.
In one embodiment, as shown in fig. 5, a method for pulling a node image of a Docker image warehouse of a container image distribution system is provided, the method comprising the following steps;
s31, receiving to-be-pulled mirror image information sent by a working node, inquiring a Docker mirror image warehouse of a Docker mirror image warehouse node according to the to-be-pulled mirror image information to obtain a to-be-pulled mirror image information list, and sending the to-be-pulled mirror image information list to the working node; the working node is any node in the Docker container node cluster; the mirror image element information to be pulled is obtained by receiving and analyzing a mirror image pulling request corresponding to the Docker client by the working node;
s32, receiving a mirror image level to be pulled sent by the working node, inquiring to obtain a corresponding seed file according to the mirror image level to be pulled, determining a mirror image pulling mode according to the workload of the Docker mirror image warehouse node, generating mirror image pulling response data according to the seed file and the mirror image pulling mode, and sending the mirror image pulling response data to the working node; the mirror image level to be pulled is determined by the working node according to the comparison result between the mirror image information list to be pulled and a local mirror image storage directory;
as shown in fig. 6, the step S32 of determining the mirror image pull manner according to the workload of the node itself includes:
s321, after the Docker mirror image warehouse node is started, continuously recording the working load of the Docker mirror image warehouse node according to a preset time interval;
s322, training to obtain a corresponding load prediction model according to the time sequence data of the working load;
the load prediction model may be a time sequence prediction model, as referred to above, and is not described herein again.
S323, obtaining a work load predicted value of a next time node of the Docker mirror image warehouse node according to the load prediction model;
s324, judging whether the working load predicted value is larger than a preset load threshold value, if so, determining that the mirror image pulling mode is P2P mirror image pulling, inquiring the P2P centralized retrieval server to obtain a corresponding P2P mirror image pulling resource table, and otherwise, determining that the mirror image pulling mode is single-point mirror image pulling.
S33, responding to the single-point mirror image pulling operation of the working node, reading the mirror image hierarchy to be pulled and the seed file in the Docker mirror image warehouse, and sending the mirror image hierarchy to be pulled and the seed file to the working node.
It should be noted that, the Docker mirror image warehouse node mirror image pulling method and the working node mirror image pulling method of the container mirror image distribution system are explained by taking a working node and a Docker mirror image warehouse node which interactively complete a mirror image pulling task in the container mirror image distribution system as main bodies, and details of the method steps can be referred to each other.
In one embodiment, as shown in fig. 7, there is provided a method for deleting a work node image of a container image distribution system, the method comprising the steps of:
s41, receiving and analyzing a mirror image deleting request corresponding to the Docker client to obtain a mirror image hierarchy to be deleted;
the mirror image deleting request of the Docker client is received by a Docker container engine of the working node, and is forwarded to a node processing module in the working node, the mirror image deleting request is analyzed by the node processing module to obtain a corresponding mirror image hierarchy to be deleted, and corresponding subsequent mirror image deleting operation is executed according to the mirror image hierarchy to be deleted.
S42, searching a seed file corresponding to the mirror image hierarchy to be deleted according to the mirror image hierarchy to be deleted;
the seed files are in one-to-one correspondence with the image levels, and in order to maintain synchronization between the image levels and the seed files in the local image working directory of the working node, the seed files corresponding to the image levels need to be found and deleted before the image levels are deleted.
S43, the seed files corresponding to the mirror image levels to be deleted and the deletion operation of the mirror image levels to be deleted are sequentially executed, and a logout application of a P2P mirror image resource provider is sent to the Docker mirror image warehouse node.
The logout application of the P2P mirror image resource provider is also realized by the fact that the node processing module of the working node is in butt joint with the P2P centralized retrieval server through the index processing module of the Docker mirror image warehouse node, the working node after the mirror image is deleted is ensured not to be listed as the corresponding P2P mirror image resource provider any more, and the accuracy and the high efficiency of the P2P mirror image pulling processing process of the subsequent container mirror image distribution system are further ensured.
It should be noted that, although the steps in the above-described flowcharts are shown in sequence as indicated by arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. In addition, the specific limitations of the above working node mirror image pushing method, working node mirror image pulling method, Docker mirror warehouse node mirror image pulling method, and working node mirror image deleting method related to the container mirror image distribution system may be mutually referred to with the limitations of the container mirror image distribution system described above.
Fig. 8 shows an internal structure diagram of a computer device in one embodiment, and the computer device may be specifically a terminal or a server. As shown in fig. 8, the computer apparatus includes a processor, a memory, a network interface, a display, and an input device, which are connected through a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. When being executed by a processor, the computer program realizes a work node mirror image pushing method, a work node mirror image pulling method, a Docker mirror image warehouse node mirror image pulling method and a work node mirror image deleting method of the container mirror image distribution system. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those of ordinary skill in the art that the architecture shown in FIG. 8 is a block diagram of only a portion of the architecture associated with the subject application, and is not intended to limit the computing devices to which the subject application may be applied, as a particular computing device may include more or less components than those shown, or may combine certain components, or have a similar arrangement of components.
In one embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the steps of the above method being performed when the computer program is executed by the processor.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the above-mentioned method.
To sum up, the embodiment of the present invention provides a container mirror image distribution system, and a corresponding work node mirror image pushing method, a work node mirror image pulling method, a Docker mirror image warehouse node mirror image pulling method, and a work node mirror image deleting method. Through the design of a container mirror image distribution system which simultaneously deploys a Docker container engine and a node processing module on a working node of a Docker container node cluster and simultaneously deploys a Docker mirror image warehouse, a P2P centralized retrieval server, a pushing processing module, a pulling processing module, an index processing module and a load processing module on a Docker mirror image warehouse node, the method realizes that after a seed file is manufactured by the node processing module of the working node according to a mirror image level to be pushed and is stored in a local mirror image directory, and after the mirror image level to be pushed and a corresponding seed file are pushed to the mirror image warehouse through interaction with the pushing processing module of the Docker mirror image warehouse node, the working node and the Docker mirror image warehouse node are both registered as a pushing method of a P2P mirror image resource provider; after the node processing module of the working node determines the mirror image level to be pulled, the interaction with the pulling processing module and the index processing module of the Docker mirror image warehouse node and the interaction between the pulling processing module and the load processing module are realized, a mirror image pulling mode is appointed to the working node by the Docker mirror image warehouse node based on the current load of the Docker mirror image warehouse node on the basis of the P2P technology and a time sequence prediction method, and the working node adopts single-point mirror image pulling or P2P mirror image pulling to obtain the mirror image level to be pulled and then registers as a pulling method of a P2P mirror image resource provider; the technical scheme of container mirror image distribution of a deletion method for dynamically deleting content mirror images through interaction of a node processing module of a working node, an index processing module of a Docker mirror image warehouse node and a P2P centralized retrieval server is characterized in that the working nodes storing target mirror images can perform high-speed fine-grained mirror image distribution for a mirror image level based on a P2P technology, the number of seed files in a cluster is reduced, the load of a mirror image warehouse is reduced, the mirror image pulling speed of a large-scale container cluster is increased, the container scheduling delay is reduced, the container deployment speed is increased, the performance bottleneck problem that the mirror image warehouse limits the container starting speed in the large-scale container cluster is effectively solved, meanwhile, the future load is predicted based on the existing load sequence by fully utilizing a time sequence prediction technology, the influence of a load wave peak on the container cluster is effectively avoided, and intelligent decision-on-demand between P2P transmission and single-point transmission is realized, the distribution flexibility is improved, the image distribution stability is improved on the premise of ensuring the image distribution speed, and unnecessary network bandwidth loss is reduced.
Based on the P2P technology, the work nodes with the target mirror images can perform high-speed fine-grained mirror image distribution on the mirror image level among each other, so that the load of a mirror image warehouse is reduced, the mirror image pulling speed of a large-scale container cluster is increased, the container scheduling time delay is reduced, and the container deployment speed is increased. Meanwhile, each embodiment in the present specification is described in a progressive manner, and each embodiment may refer to the same or similar parts directly, and each embodiment focuses on the difference from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment. It should be noted that, the technical features of the embodiments may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express some preferred embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for those skilled in the art, various modifications and substitutions can be made without departing from the technical principle of the present invention, and these should be construed as the protection scope of the present application. Therefore, the protection scope of the present patent shall be subject to the protection scope of the claims.

Claims (10)

1. A container mirror image distribution system, the system comprising a Docker container node cluster and a Docker mirror image warehouse node, characterized in that:
the Docker container node cluster comprises a plurality of working nodes; the working node is used for receiving a mirror image event request of a Docker client in real time, generating a node mirror image event request according to the mirror image event request, sending the node mirror image event request to the Docker mirror image warehouse node, and executing the mirror image event request according to a mirror image request event response returned by the Docker mirror image warehouse node;
the Docker mirror image warehouse node is used for operating a mirror image warehouse program and a P2P centralized retrieval server, responding to the node mirror image event request of any one working node, matching the corresponding working node to complete the mirror image event request, and maintaining the mirror image change of the Docker mirror image warehouse node through the P2P centralized retrieval server.
2. The container image distribution system of claim 1, wherein the working nodes comprise a Docker container engine and a node processing module deployed concurrently with the Docker container engine;
the Docker container engine is used for receiving the mirror event request and forwarding the mirror event request to the node processing module; the mirror image event request comprises a mirror image pushing request, a mirror image pulling request and a mirror image deleting request;
the node processing module is used for receiving and analyzing the mirror image pushing request, sending mirror pixel information to be pushed of the mirror image pushing request to the Docker mirror image warehouse node, making a corresponding seed file according to a mirror image level list to be pushed returned by the Docker mirror image warehouse node, sending the stored mirror image level list to be pushed and the corresponding seed file to the Docker mirror image warehouse node, and sending a registration application of a P2P mirror image resource provider to the Docker mirror image warehouse node in response to a successful mirror image pushing result of the Docker mirror image warehouse node; or the like, or, alternatively,
the system comprises a Docker mirror image warehouse node, a mirror image pull request receiving and analyzing module, a mirror image information list sending module, a mirror image pull request receiving and analyzing module, a mirror image pull request sending module, a mirror image information list sending module, a mirror image pull request sending module and a mirror image pull request sending module, wherein the mirror image pull request sending module is used for sending mirror image information to be pulled of the mirror image pull request to the Docker mirror image warehouse node, comparing the mirror image information list to be pulled returned by the Docker mirror image warehouse node with a mirror image storage directory of the working node, determining a mirror image level to be pulled, sending the mirror image pull request to be pulled, sending the mirror image pull request to the mirror image level to be pulled to the Docker mirror image warehouse node, sending to be pulled to the Docker mirror image warehouse node to be pulled to be; or the like, or, alternatively,
the system is used for receiving and analyzing the mirror image deleting request, sequentially deleting the seed file of the mirror image hierarchy to be deleted and the mirror image hierarchy to be deleted according to the mirror image hierarchy to be deleted of the mirror image deleting request, and sending a logout application of a P2P mirror image resource provider to the Docker mirror image warehouse node.
3. The container mirror distribution system according to claim 1, wherein the Docker mirror warehouse nodes include a Docker mirror warehouse, and a push processing module, a pull processing module, and an index processing module that are deployed simultaneously with the Docker mirror warehouse;
the Docker mirror image warehouse is used for storing mirror images required by the operation of the Docker container node cluster;
the pushing processing module is used for receiving the mirror image pushing request of any working node, comparing the mirror image information to be pushed of the mirror image pushing request with the existing mirror image level of the Docker mirror image warehouse, returning to the corresponding mirror image level list to be pushed, receiving and storing the mirror image level list to be pushed and the corresponding seed file sent by the working node, creating a corresponding seed index according to the seed file, and registering the Docker mirror image warehouse node as a P2P mirror image resource provider;
the pull processing module is used for receiving the mirror image pull request of any one working node, returning a mirror image information list to be pulled of the Docker mirror image warehouse according to the mirror image pull request, inquiring a seed file corresponding to the working node according to the mirror image level to be pulled of the working node responding to the mirror image information list to be pulled, determining a mirror image pull mode according to the working load of the Docker mirror image warehouse node, sending mirror image pull response data generated according to the seed file and the mirror image pull mode to the working node, and responding to the single-point pull action of the working node to obtain a mirror image level file to be pulled for transmission;
the index processing module is used for connecting the P2P centralized retrieval server and realizing registration and logout of the P2P mirror image resource provider according to the mirror image event request.
4. The container mirror distribution system of claim 3, wherein the Docker mirror warehouse node further comprises a load handling module deployed concurrently with the Docker mirror warehouse;
the load processing module is used for recording the workload of the Docker mirror image warehouse node at preset time intervals, establishing a corresponding load prediction model according to the time sequence data of the workload, and predicting the workload of the Docker mirror image warehouse node at the next time node in real time according to the load prediction model.
5. A method for pushing a working node image of any container image distribution system according to claims 1-4, wherein the method comprises the steps of:
receiving and analyzing a mirror image pushing request corresponding to a Docker client to obtain mirror image information to be pushed, and sending the mirror image information to be pushed to a Docker mirror image warehouse node; the working node is any node in the Docker container node cluster;
receiving a mirror image hierarchical list to be pushed, manufacturing a corresponding seed file according to the mirror image hierarchical list to be pushed, and pushing the mirror image hierarchical list to be pushed and the corresponding seed file to the Docker mirror image warehouse node after index storage of the mirror image hierarchical list to be pushed and the corresponding seed file; the mirror image level list to be pushed is determined by the Docker mirror image warehouse node according to the comparison result between the mirror image information to be pushed and the existing mirror image level of the Docker mirror image warehouse;
responding to a successful mirror image pushing result, and sending a registration application of a P2P mirror image resource provider to the Docker mirror image warehouse node; the successful mirror image pushing result is received by the Docker mirror image warehouse node, the to-be-pushed mirror image hierarchical list and the corresponding seed file are stored, the corresponding seed index is created, and the Docker mirror image warehouse node is registered as a P2P mirror image resource provider and then generated.
6. A method for pulling a work node image of a container image distribution system according to claims 1-4, wherein the method comprises the steps of:
receiving and analyzing a mirror image pulling request corresponding to a Docker client to obtain mirror image information to be pulled, and sending the mirror image information to be pulled to a Docker mirror image warehouse node;
receiving a mirror image information list to be pulled, comparing the mirror image information list to be pulled with a local mirror image storage directory, determining a mirror image level to be pulled, and sending the mirror image level to be pulled to the Docker mirror image warehouse node; the mirror image level meta-information list is obtained by the Docker mirror image warehouse node inquiring a Docker mirror image warehouse of the Docker mirror image warehouse node according to the mirror image information to be pulled;
receiving and analyzing mirror image pulling response data, acquiring a mirror image pulling mode in the mirror image pulling response data, executing mirror image pulling operation according to the mirror image pulling mode, and sending a P2P mirror image resource provider registration application to the Docker mirror image warehouse node after the pulling operation is completed; and the mirror image pulling response data is inquired by the Docker mirror image warehouse node according to the mirror image level to be pulled to obtain a corresponding seed file, and after a mirror image pulling mode is determined according to the workload of the Docker mirror image warehouse node, the mirror image pulling response data is generated according to the seed file and the mirror image pulling mode.
7. The method for pulling a mirror image of a working node according to claim 6, wherein the step of receiving and analyzing the mirror image pull response data to obtain a mirror image pull mode in the mirror image pull response data, and the step of executing the mirror image pull operation according to the mirror image pull mode includes:
analyzing the mirror image pulling response data to obtain the mirror image pulling mode;
judging whether the mirror image pulling mode is P2P mirror image pulling;
if the mirror image pulling mode is the P2P mirror image pulling, acquiring a corresponding P2P mirror image pulling resource table, selecting any P2P resource provider to be pulled on the P2P mirror image pulling resource table to establish a chain, executing the mirror image pulling operation, and otherwise, sending a single-point mirror image pulling operation to the Docker mirror image warehouse node.
8. A method for pulling a Docker mirror warehouse node mirror of a container mirror distribution system according to claims 1-4, wherein the method comprises the steps of:
receiving to-be-pulled mirror pixel information sent by a working node, inquiring a Docker mirror image warehouse of a Docker mirror image warehouse node according to the to-be-pulled mirror pixel information to obtain a to-be-pulled mirror pixel information list, and sending the to-be-pulled mirror pixel information list to the working node; the working node is any node in the Docker container node cluster; the mirror image element information to be pulled is obtained by receiving and analyzing a mirror image pulling request corresponding to the Docker client by the working node;
receiving a mirror image level to be pulled sent by the working node, inquiring to obtain a corresponding seed file according to the mirror image level to be pulled, determining a mirror image pulling mode according to the working load of the Docker mirror image warehouse node, generating mirror image pulling response data according to the seed file and the mirror image pulling mode, and sending the mirror image pulling response data to the working node; the mirror image level to be pulled is determined by the working node according to the comparison result between the mirror image information list to be pulled and a local mirror image storage directory;
and responding to the single-point mirror image pulling operation of the working node, reading the mirror image hierarchy to be pulled and the seed file in the Docker mirror image warehouse, and sending the mirror image hierarchy to be pulled and the seed file to the working node.
9. The method for pulling a mirror image of a node in a Docker mirror warehouse according to claim 8, wherein the step of determining a mirror image pulling manner according to a workload of the node itself comprises:
after the Docker mirror image warehouse node is started, continuously recording the workload of the Docker mirror image warehouse node according to a preset time interval;
training to obtain a corresponding load prediction model according to the time sequence data of the working load;
obtaining a working load prediction value of a next time node of the Docker mirror image warehouse node according to the load prediction model;
and judging whether the working load predicted value is greater than a preset load threshold value, if so, determining that the mirror image pulling mode is P2P mirror image pulling, and inquiring the P2P centralized retrieval server to obtain a corresponding P2P mirror image pulling resource table, otherwise, determining that the mirror image pulling mode is single-point mirror image pulling.
10. A method for deleting a working node image of a container image distribution system according to claims 1 to 4, wherein the method comprises the steps of:
receiving and analyzing a mirror image deleting request corresponding to the Docker client to obtain a mirror image hierarchy to be deleted;
searching a seed file corresponding to the mirror image level to be deleted according to the mirror image level to be deleted;
and sequentially executing the seed file corresponding to the mirror image hierarchy to be deleted and the deletion operation of the mirror image hierarchy to be deleted, and sending a logout application of the P2P mirror image resource provider to the Docker mirror image warehouse node.
CN202110894323.XA 2021-08-04 2021-08-04 Container mirror image distribution system and container mirror image pushing, pulling and deleting method Active CN113760453B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110894323.XA CN113760453B (en) 2021-08-04 2021-08-04 Container mirror image distribution system and container mirror image pushing, pulling and deleting method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110894323.XA CN113760453B (en) 2021-08-04 2021-08-04 Container mirror image distribution system and container mirror image pushing, pulling and deleting method

Publications (2)

Publication Number Publication Date
CN113760453A true CN113760453A (en) 2021-12-07
CN113760453B CN113760453B (en) 2024-05-28

Family

ID=78788591

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110894323.XA Active CN113760453B (en) 2021-08-04 2021-08-04 Container mirror image distribution system and container mirror image pushing, pulling and deleting method

Country Status (1)

Country Link
CN (1) CN113760453B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114860344A (en) * 2022-05-26 2022-08-05 中国工商银行股份有限公司 Container starting method and device, computer equipment and storage medium
CN115695453A (en) * 2022-10-31 2023-02-03 浪潮云信息技术股份公司 Method, device, equipment and medium for improving stability of mirror image warehouse
CN115766739A (en) * 2022-10-14 2023-03-07 济南浪潮数据技术有限公司 Container mirror image distribution method, device, system and medium thereof
CN117270897A (en) * 2023-11-21 2023-12-22 厦门泛卓信息科技有限公司 Computer system based on low-code business system

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104735164A (en) * 2015-04-10 2015-06-24 网易（杭州）网络有限公司 Method and device for saving file information
CN107819802A (en) * 2016-09-13 2018-03-20 华为软件技术有限公司 A kind of mirror image acquisition methods, node device and server in node cluster
CN109144485A (en) * 2018-09-10 2019-01-04 南方电网科学研究院有限责任公司 Micro-service deployment method, device, equipment and readable storage medium
CN109491758A (en) * 2018-10-11 2019-03-19 深圳市网心科技有限公司 Docker mirror image distribution method, system, data gateway and computer readable storage medium
CN109543415A (en) * 2018-11-20 2019-03-29 南方电网科学研究院有限责任公司 Safe operating system architecture
CN110099076A (en) * 2018-01-29 2019-08-06 中兴通讯股份有限公司 A kind of method and its system that mirror image pulls
CN110647580A (en) * 2019-09-05 2020-01-03 南京邮电大学 Distributed container cluster mirror image management main node, slave node, system and method
CN111045786A (en) * 2019-11-28 2020-04-21 北京大学 Container creation system and method based on mirror image layering technology in cloud environment
CN111190547A (en) * 2019-12-30 2020-05-22 中国电子科技集团公司信息科学研究院 Distributed container mirror image storage and distribution system and method
CN111385325A (en) * 2018-12-28 2020-07-07 中国移动通信集团浙江有限公司 File distribution system and method based on P2P
CN111432006A (en) * 2020-03-30 2020-07-17 中科九度(北京)空间信息技术有限责任公司 Lightweight resource virtualization and distribution method
CN112231052A (en) * 2020-09-29 2021-01-15 中山大学 High-performance distributed container mirror image distribution system and method
CN112433810A (en) * 2020-11-05 2021-03-02 北京浪潮数据技术有限公司 Docker container mirror image storage and pulling method and device and computer readable storage medium
WO2021143590A1 (en) * 2020-01-14 2021-07-22 南京邮电大学 Distributed container image construction scheduling system and method

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104735164A (en) * 2015-04-10 2015-06-24 网易（杭州）网络有限公司 Method and device for saving file information
CN107819802A (en) * 2016-09-13 2018-03-20 华为软件技术有限公司 A kind of mirror image acquisition methods, node device and server in node cluster
CN110099076A (en) * 2018-01-29 2019-08-06 中兴通讯股份有限公司 A kind of method and its system that mirror image pulls
CN109144485A (en) * 2018-09-10 2019-01-04 南方电网科学研究院有限责任公司 Micro-service deployment method, device, equipment and readable storage medium
CN109491758A (en) * 2018-10-11 2019-03-19 深圳市网心科技有限公司 Docker mirror image distribution method, system, data gateway and computer readable storage medium
CN109543415A (en) * 2018-11-20 2019-03-29 南方电网科学研究院有限责任公司 Safe operating system architecture
CN111385325A (en) * 2018-12-28 2020-07-07 中国移动通信集团浙江有限公司 File distribution system and method based on P2P
CN110647580A (en) * 2019-09-05 2020-01-03 南京邮电大学 Distributed container cluster mirror image management main node, slave node, system and method
CN111045786A (en) * 2019-11-28 2020-04-21 北京大学 Container creation system and method based on mirror image layering technology in cloud environment
CN111190547A (en) * 2019-12-30 2020-05-22 中国电子科技集团公司信息科学研究院 Distributed container mirror image storage and distribution system and method
WO2021143590A1 (en) * 2020-01-14 2021-07-22 南京邮电大学 Distributed container image construction scheduling system and method
CN111432006A (en) * 2020-03-30 2020-07-17 中科九度(北京)空间信息技术有限责任公司 Lightweight resource virtualization and distribution method
CN112231052A (en) * 2020-09-29 2021-01-15 中山大学 High-performance distributed container mirror image distribution system and method
CN112433810A (en) * 2020-11-05 2021-03-02 北京浪潮数据技术有限公司 Docker container mirror image storage and pulling method and device and computer readable storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
张浩等: "RainbowD:一种异构云环境下高效的Docker镜像分发系统", 《计算机学报》, vol. 43, no. 11, 30 November 2020 (2020-11-30), pages 2067 - 2083 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114860344A (en) * 2022-05-26 2022-08-05 中国工商银行股份有限公司 Container starting method and device, computer equipment and storage medium
CN115766739A (en) * 2022-10-14 2023-03-07 济南浪潮数据技术有限公司 Container mirror image distribution method, device, system and medium thereof
CN115695453A (en) * 2022-10-31 2023-02-03 浪潮云信息技术股份公司 Method, device, equipment and medium for improving stability of mirror image warehouse
CN117270897A (en) * 2023-11-21 2023-12-22 厦门泛卓信息科技有限公司 Computer system based on low-code business system
CN117270897B (en) * 2023-11-21 2024-03-08 北京祝融视觉科技股份有限公司 Computer system based on low-code business system

Also Published As

Publication number Publication date
CN113760453B (en) 2024-05-28

Similar Documents

Publication Publication Date Title
CN113760453B (en) Container mirror image distribution system and container mirror image pushing, pulling and deleting method
CN109976667B (en) Mirror image management method, device and system
CN111897638B (en) Distributed task scheduling method and system
CN111460298A (en) Service data processing method and device, computer equipment and storage medium
CN104981768A (en) Cloud-based streaming data receiver and persister
KR101545146B1 (en) System and method for job execution in conjunction with cloud storage
CN109643305A (en) Data query method, application and database server, middleware and system
JP7130803B2 (en) SEARCH METHOD, SEARCH DEVICE, ELECTRONIC DEVICE AND STORAGE MEDIUM
JP4336363B2 (en) Business process execution method, business process execution system, and program
CN113760638A (en) Log service method and device based on kubernets cluster
CN111193789A (en) Subscription information pushing method and device, computer equipment and readable storage medium
CN111131487B (en) Deep learning platform container mirror image obtaining and sharing method and system
CN110168513A (en) The part of big file is stored in different storage systems
EP1079318A2 (en) Message processing system
US20230401235A1 (en) Low latency ingestion into a data system
CN114968495B (en) Method and system for realizing ground science data sharing based on kubernetes platform
CN106302641A (en) A kind of methods, devices and systems of upper transmitting file
CN103425707A (en) Data analyzing method and data analyzing device
CN106936911B (en) Lightweight distributed computing platform and computing resource management and control method thereof
CN115525440A (en) Method, device, medium and equipment for processing microservice calling data
CN112099933B (en) Task operation and query method and device, electronic equipment and storage medium
KR20210095096A (en) Method and apparatus for identifying user, storage medium, and electronic device
CN111078736A (en) Data aggregation processing method and device, terminal and storage medium
CN108733805B (en) File interaction method, system, computer equipment and storage medium
CN112307289A (en) Search method, node, and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant