WO2019144965A1 - 一种镜像拉取的方法及其系统 - Google Patents

一种镜像拉取的方法及其系统 Download PDF

Info

Publication number
WO2019144965A1
WO2019144965A1 PCT/CN2019/073581 CN2019073581W WO2019144965A1 WO 2019144965 A1 WO2019144965 A1 WO 2019144965A1 CN 2019073581 W CN2019073581 W CN 2019073581W WO 2019144965 A1 WO2019144965 A1 WO 2019144965A1
Authority
WO
WIPO (PCT)
Prior art keywords
client
pull
image
mirror
subnet
Prior art date
Application number
PCT/CN2019/073581
Other languages
English (en)
French (fr)
Inventor
李夏隆
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Priority to EP19743986.2A priority Critical patent/EP3748934B1/en
Priority to US16/965,560 priority patent/US20210051211A1/en
Publication of WO2019144965A1 publication Critical patent/WO2019144965A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1029—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers using data related to the state of servers by a load balancer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G06F8/61—Installation
    • G06F8/63—Image based installation; Cloning; Build to order
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00—Network arrangements, protocols or services for addressing or naming
    • H04L61/09—Mapping addresses
    • H04L61/25—Mapping addresses of the same type
    • H04L61/2503—Translation of Internet protocol [IP] addresses
    • H04L61/256—NAT traversal
    • H04L61/2571—NAT traversal for identification, e.g. for authentication or billing 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1087—Peer-to-peer [P2P] networks using cross-functional networking aspects
    • H04L67/1091—Interfacing with client-server systems or between P2P systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/2866—Architectures; Arrangements
    • H04L67/289—Intermediate processing functionally located close to the data consumer application, e.g. in same machine, in same home or in same sub-network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/34—Network arrangements or protocols for supporting network services or applications involving the movement of software or configuration parameters 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/563—Data redirection of data network streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/566—Grouping or aggregating service requests, e.g. for unified processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • H04L67/61—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources taking into account QoS or priority requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • H04L67/63—Routing a service request depending on the request content or context
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/12—Shortest path evaluation
    • H04L45/122—Shortest path evaluation by minimising distances, e.g. by selecting a route with minimum of number of hops
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00—Network arrangements, protocols or services for addressing or naming
    • H04L61/09—Mapping addresses
    • H04L61/25—Mapping addresses of the same type
    • H04L61/2503—Translation of Internet protocol [IP] addresses
    • H04L61/2514—Translation of Internet protocol [IP] addresses between local and global IP addresses
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004—Server selection for load balancing
    • H04L67/1021—Server selection for load balancing based on client or server locations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Definitions

  • the invention relates to the field of containerized platforms, and in particular to a method and a system for mirror image drawing.
  • Container technology virtualization technology has become a widely recognized method of server resource sharing, which provides system administrators with great flexibility in building operating system instances on demand.
  • Containers are completely sandboxed and do not have any interfaces to each other.
  • Docker is an open source application container engine that allows developers to package their applications and dependencies into a portable container, then publish them to a Linux machine, or virtualize them.
  • the emergence of container technology has led to the gradual containerization of distributed systems based on virtual machine node deployment. For example, google's kubernetes, swarm, etc. have realized containerized deployment, and the use of container technology has become increasingly widespread.
  • the number of nodes used is hundreds or thousands (for example, a kubernetes cluster system claims to support 2000 nodes), and the deployment process is mainly spent in the image distribution process. Therefore, when a large number of nodes deploy container application clusters and update images for the first time, they will face the bottleneck of the Registry of the private warehouse, including the processing bottleneck of the server where the private warehouse is located and the network bandwidth bottleneck faced by the server. In the case of limited network bandwidth and processing capacity, the Registry is faced with a large number of concurrent operations, and it is inevitable that queues are waiting for the mirror to be pulled, which is relatively slow.
  • the main principle is to share the mirrors of each node through the Bit Torrent protocol to implement P2P acceleration. Due to the principle of P2P technology, the technology needs to split and compress the image file, and transmit and accelerate all the nodes in the cluster, which will consume the bandwidth resources of the network to some extent. In addition, the complexity of the system is increased by the need to split the image file and maintain the synchronization of each split file.
  • a method for mirroring is applied to a server, including:
  • the method after receiving the image pull request sent by the at least one requesting client, the method includes:
  • one or more of the image pull requests are selected according to a preset priority pull principle for priority processing.
  • the selecting a plurality of the image pull requests according to a preset priority pull principle for priority processing includes:
  • the mirror pull request is selected in different subnets.
  • the image pull request selected in different subnets includes:
  • the mirror pull request is selected in the subnet having the larger number of online clients.
  • the determining the first client comprises:
  • the client having the mirror image corresponding to the mirror pull request in the other subnet is determined, and at least one client closest to the server is selected as the first client.
  • the method before the sending the redirect pull image message to the requesting client, the method includes:
  • the image pull request sent by the receiving request client includes:
  • the present invention also provides a system for mirroring, applied to a server, comprising: a memory, a processor, and a computer program stored on the memory and operable on the processor, the computer program being The steps of the method for mirroring the image provided by the embodiment of the present invention are implemented when the processor is executed.
  • the present invention also provides a computer readable storage medium having a mirrored pull program stored thereon, and the image pull method provided by the embodiment of the present invention is implemented when the image pull program is executed by the processor A step of.
  • the present invention also provides a system for mirroring, comprising: a server and at least one client; the server is configured to send a redirect pull to the client according to the image pull request sent by the client Obtaining a mirroring message, or establishing a mirrored transmission with the client; wherein the redirecting pull mirroring message is used by the requesting client to pull a mirror from the first client, where the first client is configured with The image corresponding to the mirror pull request.
  • the image capturing method and system thereof utilize the network bandwidth of each subnet to realize the shared transmission of images in each subnet; the transmission protocol used is simple, and only a single point to single point transmission protocol is needed. Yes, solve the problem of network and processing capacity bottlenecks of the services of the private warehouse (Registry).
  • the invention can also optimize the priority order of the push mirror according to the network condition of the specific cluster, and maximize the use of the switch network to accelerate the pull of the cluster image.
  • FIG. 1 is a schematic flowchart of a method for applying image capture to a server according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart diagram of a method for mirroring a server to be applied to a server according to another embodiment of the present invention
  • FIG. 3 is a schematic flowchart of a method for applying image capture to a server according to another embodiment of the present invention.
  • FIG. 4 is a schematic flowchart of a system for mirroring and pulling according to an embodiment of the present invention
  • both the server and the client provide a warehouse service, so that the container engine (such as Docker) can pull the image from the server of the server or the local client.
  • the client itself is also a mirror, which needs to be pulled from the server and run the client container.
  • an embodiment of the present invention provides a method for mirroring, which is applied to a server, and includes the following steps:
  • the client After the client runs, it will connect to the server (can be understood as the login operation), and send the basic information of the local node, such as the local IP.
  • the connection is successful, and a heartbeat connection is maintained between them, similar to the client and server heartbeat mechanism of the communication software, which occupies only a small amount of resources. As long as the heartbeat continues, the server considers these clients to be online and is an online client.
  • the online client When the online client issues a mirror pull request, it is the requesting client.
  • step 103 Determine whether the current load status of the server is a high load operation state. If yes, proceed to step 104; if no, proceed to step 105.
  • the first client is configured with a mirror corresponding to the mirror pull request.
  • the redirect pull image message is used by the requesting client to pull a mirror from the first client.
  • the image corresponding to the image pull request is directly transmitted to the request client.
  • the server waits in line.
  • the server After the server completes a push task, it records which images are currently stored by the client, and maintains synchronization through a regular synchronization mechanism.
  • the server When a new request to pull the image comes, if the server is in the high load phase, it will search for the client that owns the image in the current online client, and send a message to the requesting client to have the image. The first client pulls the image.
  • the embodiment of the present invention provides a method for mirroring, which is applied to a server, and includes the following steps:
  • the client After the client runs, it will connect to the server (can be understood as the login operation), and send the basic information of the local node, such as the local IP.
  • the connection is successful, and a heartbeat connection is maintained between them, similar to the client and server heartbeat mechanism of the communication software, which occupies only a small amount of resources. As long as the heartbeat continues, the server considers these clients to be online and is an online client.
  • the online client When the online client issues a mirror pull request, it is the requesting client.
  • step 203 Determine whether the current load status of the server is a high load running status. If yes, go to step 204; if no, go to step 206.
  • step 204 Search for a first client, where the first client is configured with an image corresponding to the image pull request; if yes, go to step 205; if no, go to step 206.
  • step 206 Determine whether the number of the received mirror pull requests exceeds a preset threshold; if yes, go to step 207; if no, go to step 208.
  • the default priority pull principle is prioritized to cover all subnets.
  • the server maintains a table of client network information according to the obtained IP information of the client, and classifies the node where the client is located according to the subnet and the routing path.
  • the image corresponding to the image pull request is directly transmitted to the request client.
  • the transmission priority is adjusted by using a preset priority pull principle, and the request that needs to be processed preferentially is selected. For the remaining requests, if the server is in a heavy load period, then the online client is asked whether it has the same image, and if so, a resource redirect message is sent to the requesting client, and the client is sent to the other client. The node that has the image pulls the image. The server will give priority to the host that is the same as or adjacent to the subnet where the client requesting the mirror is located, usually a list of hosts for the client to try to connect multiple times.
  • the embodiment of the present invention provides a method for performing image capture.
  • the method of selecting a plurality of the image pull requests according to a preset priority pull principle for priority processing includes:
  • the mirror pull request is selected in different subnets.
  • deploying and updating patches means replacing the boot image.
  • the server When the method for accelerating large-scale image pull is provided by the present invention, it is first necessary to have a main warehouse server, and the server provides a private warehouse service for all nodes to pull the corresponding client image for the first time and run.
  • the client image itself also provides the warehouse service.
  • the client sends the network information of the node to the server, including the IP address, mask and gateway of the external communication network. And to maintain a heartbeat connection with the server, as long as the heartbeat is continuous, the server considers these clients to be online.
  • the server After receiving the information of the client, the server maintains the information of the client, and the source IP address of the client is combined with the routing path of the client to each client, and the client is grouped according to the subnet. Let's assume that these 300 nodes belong to three subnets: subnet A, subnet B, and subnet C.
  • each node will first access the local client's warehouse service.
  • the local repository has the image and pulls it directly. If not, it pulls the image request from the server. In the case of the first deployment, each node does not have the image.
  • the client needs to pull the image from the server.
  • the server selects multiple of the mirror pull requests according to the preset priority pull principle to prioritize the processing of each node. request.
  • the preset priority pull principle includes: determining a different subnet priority principle according to the number of the subnets. Assuming that there are requests from x1, x2, and x3 nodes at the same time, the current server can only prioritize two requests after considering the current bandwidth usage and efficiency. According to the information of x1, x2 and x3, x1 and x2 belong to the same subnet A, then the server will preferentially satisfy the pull requests of x1 and x3 or x2 and x3, that is, the requests from the nodes of different subnets are preferentially satisfied. , try to cover all subnets first.
  • the mirror pull request selected in different subnets includes:
  • the mirror pull request is selected in the subnet having the larger number of online clients.
  • the threshold is n, then one mirror pull request is selected in each of the n subnets having a larger number of online clients.
  • the preset priority pull principle further includes: a large subnet priority principle determined according to the number of online clients owned by the subnet.
  • a large subnet priority principle determined according to the number of online clients owned by the subnet.
  • the government enterprise network consists of three subnets, subnet A, subnet B, and subnet C. There is no NAT mapping, and the relationship between the client IP and the subnet is as follows:
  • Subnet A 10.113.4.2 ⁇ 10.113.4.241
  • a total of 240 nodes are connected to the server online.
  • Subnet B 10.113.3.2 ⁇ 10.113.3.201 A total of 200 nodes online
  • Subnet C 10.115.2.2 ⁇ 10.115.2.101 A total of 100 nodes online
  • these nodes can send mirror pull requests to the server at the same time or during the period, according to the principle of different subnets and the principle of large subnet priority; if it is within a short period of time (assumed to be within 3 seconds) 3
  • Each subnet has two requests (a total of six requests) to the server, and the server (assuming that the server can only satisfy n concurrent requests at a time) will first satisfy subnet A (assuming that the request of 10.113.4.2 is met) ), subnet B (assumed to meet 10.113.3.2) and subnet C (assumed to meet 10.115.2.2) request (different subnet priority principle, that is, as much as possible to cover different subnets as much as possible).
  • n is 4.
  • the node request of subnet A, the node request of subnet B, and the node request of subnet C are respectively satisfied first.
  • the remaining connection capability will be given to subnet A. This is based on the principle of large subnets. Since the number of online clients of subnet A is large, the remaining connections are temporarily queued. That is to say, two requests from subnet A will be processed first, and one request from each of subnets B and C will be prioritized, that is, at least one request per subnet is selected.
  • the remaining connection capability can be allocated according to the time when the request arrives, that is, the preset.
  • the principle of priority pull can also include: the principle of time priority determined according to chronological order.
  • n is 2.
  • the node requests of subnet A are respectively satisfied first, and the nodes of subnet B are requested. This is because subnet A and subnet.
  • the number of online clients of B is greater than the number of online clients of subnet C.
  • the two connection capabilities can be allocated according to the time of the request, that is, the preset priority.
  • the principle of pulling can also include: the principle of time priority determined according to chronological order.
  • the embodiment of the present invention provides a method for mirroring a pull.
  • the determining the first client includes:
  • step 302. Determine whether an online client exists in the subnet where the requesting client is located; if yes, go to step 303; if no, go to step 304.
  • the online client is configured with a mirror corresponding to the mirror pull request.
  • the node that is in the same subnet as the requested node is preferentially searched, and whether the online client of the subnet also has the same image, and if yes, sends a “redirect pull” to the requesting client. Take the mirror message, so that the requesting client pulls the image to the first client that has the same image in its subnet.
  • the first client can have multiple.
  • x1, x2, and x3 nodes belong to three different subnets A, B, and C
  • x1, x2, and x3 all have a mirror image X, and now there is a new pull.
  • the request for mirror X is from x4, x5 and x6, where x4, x5 belong to subnet A and x6 belongs to subnet C.
  • the server is only able to satisfy the x4 request because of the large load.
  • x5 and x6 by querying the client information, it is known that x1 and x3 also have mirror X, and send a message confirmation to x1 and x3, and because x1 And x5 belongs to subnet A, and x3 and x6 belong to subnet C, then the server will send "redirect pull mirror message" to x5 and x6, let x5 pull mirror X from x1, x6 pulls from x3 Mirroring, so that the traffic of x5 and x6 pull mirrors is limited to subnets A and C. At the same time, the speed of the switch network using subnets is greatly improved. When x5 and x6 pull the image, it will send feedback to the server to inform that it has completed the pull of the mirror X.
  • the server queries the subnet A to which x5 belongs and does not have the image X, but by querying the online client, the node x7 and the subnet C of the subnet B are known. Node x8 has a mirror X. Then, at this time, the server preferentially selects a node of a subnet that is closer to the subnet A to which the x5 belongs, and causes the x5 to redirect the pull request.
  • the routing path is r 1 , r 2 , r 3
  • the routing path to subnet B is r 1 , r 2 , r 4 , to subnet C.
  • the routing path of the node is r 1 , r 2 , r 5 , r 6
  • the routing path selection is basically Certainly, in this scenario, it can be considered that the number of route hops passed by the nodes of subnet B and subnet A is less than the number of hops of subnet C and subnet A.
  • the server will preferentially let x5 pull the image to node x7 of subnet B.
  • Sr i and Sr j are ordered sets of routing paths of subnets i and j respectively.
  • the routing path set from the server to the subnet A is ⁇ r 1 , r 2 , r 3 ⁇
  • the set of routing paths to subnet B is ⁇ r 1 , r 2 , r 4 ⁇
  • the set of routing paths to subnet C is ⁇ r 1 , r 2 , r 5 , r 6 ⁇ , then according to formula (1)
  • the routing distance between subnets A and B is
  • the method before the sending the redirected pull image message to the requesting client, the method further includes:
  • the server considers that it is not suitable for file transfer between the two subnets at this time. The efficiency improvement is not high. At this point, the node needs to wait for the server to establish a mirrored transmission.
  • the server When the first batch of requests are processed and there are more requests for the concurrent processing capability of the server, the server will perform a redirect pull request. Assume that there is a node 10.113.4.5 from subnet A requesting image pull. At this time, the server does not have redundant connection capability.
  • the client management analysis module tries to analyze the existing online client. The server first goes to the same 10.113. 4.5 The online client query on the same subnet (subnet A) has the image. If the 10.113.4.2 node that is also located in subnet A has the requested image, send a resource redirection message to 10.113. The client of 4.5 requests the client of 10.113.4.2 to pull the image.
  • routing path of subnet A is:
  • the routing path of subnet B is:
  • the routing path of subnet C is:
  • subnet B is closer to subnet A. Therefore, when subnet A does not search for the node that owns the mirror, it searches for the client in the adjacent subnet B to have the image. And so on, if there is no subnet B, it will specifically search for nodes in subnet C.
  • the server may not redirect the pull request, but the server provides the mirror transfer.
  • the receiving the image requesting request by the client includes:
  • a host in a subnet communicates using a C-type URL (IP starting with 192).
  • the host accessing the subnet is mapped by NAT.
  • the mapped value is 10.116.53.22, then the server is used. Said that it sees the IP source of the request is 10.116.53.22, which is inconsistent with the IP information 192.xxx transmitted by the client. It can be considered that the network exit of the client is NAT mapping.
  • the processing method of the server is also the same as before. The same way, if the local IP is 192.168.2.2, the client with the source IP address of 10.116.53.22 requests to pull the mirror X, the server searches for the same routing path, there is an online client, the source IP is 10.116.
  • Subnet D 192.168.2.2 ⁇ 192.168.2.61 A total of 60 nodes online
  • the subnet D internally uses a class C address to perform NAT mapping externally. It is assumed that the routing path of subnet D is:
  • the IP of the node of subnet D is 10.114.6.128, but because the client also provides the IP information of the local machine, the information seen by the server is as follows:
  • the server will query the client of subnet D for the presence of the image. If it is found that the 192.168.2.2 node has the image, then the 192.168.2.5 node pulls the image to 192.168.2.2, thereby achieving the purpose of utilizing the network acceleration of the switch.
  • the server After the mirror transmission is completed, the server will check the integrity of the image. If the client attempts to pull the image to another client, it will feed back to the server, and the server will redirect to other clients. The end download or the server itself provides mirrored transmission.
  • the present invention also provides a system for mirroring, applied to a server, comprising: a memory, a processor, and a computer program stored on the memory and operable on the processor, the computer program being The steps of the method for performing image capture provided by any embodiment of the present invention are implemented when the processor is executed.
  • system embodiment of the image capture is the same as the method embodiment, and the specific implementation process is described in the method embodiment, and the technical features in the method embodiment are corresponding to the system embodiment of the image capture. Applicable, no more details here.
  • the embodiment of the present invention further provides a computer readable storage medium, where the image readable storage medium stores a mirror image pull program, and the image pull program is executed by the processor to implement the image provided by the embodiment of the present invention.
  • the steps of the pull method are not limited to a computer readable storage medium, where the image readable storage medium stores a mirror image pull program, and the image pull program is executed by the processor to implement the image provided by the embodiment of the present invention. The steps of the pull method.
  • the image capturing program embodiment on the above computer readable storage medium belongs to the same concept as the method embodiment, and the specific implementation process is described in the method embodiment, and the technical features in the method embodiment are readable by the above computer.
  • the embodiments of the storage medium are correspondingly applicable, and are not described herein again.
  • the present invention further provides a system for mirroring, including: a server 401 and at least one client 402.
  • the server 401 is configured to perform a mirror pull request sent by the client. Sending a redirect pull image message to the client, or establishing a mirrored transmission with the client; wherein the redirect pull image message is used by the request client to pull a mirror from the first client .
  • the client 402 is configured to pull a mirror from the server or pull the image from the first client according to the redirected pull image message; wherein the first client is configured to correspond to the image pull request Mirror.
  • the invention provides a method for mirroring and a system thereof, in particular to a cluster composed of a plurality of subnet nodes in a government enterprise network.
  • the invention fully utilizes the internal switch network of the subnet, does not occupy the bandwidth of the main road, greatly reduces the pressure on the service end of the main warehouse, and avoids the bandwidth congestion of the main road.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Computer And Data Communications (AREA)
  • Information Transfer Between Computers (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本发明公开了一种镜像拉取的方法及其系统，该方法应用于服务端，包括：接收请求客户端发出的镜像拉取请求；判断所述服务端的当前负载状态是否为高负载运行状态；在判断结果为是的情况下，确定第一客户端，其中，所述第一客户端配置有与所述镜像拉取请求对应的镜像；向所述请求客户端发送重定向拉取镜像消息，其中，所述重定向拉取镜像消息用于所述请求客户端从所述第一客户端拉取镜像。该系统应用于服务端，包括：存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现本发明提供的镜像拉取的方法的步骤。本发明解决了私有仓库的服务的网络和处理能力瓶颈的问题，加速集群镜像的拉取。

Description

一种镜像拉取的方法及其系统
本申请要求享有2018年1月29日提交的名称为“一种镜像拉取的方法及其系统”的中国专利申请CN201810082188.7的优先权，其全部内容通过引用并入本文中。
技术领域
本发明涉及容器化平台领域，尤其涉及一种镜像拉取的方法及其系统。
背景技术
容器技术虚拟化技术已经成为一种被大家广泛认可的服务器资源共享方式，它可以在按需构建操作系统实例的过程当中为系统管理员提供极大的灵活性。容器是完全使用沙箱机制，相互之间不会有任何接口。
Docker是一个开源的应用容器引擎，也是让开发者可以打包他们的应用以及依赖包到一个可移植的容器中，然后发布到Linux机器上，也可以实现虚拟化。容器技术出现，使得原先基于虚拟机节点部署的分布式系统也逐渐进行了容器化操作，如google的kubernetes，swarm等都实现了容器化的部署，容器技术的使用日渐广泛。
运行一个容器化应用时，通常的做法是从仓库(Registry)上拉取该镜像，例如从公有仓库docker hub上下载镜像。而对于企业内部系统而言，由于采用的往往是定制化镜像，且出于安全以及效率的角度考虑，会使用私有仓库服务进行镜像的分发。
在部署较大的容器应用集群时，使用到的节点动辄成百上千个(比如一个kubernetes集群系统声称能支持2000个node)，部署过程主要时间都花费在镜像的分发过程中。因此在大量节点首次部署容器应用集群以及更新镜像时，都会面临私有仓库的Registry的处理瓶颈问题，这其中包括私有仓库所在服务器的处理瓶颈，以及服务器面临的网络带宽瓶颈。在网络带宽和处理能力有限的情况下，Registry面对大量并发操作，就免不了出现排队等待拉取镜像的情形，效率比较慢。
目前有一种结合了Bit Torrent协议来加速镜像分发的技术，主要原理就是通过Bit Torrent协议，对各个节点的镜像进行共享，实现P2P加速。由于P2P技术的原理，该技术需要将镜像文件进行分割和压缩，在集群的所有节点进行传输和加速，一定程度上会消耗网络的带宽资源。此外，由于要分割镜像文件，并需要维护每个分割文件的同步，也增 加了系统的复杂度。
发明内容
有鉴于此，本发明的目的在于提供一种镜像拉取的方法及其系统,以提高拉取镜像的效率。
本发明解决上述技术问题所采用的技术方案如下：
根据本发明的一个方面，提供一种镜像拉取的方法，应用于服务端，包括：
接收请求客户端发出的镜像拉取请求；
判断所述服务端的当前负载状态是否为高负载运行状态；
在判断结果为是的情况下，确定第一客户端，其中，所述第一客户端配置有与所述镜像拉取请求对应的镜像；
向所述请求客户端发送重定向拉取镜像消息，其中，所述重定向拉取镜像消息用于所述请求客户端从所述第一客户端拉取镜像。
在一个可能的设计中，所述接收至少一个请求客户端发出的镜像拉取请求之后，包括：
判断接收到的所述镜像拉取请求的个数是否超过预设阈值；
在判断结果为是的情况下，按照预设的优先拉取原则选取一个或多个所述镜像拉取请求进行优先处理。
在一个可能的设计中，所述按照预设的优先拉取原则选取多个所述镜像拉取请求进行优先处理包括：
确定所述镜像拉取请求的请求客户端的所属子网；
在不同的所属子网中选取所述镜像拉取请求。
在一个可能的设计中，所述在不同的所属子网中选取的所述镜像拉取请求包括：
若所述所属子网的个数小于或等于所述预设阈值，则在每一个不同的所属子网中选取至少一个所述镜像拉取请求；
若所述所属子网的个数大于所述预设阈值，则优先在拥有较大在线客户端数量的所述所属子网中选取一个镜像拉取请求。
在一个可能的设计中，所述确定第一客户端包括：
判断所述请求客户端所处的子网中是否存在在线客户端，其中，所述在线客户端配置有与所述镜像拉取请求对应的镜像；
在判断结果为是的情况下，确定所述在线客户端为第一客户端；
在判断结果为否的情况下，确定其他子网中拥有与所述镜像拉取请求对应的镜像的客 户端，并从中选取距离服务端最近的至少一个客户端作为所述第一客户端。
在一个可能的设计中，所述向所述请求客户端发送重定向拉取镜像消息之前，包括：
判断所述请求客户端与第一客户端之间的距离是否大于预定值；
若是，则等待服务端建立镜像传输。
在一个可能的设计中，所述接收请求客户端发出的镜像拉取请求包括：
获取所述请求客户端的IP信息，以判断所述请求客户端的网络出口是否做了NAT映射。
本发明还提供一种镜像拉取的系统，应用于服务端，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现本发明实施例提供的镜像拉取的方法的步骤。
本发明还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有镜像拉取程序，所述镜像拉取程序被处理器执行时实现本发明实施例提供的镜像拉取的方法的步骤。
本发明还提供一种镜像拉取的系统，包括：服务端以及至少一个客户端；所述服务端，用于根据所述客户端发出的镜像拉取请求，向所述客户端发送重定向拉取镜像消息，或与所述客户端建立镜像传输；其中，所述重定向拉取镜像消息用于所述请求客户端从第一客户端拉取镜像，所述第一客户端配置有与所述镜像拉取请求对应的镜像。
本发明实施例提供的镜像拉取的方法及其系统，利用各个子网的网络带宽，实现各个子网内镜像的共享传输；使用的传输协议简单，只需要单点对单点的传输协议即可，解决私有仓库(Registry)的服务的网络和处理能力瓶颈的问题。本发明还能根据特定集群的网络情况，优化推送镜像的优先级顺序，最大化的利用交换机网络实现加速集群镜像的拉取。
附图说明
附图用来提供对本申请技术方案的进一步理解，并且构成说明书的一部分，与本申请的实施例一起用于解释本申请的技术方案，并不构成对本申请技术方案的限制。
图1为本发明实施例的应用于服务端的镜像拉取的方法的流程示意图；
图2为本发明另一实施例的应用于服务端的镜像拉取的方法的流程示意图；
图3为本发明另一实施例的应用于服务端的镜像拉取的方法的流程示意图；
图4为本发明实施例的镜像拉取的系统的流程示意图；
本发明目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
具体实施方式
为了使本发明所要解决的技术问题、技术方案及有益效果更加清楚、明白，以下结合附图和实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
本发明中，服务端和客户端都提供仓库服务，以便让容器引擎(比如Docker)都能从服务端或本地客户端的仓库拉取镜像。客户端本身也是一个镜像，需要先从服务端上拉取下来并运行客户端容器。
如图1所示，本发明实施例提供一种镜像拉取的方法，应用于服务端，包括以下步骤：
101、开始。
客户端运行起来后，会连接服务端(可理解为登陆操作)，发送本机节点基础信息，如本机IP等。连接成功，它们之间保持一种心跳连接，类似即使通信软件的客户端与服务端心跳机制，只占用少量资源。只要心跳不断，服务端就认为这些客户端处于在线状态，为在线客户端。
102、接收至少一个请求客户端发出的镜像拉取请求。
当在线客户端发出镜像拉取请求，即为请求客户端。
103、判断所述服务端的当前负载状态是否为高负载运行状态，若是，则进入步骤104；若否，则进入步骤105。
104、确定第一客户端，向所述请求客户端发送重定向拉取镜像消息。
其中，所述第一客户端配置有与所述镜像拉取请求对应的镜像。所述重定向拉取镜像消息用于所述请求客户端从所述第一客户端拉取镜像。
105、将与所述镜像拉取请求对应的镜像直接传输给所述请求客户端。
此时，若服务端处于高负载运行状态，则排队等候。
106、结束。
本实施例中，当服务端完成一个推送任务后，会记录该客户端现在存放了哪些镜像，并通过定期同步机制保持同步。当有新的拉取镜像的请求到来时，若服务端处于高负载阶段时，则会将搜索当前在线客户端中拥有该镜像的客户端，向请求客户端发送消息，让其 向拥有该镜像的第一客户端拉取镜像。
如图2所示，本发明实施例提供一种镜像拉取的方法，应用于服务端，包括以下步骤：
201、开始。
客户端运行起来后，会连接服务端(可理解为登陆操作)，发送本机节点基础信息，如本机IP等。连接成功，它们之间保持一种心跳连接，类似即使通信软件的客户端与服务端心跳机制，只占用少量资源。只要心跳不断，服务端就认为这些客户端处于在线状态，为在线客户端。
202、接收至少一个请求客户端发出的镜像拉取请求。
当在线客户端发出镜像拉取请求，即为请求客户端。
203、判断所述服务端当前负载状态是否为高负载运行状态，若是，则进入步骤204；若否，则进入步骤206。
204、搜索是否有第一客户端，所述第一客户端配置有与所述镜像拉取请求对应的镜像；若是，则进入步骤205；若否，则进入步骤206。
205、向所述请求客户端发送重定向拉取镜像消息，以让所述请求客户端能从所述第一客户端拉取镜像。
206、判断接收到的所述镜像拉取请求的个数是否超过预设阈值；若是，则进入步骤207；若否，则进入步骤208。
207、按照预设的优先拉取原则选取一个或多个所述镜像拉取请求进行优先处理。
预设的优先拉取原则优先考虑的是覆盖到所有子网。本实施例中，服务端根据已经获取的客户端的IP信息，维护一张客户端网络信息的表，按照子网以及路由路径为客户端所在的节点进行分类。
208、将与所述镜像拉取请求对应的镜像直接传输给所述请求客户端。
209、结束。
本实施例中，服务端面对大量的并发拉取镜像操作时，采用预设的优先拉取原则调整发送权重，选取需要优先处理的请求。对于剩余的请求，若服务端处于较大负载期时，那么会询问在线的客户端是否拥有相同的镜像，若有，则发送资源重定向消息给所请求的客户端，让该客户端向其他存在该镜像的节点拉取镜像。服务端会优先提供与请求镜像的客 户端所在子网相同或临近的主机，通常为一个主机列表，以供客户端多次尝试连接。
这样一来，由于充分利用了各个子网的交换机网络带宽，一方面能极大地减轻了主仓库的压力，一方面也加快了镜像的分发速度。从理论上能看出来，镜像分发的速度能随着时间增加而不断加倍，镜像文件传输的流量也会逐渐被控制在子网的交换机网络当中。
在图2对应的实施例的基础上，本发明实施例提供一种镜像拉取的方法中，所述按照预设的优先拉取原则选取多个所述镜像拉取请求进行优先处理包括：
确定所述镜像拉取请求的请求客户端的所属子网；
在不同的所属子网中选取所述镜像拉取请求。
例如，某政企网络中有300台节点，需要部署一个大型的容器应用集群，并且还有后续更新、打补丁的需求。对于容器应用集群来说，部署、更新打补丁就意味着要更换启动的镜像。
采用本发明提供的加速大规模镜像拉取的方法时，首先需要有一个主仓库服务端，该服务端提供一个私有仓库服务，以供所有节点首次拉取对应的客户端镜像，并运行。
客户端镜像本身也提供仓库服务，当300台节点都运行好客户端镜像时，客户端会向服务端服务端发送本节点的网络信息，包括对外通信的网卡IP地址、掩码和网关等，并与服务端保持心跳连接，只要心跳不断，服务端就认为这些客户端处于在线状态。服务端接收到客户端的信息后，会维护这些客户端的信息，对客户端的IP地址来源，结合服务端到各个客户端路由路径，对客户端按照子网进行分组。这里先假设这300个节点分别属于3个子网：子网A，子网B和子网C。
当300台节点都运行好客户端后，为了部署或和新应用集群，可通过运维工具对向300台节点执行批量拉取镜像的操作，每个节点会先访问本地客户端的仓库服务，若本地仓库有该镜像，直接拉取，若没有则向服务端进行拉取镜像请求。在首次部署的情况下，各个节点都没有该镜像，此时的客户端都需要向服务端拉取镜像。对于服务端来说，在一个时间段内，可能会有多个拉取镜像的请求，那么服务端就会按照预设的优先拉取原则选取多个所述镜像拉取请求优先处理各个节点的请求。
本实施例中，预设的优先拉取原则包括：根据所述子网的个数的确定异子网优先原则。假设同时有来自x1、x2和x3节点的请求，而目前服务端考虑了当前带宽使用和效率后，只能优先处理2个请求。又根据x1、x2和x3的信息得知，x1和x2属于同一个子网A， 那么服务端会优先满足x1和x3或者x2和x3的拉取请求，即优先满足来自不同子网的节点的请求，尽量先覆盖所有子网。
在具体实施时，所述在不同的所属子网中选取的所述镜像拉取请求包括：
若所述所属子网的个数小于或等于所述预设阈值，则在每一个不同的所属子网中选取至少一个所述镜像拉取请求；
若所述所属子网的个数大于所述预设阈值，则优先在拥有较大在线客户端数量的所述所属子网中选取一个镜像拉取请求。一般地，若阈值为n，则优先在拥有较大在线客户端数量的n个所属子网中各选取一个镜像拉取请求。
即本实施例中，预设的优先拉取原则还包括：根据所述子网拥有的在线客户端的个数确定的大子网优先原则。此时，若x1、x2和x3节点分属于3个不同子网A、B和C，但根据当前客户端的子网信息得知，x1所在的子网有100个客户端保持在线，x2的子网有80个客户端在线，x3的子网有60个客户端在线，那么按照大子网优先的原则，优先满足x1和x2的请求。
更具体地，假设该政企网络由3个子网组成，子网A、子网B和子网C，不存在NAT映射，其节点即客户端IP与子网的关系如下：
子网A：10.113.4.2～10.113.4.241共240个节点在线连接到服务端
子网B：10.113.3.2～10.113.3.201共200个节点在线
子网C：10.115.2.2～10.115.2.101共100个节点在线
利用运维工具，可以让这些节点同时或段时间段内向服务端发送镜像拉取请求，根据异子网优先原则以及大子网优先原则；若在短时间段内(假设为3秒内)3个子网先后各有2个请求(总共6个请求)到达服务端，服务端(假设服务端一次只能同时满足n次并发请求)会首先分别满足子网A(假设满足了10.113.4.2的请求)、子网B(假设满足10.113.3.2)和子网C(假设满足10.115.2.2)的请求(异子网优先原则，即尽可能将连接能力尽可能多的覆盖不同子网)。
举例，假设n为4，此时，由于子网个数为3，小于n，因此，会先分别满足子网A的节点请求，子网B的节点请求和子网C的节点请求。剩下一个连接能力则会给子网A，这是根据大子网原则，由于子网A的在线客户端数量较多，剩余的连接则暂时排队。也就是说，子网A有2个请求会优先处理，子网B和C各有1个请求会优先处理，即，每 个子网至少有一个请求被选取。
若有多个请求的条件都满足上述要求，例如子网A、B、C的在线客户端数量一样多，则此时，剩余的一个连接能力可以按照请求到来的时间进行分配，即预设的优先拉取原则还可以包括：根据时间先后确定的时间优先原则。
再举例，假设n为2，此时，由于子网个数为3，大于n，因此，先分别满足子网A的节点请求，子网B的节点请求，这是因为子网A、子网B的在线客户端数量比子网C的在线客户端数量多。
若有多个请求的条件都满足上述要求，例如子网A、B、C的在线客户端数量一样多，则此时，两个连接能力可以按照请求到来的时间进行分配，即预设的优先拉取原则还可以包括：根据时间先后确定的时间优先原则。
在上述任一实施例的基础上，本发明实施例提供一种镜像拉取的方法中，如图3所示，所述确定第一客户端包括：
301、开始。
302、判断所述请求客户端所处的子网中是否存在在线客户端；若是，则进入步骤303；若否，则进入步骤304。
所述在线客户端配置有与所述镜像拉取请求对应的镜像。
303、确定所述在线客户端为第一客户端；并进入步骤305。
304、确定其他子网中拥有与所述镜像拉取请求对应的镜像的客户端，并从中选取距离服务端最近的至少一个客户端作为所述第一客户端；
305、结束。
可见，本实施例中，优先搜索与该请求的节点处于同一子网的节点，查询该子网的在线客户端是否也拥有同样的镜像，假设若存在，则向请求客户端发送“重定向拉取镜像消息”，让请求客户端向其子网内拥有同样镜像的第一客户端拉取镜像，第一客户端可以为多个。
举例说明：如前面所述，若x1、x2和x3节点分属于3个不同子网A、B和C，经过首次传输后，x1、x2和x3都有了某镜像X，现在有新的拉取镜像X的请求，来自x4，x5和x6，其中x4，x5属于子网A，x6属于子网C。假设此时服务端由于负载比较大，只能满足x4的请求，然后对于x5和x6，通过查询客户端信息，得知x1和x3也有镜像 X，并向x1和x3发送消息确认，又因为x1和x5属于子网A，而x3和x6属于子网C，则服务端会向x5和x6发送“重定向拉取镜像消息”，让x5从x1处拉取镜像X，x6从x3处拉取镜像，这样x5和x6拉取镜像的流量就限制在了子网A和C中，同时，利用子网的交换机网络，速度得到了很大的提升。当x5和x6拉取完镜像后，会向服务端发送反馈，告知自己已经完成了镜像X的拉取。
和上述同样地的来自x4，x5和x6的拉取镜像请求，但服务端查询x5所属子网A并没有镜像X，但通过查询在线客户端得知，子网B的节点x7和子网C的节点x8都有镜像X。那么此时服务端优先选择距离x5所属子网A更相近的子网的节点，让x5重定向拉取请求。
举例说明，假如服务端到子网A的节点，经过的路由路径为r 1,r 2,r 3，到子网B经过的路由路径为r 1,r 2,r 4，到子网C的节点经过的路由路径为r 1,r 2,r 5,r 6，那么能看出子网B与子网A更相近，特别是在政企的网络专线和网络情况中，路由路径选择基本是确定的，在这种场景，可以认为子网B和子网A的节点传输流量经过的路由跳数要比子网C和子网A的跳数少。这样，在上述场景下，服务端会优先让x5向子网B的节点x7拉取镜像。
判断子网之间的距离可以通过下述公式判定：
d(Sr i,Sr j)＝card(Sr i∪Sr j)-card(Sr i∩Sr j)     (1)
其中，Sr i,Sr j分别为子网i、j是的路由路径的有序集合，以上述情况举例，服务端到子网A的路由路径集合为{r 1,r 2,r 3}，到子网B的的路由路径集合为{r 1,r 2,r 4}，到子网C的路由路径集合为{r 1,r 2,r 5,r 6}，那么根据公式(1)，子网A和B的路由距离为
d(Sr A,Sr B)＝card(Sr A∪Sr B)-card(Sr A∩Sr B)
＝card({r 1,r 2,r 3,r 4})-card({r 1,r 2})
＝4-2＝2
子网A和C的距离为
d(Sr A,Sr C)＝card(Sr A∪Sr C)-card(Sr A∩Sr C)
＝card({r 1,r 2,r 4,r 5,r 6})-card({r 1,r 2})
＝5-2＝3
因此子网B和子网A的距离更近。
在上述任一实施例的基础上，所述向所述请求客户端发送重定向拉取镜像消息之前，还包括：
判断所述请求客户端与第一客户端之间的距离是否大于预定值；
若是，则等待服务端建立镜像传输。
一般地，若请求客户端与第一客户端属于同一子网，则两者之间的距离肯定不会大于预定值。若属于不同子网，则当两者之间距离大于预定值后，即使该子网有节点拥有请求节点所需要镜像，服务端会认为此时不适合在这两个子网之间进行文件传输，效率提升不高。此时该节点需要等待服务端建立镜像传输。
下面辅以实例进一步说明本实施例：
当第一批请求都处理完后又有超过服务端并发处理能力的个数的请求陆续到来，那么服务端会进行重定向拉取请求。假设此时有来自子网A的节点10.113.4.5请求镜像拉取，此时服务端没有多余的连接能力，会通过客户端管理分析模块尝试分析现有在线客户端，服务端首先向同10.113.4.5节点处于相同子网(子网A)的在线客户端查询，是否有该镜像，若发现同样位于子网A的10.113.4.2节点有该请求的镜像，则发送资源重定向消息，让10.113.4.5的客户端请求10.113.4.2的客户端，拉取镜像。
若查询了子网A的客户端都没有发现，则转向临近子网查询
若子网A的路由路径为：
＝{10.113.23.169，10.114.51.121，10.114.4.128}
子网B的路由路径为：
＝{10.113.23.169，10.114.51.121，10.114.3.128}
子网C的路由路径为：
＝{10.113.23.169，10.114.51.121，10.114.55.1，10.114.5.128}
根据公式(1)，可以知道子网B距离子网A距离更近一些，因此在子网A内搜索不到拥有该镜像的节点时，会搜索临近子网B内的客户端是否拥有该镜像，以此类推，若子网B也没有，会专向搜索子网C内的节点。
当临近子网与请求节点所在的子网距离较大时，此时即便临近子网有节点拥有该镜像时，这种跨子网传输对效率的提升也是不明显的，因为此时的传输要跨过很多路由点。故在这种情况下，服务端可能不会重定向拉取请求，而是由服务端提供镜像传输。
在上述任一实施例的基础上，所述接收请求客户端发出的镜像拉取请求包括：
获取所述请求客户端的IP信息，以判断所述请求客户端的网络出口是否做了NAT映 射。
考虑到一些子网环境会做NAT映射，因此在服务端看来，这些客户端的源IP都是相同的，只是端口不同，因此这也是为什么需要客户端向服务端发送客户端本机的IP信息的原因，在此情况下，只要NAT映射层数不多，依然也能做到镜像分发加速功能。
例如，一个子网内的主机，使用C类网址(192开头的IP)进行通信，访问子网外的主机，都是通过路由进行NAT映射，映射的值都10.116.53.22，那么对于服务端来说，它看到的请求IP来源都是10.116.53.22，与客户端传递的IP信息192.x.x.x不一致，那可以认为该客户端的网络出口是做了NAT映射，此时服务端的处理方式也和先前的方式一样，若本机IP为192.168.2.2，源IP为10.116.53.22的客户端请求拉取镜像X，服务端搜索得知在相同的路由路径下，有一个在线客户端，源IP为10.116.53.22，本机IP为192.168.2.3的主机也拥有该镜像X，那么就会发送“重定向拉取镜像消息”给192.168.2.2,让其向192.168.2.3拉取镜像X。重复上述步骤，最大程度地利用各个子网内部的局域网网络带宽，极大地加速了整个集群节点镜像分发的速度。
更具体地，若现在有一个子网D，如下
子网D：192.168.2.2～192.168.2.61 共60个节点在线
子网D内部使用了C类地址，对外做NAT映射，假设子网D的路由路径为：
＝{10.113.23.169，10.114.51.121，10.114.6.128}
那么在服务端看来，子网D的节点的IP都是10.114.6.128，但又因为客户端还会提供本机的IP信息，因此服务端看到的信息如下：
客户端1：来源IP:10.114.51.121 上报IP:192.168.2.2
客户端2：来源IP:10.114.51.121 上报IP:192.168.2.3
客户端3：来源IP:10.114.51.121 上报IP:192.168.2.4
......
客户端60：来源IP:10.114.51.121 上报IP:192.168.2.61
在这种场景下，若子网D的192.168.2.5的节点请求下载镜像，由于其IP显示为10.114.51.121，且路由路径为，那么服务端会在子网D的客户端中查询是否存在该镜像， 若查到192.168.2.2节点有该镜像，则让192.168.2.5节点向192.168.2.2拉取镜像，从而达到利用交换机网络加速的目的。
当镜像传输完成后，服务端还会对镜像的完整性做检查，同时若客户端尝试向别的客户端拉取镜像失败时，会反馈给服务端，服务端会重新重定向到其他的客户端下载或者有服务端本身提供镜像传输。
本发明还提供一种镜像拉取的系统，应用于服务端，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现本发明任一实施例提供的镜像拉取的方法的步骤。
需要说明的是，上述镜像拉取的系统实施例与方法实施例属于同一构思，其具体实现过程详见方法实施例，且方法实施例中的技术特征在镜像拉取的系统实施例中均对应适用，这里不再赘述。
另外，本发明实施例还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有镜像拉取程序，所述镜像拉取程序被处理器执行时实现本发明实施例提供的镜像拉取的方法的步骤。
需要说明的是，上述计算机可读存储介质上的镜像拉取程序实施例与方法实施例属于同一构思，其具体实现过程详见方法实施例，且方法实施例中的技术特征在上述计算机可读存储介质的实施例中均对应适用，这里不再赘述。
如图4所示，本发明还提供一种镜像拉取的系统，包括：服务端401以及至少一个客户端402；所述服务端401，用于根据所述客户端发出的镜像拉取请求，向所述客户端发送重定向拉取镜像消息，或与所述客户端建立镜像传输；其中，所述重定向拉取镜像消息用于所述请求客户端从所述第一客户端拉取镜像。所述客户端402用于从服务端拉取镜像或根据所述重定向拉取镜像消息从第一客户端拉取镜像；其中，所述第一客户端配置有与所述镜像拉取请求对应的镜像。
本发明提供一种镜像拉取的方法及其系统，特别针对政企网络内多个子网节点组成的集群。本发明充分利用了子网内部交换机网络，不占用主干道带宽，极大地减轻了主仓库 服务端的压力，避免主干道带宽拥堵。
以上参照附图说明了本发明的优选实施例，并非因此局限本发明的权利范围。本领域技术人员不脱离本发明的范围和实质，可以有多种变型方案实现本发明，比如作为一个实施例的特征可用于另一实施例而得到又一实施例。凡在运用本发明的技术构思之内所作的任何修改、等同替换和改进，均应在本发明的权利范围之内。

Claims (10)

  1. 一种镜像拉取的方法，应用于服务端，其特征在于，包括：
    接收请求客户端发出的镜像拉取请求；
    判断所述服务端的当前负载状态是否为高负载运行状态；
    在判断结果为是的情况下，确定第一客户端，其中，所述第一客户端配置有与所述镜像拉取请求对应的镜像；
    向所述请求客户端发送重定向拉取镜像消息，其中，所述重定向拉取镜像消息用于所述请求客户端从所述第一客户端拉取镜像。
  2. 根据权利要求1所述镜像拉取的方法，其特征在于，所述接收至少一个请求客户端发出的镜像拉取请求之后，包括：
    判断接收到的所述镜像拉取请求的个数是否超过预设阈值；
    在判断结果为是的情况下，按照预设的优先拉取原则选取一个或多个所述镜像拉取请求进行优先处理。
  3. 根据权利要求2所述镜像拉取的方法，其特征在于，所述按照预设的优先拉取原则选取多个所述镜像拉取请求进行优先处理包括：
    确定所述镜像拉取请求的请求客户端的所属子网；
    在不同的所属子网中选取所述镜像拉取请求。
  4. 根据权利要求3所述镜像拉取的方法，其特征在于，所述在不同的所属子网中选取的所述镜像拉取请求包括：
    若所述所属子网的个数小于或等于所述预设阈值，则在每一个不同的所属子网中选取至少一个所述镜像拉取请求；
    若所述所属子网的个数大于所述预设阈值，则优先在拥有较大在线客户端数量的所述所属子网中选取一个镜像拉取请求。
  5. 根据权利要求1所述镜像拉取的方法，其特征在于，所述确定第一客户端包括：
    判断所述请求客户端所处的子网中是否存在在线客户端，其中，所述在线客户端配置有与所述镜像拉取请求对应的镜像；
    在判断结果为是的情况下，确定所述在线客户端为第一客户端；
    在判断结果为否的情况下，确定其他子网中拥有与所述镜像拉取请求对应的镜像的客 户端，并从中选取距离服务端最近的至少一个客户端作为所述第一客户端。
  6. 根据权利要求1所述镜像拉取的方法，其特征在于，所述向所述请求客户端发送重定向拉取镜像消息之前，包括：
    判断所述请求客户端与所述第一客户端之间的距离是否大于预定值；
    若是，则等待服务端建立镜像传输。
  7. 根据权利要求1所述镜像拉取的方法，其特征在于，所述接收请求客户端发出的镜像拉取请求包括：
    获取所述请求客户端的IP信息，以判断所述请求客户端的网络出口是否做了NAT映射。
  8. 一种镜像拉取的系统，应用于服务端，其特征在于，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现如权利要求1至7中任一项所述的镜像拉取的方法的步骤。
  9. 一种计算机可读存储介质，所述计算机可读存储介质上存储有镜像拉取程序，所述镜像拉取程序被处理器执行时实现本发明实施例提供的镜像拉取的方法的步骤。
  10. 一种镜像拉取的系统，其特征在于，包括：服务端以及至少一个客户端；所述服务端，用于根据所述客户端发出的镜像拉取请求，向所述客户端发送重定向拉取镜像消息，或与所述客户端建立镜像传输；其中，所述重定向拉取镜像消息用于所述请求客户端从第一客户端拉取镜像，所述第一客户端配置有与所述镜像拉取请求对应的镜像。
PCT/CN2019/073581 2018-01-29 2019-01-29 一种镜像拉取的方法及其系统 WO2019144965A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19743986.2A EP3748934B1 (en) 2018-01-29 2019-01-29 Mirror pull method and system therefor
US16/965,560 US20210051211A1 (en) 2018-01-29 2019-01-29 Method and system for image pulling

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810082188.7A CN110099076A (zh) 2018-01-29 2018-01-29 一种镜像拉取的方法及其系统
CN201810082188.7 2018-01-29

Publications (1)

Publication Number Publication Date
WO2019144965A1 true WO2019144965A1 (zh) 2019-08-01

Family

ID=67396001

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/073581 WO2019144965A1 (zh) 2018-01-29 2019-01-29 一种镜像拉取的方法及其系统

Country Status (4)

Country Link
US (1) US20210051211A1 (zh)
EP (1) EP3748934B1 (zh)
CN (1) CN110099076A (zh)
WO (1) WO2019144965A1 (zh)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111221623B (zh) * 2019-12-26 2024-01-19 中科曙光国际信息产业有限公司 镜像库管理系统
CN111131487B (zh) * 2019-12-30 2022-08-12 广东浪潮大数据研究有限公司 一种深度学习平台容器镜像获取、共享方法及系统
JP2021157461A (ja) * 2020-03-26 2021-10-07 富士フイルムビジネスイノベーション株式会社 情報処理装置、情報処理システム、及び情報処理プログラム
WO2021232289A1 (zh) * 2020-05-20 2021-11-25 深圳市欢太科技有限公司 镜像拉取方法及相关产品
CN113746881A (zh) * 2020-05-29 2021-12-03 电科云(北京)科技有限公司 容器镜像下载方法及系统
CN112418447B (zh) * 2021-01-22 2021-04-13 北京瑞莱智慧科技有限公司 提供机器学习服务的系统、方法、介质和设备
CN113535324B (zh) * 2021-06-25 2024-04-05 深圳软通动力信息技术有限公司 边缘侧拉取镜像的方法和系统
CN113760453B (zh) * 2021-08-04 2024-05-28 南方电网科学研究院有限责任公司 容器镜像分发系统及容器镜像推送、拉取和删除方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105260229A (zh) * 2015-10-28 2016-01-20 北京百度网讯科技有限公司 拉取虚拟机镜像文件的方法和装置
CN106790660A (zh) * 2017-01-18 2017-05-31 咪咕视讯科技有限公司 一种实现分布式存储系统的部署方法及装置
CN107247793A (zh) * 2017-06-21 2017-10-13 平安科技（深圳）有限公司 Docker镜像仓库的镜像同步方法和镜像同步系统
CN107256178A (zh) * 2017-04-27 2017-10-17 北京数人科技有限公司 一种容器管理平台

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2989241B1 (fr) * 2012-04-05 2018-01-26 Easybroadcast Procede de diffusion d'un contenu dans un reseau informatique.
CN105740048B (zh) * 2016-01-26 2019-03-08 华为技术有限公司 一种镜像管理方法、装置及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105260229A (zh) * 2015-10-28 2016-01-20 北京百度网讯科技有限公司 拉取虚拟机镜像文件的方法和装置
CN106790660A (zh) * 2017-01-18 2017-05-31 咪咕视讯科技有限公司 一种实现分布式存储系统的部署方法及装置
CN107256178A (zh) * 2017-04-27 2017-10-17 北京数人科技有限公司 一种容器管理平台
CN107247793A (zh) * 2017-06-21 2017-10-13 平安科技（深圳）有限公司 Docker镜像仓库的镜像同步方法和镜像同步系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3748934A4 *

Also Published As

Publication number Publication date
CN110099076A (zh) 2019-08-06
EP3748934B1 (en) 2023-07-12
US20210051211A1 (en) 2021-02-18
EP3748934A1 (en) 2020-12-09
EP3748934A4 (en) 2021-11-10

Similar Documents

Publication Publication Date Title
WO2019144965A1 (zh) 一种镜像拉取的方法及其系统
AU2020239763B2 (en) Virtual network, hot swapping, hot scaling, and disaster recovery for containers
JP6544872B2 (ja) 負荷バランシングコンピュータデバイス、システム、および方法
EP3355553B1 (en) Reliable load-balancer using segment routing and real-time application monitoring
CN113826363B (zh) 全局网络接入点中冗余控制器之间的一致路由公告
EP2922238B1 (en) Resource allocation method
US10318550B2 (en) Systems and methods for autonomous resource discovery, management, and stitching
US6996617B1 (en) Methods, systems and computer program products for non-disruptively transferring a virtual internet protocol address between communication protocol stacks
US6941384B1 (en) Methods, systems and computer program products for failure recovery for routed virtual internet protocol addresses
US7315896B2 (en) Server network controller including packet forwarding and method therefor
US9203762B2 (en) Information handling system data center bridging features with defined application environments
JP2010527561A (ja) エッジルーティングを用いたピアツーピアコラボレーションシステム
JP2002324056A (ja) 分散型ネットワーク環境においてソフトウェアコンポーネントにアクセスするためのシステム及び方法
WO2014190791A1 (zh) 一种网关设备身份设置的方法及管理网关设备
CN111756830A (zh) 公有云网络的内网负载均衡实现方法
US9942108B2 (en) Network service aware routers, and applications thereof
WO2021120633A1 (zh) 一种负载均衡方法及相关设备
CN112994937A (zh) 智融标识网络中虚拟cdn的部署与迁移系统
US20220166715A1 (en) Communication system and communication method
WO2010069261A1 (zh) 边界网关协议路由处理方法、装置和系统
US9736027B2 (en) Centralized enterprise image upgrades for distributed campus networks
CN112655185B (zh) 软件定义网络中的服务分配的设备、方法和存储介质
He et al. Firebird: Network-aware task scheduling for spark using sdns
JP7306480B2 (ja) 制御装置、制御方法、及びプログラム
JP5482783B2 (ja) セキュリティ管理装置、方法、及びプログラム、並びにセキュリティ分散システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19743986

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019743986

Country of ref document: EP

Effective date: 20200831