CN110196679B

CN110196679B - Distributed deployment system, object management method, device, medium and electronic equipment

Info

Publication number: CN110196679B
Application number: CN201810236908.0A
Authority: CN
Inventors: 马梓扬; 陈杰
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2018-03-21
Filing date: 2018-03-21
Publication date: 2023-01-17
Anticipated expiration: 2038-03-21
Also published as: CN110196679A

Abstract

The embodiment of the invention provides a distributed deployment system of a container warehouse, an object management method, an object management device, an object management medium and electronic equipment applied to a distributed deployment scene of the container warehouse. The object management method comprises the following steps: if the container warehouse receives the object uploaded by the object uploading party, storing the object in a local storage space of the container warehouse; and sending the object to an object storage server connected with a plurality of container warehouses deployed in a distributed mode for storage, so that other container warehouses can acquire the object from the object storage server. The technical scheme of the embodiment of the invention solves the problem of capacity limitation caused by the need of synchronization among container warehouses, and can conveniently perform capacity expansion and reduce the capacity expansion cost.

Description

Distributed deployment system, object management method, device, medium, and electronic device

Technical Field

The invention relates to the technical field of computers and communication, in particular to a distributed deployment system of a container warehouse, an object management method, a device, a medium and electronic equipment applied to the distributed deployment scene of the container warehouse.

Background

In a distributed deployment scenario of a container warehouse (Docker registry), there may be a plurality of registrations deployed in different regions, and the same region may also have a plurality of registrations deployed, so how to synchronize the registrations (i.e., the registrations deployed in the same region and the registrations deployed in different regions) and how to perform capacity expansion are problems that must be faced. But there is currently no effective solution.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present invention and therefore may include information that does not constitute prior art known to a person of ordinary skill in the art.

Disclosure of Invention

An object of the embodiments of the present invention is to provide a distributed deployment system for a container warehouse, and an object management method, an object management device, an object management medium, and an electronic device applied in a distributed deployment scenario of the container warehouse, so as to solve, at least to a certain extent, a synchronization problem and a problem of inconvenience in capacity expansion in the distributed deployment scenario of the container warehouse.

Additional features and advantages of the invention will be set forth in the detailed description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

According to an aspect of the embodiments of the present invention, an object management method applied in a distributed deployment scenario of a container warehouse is provided, including: if the container warehouse receives the object uploaded by the object uploading party, storing the object in a local storage space of the container warehouse; and sending the object to an object storage server connected with a plurality of container warehouses deployed in a distributed mode for storage, so that other container warehouses can acquire the object from the object storage server.

According to an aspect of an embodiment of the present invention, there is provided a distributed deployment system of a container warehouse, including: a plurality of container warehouses, one or more of which are used to implement the object management method as described in the above embodiments; and the object storage server is used for receiving and storing the objects uploaded by the container warehouses and providing the required objects to the container warehouses.

According to an aspect of the embodiments of the present invention, there is provided an object management apparatus applied in a distributed deployment scenario of a container warehouse, including: the storage unit is used for storing the objects in the local storage space of the container warehouse when the objects uploaded by the object uploading party are received; and the sending unit is used for sending the object to an object storage server connected with a plurality of container warehouses deployed in a distributed mode for storage, so that other container warehouses can acquire the object from the object storage server.

In some embodiments of the present invention, based on the foregoing solution, the object management apparatus applied in a distributed deployment scenario of a container warehouse further includes: the device comprises a judging unit, a storage unit and a processing unit, wherein the judging unit is used for judging whether a local storage space of a container warehouse stores a target object or not when an acquisition request of an object acquirer for the target object is received; a response unit, configured to return the target object stored in the local storage space to the object acquirer when the determination unit determines that the target object is stored in the local storage space, and configured to acquire the target object from the object storage server and return to the object acquirer when the determination unit determines that the target object is not stored in the local storage space.

In some embodiments of the present invention, based on the foregoing solution, the storage unit is further configured to store the target object obtained from the object storage server into the local storage space.

In some embodiments of the present invention, based on the foregoing solution, the object management apparatus applied in a distributed deployment scenario of a container warehouse further includes: a detecting unit, configured to detect whether a remaining capacity of the local storage space is lower than a first predetermined threshold; and the cleaning unit is used for cleaning the local storage space when the detection unit detects that the residual capacity of the local storage space is lower than the first preset threshold value, so that the residual capacity in the local storage space reaches a second preset threshold value.

In some embodiments of the present invention, based on the foregoing solution, the cleaning unit is configured to: counting the access frequency of each object stored in the local storage space; and deleting the preset number of objects stored in the local storage space according to the sequence of the access frequency from low to high.

In some embodiments of the present invention, based on the foregoing solution, the cleaning unit is configured to: determining the access time of each object stored in the local storage space; and deleting the preset number of objects stored in the local storage space according to the sequence of the access time from far to near.

In some embodiments of the present invention, based on the foregoing solution, the cleaning unit is configured to: receiving an object cleaning instruction sent by a designated device; and deleting the corresponding object stored in the local storage space based on the object cleaning instruction.

In some embodiments of the present invention, based on the foregoing solution, the object management apparatus applied in a distributed deployment scenario of a container warehouse further includes: a receiving unit, configured to receive a configuration file, where the configuration file includes parameter information of the object storage server; and the processing unit is used for connecting to the object storage server based on the parameter information.

In some embodiments of the present invention, based on the foregoing solution, the parameter information includes a network access address, an access ip, a uniform resource locator http url of a hypertext transfer protocol, and a container packet for storing an object.

According to an aspect of the embodiments of the present invention, there is provided a computer readable medium, on which a computer program is stored, which when executed by a processor implements the object management method applied to the distributed deployment scenario of a container warehouse as described in the above embodiments.

According to an aspect of an embodiment of the present invention, there is provided an electronic apparatus including: one or more processors; a storage device, configured to store one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the object management method in the distributed deployment scenario applied to the container warehouse as described in the foregoing embodiments.

The technical scheme provided by the embodiment of the invention has the following beneficial effects:

in the technical solutions provided in some embodiments of the present invention, when receiving an object uploaded by an object uploading party, a container warehouse stores the object in a local storage space, and sends the object to an object storage server for storage, so that other container warehouses acquire the object from the object storage server, and thus in a distributed deployment system of the container warehouse, each container warehouse may acquire a required object through the object storage server, and further each container warehouse does not need to synchronize the stored objects with each other, thereby solving the problem of capacity limitation caused by the need to synchronize the objects between the container warehouses. Meanwhile, the container warehouse can be stored in the local storage space when receiving the objects uploaded by the object uploading party, so that if an object acquiring party needs to acquire the objects, the container warehouse can directly provide the locally stored objects to the object acquiring party without acquiring the objects from the object storage server, the object acquiring efficiency is improved, and the flow consumption caused by acquiring the objects from the object storage server is reduced. In addition, according to the technical scheme of the embodiment of the invention, as the object synchronization among the container warehouses is not needed, the capacity expansion can be conveniently carried out, and the capacity expansion cost is reduced.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:

fig. 1 is a schematic diagram illustrating an exemplary system architecture of an object management method applied in a distributed deployment scenario of a container warehouse or an object management apparatus applied in a distributed deployment scenario of a container warehouse, to which an embodiment of the present invention may be applied;

FIG. 2 illustrates a schematic structural diagram of a computer system suitable for use with the electronic device to implement an embodiment of the invention;

FIG. 3 schematically illustrates a block diagram of a distributed system according to one embodiment of the invention;

FIG. 4 schematically illustrates a block diagram of a distributed system according to another embodiment of the invention;

FIG. 5 is a flow diagram schematically illustrating an object management method applied in a distributed deployment scenario of a container warehouse, in accordance with an embodiment of the present invention;

FIG. 6 is a flow diagram schematically illustrating an object management method applied in a distributed deployment scenario of a container warehouse, in accordance with another embodiment of the present invention;

FIG. 7 schematically illustrates a flow diagram of an object management method applied in a distributed deployment scenario of a container warehouse, according to yet another embodiment of the present invention;

FIG. 8 illustrates a scenario diagram of a distributed deployment of registrations according to an embodiment of the present invention;

FIG. 9 is a diagram illustrating a process for uploading data by a user according to an embodiment of the invention;

FIG. 10 is a diagram illustrating a process for a user to download data according to one embodiment of the invention;

FIG. 11 is a block diagram schematically illustrating an object management apparatus applied in a distributed deployment scenario of a container warehouse, according to an embodiment of the present invention;

FIG. 12 schematically illustrates a block diagram of an object management apparatus in a distributed deployment scenario applied to a container warehouse, in accordance with another embodiment of the present invention;

FIG. 13 schematically illustrates a block diagram of an object management apparatus in a distributed deployment scenario applied to a container warehouse, in accordance with yet another embodiment of the present invention;

fig. 14 schematically shows a block diagram of an object management apparatus applied in a distributed deployment scenario of a container warehouse according to still another embodiment of the present invention.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations or operations have not been shown or described in detail to avoid obscuring aspects of the invention.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.

The flowcharts shown in the figures are illustrative only and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

Fig. 1 is a schematic diagram illustrating an exemplary system architecture 100 of an object management method applied in a distributed deployment scenario of a container warehouse or an object management apparatus applied in a distributed deployment scenario of a container warehouse, to which an embodiment of the present invention may be applied.

As shown in fig. 1, system architecture 100 may include one or more of

end devices

101, 102, 103, a plurality of distributed nodes (e.g.,

distributed nodes

104a, 104b, \ 8230; \8230;, 104n shown in fig. 1), and a data storage server 105. Communication may be established between the

terminal devices

101, 102, 103 and the server 105 by means of, for example, wired, wireless communication links, or fiber optic cables.

It should be understood that the number of terminal devices, distributed nodes, and data storage servers in fig. 1 is merely illustrative. There may be any number of end devices, distributed nodes, and data storage servers, as desired for an implementation.

A user may interact with the distributed nodes using the

terminal devices

101, 102, 103 to receive or transmit data, etc. The

terminal devices

101, 102, 103 may be various electronic devices having a display screen, including smart phones, tablet computers, portable computers, desktop computers, and the like.

In an embodiment of the present invention, the

terminal devices

101, 102, and 103 may be terminal devices running a docker client, the distributed nodes may be multiple container warehouses deployed in a distributed manner, and the data storage server 105 may be an object storage server, so that an object management scheme in a distributed deployment scenario in an embodiment of the present invention may be implemented, which is specifically described as follows:

in an embodiment of the present invention, a user uploads data to a container warehouse (for convenience of distinguishing from other container warehouses, hereinafter, an example of uploading an object to the container warehouse 104b by the user is taken as an example for convenience of explanation) by using the terminal device 103 (which may also be the terminal device 101 or 102), after receiving the object uploaded by the terminal device 103 (for convenience of explanation, the object is taken as the object 1 for convenience of explanation), the container warehouse 104b may store the object 1 in the local storage space on one hand, and may send the object 1 to the object storage server 105 on the other hand, and when there is another user requesting to acquire the object 1, the container warehouse 104b may directly send the object 1 stored in the local storage space to a corresponding object acquirer, so that the object acquisition efficiency is improved. Meanwhile, when the other container warehouse needs to acquire the object 1 to provide it to the object acquirer, if the object 1 is stored in the local storage space, the object 1 stored in the local storage space is sent to the object acquirer, and if the object 1 is not stored in the local storage space, the object 1 may be acquired from the object storage server 105. Therefore, the technical scheme of the embodiment of the invention does not need to carry out object synchronization among the container warehouses, solves the problem of capacity limitation caused by the need of object synchronization among the container warehouses, is convenient for capacity expansion, and reduces the capacity expansion cost.

In an embodiment of the present invention, since the container warehouse stores the objects uploaded by the terminal device and the objects acquired from the object storage server 105, when the container warehouse detects that the remaining capacity in the local storage space is lower than a certain threshold, the local storage space may be cleared, for example, the objects with lower access frequency or the objects with longer access time are deleted, so as to ensure that the local storage space has sufficient capacity.

It should be noted that the "Object" in the embodiment of the present invention is an Object, and each Object is an integrated body of data and a data attribute set, or is separate data. In other words, the object contains the file data and the associated attribute information, or only the file data. The object management method applied to the distributed deployment scenario of the container warehouse provided by the embodiment of the present invention is generally executed by the container warehouse, and accordingly, the object management apparatus applied to the distributed deployment scenario of the container warehouse is generally disposed in the container warehouse.

FIG. 2 illustrates a schematic structural diagram of a computer system suitable for use with the electronic device to implement an embodiment of the invention.

It should be noted that the computer system 200 of the electronic device shown in fig. 2 is only an example, and should not bring any limitation to the functions and the scope of the application of the embodiment of the present invention.

As shown in fig. 2, the computer system 200 includes a Central Processing Unit (CPU) 201 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM) 202 or a program loaded from a storage section 208 into a Random Access Memory (RAM) 203. In the RAM 203, various programs and data necessary for system operation are also stored. The CPU 201, ROM202, and RAM 203 are connected to each other via a bus 204. An input/output (I/O) interface 205 is also connected to bus 204.

The following components are connected to the I/O interface 205: an input portion 206 including a keyboard, a mouse, and the like; an output section 207 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 208 including a hard disk and the like; and a communication section 209 including a network interface card such as a LAN card, a modem, or the like. The communication section 209 performs communication processing via a network such as the internet. A drive 210 is also connected to the I/O interface 205 as needed. A removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 210 as necessary, so that the computer program read out therefrom is mounted into the storage section 208 as necessary.

In particular, according to an embodiment of the present invention, the processes described below with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the invention include a computer program product comprising a computer program embodied on a computer-readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 209 and/or installed from the removable medium 211. When the computer program is executed by a Central Processing Unit (CPU) 201, various functions defined in the system of the present application are executed.

It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present invention, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present invention may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.

As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method as described in the embodiments below. For example, the electronic device may implement the steps shown in fig. 5 to 7.

The implementation details of the technical scheme of the embodiment of the invention are explained in detail as follows:

fig. 3 schematically shows a block diagram of a distributed system according to an embodiment of the invention.

Referring to fig. 3, a distributed system according to an embodiment of the present invention includes: a plurality of distributed nodes (e.g., distributed

nodes

301, 302, \8230;, 30n shown in fig. 3) and a data storage server 31. The data storage server 31 is used for receiving and storing data uploaded by the distributed nodes, and for providing data sources for the distributed nodes; the distributed nodes are used for directly interacting with the data provider and the data acquirer, for example, receiving data uploaded by the data provider and a data acquisition request sent by the data provider, and sending the data to the data acquirer.

Referring to fig. 4, a distributed system according to another embodiment of the present invention may further include a control node 32, on the basis of the plurality of distributed nodes and the data storage server 31 shown in fig. 3, wherein the control node 32 is connected to a plurality of distributed nodes (e.g., distributed

nodes

301, 302, \ 8230; \8230;, 30 n), and the control node 32 is configured to receive a request message sent by a data provider and/or a data acquirer and distribute distributed nodes responding to the request message based on a load balancing manner.

In an embodiment of the present invention, the control node 32 shown in fig. 4 is further configured to control a bandwidth of uploading data to the data storage server 31 by the plurality of distributed nodes and/or a bandwidth of downloading data from the data storage server 31 by the plurality of distributed nodes.

In an embodiment of the present invention, the distributed system shown in fig. 3 and fig. 4 may be specifically a distributed deployment system of a container warehouse, where the distributed nodes are the container warehouse, and the data storage server 31 is an object storage server. One or more container warehouses in the distributed deployment system of container warehouses may execute the object management method in the distributed deployment scenario applied to the container warehouses shown in fig. 5 to 7, and a specific object management method is set forth in detail below.

Fig. 5 schematically shows a flowchart of an object management method applied in a distributed deployment scenario of a container warehouse according to an embodiment of the present invention, where an execution subject of the object management method may be a certain or some container warehouses in a distributed deployment system of the container warehouse, such as any one of the container warehouses. Referring to fig. 5, the object management method at least includes step S510 and step S520, and is described in detail as follows:

in step S510, if the container warehouse receives an object uploaded by an object uploading party, storing the object in a local storage space of the container warehouse;

in step S520, the object is sent to an object storage server connected to a plurality of container warehouses deployed in a distributed manner for storage, so that other container warehouses can obtain the object from the object storage server.

The technical solution of the embodiment shown in fig. 5 enables each container warehouse to obtain a required object through the object storage server in the distributed deployment system of the container warehouse, so that each container warehouse does not need to synchronize objects stored therebetween, thereby solving the problem of limited capacity caused by the need to synchronize objects between container warehouses, and also being capable of conveniently performing capacity expansion and reducing capacity expansion cost. Meanwhile, the container warehouse can be stored in the local storage space when receiving the objects uploaded by the object uploading party, so that if an object acquiring party needs to acquire the objects, the container warehouse can directly provide the locally stored objects to the object acquiring party without acquiring the objects from the object storage server, the object acquiring efficiency is improved, and the flow consumption caused by acquiring the objects from the object storage server is reduced.

Fig. 6 schematically shows a flowchart of an object management method applied in a distributed deployment scenario of a container warehouse according to another embodiment of the present invention, where an execution subject of the object management method may be some or some container warehouse in a distributed deployment system of the container warehouse, such as any one of the container warehouses. Referring to fig. 6, the object management method at least includes steps S610 to S630, which are described in detail as follows:

in step S610, if the container warehouse receives an acquisition request of an object acquirer for a target object, it is determined whether the target object is stored in the local storage space;

in step S620, if it is determined that the target object is stored in the local storage space, returning the target object stored in the local storage space to the object acquirer;

in step S630, if it is determined that the target object is not stored in the local storage space, the target object is obtained from the object storage server and returned to the object obtaining party.

The technical solution of the embodiment shown in fig. 6 enables the container warehouse to preferentially consider returning the target object stored in the local storage space to the object acquirer when receiving the acquisition request of the object acquirer for the target object, so as to improve the object acquisition efficiency and avoid increasing unnecessary traffic consumption caused by directly acquiring the object from the object storage server each time. Meanwhile, when the target object is not stored in the local storage space, the target object can be acquired from the object storage server and returned to the object acquirer, so that the required object can be provided to the object acquirer.

Based on the technical solution of the embodiment shown in fig. 6, after the container warehouse acquires the target object from the object storage server, the target object may also be stored in the local storage space, and further, when an acquisition request for the target object is received again, the object stored in the local storage space may be directly provided to the object acquirer. In other words, the technical solution of the embodiment of the present invention generates traffic consumption only when the target object is first acquired from the object storage server, and solves the problem of increasing traffic consumption when the target object is acquired from the object storage server each time when an acquisition request for the target object is received.

Fig. 7 schematically shows a flowchart of an object management method applied in a distributed deployment scenario of a container warehouse according to yet another embodiment of the present invention, where an execution subject of the object management method may be a certain or some container warehouse in a distributed deployment system of the container warehouse, such as any one of the container warehouses. Referring to fig. 7, the object management method at least includes step S710 and step S720, which are described in detail as follows:

in step S710, detecting whether the remaining capacity of the local storage space is lower than a first predetermined threshold;

in step S720, if it is detected that the remaining capacity of the local storage space is lower than the first predetermined threshold, the local storage space is cleared, so that the remaining capacity of the local storage space reaches a second predetermined threshold.

The technical solution of the embodiment shown in fig. 7 enables the local storage space of the container warehouse to be cleaned in time, so as to ensure that the local storage space of the container warehouse has sufficient capacity to store the objects uploaded by the object uploading party and the objects acquired from the object storage server.

In an embodiment of the present invention, the scheme for cleaning the local storage space in step S720 may have the following specific embodiments:

example 1 for cleaning up local storage space：

In one embodiment of the invention, the access frequency of each object stored in the local storage space is counted; and deleting the preset number of objects stored in the local storage space according to the sequence of the access frequency from low to high.

The technical scheme of the embodiment enables the objects which are accessed infrequently to be cleaned preferentially, and prevents the objects which are accessed infrequently from occupying more storage space.

Example 2 for cleaning local storage space：

In one embodiment of the invention, the access time of each object stored in the local storage space is determined; and deleting the preset number of objects stored in the local storage space according to the sequence of the access time from far to near.

According to the technical scheme of the embodiment, the objects with longer access time can be cleared preferentially, and the objects with longer access time are prevented from occupying more storage space.

Example 3 of scrubbing local storage space：

In one embodiment of the invention, an object cleaning instruction sent by a designated device is received; and deleting the corresponding object stored in the local storage space based on the object cleaning instruction.

The technical solution of this embodiment enables deleting the corresponding object based on the object cleaning instruction sent by the specified device. The designated device may be a bypass module independent of the container warehouse, and the object cleaning instruction sent by the designated device may also be to preferentially delete an object with a lower access frequency and/or a longer access time.

Based on the explanation of the above embodiments, it can be seen that the object management method according to the embodiments of the present invention does not need objects stored in synchronization in each container warehouse, and thus capacity expansion can be performed conveniently. In particular, the container repository may receive a configuration file comprising parameter information of the object storage server, and may in turn connect to the object storage server based on the parameter information.

In an embodiment of the present invention, the configuration file may be configured by a technician, where the parameter information may include a network access address, an http url, a container packet for storing data, and the like.

It should be noted that the technical solutions of the embodiments shown in fig. 5 to fig. 7 can be combined or implemented separately according to actual situations. The object management method applied to the distributed deployment scenario of the container warehouse in the above embodiment of the present invention may be generalized to a general distributed system, for example, the distributed system may include a plurality of distributed nodes and a data storage server, and each distributed node is connected to the data storage server. After a certain distributed node receives data uploaded by a data provider (for convenience of description, the data is referred to as data 1 in the following), on one hand, the data 1 may be stored in a local storage space, on the other hand, the data 1 may be sent to the data storage server 105, and when other users request to acquire the data 1, the distributed node may directly send the data 1 stored in the local storage space to a corresponding data acquirer, thereby improving data acquisition efficiency. Meanwhile, when the other distributed nodes need to acquire the data 1 to provide the data acquiring party, if the data 1 is stored in the local storage space, the data 1 stored in the local storage space is sent to the data acquiring party, and if the data 1 is not stored in the local storage space, the data 1 can be acquired from the data storage server 105. Therefore, the technical scheme of the embodiment of the invention does not need to carry out data synchronization among all the distributed nodes, solves the problem of capacity limitation caused by the need of data synchronization among the distributed nodes, is convenient for capacity expansion, and reduces the capacity expansion cost.

Implementation details of the embodiment of the present invention are further described below with reference to fig. 8 to fig. 10, where the Object storage server may be a server that provides Cos (Object storage Service, which is an Object storage Service provided by the tenuous Cloud platform) Service.

As shown in fig. 8, in the distributed deployment scenario of the registrations, one or more registrations can be deployed in different regions (e.g., region a, region B, \8230;) as shown in fig. 8, and each registration is connected to the Cos service, that is, in the embodiment of the present invention, the registrations in different regions and the registration in the same region are both connected to the Cos service without direct connection between the registrations. One Registry internally includes a storage drive (storage driver) and a local storage (local storage).

In one embodiment of the present invention, before installing the Registry, a suitable host needs to be selected, such as selecting a PC (personal computer) as the host, after which a Docker can be installed on the host, and then installing the Registry by obtaining an official Registry image. After installation of the Registry, local storage may be initiated for storing data received by the Registry or downloaded from Cos locally (i.e., the host where the Registry is installed).

In one embodiment of the invention, after installation of the Registry, data may be uploaded to (e.g., uploading an image file) or downloaded from (e.g., downloading an image file) the Registry via the Docker client. The Docker client is a client that a user initiates a request (such as an upload mirror request, a download mirror request, etc.) through the Docker command line tool, for example, a computer installed with Docker is used as the client, and the request is initiated on the computer through the Docker command line.

In an embodiment of the present invention, for a distributed deployment scenario of the Registry, if an object storage manner is adopted to store an object, when a user uses a client to upload data, the object can be sent to an object storage service through the Registry, in the process, the Registry is only used as a bridge between the client and the object storage service, in this case, the Registry needs to acquire a corresponding object from the object storage service every time an object acquisition request is received, and further, a problem of traffic consumption inevitably arises. Based on this problem, in another embodiment of the present invention, when a user uploads data (which may be an object) through a client, registry may first save a copy locally, and send the uploaded data to an object storage service (i.e., cos) for storage, for example, a storage driver (storage driver) may send the data to a local storage (local storage) and send the data to the object storage service for storage.

As shown in fig. 9, a user initiates a push request to a Registry through a client (e.g., a Docker client), after receiving the push request, the Registry starts to receive data transmitted by the client, and calls a storage driver to store the received data to the local; after the Registry receives the data, an upload request is sent to the Cos to acquire an upload address, and a storage driver is called to upload the data just received to the address, so that one mirror image upload is completed.

In the foregoing embodiment of the present invention, since the Registry stores the data uploaded by the client locally after receiving the data, when the user downloads the data (the data may be an object), the Registry may preferentially obtain the data locally, and when the corresponding data is not found locally, pull the corresponding data from the object storage service and transmit the data to the caller, and simultaneously store the pulled data locally for subsequent use, so as to avoid excessive traffic consumption caused by pulling the same data from the object storage server each time.

As shown in fig. 10, a user initiates a pull request to a Registry through a client (e.g., a Docker client), and after receiving the pull request, the Registry first checks whether there is data required by the client in a local storage. If yes, calling a storage driver to directly return the corresponding data in the local storage to the requester; if not, the storage driver is called to initiate a downloading request to the Cos, the required file is downloaded to the local storage, and then the file is returned to the requesting party.

In the technical solution of the above embodiment of the present invention, because the registers in different regions and the registers in the same region are both connected to the object storage service, and there is no need to directly connect the registers, there is no need to consider the connection and data synchronization problem between the registers when performing capacity expansion, which facilitates the capacity expansion of the registers. And the data uploaded by the user is finally stored in the object storage service, so that all the registers can acquire the required data from the object storage service, and the problem of capacity limitation caused by the need of synchronizing local storage contents among all the registers during capacity expansion is solved. Meanwhile, since the data is cached locally in the Registry, when the data in different regions is pulled from the object storage service, the pulling flow is generated only for the first time, the data can be directly obtained from the local subsequently, and the problem that the excessive flow is consumed because the pulling is required every time in the traditional scheme is solved.

In an embodiment of the present invention, a scheme for clearing a local storage space of a Registry is further provided, and specifically, when the Registry downloads data from Cos or receives data uploaded by a user each time, the local storage space is checked, and if an occupied space reaches a certain threshold, a part of data is deleted, so that a space is released to store new data until a certain amount of space is cleared. When deleting data, data which is not accessed frequently can be deleted, for example, when an uploading/downloading interface is called each time, files which are operated and the access times of the files are recorded, and when cleaning is carried out, the files with the lower access times are cleaned preferentially. Of course, in other embodiments of the invention, data with longer access times may also be deleted.

In addition, in an embodiment of the present invention, a new Registry only used for forwarding a client request may be added, which is used for a user to uniformly access and forward a request, so as to implement load balancing and bandwidth expansion and capacity of uploading and downloading.

Based on the technical solutions of the above embodiments, in the embodiments of the present invention, since each Registry can obtain the required data through the data storage service, the data stored synchronously between the registries is not needed, so that the capacity expansion can be performed conveniently, and the capacity expansion cost is reduced. Specifically, when performing capacity expansion, a technician may fill in unified access, httpurl, and bucket (container for storing files) in the configuration file of the Registry, that is, may access the same bucket anywhere. In other words, the technician completes one expansion only by filling the configuration file with the correct bucket information (where the docker image file is located) and starting the Registry, the service is immediately available, and all images are shared with other registries without manually synchronizing the images. When the capacity is reduced, the corresponding service is only needed to be stopped. Therefore, the technical scheme of the embodiment of the invention enables the distributed deployment of the Docker Registry to be efficient and convenient, and greatly improves the efficiency of capacity expansion and capacity reduction.

The following describes an embodiment of an apparatus of the present invention, which may be used to execute an object management method applied in a distributed deployment scenario of a container warehouse in the foregoing embodiment of the present invention. For details that are not disclosed in the embodiment of the apparatus of the present invention, please refer to the embodiment of the object management method applied in the distributed deployment scenario of the container warehouse of the present invention.

Fig. 11 schematically shows a block diagram of an object management apparatus applied in a distributed deployment scenario of a container warehouse, where the object management apparatus is applicable to some or some container warehouses, such as any one of the container warehouses, in a distributed deployment system of the container warehouse according to an embodiment of the present invention.

Referring to fig. 11, an object management apparatus 110 applied in a distributed deployment scenario of a container warehouse according to an embodiment of the present invention includes: a storage unit 111 and a transmission unit 112.

The storage unit 111 is configured to store the object in a local storage space of the container warehouse when receiving the object uploaded by the object uploading party; the sending unit 112 is configured to send the object to an object storage server connected to a plurality of container warehouses deployed in a distributed manner for storage, so that other container warehouses acquire the object from the object storage server.

Referring to fig. 12, an object management apparatus 120 applied in a distributed deployment scenario of a container warehouse according to another embodiment of the present invention further includes, on the basis of the object management apparatus 110 shown in fig. 11: a judging unit 113 and a responding unit 114.

The judging unit 113 is configured to, when receiving an acquisition request of an object acquirer for a target object, judge whether the target object is stored in a local storage space of the container warehouse; the response unit 114 is configured to return the target object stored in the local storage space to the object acquirer when the determination unit 113 determines that the target object is stored in the local storage space, and is configured to acquire the target object from the object storage server and return the target object to the object acquirer when the determination unit 113 determines that the target object is not stored in the local storage space.

In some embodiments of the present invention, based on the foregoing solution, the storage unit 111 is further configured to store the target object obtained from the object storage server into the local storage space.

Referring to fig. 13, an object management apparatus 130 applied in a distributed deployment scenario of a container warehouse according to another embodiment of the present invention further includes, on the basis of the object management apparatus 120 shown in fig. 12: a detection unit 115 and a cleaning unit 116.

Wherein the detecting unit 115 is configured to detect whether a remaining capacity of the local storage space is lower than a first predetermined threshold; the cleaning unit 116 is configured to, when the detecting unit 115 detects that the remaining capacity of the local storage space is lower than the first predetermined threshold, clean the local storage space so that the remaining capacity in the local storage space reaches a second predetermined threshold.

In some embodiments of the present invention, based on the foregoing solution, the cleaning unit 116 is configured to: counting the access frequency of each object stored in the local storage space; and deleting the preset number of objects stored in the local storage space according to the sequence of the access frequency from low to high.

In some embodiments of the present invention, based on the foregoing solution, the cleaning unit 116 is configured to: determining access time of each object stored in the local storage space; and deleting the preset number of objects stored in the local storage space according to the sequence of the access time from far to near.

In some embodiments of the present invention, based on the foregoing solution, the cleaning unit 116 is configured to: receiving an object cleaning instruction sent by a designated device; and deleting the corresponding object stored in the local storage space based on the object cleaning instruction.

Referring to fig. 14, an object management apparatus 140 applied in a distributed deployment scenario of a container warehouse according to still another embodiment of the present invention further includes, on the basis of the object management apparatus 110 shown in fig. 11: a detection unit 115 and a cleaning unit 116. The specific functions of the detection unit 115 and the cleaning unit 116 are similar to those described in the above embodiments.

In addition, on the basis of the object management apparatus applied to the distributed deployment scenario of the container warehouse shown in fig. 11 to 14, in other embodiments of the present invention, a receiving unit and a processing unit may also be included. The receiving unit is used for receiving a configuration file, wherein the configuration file comprises parameter information of the object storage server; the processing unit is used for connecting to the object storage server based on the parameter information.

In some embodiments of the present invention, based on the foregoing scheme, the parameter information includes a network access address, an http url, and a container packet for storing data.

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the invention. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.

Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

1. An object management method applied to a container warehouse under a distributed deployment scene is characterized in that under the distributed deployment scene, a plurality of container warehouses deployed in a distributed manner are respectively connected with an object storage server, each container warehouse internally comprises a storage drive and a local storage space, when receiving an object uploaded by an object uploading party, the container warehouses call the storage drives to directly store the object in the local storage space of the container warehouse and upload the object to the object storage server, and the container warehouses are not synchronous with each other for storing the object and are not directly connected with each other; when capacity expansion is carried out, the added container warehouse is connected to the object storage server, and the added container warehouse is not connected to other container warehouses and does not need to synchronize data with other container warehouses;

the object management method comprises the following steps:

when an object uploaded by an object uploading party is received or an object needs to be acquired from the object storage server, detecting whether the residual capacity of the local storage space of the container warehouse is lower than a first preset threshold value;

if the fact that the residual capacity of the local storage space is lower than the first preset threshold value is detected, cleaning the local storage space to enable the residual capacity of the local storage space to reach a second preset threshold value;

wherein, cleaning the local storage space comprises: deleting the preset number of objects stored in the local storage space according to the sequence of the access frequency from low to high; and/or deleting the preset number of objects stored in the local storage space according to the sequence of the access time from far to near.

2. The object management method applied to the distributed deployment scenario of the container warehouse according to claim 1, further comprising:

if the container warehouse receives an acquisition request of an object acquirer for a target object, judging whether the target object is stored in a local storage space of the container warehouse or not;

if the target object is judged to be stored in the local storage space, returning the target object stored in the local storage space to the object acquirer;

and if the target object is not stored in the local storage space, acquiring the target object from the object storage server and returning the target object to the object acquirer.

3. The object management method applied to the distributed deployment scenario of the container warehouse according to claim 2, further comprising, after obtaining the target object from the object storage server:

and storing the target object into the local storage space.

4. The object management method applied to the distributed deployment scenario of the container warehouse according to claim 1, wherein the cleaning of the local storage space further comprises:

receiving an object cleaning instruction sent by a designated device;

and deleting the corresponding object stored in the local storage space based on the object cleaning instruction.

5. The object management method applied to the distributed deployment scenario of the container warehouse according to any one of claims 1 to 4, further comprising:

receiving a configuration file, wherein the configuration file comprises parameter information of the object storage server;

connecting to the object storage server based on the parameter information.

6. The object management method applied to the distributed deployment scenario of the container warehouse according to claim 5, wherein the parameter information includes a network access address, a uniform resource locator http url of a hypertext transfer protocol, and a container button for storing data.

7. A distributed deployment system for a container warehouse, comprising:

the system comprises a plurality of container warehouses which are deployed in a distributed mode, wherein each container warehouse internally comprises a storage drive and a local storage space, when receiving an object uploaded by an object uploading party, the container warehouses call the storage drives to directly store the object in the local storage space of the container warehouse and upload the object to an object storage server, the container warehouses are not synchronous with each other for storing the object, and the container warehouses are not directly connected with each other; when capacity expansion is carried out, the added container warehouse is connected to the object storage server, and the added container warehouse is not connected to other container warehouses and does not need to synchronize data with other container warehouses; one or more of the plurality of container stores are for implementing an object management method as claimed in any one of claims 1 to 6;

and the object storage server is used for receiving and storing the objects uploaded by the container warehouses and providing the required objects to the container warehouses.

8. The distributed deployment system of a container warehouse of claim 7, further comprising:

and the control node is connected to the plurality of container warehouses and is used for receiving the request message sent by the object uploading party and/or the object acquiring party and distributing the container warehouses responding to the request message based on a load balancing mode.

9. The distributed deployment system of container warehouses of claim 8, wherein the control node is further configured to control the bandwidth of the plurality of container warehouses uploading objects to the object storage server and/or the bandwidth of the plurality of container warehouses downloading objects from the object storage server.

10. An object management device applied to a container warehouse in a distributed deployment scenario is characterized in that in the distributed deployment scenario, a plurality of container warehouses deployed in a distributed manner are respectively connected with an object storage server, each container warehouse internally comprises a storage drive and a local storage space, when receiving an object uploaded by an object uploading party, each container warehouse calls the storage drive to directly store the object in the local storage space of the container warehouse and uploads the object to the object storage server, the container warehouses are not synchronous with each other for storing the object, and the container warehouses are not directly connected with each other; when capacity expansion is carried out, the added container warehouse is connected to the object storage server, and the added container warehouse is not connected to other container warehouses and does not need to synchronize data with other container warehouses; the object management apparatus includes:

the device comprises a detection unit, a storage unit and a processing unit, wherein the detection unit is used for detecting whether the residual capacity of the local storage space of the container warehouse is lower than a first preset threshold value when an object uploaded by an object uploading party is received or the object needs to be acquired from the object storage server;

a cleaning unit, configured to clean the local storage space when the detection unit detects that the remaining capacity of the local storage space is lower than the first predetermined threshold, so that the remaining capacity in the local storage space reaches a second predetermined threshold;

wherein the cleaning unit is configured to: deleting the preset number of objects stored in the local storage space according to the sequence of the access frequency from low to high; and/or deleting the preset number of objects stored in the local storage space according to the sequence of the access time from far to near.

11. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, implements the object management method applied in a distributed deployment scenario of a container warehouse according to any one of claims 1 to 6.

12. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the object management method as claimed in any one of claims 1 to 6 applied in a distributed deployment scenario of a container warehouse.