CN110874255A - Garbage cleaning method and device, equipment and storage medium - Google Patents

Garbage cleaning method and device, equipment and storage medium Download PDF

Info

Publication number
CN110874255A
CN110874255A CN201810996685.8A CN201810996685A CN110874255A CN 110874255 A CN110874255 A CN 110874255A CN 201810996685 A CN201810996685 A CN 201810996685A CN 110874255 A CN110874255 A CN 110874255A
Authority
CN
China
Prior art keywords
mirror image
request
garbage cleaning
target
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810996685.8A
Other languages
Chinese (zh)
Inventor
白淼淼
杨武军
郭天
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to CN201810996685.8A priority Critical patent/CN110874255A/en
Publication of CN110874255A publication Critical patent/CN110874255A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5022Mechanisms to release resources
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45562Creating, deleting, cloning virtual machine instances

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the invention discloses a garbage cleaning method, a garbage cleaning device, garbage cleaning equipment and a garbage cleaning storage medium, wherein the method comprises the following steps: acquiring a deletion request aiming at a target mirror image in a mirror image warehouse; after the target mirror image is deleted, according to a reference counting strategy, the reference counting of the storage block referred by the target mirror image is decreased; and after the reference count is decreased to zero, performing garbage cleaning once according to a preset garbage cleaning strategy, and deleting the memory block referenced by the target mirror image.

Description

Garbage cleaning method and device, equipment and storage medium
Technical Field
The present invention relates to computer technologies, and in particular, to a garbage cleaning method, apparatus, device, and storage medium.
Background
Docker is an open source application container engine. The mirror repository registry is one of the core components of Docker and is responsible for storing the mirror. The mirror image is a basic unit which can run in the Docker, the mirror image is composed of layers, and each layer of the mirror image is a file system. When the memory mirror image is registered, a blob is allocated for each layer of the mirror image and a description file containing the reference relation of the layer of the mirror image for storage, and the blob is a storage block. For each blob, the registry performs hash calculation according to the stored content, and the hash value is used as the key value of the blob. And the key value of the mirror image is a hash value of the blob where the description file is located. When a new image is pushed to the registry, the registry judges whether the image layer exists in the registry according to the hash value, if so, the step of uploading through the request is skipped, and the reference relationship is directly established, so that the storage of the registry can be saved, and the network flow can be reduced.
As users continually upload images, the system memory occupied by the registry will gradually increase. But the user cannot achieve the purpose of immediately releasing the storage space after deleting the mirror image in the registry. And providing a garbage cleaning tool for the registry, wherein the using process comprises the steps of stopping registry service, then scanning the existing images and blobs of the system in a full amount, analyzing the reference relation of the blobs one by one, marking the blobs referenced by the images, and deleting the blob storage file aiming at the blobs which are not referenced by the images, so that the storage space of the system is released.
Disclosure of Invention
In view of this, embodiments of the present invention provide a garbage cleaning method, apparatus, device, and storage medium for solving at least one problem in the prior art, so that on the premise of starting a registry service, a user does not need to trigger, dynamic garbage cleaning without stopping the registry service is realized, and meanwhile, mirror image deletion using a mirror image name plus Tag is realized, where Tag is a label; therefore, during garbage cleaning, the registration service does not need to be stopped, only the full scanning can be carried out when the service is started, all operations and records on the mirror image and the blob are incremental during operation, the system burden is smaller, and the cleaning speed is higher. The technical scheme of the embodiment of the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a method for cleaning garbage, where the method includes:
acquiring a deletion request aiming at a target mirror image in a mirror image warehouse;
after the target mirror image is deleted, according to a reference counting strategy, the reference counting of the storage block referred by the target mirror image is decreased;
and after the reference count is decreased to zero, performing garbage cleaning once according to a preset garbage cleaning strategy, and deleting the memory block referenced by the target mirror image.
In a second aspect, an embodiment of the present invention provides a garbage disposal apparatus, where the apparatus includes: agent control module, mirror image management module, quote count module and rubbish clearance control module, wherein:
the agent control module is used for acquiring a deletion request aiming at a target mirror image in the mirror image warehouse;
the mirror image management module is used for deleting the target mirror image;
the reference counting module is used for counting down the reference of the storage block referenced by the target mirror image according to a reference counting strategy;
and the garbage cleaning control module is used for performing garbage cleaning once according to a preset garbage cleaning strategy and deleting the memory block referred by the target mirror image after the reference count is decreased to zero.
In a third aspect, an embodiment of the present invention provides an apparatus, including a memory and a processor, where the memory stores a computer program operable on the processor, and the processor implements the steps in the garbage cleaning method when executing the program.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps in the garbage cleaning method.
In the embodiment of the invention, a deletion request for a target mirror image in a mirror image warehouse is acquired; after the target mirror image is deleted, according to a reference counting strategy, the reference counting of the storage block referred by the target mirror image is decreased; when the reference count is decreased to zero, performing garbage cleaning once according to a preset garbage cleaning strategy, and deleting the memory block referenced by the target mirror image; therefore, during garbage cleaning, the registration service does not need to be stopped, only full scanning can be carried out when the service is started, all operations and records on the mirror image and the blob are incremental during operation, the system load is smaller, the cleaning speed is higher, a cleaning strategy can be customized according to the registration operation environment, the applicability is wider, an interface for deleting the mirror image by adding the Tag to the mirror image name is increased, and the operation for deleting the mirror image is simplified.
Drawings
FIG. 1 is a schematic diagram of a process for implementing a garbage disposal method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a process of querying a request for specifying whether a memory block exists according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a process of creating a request for creating an image according to an embodiment of the present invention;
FIG. 4 is a schematic view of a processing flow of a request for deleting a mirror image by a mirror image key value according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating a process of deleting a mirror request via a mirror tag according to an embodiment of the present invention;
FIG. 6 is a schematic view of a garbage disposal control flow according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a garbage disposal apparatus according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of the garbage cleaning apparatus and the mirror warehouse according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of the structure of the apparatus according to the embodiment of the present invention.
Detailed Description
The technical solution of the present invention is further elaborated below with reference to the drawings and the embodiments.
Example one
Fig. 1 is a schematic view of an implementation flow of a garbage disposal method according to an embodiment of the present invention, as shown in fig. 1, the method includes:
step S101, a deletion request for a target mirror in a mirror repository is acquired.
And step S102, after the target mirror image is deleted, according to the reference counting strategy, the reference counting of the storage block referred by the target mirror image is decreased.
And step S103, when the reference count is decreased to zero, performing garbage cleaning once according to a preset garbage cleaning strategy, and deleting the memory block referenced by the target mirror image.
For the technical solution shown in fig. 1, it should be noted that a delete request for a target mirror in a mirror repository is obtained; after the target mirror image is deleted, according to a reference counting strategy, the reference counting of the storage block referred by the target mirror image is decreased; when the reference count is decreased to zero, performing garbage cleaning once according to a preset garbage cleaning strategy, and deleting the memory block referenced by the target mirror image; on the premise of starting a registry service, a user is not required to trigger, dynamic garbage cleaning of the registry service without stopping the registry service is realized, and meanwhile, image deletion using an image name and Tag is realized.
For the technical solution shown in fig. 1, in a possible implementation, performing garbage cleaning once according to a preset garbage cleaning policy after the reference count is decremented to zero, and deleting the memory block referenced by the target image includes:
if the time interval between the first moment and the previous garbage cleaning starting moment is equal to a preset first time interval, performing garbage cleaning once in a preset second time interval, and deleting the memory blocks quoted by the target mirror image, wherein the first moment comprises that the quote count is decreased to zero and the count of the requests of the mirror image warehouse is decreased to zero; the request for the mirror image warehouse comprises a deletion request, a storage block access request and a mirror image information modification request. It should be noted that the request for accessing the storage block is a HEAD request, and the request for modifying the mirror image information is a PUT request.
Preferably, before performing garbage cleaning once within a preset second time interval and deleting the memory block referred by the target image, the method further includes:
and caching the request of the mirror image warehouse after the first moment.
Then, preferably, after caching the request for the mirror repository after the first time, the method further includes:
and releasing the cache of the request for the mirror image warehouse after the current moment when the preset second time interval is exceeded.
For the technical solution shown in fig. 1, in a possible implementation, after the obtaining a deletion request for a target image in an image repository, the method further includes:
and analyzing the deletion request aiming at the target mirror image in the mirror image warehouse to acquire the mirror image name and the mirror image label information.
Then, preferably, deleting the target image comprises:
and deleting the target mirror image according to the mirror image name and the mirror image label information.
After the target image is deleted, the decrementing of the reference count of the storage block referenced by the target image according to a preset reference count policy includes:
the storage block referred by the target mirror image is referred by N mirror images, wherein the N mirror images comprise the target mirror image, and N is a positive integer greater than or equal to 1;
when any one of the N images is deleted, the reference count is decremented;
when all of the N images are deleted, the reference count is decremented to zero.
Based on the same inventive concept of the foregoing embodiments, the present embodiment explains the technical solutions of the foregoing embodiments by specific examples. Fig. 2 is a schematic view of a processing flow of a request for querying whether a specified storage block exists according to an embodiment of the present invention, as shown in fig. 2, specifically including:
step S201, a HEAD request is acquired.
It should be noted that the server receives a HEAD request sent by a docker daemon, where a URL of the HEAD request carries a name of an upload mirror and ID information of a mirror layer, where the docker daemon is a service end of the docker. The HEAD request is a request to the mirror repository.
Step S202, determining whether the HEAD request needs to be cached, if not, going to step S203 for processing.
Step S203, increment the request counter and then forward the HEAD request to the mirror repository registry.
Step S204, judging whether the mirrored layer needs to be uploaded or not according to the situation of the current mirror image warehouse registry storage blob, and if not, turning to the step S205 for processing; if the upload is required, the process goes to step S208.
Note that the blob is a storage block.
Step S205, sending a message OK to reply to the HEAD request to the docker daemon, and decrementing the request counter.
Step S206, if the reference relation of the storage block is identified, the storage block is referenced.
It should be noted that, performing a reference operation on the storage block, that is, counting the reference of the storage block referenced by the to-be-uploaded mirror, and incrementing the reference count.
Step S207, after the reference period, erasing the reference relationship to the storage block.
It should be noted that step S207 may prevent a failure of the whole uploading action due to garbage cleaning deleting the layer before the mirror image is not uploaded successfully; meanwhile, if the uploading request is interrupted, the uploaded storage blocks need to be cleared in time by relying on the mechanism. The reference count of the memory block is decremented. The reference period is a preset value, and preferably, the reference period is set to 10 minutes. Erasing the reference relationship to the memory block, i.e., the reference count of the memory block, is decremented.
It should be noted that, in step S206 and step S207, the reference count of the storage block is incremented, and after the preset reference period, the reference count of the storage block is correspondingly decremented no matter whether the uploading of the uploaded mirror image is successful or not. When the memory block reference is successful, the reference count of the memory block is additionally incremented.
Step S208, replying a failure message FAIL to the HEAD request to the docker daemon, and the request counter is decremented.
Based on the same inventive concept of the foregoing embodiments, the present embodiment explains the technical solutions of the foregoing embodiments by specific examples. Fig. 3 is a schematic processing flow diagram of a request for creating a new image according to an embodiment of the present invention, and as shown in fig. 3, the processing flow specifically includes:
step S301, a PUT request is acquired.
It should be noted that the PUT request URL carries a mirror name and a mirror Tag, or the PUT request URL carries information of a mirror key, and the PUT request body includes detailed mirror description information. After the completion of uploading of the dockerdaemon on all the image layers, a PUT request needs to be sent, the image configuration file is uploaded, and the whole uploading process is completed. Wherein, the mirror image configuration file is a description file containing the layer reference relation of the mirror image. The PUT request is a request to the mirror store.
Step S302, determine whether the PUT request needs to be cached, if the PUT request does not need to be cached, go to step S303 to process.
Step S303, increment the request counter, and then forward the PUT request to the mirror repository registry.
And step S304, judging whether the image is successfully saved, if the image is successfully saved, turning to the step S305 for processing, and if the image is not successfully saved, turning to the step S308 for processing.
Step S305, sending a message SUCC for replying to the PUT request to the docker daemon, and decreasing the request counter.
It should be noted that the registry successfully processes the PUT request, which indicates that the registry has successfully saved the image, and replies the SUCC message. And the server sends a message SUCC for replying the PUT request to the docker daemon, and the request counter is decreased.
And S306, recognizing that the registration storage mirror image is successful, and judging whether the original mirror image needs to be updated or a new mirror image is added according to the transmitted mirror image information.
Step S307, updating the reference relation of the corresponding storage block for the reference change of the storage block caused by the mirror image change.
It should be noted that, the reference relationship of the corresponding storage block is updated, that is, the reference count of the storage block referred by the image is incremented or decremented.
Step S308, sending a message FAIL for responding to the PUT request to the docker daemon, and decreasing the request counter.
It should be noted that, due to failure of the PUT request for some reason, the registry FAILs to save the image successfully and replies to the FAIL message. And the server sends a message FAIL for replying the PUT request to the docker daemon, and the request counter is decreased.
It should be noted that, by uploading the mirror image through the docker, the docker daemon firstly takes the layer information of the mirror image from the local, then sequentially judges whether the layer of the mirror image needs to be uploaded, if so, the uploading operation is performed, and the uploading operation is not skipped, and after all the layers are uploaded, the docker daemon sends a new requests to upload the relationship between the mirror image layers, and at this time, the mirror image uploading action is completed. Wherein the artifacts exist in the registry as metadata files of the docker image.
Based on the same inventive concept of the foregoing embodiments, the present embodiment explains the technical solutions of the foregoing embodiments by specific examples. Fig. 4 is a schematic view of a processing flow of a request for deleting a mirror image by a mirror image key value according to an embodiment of the present invention, and as shown in fig. 4, the processing flow specifically includes:
step S401, obtain DELETE request.
It should be noted that the user deletes the mirror image through the registration native interface. The deletion request is a DELETE request, and the URL of the DELETE request carries information of mirror names and mirror key values. The native registry DELETE mirror interface sends a DELETE request. Deleting the key-value information of the mirror poses the risk that all tags referring to the key-value of the mirror are not available. The DELETE request is a request to a mirror repository.
Step S402, judging whether the DELETE request needs to be cached, if not, turning to step S403 for processing.
Step S403, increment the request counter, and forward the DELETE request to the mirror repository registry.
And step S404, judging whether the mirror image is successfully deleted, if so, turning to the step S405, and if not, turning to the step S408.
Step S405, sending the message SUCC for the DELETE request reply to the user, and the request counter is decreased.
It should be noted that the registration successfully deletes the specified mirror image, and replies to the successful SUCC message. The server sends to the user a message SUCC in reply to the DELETE request, the request counter being decremented.
Step S406, when the registration message SUCC is received, the mirror image record specified by the user is deleted.
In step S407, the reference count of the memory block referenced by the mirror is decremented.
It should be noted that, the reference count of the memory block referred by the image is decremented, that is, a dereference operation is performed on the memory block.
Step S408, a message NOT FOUND in reply to the DELETE request is sent to the user, and the request counter is decremented.
It should be noted that the registry considers that the user deletes the image, or the image deletion fails due to other reasons. The registry replies with the delete failure NOT FOUND message and takes the reason for the failure. The server sends a message NOT FOUND in reply to the DELETE request to the user, the request counter being decremented.
Based on the same inventive concept of the foregoing embodiments, the present embodiment explains the technical solutions of the foregoing embodiments by specific examples. Fig. 5 is a schematic processing flow diagram of a request for deleting a mirror image through a mirror image tag according to an embodiment of the present invention, and as shown in fig. 5, the processing flow specifically includes:
step S501, obtain DELETE request.
It should be noted that, the user deletes the mirror image through the mirror image name plus Tag deletion interface, where the Tag is a label. The user sends a DELETE request to DELETE the mirror image, the URL of the DELETE request needs to carry the mirror image name and the mirror image Tag information, and the DELETE request is sent to the proxy control module. The add-delete interface allows the user to delete the specified image by specifying the image name and Tag in the URL. The DELETE request is a request to a mirror repository.
Step S502, when the DELETE request is identified to carry Tag information, the processing is transferred to step S503, and if the mirror image specified by the user cannot be found, the processing is transferred to step S506.
In step S503, the specified mirror is deleted.
Step S504, the reference count of the storage block of the specified mirror is decreased.
And step S505, replying a successful deletion SUCC message to the user.
In step S506, the mirror image specified by the user cannot be FOUND, and a NOT FOUND message indicating the deletion failure is replied to the user.
Based on the same inventive concept of the foregoing embodiments, the present embodiment explains the technical solutions of the foregoing embodiments by specific examples. Fig. 6 is a schematic view of a garbage disposal control flow according to an embodiment of the present invention, as shown in fig. 6, specifically including:
step S601, when the request counter is decremented, determines whether the current traffic of the registration is 0, and if so, goes to step S602 to process.
It should be noted that, when the reference count of the storage block is decremented to zero, it is determined whether the current traffic of the registry is 0, where the current traffic of the registry is the total number of various requests to the mirror warehouse, and the various requests include DELETE request, HEAD request, and PUT request.
Step S602, determining whether the first time meets the immediate cleaning condition, if so, going to step S603, and if not, going to step S604.
It should be noted that when the current traffic of the registry is 0, whether garbage cleaning can be performed at the first time is judged according to whether the interval between the first time and the previous garbage cleaning start time is equal to a preset first time interval; cleaning conditions immediately: the interval between the first moment and the last garbage cleaning starting moment is equal to a preset first time interval; and when the clearing condition is met, performing garbage clearing, otherwise, waiting for the time when the next registration traffic is 0.
Step S603, when the requirement of immediate cleaning is satisfied, enter HOLD mode.
It should be noted that the HOLD mode buffers the received various requests for the mirror repository registry and starts a buffer request timeout timer, where the HOLD mode is a service suspension mode.
And step S604, the garbage cleaning requirement is not met at the first moment, the whole cleaning process is finished, and the next triggering of the registration idle event is waited.
In step S605, all the memory blocks to be cleaned are acquired.
Step S606, one of all the memory blocks to be cleaned is acquired, if the acquisition is successful, the processing goes to step S608, and if the acquisition is failed, it indicates that there is no memory block that can be cleaned, the processing goes to step S607.
In step S607, the HOLD mode is exited.
It should be noted that exiting HOLD mode will release all buffered requests and clear the buffer request timeout timer.
Step S608, after the storage block information that can be cleaned is acquired, it is determined whether the cache request timeout timer is triggered overtime, if yes, the process goes to step S607, and if not, the process goes to step S609.
In step S609, the storage block and the recording information of the storage block are deleted.
It should be noted that, after the deletion is finished, the process proceeds to step S606 to apply for the next storage block information to be deleted until all the storage block information to be deleted is cleared.
Example two
Based on the same inventive concept of the foregoing embodiment, fig. 7 is a schematic structural diagram of a garbage disposal apparatus according to an embodiment of the present invention, and as shown in fig. 7, the garbage disposal apparatus 700 includes: an agent control module 701, a mirror management module 702, a reference counting module 703 and a garbage cleaning control module 704, wherein:
the agent control module 701 is configured to obtain a deletion request for a target mirror in a mirror repository;
the mirror image management module 702 is configured to delete the target mirror image;
the reference counting module 703 is configured to decrement a reference count of the memory block referred to by the target image according to a reference counting policy;
the garbage cleaning control module 704 is configured to perform garbage cleaning once according to a preset garbage cleaning policy and delete the storage block referenced by the target mirror image after the reference count is decremented to zero.
In the foregoing solution, preferably, the garbage cleaning control module 704 is configured to perform garbage cleaning once in a preset second time interval if a time interval between a first time and a previous garbage cleaning start time is equal to a preset first time interval, and delete a storage block referenced by the target image, where the first time includes that the reference count is decremented to zero and a count of requests to the image warehouse is decremented to zero; the request for the mirror image warehouse comprises a deletion request, a storage block access request and a mirror image information modification request. In the foregoing solution, preferably, the agent control module 701 is configured to parse a deletion request for a target mirror in a mirror repository, and obtain a mirror name and mirror label information.
Preferably, the agent control module 701 is configured to cache the request for the mirror warehouse after the first time.
Preferably, the agent control module 701 is configured to release the cache of the request for the mirror image warehouse after the first time when the preset second time interval is exceeded.
In the foregoing solution, preferably, the mirror image management module 702 is configured to delete the target mirror image according to the mirror image name and the mirror image tag information.
In the above solution, preferably, the reference counting module 703 is configured to reference the memory block referenced by the target image by N images, where the N images include the target image, and N is a positive integer greater than or equal to 1; when any one of the N images is deleted, the reference count is decremented; when all of the N images are deleted, the reference count is decremented to zero.
It should be noted that the agent control module 701 is configured to complete forwarding of a registry request, real-time statistics of the current service quantity, caching of a service request, identify a service request, and notify other related modules, where the registry request is a request for a mirror repository. The mirror management module 702 is responsible for maintaining the reference relationships of all mirrors and storage blocks in the current registry, and providing interfaces for adding, updating and deleting the reference relationships of the mirrors. And a reference count module 703, configured to maintain reference counts of all the memory blocks, and provide an interface for adding and deleting reference counts. And a garbage cleaning control module 704, configured to decide whether garbage cleaning can be performed at the current time, and perform flow control of garbage cleaning. When the system is powered on, the image management module 702 and the reference counting module 703 perform data initialization operation to obtain the reference relationship between the current image and the storage block; the proxy control and registration services are then started in turn. The agent control module 701 intercepts all requests for registry and thereby recognizes events for adding, deleting images and accessing memory blocks to notify the image management module 702 and the reference counting module 703. When the number of registered services is 0, the agent control module 701 notifies the garbage cleaning control module 704, and the garbage cleaning control module 704 determines whether to perform garbage cleaning currently according to the reference relationship of the current storage block. When performing garbage cleaning, the garbage cleaning control module 704 notifies the proxy control module 701 to cache all requests, and acquires the memory blocks that are not referenced currently from the reference counting module 703 to perform cleaning one by one.
Fig. 8 is a schematic structural diagram of a garbage cleaning apparatus and a mirror repository according to an embodiment of the present invention, and as shown in fig. 8, the garbage cleaning apparatus 700 and the mirror repository (registry)800, where the dynamic garbage cleaning apparatus 700 includes an agent control module 701, a mirror management module 702, a reference counting module 703 and a garbage cleaning control module 704, and the mirror repository 800 includes a storage block (blob) 801 and mirrors (images) 802.
It should be noted that, the process of querying the request specifying whether the memory block exists is:
s011, the agent control module 701 receives a message HEAD/v2/< image >/blob/< digst > sent by the docker daemon.
S012, the agent control module 701 forwards message HEAD/v2/< image >/blob/< digst > to the mirror repository 800.
S013, the image repository 800 sends a message OK or FAIL to the proxy control module 701.
S014, the agent control module 701 forwards the message OK or FAIL.
When the agent control module 701 forwards the message OK, the following processing is continued:
s015, the agent control module 701 sends a message blob < digst > head succ to the reference counting module 703.
S016, after waiting for a reference period, the agent control module 701 sends an error blob < digst > head history to the reference counting module 703, where the reference period is a preset value, and preferably, the reference period is set to 10 minutes.
It should be noted that, the processing of the request for creating an image:
s021, the agent control module 701 receives a message PUT/v2/< image >/experiences/< reference > sent by the docker daemon.
S022, the agent control module 701 forwards the message PUT/v2/< image >/experiences/< reference > to the mirror repository 800.
S023, the mirror repository 800 sends a message SUCC or FAIL to the agent control module 701.
S024, the agent control module 701 forwards a message SUCC or FAIL.
When the agent control module 701 forwards the message SUCC, the following processing is continued:
s025, the agent control module 701 sends the message add image < reference > to the image management module 702.
S026, the image management module 702 sends a message ref by image to the reference count module 703.
It should be noted that, the processing of the request for deleting the mirror by the mirror key value:
s031, the agent control module 701 receives a message DELETE/v2/< image >/requirements/< digst > sent by a user.
S032, proxy control module 701 forwards message DELETE/v2/< image >/experiences/< digst > to mirror store 800.
S033, the mirror repository 800 sends a message SUCC or NOT FOUND to the proxy control module 701.
S034, the agent control module 701 forwards the message SUCC or NOT FOUND.
When the agent control module 701 forwards the message SUCC, the following processing is continued:
s035, the agent control module 701 sends a message delete image by < digst > to the mirror image management module 702.
S036, the image management module 702 sends a message remove ref by image to the reference counting module 703.
It should be noted that the processing of the request for deleting the mirror by the mirror tag:
s041, the agent control module 701 receives a message DELETE/v2/< image >/games/< tag > sent by the user.
S042, the agent control module 701 forwards the message delete image by < tag > to the image management module 702.
When the agent control module 701 recognizes that the DELETE request carries Tag information, the following processing is performed:
s043, the image management module 702 sends a message remove ref by image to the reference counting module 703.
S044, the image management module 702 sends a message SUCC to the agent control module 701.
S045, the agent control module 701 forwards the message SUCC.
When the agent control module 701 cannot find the mirror image specified by the user, the following processing is performed:
s046, the image management module 702 sends a message NOT FOUND to the proxy control module 701.
S047, the agent control module 701 forwards the message NOT FOUND. Note that the Tag is a label.
The above description of the apparatus embodiments, similar to the above description of the method embodiments, has similar beneficial effects as the method embodiments. For technical details not disclosed in the embodiments of the apparatus according to the invention, reference is made to the description of the embodiments of the method according to the invention for understanding.
It should be noted that, in the embodiment of the present invention, if the garbage cleaning method is implemented in the form of a software functional module and is sold or used as a standalone product, the method may also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a device to perform all or part of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.
Correspondingly, an embodiment of the present invention provides an apparatus, including a memory and a processor, where the memory stores a computer program operable on the processor, and the processor implements the steps in the garbage cleaning method when executing the program.
In general, a device may be various types of devices having information processing capabilities in the course of implementation, and for example, the device may include a server or the like.
Correspondingly, the embodiment of the invention provides a computer-readable storage medium, on which a computer program is stored, and the computer program realizes the steps in the garbage cleaning method when being executed by a processor.
Here, it should be noted that: the above description of the storage medium and device embodiments is similar to the description of the method embodiments above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the storage medium and the apparatus according to the invention, reference is made to the description of the embodiments of the method according to the invention.
Based on the garbage cleaning apparatus 700 and the computer-readable storage medium, an apparatus according to an embodiment of the present invention is provided, fig. 9 is a schematic diagram of a component structure of the apparatus according to an embodiment of the present invention, and as shown in fig. 9, the apparatus 900 at least includes a processor 901, at least one communication bus 902, a user interface 903, at least one external communication interface 904, and a memory 905. Wherein the communication bus 902 is configured to enable connective communication between these components. The user interface 903 may include a display screen, and the external communication interface 904 may include a standard wired interface and a wireless interface, among others. The Memory 905 is configured to store instructions and applications executable by the processor 901, and may also cache data to be processed or already processed by the processor 901 and modules in the device 900, and may be implemented by a FLASH Memory (FLASH) or a Random Access Memory (RAM).
Wherein the processor 901, when running the computer program, is configured to perform:
acquiring a deletion request aiming at a target mirror image in a mirror image warehouse;
after the target mirror image is deleted, according to a reference counting strategy, the reference counting of the storage block referred by the target mirror image is decreased;
and after the reference count is decreased to zero, performing garbage cleaning once according to a preset garbage cleaning strategy, and deleting the memory block referenced by the target mirror image.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present invention, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention. The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.
Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a device to perform all or part of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.
The above description is only an embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present invention, and all such changes or substitutions are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

Claims (10)

1. A method of waste disposal, the method comprising:
acquiring a deletion request aiming at a target mirror image in a mirror image warehouse;
after the target mirror image is deleted, according to a reference counting strategy, the reference counting of the storage block referred by the target mirror image is decreased;
and after the reference count is decreased to zero, performing garbage cleaning once according to a preset garbage cleaning strategy, and deleting the memory block referenced by the target mirror image.
2. The method according to claim 1, wherein performing garbage cleaning once according to a preset garbage cleaning policy after the reference count is decremented to zero to delete the memory block referenced by the target image comprises:
if the time interval between the first moment and the previous garbage cleaning starting moment is equal to a preset first time interval, performing garbage cleaning once in a preset second time interval, and deleting the memory blocks quoted by the target mirror image, wherein the first moment comprises that the quote count is decreased to zero and the count of the requests of the mirror image warehouse is decreased to zero; the request for the mirror image warehouse comprises a deletion request, a storage block access request and a mirror image information modification request.
3. The method of claim 1, wherein after obtaining the delete request for the target image in the image repository, the method further comprises:
and analyzing the deletion request aiming at the target mirror image in the mirror image warehouse to acquire the mirror image name and the mirror image label information.
4. The method of claim 1, wherein the deleting the target image comprises:
and deleting the target mirror image according to the mirror image name and the mirror image label information.
5. The method of claim 1, wherein decrementing the reference count of the memory block referenced by the target image according to the reference count policy comprises:
the storage block referred by the target mirror image is referred by N mirror images, wherein the N mirror images comprise the target mirror image, and N is a positive integer greater than or equal to 1;
when any one of the N images is deleted, the reference count is decremented;
when all of the N images are deleted, the reference count is decremented to zero.
6. The method according to claim 2, wherein before performing garbage cleaning once within a preset second time interval and deleting the memory block referenced by the target image, the method further comprises:
and caching the request of the mirror image warehouse after the first moment.
7. The method of claim 6, wherein after caching the request for the mirror store after the first time, the method further comprises:
and releasing the cache of the request for the mirror image warehouse after the first moment when the preset second time interval is exceeded.
8. A dynamic trash removal device, the device comprising: agent control module, mirror image management module, quote count module and rubbish clearance control module, wherein:
the agent control module is used for acquiring a deletion request aiming at a target mirror image in the mirror image warehouse;
the mirror image management module is used for deleting the target mirror image;
the reference counting module is used for counting down the reference of the storage block referenced by the target mirror image according to a reference counting strategy;
and the garbage cleaning control module is used for performing garbage cleaning once according to a preset garbage cleaning strategy and deleting the memory block referred by the target mirror image after the reference count is decreased to zero.
9. An apparatus comprising a memory and a processor, the memory storing a computer program operable on the processor, wherein the processor implements the steps in the garbage cleaning method of any one of claims 1 to 7 when executing the program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the garbage disposal method of any one of claims 1 to 7.
CN201810996685.8A 2018-08-29 2018-08-29 Garbage cleaning method and device, equipment and storage medium Pending CN110874255A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810996685.8A CN110874255A (en) 2018-08-29 2018-08-29 Garbage cleaning method and device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810996685.8A CN110874255A (en) 2018-08-29 2018-08-29 Garbage cleaning method and device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN110874255A true CN110874255A (en) 2020-03-10

Family

ID=69714619

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810996685.8A Pending CN110874255A (en) 2018-08-29 2018-08-29 Garbage cleaning method and device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN110874255A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021242331A1 (en) * 2020-05-29 2021-12-02 Western Digital Technologies, Inc. Storage system, host, and method for optimizing storage of a sequence of images

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021242331A1 (en) * 2020-05-29 2021-12-02 Western Digital Technologies, Inc. Storage system, host, and method for optimizing storage of a sequence of images

Similar Documents

Publication Publication Date Title
US11461296B2 (en) Systems and methods for database management using append-only storage devices
CN102782670B (en) Memory cache data center
US20210342264A1 (en) Scalable garbage collection for deduplicated storage
US10872037B2 (en) Estimating worker nodes needed for performing garbage collection operations
CN1997015B (en) Cache application method and device, and file transfer system
US20120209814A1 (en) Processes and methods for client-side fingerprint caching to improve deduplication system backup performance
US7661036B1 (en) Cache for collecting events on a monitored computer
CN111198856B (en) File management method, device, computer equipment and storage medium
CN108197270B (en) Distributed file system data recovery method
JP2008546076A (en) Efficient handling of time-limited messages
JPH08115241A (en) Cache management method and computer system
CN106331148A (en) Cache management method and cache management device for data reading by clients
US11392490B2 (en) Marking impacted similarity groups in garbage collection operations in deduplicated storage systems
CN110737388A (en) Data pre-reading method, client, server and file system
US20200310965A1 (en) Deleting data in storage systems that perform garbage collection
CN106302638B (en) Data management method, forwarding equipment and system
CN110874255A (en) Garbage cleaning method and device, equipment and storage medium
CN108958660B (en) Distributed storage system and data processing method and device thereof
CN104079600A (en) File storage method, file storage device, file access client and metadata server system
JP2016500164A5 (en)
CN105205011B (en) A kind of method, normal client end and management client obtaining blocks of files reference count
CN108604231B (en) Mirror image processing method and computing device
CN109977074B (en) HDFS-based LOB data processing method and device
US8805942B2 (en) Storing and partitioning email messaging data
CN1741016A (en) Effective synchronization of smart card data and data in external stored server

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination