CN110442480B - Mirror image data cleaning method, device and system - Google Patents
Mirror image data cleaning method, device and system Download PDFInfo
- Publication number
- CN110442480B CN110442480B CN201910699440.3A CN201910699440A CN110442480B CN 110442480 B CN110442480 B CN 110442480B CN 201910699440 A CN201910699440 A CN 201910699440A CN 110442480 B CN110442480 B CN 110442480B
- Authority
- CN
- China
- Prior art keywords
- mirror image
- application
- name
- mirror
- version
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
- G06F11/1453—Management of the data involved in backup or backup restore using de-duplication of the data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1461—Backup scheduling policy
Abstract
The embodiment of the specification discloses a method, a device and a system for cleaning mirror image data, wherein the method comprises the steps of obtaining mirror image name data from a mirror image warehouse, wherein the mirror image name data comprises an application name and a label, and the label comprises application version information; sequencing the mirror image name data according to the application name and the application version information; screening out mirror image names of the mirror images to be cleaned from the sorted mirror image name data according to a mirror image cleaning rule; and sending the mirror name of the mirror image to be cleaned to a mirror image warehouse so that the mirror image warehouse can clean the mirror image according to the mirror name of the mirror image to be cleaned. By utilizing the embodiments of the specification, the unnecessary mirror image version data can be cleared in a more targeted manner, and the required mirror image version data can be accurately reserved.
Description
Technical Field
The present invention relates to the field of computer data processing technologies, and in particular, to a method, an apparatus, and a system for cleaning mirror image data.
Background
As the number of applications on the cloud continues to grow, the number of images in the image repository also continues to grow. But the number of valid images in the image warehouse is not large, which results in a great waste of storage resources of the image warehouse and also increases the time for troubleshooting problems.
At present, the mirror image cleaning usually adopts a mode of firstly sorting according to time, then reserving a fixed number of mirror images and cleaning other redundant mirror images. However, these methods of cleaning the image have a problem that the number of patches sent by each application is uneven, some patches are applied in several or even more than ten versions, and some patches are applied in one version without patches. If the parameter setting of the reserved image is smaller, the image of the previous version may not be reserved by the application with more patches, so that the last image of the previous version cannot be found when rollback is needed; if the parameter settings of the retained image are large, the cleaning effect is greatly reduced. Therefore, there is a need in the art for a method for cleaning mirror image data more effectively.
Disclosure of Invention
An object of the embodiments of the present specification is to provide a method, an apparatus, and a system for cleaning mirror image data, which can more specifically clean unwanted mirror image data and accurately retain a needed mirror image version.
The present specification provides a method, an apparatus and a system for cleaning mirror image data, which are implemented in the following ways:
a mirror image data cleaning method comprises the following steps:
acquiring mirror image name data from a mirror image warehouse, wherein the mirror image name data comprises an application name and a label, and the label comprises application version information;
sequencing the mirror image name data according to the application name and the application version information;
screening out mirror image names of the mirror images to be cleaned from the sorted mirror image name data according to a mirror image cleaning rule;
and sending the mirror name of the mirror image to be cleaned to a mirror image warehouse so that the mirror image warehouse can clean the mirror image according to the mirror name of the mirror image to be cleaned.
In another embodiment of the method provided in this specification, the obtaining mirror name data from a mirror repository includes:
receiving production information of a production application corresponding to a current production time node, wherein the production information comprises an application name of the production application;
and acquiring mirror image name data corresponding to the production application from a mirror image warehouse according to the application name.
In another embodiment of the method provided in this specification, the mirror name of the mirror to be cleaned is screened from the sorted mirror name data according to a mirror cleaning rule:
acquiring a mirror image cleaning rule corresponding to a target application, wherein the mirror image cleaning rule is determined according to application version information and/or version updating characteristics corresponding to the target application;
and screening out the mirror image name of the mirror image to be cleaned corresponding to the target application from the sorted mirror image name data corresponding to the target application according to the mirror image cleaning rule.
In another embodiment of the method provided in this specification, the tag includes a period version field, a patch version field, and a timestamp field, which are arranged in sequence;
the period version field is determined according to a preset updating period and the number of updated period versions in the preset updating period, the patch version field is determined according to the number of patch versions added to the period version, and the timestamp field is determined according to the updating time of the application.
In another aspect, an embodiment of the present specification further provides a mirror image data cleaning apparatus, where the apparatus includes:
the data acquisition module is used for acquiring mirror image name data from a mirror image warehouse, wherein the mirror image name data comprises an application name and a label, and the label comprises application version information;
the data sorting module is used for sorting the mirror image name data according to the application name and the application version information;
the data screening module is used for screening out mirror image names of the mirror images to be cleaned from the sorted mirror image name data according to the mirror image cleaning rule;
and the data sending module is used for sending the mirror name of the mirror image to be cleaned to the mirror image warehouse so as to enable the mirror image warehouse to carry out mirror image cleaning according to the mirror name of the mirror image to be cleaned.
In another embodiment of the apparatus provided in this specification, the apparatus further comprises a data receiving module, wherein,
the data receiving module is used for receiving production information of the production application corresponding to the current production time node, and the production information comprises an application name of the production application;
the data acquisition module is further used for acquiring mirror image name data corresponding to the production application from a mirror image warehouse according to the application name.
In another embodiment of the apparatus provided in this specification, the data filtering module includes:
the system comprises a rule determining unit, a judging unit and a judging unit, wherein the rule determining unit is used for acquiring a mirror image cleaning rule corresponding to a target application, and the mirror image cleaning rule is determined according to application version information and/or version updating characteristics corresponding to the target application;
and the data screening unit is used for screening the mirror image name of the mirror image to be cleaned corresponding to the target application from the sorted mirror image name data corresponding to the target application according to the mirror image cleaning rule.
In another embodiment of the apparatus provided in this specification, the tag includes a period version field, a patch version field, and a timestamp field, which are arranged in sequence;
the period version field is determined according to a preset updating period and the number of updated period versions in the preset updating period, the patch version field is determined according to the number of patch versions added to the period version, and the timestamp field is determined according to the updating time of the application.
In another aspect, an embodiment of the present specification further provides an image data cleaning device, including a processor and a memory for storing processor-executable instructions, where the instructions, when executed by the processor, implement the following steps:
acquiring mirror image name data from a mirror image warehouse, wherein the mirror image name data comprises an application name and a label, and the label comprises application version information;
sequencing the mirror image name data according to the application name and the application version information;
screening out mirror image names of the mirror images to be cleaned from the sorted mirror image name data according to a mirror image cleaning rule;
and sending the mirror name of the mirror image to be cleaned to a mirror image warehouse so that the mirror image warehouse can clean the mirror image according to the mirror name of the mirror image to be cleaned.
In another aspect, the present specification further provides an image data cleaning system, where the system includes at least one processor and a memory storing computer-executable instructions, and the processor executes the instructions to implement the steps of the method according to any one of the embodiments.
According to the method, the device and the system for cleaning the mirror image data, provided by one or more embodiments of the present specification, the application corresponding to each mirror image can be determined according to the application name in the mirror image name, and then, the version characteristics of the application stored in the mirror image repository, the patch version corresponding to each version, the version release time and the like can be determined according to the application version information in the mirror image name. Then, the image data of the application can be cleaned based on the version characteristics, so that the image version data which is not needed is cleaned in a more targeted manner, and the needed image version data is accurately reserved.
Drawings
In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort. In the drawings:
FIG. 1 is a schematic flowchart of an embodiment of a method for cleaning mirror image data provided in the present specification;
FIG. 2 is a flow chart illustrating a method for cleaning mirror image data according to an embodiment of the present disclosure;
FIG. 3 is a block diagram of an embodiment of a mirror image data cleaning apparatus provided in the present specification;
fig. 4 is a schematic block diagram of a server according to an exemplary embodiment of the present description.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in one or more embodiments of the present specification will be clearly and completely described below with reference to the drawings in one or more embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the specification, and not all embodiments. All other embodiments obtained by a person skilled in the art based on one or more embodiments of the present specification without making any creative effort shall fall within the protection scope of the embodiments of the present specification.
As the number of applications on the cloud continues to grow, the number of images in the image repository also continues to grow. But the number of valid images in the image warehouse is not large, which results in a great waste of storage resources of the image warehouse and also increases the time for troubleshooting problems. Accordingly, embodiments of the present specification provide a method for cleaning mirror image data, which may determine an application corresponding to each mirror image according to an application name in a mirror image name, and then determine version characteristics of the application stored in a mirror image repository, a patch version corresponding to each version, version release time, and the like according to application version information in the mirror image name. Then, the image data of the application can be cleaned based on the version characteristics, so that the image version data which is not needed is cleaned in a more targeted manner, and the needed image version data is accurately reserved.
Fig. 1 is a schematic flowchart of an embodiment of a method for cleaning mirror image data provided in this specification. Although the present specification provides the method steps or apparatus structures as shown in the following examples or figures, more or less steps or modules may be included in the method or apparatus structures based on conventional or non-inventive efforts. In the case of steps or structures which do not logically have the necessary cause and effect relationship, the execution order of the steps or the block structure of the apparatus is not limited to the execution order or the block structure shown in the embodiments or the drawings of the present specification. When the described method or module structure is applied to a device, a server or an end product in practice, the method or module structure according to the embodiment or the figures may be executed sequentially or in parallel (for example, in a parallel processor or multi-thread processing environment, or even in an implementation environment including distributed processing and server clustering).
In a specific embodiment, as shown in fig. 1, in an embodiment of the mirror data cleaning method provided in this specification, the method may include:
s0: mirror image name data is obtained from a mirror image warehouse, the mirror image name data comprises an application name and a label, and the label comprises application version information.
The mirror repository may be a database for storing mirrored data for one or more versions of an application. The application may refer to computer program data developed to run on top of an operating system in order to accomplish a particular task or tasks. After each update of the application, the application data corresponding to the updated version is uploaded to the mirror image warehouse for backup. The image data for each version of the application may be stored in the image repository in the form of image files, each image file corresponding to an image name. The mirror name is unique in the mirror repository, and if a mirror file with the same name is newly added, the previous mirror file is covered.
In some embodiments, the mirror name may include an application name and a tag, and the format may be, for example, image: tag, image is the application name, tag is the label. The application name may be used to characterize an application corresponding to the image file, and the tag may include application version information, which may be used to characterize version characteristics of the application, such as version characteristics corresponding to the application, patch version corresponding to each version, and version release time. The field characteristics of the tag used for characterizing the version information of the application can be determined according to the naming format of each application when generated.
Preferably, in an embodiment of the present specification, the tag may include a period version field, a patch version field, and a timestamp field, which are arranged in sequence. The period version field may be determined according to a preset update period and a number of period versions updated in the preset update period, where the preset update period may be, for example, a month, a quarter, or a year. The patch version field may include a patch version number determination for a corresponding periodic version increment. The timestamp field may be determined according to an update time of an application.
For example, the image name of an image file may be represented as: bj1/wapb-b 2-app: v1.84.01.0.19051915014957, where bj1/wapb-b2-app is the application name, V1 indicates the major version number, field "84.01" indicates the periodic version field, "0" indicates the patch version field, and "19051915014957" indicates the timestamp field. If the preset updating period is month, every month, the first bit of the period version field is added with 1, and the period version field is changed from '84' to '85'; the second bit of the periodic version field is the sequence number of the periodic version transmitted each month, the first version is '01', the second version is '02', and so on, and the calculation is restarted after changing one month. The patch version field is the serial number of the patch of each period version, the patch version field corresponding to the original period version is '0', when the first patch version of the period version is released, the patch version field corresponding to the first patch version is '1', and so on, and the calculation is restarted when one period version is changed. "19051915014957" is a timestamp field, determined by the specific update time of each version of the application.
The named format can be adopted when each version of application is generated, so that data query and operation processing are facilitated. Of course, when the application data of each version is backed up to the mirror image warehouse, the mirror image data corresponding to each application can be changed into the named format uniformly. The mirror image name of the mirror image is set by adopting the naming format, so that the pertinence and the accuracy of mirror image cleaning by using the application version information in the mirror image name can be further improved.
The mirror image name data of all mirror image files in the mirror image warehouse can be obtained, the mirror image name data of the target mirror image file can also be obtained, and the target mirror image file can be set according to the needs. In one embodiment of the present description, the mirror name data may be obtained from the mirror repository in the following manner:
receiving production information of a production application corresponding to a current production time node, wherein the production information comprises an application name of the production application;
and acquiring mirror image name data corresponding to the production application from a mirror image warehouse according to the application name.
The commissioning time node may include a point in time at which an updated version of the application is released into each platform to operate in place of the original version. The commissioning application may include generating version updates and publishing to the platforms to replace the application running in the original version. In some embodiments, the production time node data may be imported into the image cleaning device in advance, and then the image cleaning operation instruction may be triggered based on the production time node. For example, the mirror cleaning operation instruction can be triggered on the next day of the production time node to clean the data in the mirror warehouse.
After triggering the mirror image cleaning operation instruction, the mirror image cleaning device may send a query request to the mirror image warehouse, query mirror image data in the mirror image warehouse, and obtain mirror image name data of each mirror image file from the mirror image warehouse. After the updated version data of a part of newly added applications in the mirror image warehouse, part of old version data corresponding to the part of applications can be cleaned, and only effective version data is reserved so as to reduce the storage pressure of the mirror image warehouse. The application without version update usually does not need to perform mirror image data cleaning, so that the mirror image cleaning efficiency can be further improved by acquiring the application in each production and then performing mirror image cleaning on the batch of applications.
Of course, on the basis of performing mirror image cleaning based on the production time node, a cleaning cycle can be set to perform comprehensive cleaning on all mirror image data in the mirror image warehouse, so as to further realize cleaning of the mirror image data of the partially invalid application.
S2: and sequencing the mirror image name data according to the application name and the application version information.
The mirror name data may be sorted according to the application name and application version information in each mirror name data. Mirror name data of the same application name may be extracted and then sorted according to application version information. For example, the versions may be sorted in the order of their release:
mirror name example: bj1/wapb-b 2-app: v1.86.01.0.19051915014957
bj1/wapb-b2-app：V1.86.02.0.19051915314957
bj1/wapb-b2-app：V1.86.01.0.19041915014957
bj1/wapb-b2-app：V1.87.01.0.19051915014957
After sorting may be:
bj1/wapb-b2-app：V1.87.01.0.19051915014957
bj1/wapb-b2-app：V1.86.02.0.19051915314957
bj1/wapb-b2-app：V1.86.01.0.19051915014957
bj1/wapb-b2-app：V1.86.01.0.19041915014957
s4: and screening out the mirror image name of the mirror image to be cleaned from the sorted mirror image name data according to the mirror image cleaning rule.
The image cleaning rule may be preset according to actual needs, for example, only image data corresponding to the latest several periodic versions may be set, or, if there are more patch versions, only the latest N patch versions of the last periodic version and all versions corresponding to the periodic version may be reserved, or of course, only one or more specific versions may be reserved according to needs. In an actual application scenario, mirror names of the mirror images to be cleaned can be screened out from the sorted mirror name data according to a preset mirror image cleaning rule. The cleaning rules corresponding to each application name may be the same or different, and are not limited herein.
In an embodiment of the present specification, the mirror name of the mirror image to be cleaned may be further screened from the sorted mirror name data by the following method:
acquiring a mirror image cleaning rule corresponding to a target application, wherein the mirror image cleaning rule is determined according to application version information and/or version updating characteristics corresponding to the target application;
and screening out the mirror image name of the mirror image to be cleaned corresponding to the target application from the sorted mirror image name data corresponding to the target application according to the mirror image cleaning rule.
The version update characteristics of each application and/or the field data characteristics characterizing the version information, etc. may be analyzed in advance to determine the image cleaning rules of each application. For example, some applications may have a higher frequency of update cycles (e.g., once a month), while some applications may have longer update cycles (e.g., once a year), but may have more patch updates. The image cleaning rule of each application is determined by analyzing the version update and the version field characteristics of each application in advance, so that the image data can be cleaned in a more targeted manner. During actual mirror image cleaning, a mirror image cleaning rule corresponding to a target application can be obtained, and a mirror image name of a mirror image to be cleaned corresponding to the target application is screened out from sorted mirror image name data corresponding to the target application according to the mirror image cleaning rule.
S6: and sending the mirror name of the mirror image to be cleaned to a mirror image warehouse so that the mirror image warehouse can clean the mirror image according to the mirror name of the mirror image to be cleaned.
And the mirror image warehouse cleans the corresponding mirror image data in the warehouse according to the deletion request and the mirror image name of the mirror image to be cleaned.
Fig. 2 shows a flowchart of a mirror cleaning method provided in the present specification. As shown in fig. 2, in an application scenario of the present specification, the mirror cleaning may include:
s101, leading the production time points into a mirror image cleaning system by each node;
s102: triggering a mirror image cleaning instruction on the next day of the production time point;
s103: the mirror image cleaning system sends a query request to a mirror image warehouse to obtain mirror image name data of a production mirror image;
s104: sequencing the mirror image name data according to the application name and the application version information in the mirror image name;
s105: screening out a mirror name list needing to be cleaned;
s106: sending a deletion request and a mirror name list to a mirror repository;
s107: and the mirror image warehouse deletes the mirror image data according to the deletion request and the mirror image name list.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. For details, reference may be made to the description of the related embodiments of the related processing, and details are not repeated herein.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
In the method for cleaning mirror image data provided in one or more embodiments of the present specification, an application corresponding to each mirror image may be determined according to an application name in a mirror image name, and then version characteristics of the application stored in a mirror image repository, a patch version corresponding to each version, version release time, and the like may be determined according to application version information in the mirror image name. Then, the image data of the application can be cleaned based on the version characteristics, so that the image version data which is not needed is cleaned in a more targeted manner, and the needed image version data is accurately reserved.
Based on the mirror image data cleaning method, one or more embodiments of the present specification further provide a mirror image data cleaning device. The apparatus may include systems, software (applications), modules, components, servers, etc. that utilize the methods described in the embodiments of the present specification in conjunction with hardware implementations as necessary. Based on the same innovative conception, embodiments of the present specification provide an apparatus as described in the following embodiments. Since the implementation scheme of the apparatus for solving the problem is similar to that of the method, the specific implementation of the apparatus in the embodiment of the present specification may refer to the implementation of the foregoing method, and repeated details are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated. Specifically, fig. 3 shows a schematic block structure diagram of an embodiment of a mirror image data cleaning apparatus provided in the specification, and as shown in fig. 3, the apparatus may include:
a data obtaining module 802, configured to obtain mirror name data from a mirror repository, where the mirror name data includes an application name and a tag, and the tag includes application version information;
a data sorting module 804, configured to sort the mirror name data according to the application name and the application version information;
the data screening module 806 may be configured to screen mirror names of the mirrors to be cleaned from the sorted mirror name data according to a mirror cleaning rule;
the data sending module 808 may be configured to send the mirror name of the mirror to be cleaned to a mirror warehouse, so that the mirror warehouse performs mirror cleaning according to the mirror name of the mirror to be cleaned.
In another embodiment of the present specification, the apparatus further comprises a data receiving module, wherein,
the data receiving module can be used for receiving production information of a production application corresponding to a current production time node, wherein the production information comprises an application name of the production application;
the data acquisition module can be further used for acquiring mirror name data corresponding to the production application from a mirror repository according to the application name.
In another embodiment of the present disclosure, the data filtering module 806 may include:
the system comprises a rule determining unit, a processing unit and a processing unit, wherein the rule determining unit can be used for acquiring a mirror image cleaning rule corresponding to a target application, and the mirror image cleaning rule is determined according to application version information and/or version updating characteristics corresponding to the target application;
and the data screening unit can be used for screening the mirror image name of the mirror image to be cleaned corresponding to the target application from the sorted mirror image name data corresponding to the target application according to the mirror image cleaning rule.
In another embodiment of the present specification, the tag may include a period version field, a patch version field, and a timestamp field, which are arranged in sequence; the period version field may be determined according to a preset update period and the number of period versions updated in the preset update period, the patch version field may be determined according to the number of patch versions added to the period version, and the timestamp field may be determined according to the update time of the application.
It should be noted that the above-described apparatus may also include other embodiments according to the description of the method embodiment. The specific implementation manner may refer to the description of the related method embodiment, and is not described in detail herein.
The image data cleaning apparatus provided in one or more embodiments of the present specification may determine, according to an application name in an image name, an application corresponding to each image, and then determine, according to application version information in the image name, version characteristics such as a version of the application stored in an image repository, a patch version corresponding to each version, and version release time. Then, the image data of the application can be cleaned based on the version characteristics, so that the image version data which is not needed is cleaned in a more targeted manner, and the needed image version data is accurately reserved.
The method or apparatus provided by the present specification and described in the foregoing embodiments may implement service logic through a computer program and record the service logic on a storage medium, where the storage medium may be read and executed by a computer, so as to implement the effect of the solution described in the embodiments of the present specification. Accordingly, the present specification also provides an image data cleansing apparatus comprising a processor and a memory storing processor-executable instructions which, when executed by the processor, implement steps comprising:
acquiring mirror image name data from a mirror image warehouse, wherein the mirror image name data comprises an application name and a label, and the label comprises application version information;
sequencing the mirror image name data according to the application name and the application version information;
screening out mirror image names of the mirror images to be cleaned from the sorted mirror image name data according to a mirror image cleaning rule;
and sending the mirror name of the mirror image to be cleaned to a mirror image warehouse so that the mirror image warehouse can clean the mirror image according to the mirror name of the mirror image to be cleaned.
The method embodiments provided by the embodiments of the present specification can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking the example of running on a server, fig. 4 is a hardware structure block diagram of a mirror data cleaning server to which the embodiments of the present specification are applied. As shown in fig. 4, the server 10 may include one or more (only one shown) processors 20 (the processors 20 may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 30 for storing data, and a transmission module 40 for communication functions. It will be understood by those skilled in the art that the structure shown in fig. 4 is merely an illustration and is not intended to limit the structure of the electronic device. For example, the server 10 may also include more or fewer components than shown in FIG. 4, and may also include other processing hardware, such as a database or multi-level cache, a GPU, or have a different configuration than shown in FIG. 4, for example.
The memory 30 may be used to store software programs and modules of application software, such as program instructions/modules corresponding to the search method in the embodiment of the present invention, and the processor 20 executes various functional applications and data processing by executing the software programs and modules stored in the memory 30. The memory 30 may include high speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 30 may further include memory located remotely from the processor 20, which may be connected to a computer terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission module 40 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal. In one example, the transmission module 40 includes a Network adapter (NIC) that can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission module 40 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
The storage medium may include a physical device for storing information, and typically, the information is digitized and then stored using an electrical, magnetic, or optical media. The storage medium may include: devices that store information using electrical energy, such as various types of memory, e.g., RAM, ROM, etc.; devices that store information using magnetic energy, such as hard disks, floppy disks, tapes, core memories, bubble memories, and usb disks; devices that store information optically, such as CDs or DVDs. Of course, there are other ways of storing media that can be read, such as quantum memory, graphene memory, and so forth.
It should be noted that the above description of the apparatus according to the method embodiment may also include other embodiments. The specific implementation manner may refer to the description of the related method embodiment, and is not described in detail herein.
The image data cleaning device in the foregoing embodiment may determine, according to the application name in the image name, the application corresponding to each image, and then may determine, according to the application version information in the image name, version characteristics such as the version of the application stored in the image repository, the patch version corresponding to each version, and the version release time. Then, the image data of the application can be cleaned based on the version characteristics, so that the image version data which is not needed is cleaned in a more targeted manner, and the needed image version data is accurately reserved.
The present specification also provides a mirror image data cleaning system, which may be a single mirror image data cleaning system, or may be applied to a variety of computer data processing systems. The system may be a single server, or may include a server cluster, a system (including a distributed system), software (applications), an actual operating device, a logic gate device, a quantum computer, etc. using one or more of the methods or one or more of the example devices of the present specification, in combination with a terminal device implementing hardware as necessary. The mirrored data cleaning system may include at least one processor and memory storing computer-executable instructions that when executed by the processor implement the steps of the method described in any one or more of the embodiments described above.
It should be noted that the above-mentioned system may also include other implementation manners according to the description of the method or apparatus embodiment, and specific implementation manners may refer to the description of the related method embodiment, which is not described in detail herein.
The image data cleaning system according to the above embodiment may determine the application corresponding to each image according to the application name in the image name, and then determine the version of the application stored in the image repository, the patch version corresponding to each version, the version release time, and other version characteristics according to the application version information in the image name. Then, the image data of the application can be cleaned based on the version characteristics, so that the image version data which is not needed is cleaned in a more targeted manner, and the needed image version data is accurately reserved.
It should be noted that, the above-mentioned apparatus or system in this specification may also include other implementation manners according to the description of the related method embodiment, and a specific implementation manner may refer to the description of the method embodiment, which is not described herein in detail. The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the hardware + program class, storage medium + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant points, refer to the partial description of the method embodiment.
The embodiments of the present description are not limited to what must be consistent with a standard data model/template or described in the embodiments of the present description. Certain industry standards, or implementations modified slightly from those described using custom modes or examples, may also achieve the same, equivalent, or similar, or other, contemplated implementations of the above-described examples. The embodiments using these modified or transformed data acquisition, storage, judgment, processing, etc. may still fall within the scope of the alternative embodiments of the present description.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, when implementing one or more of the present description, the functions of each module may be implemented in one or more software and/or hardware, or a module implementing the same function may be implemented by a combination of multiple sub-modules or sub-units, etc. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may therefore be considered as a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method or apparatus that comprises the element.
As will be appreciated by one skilled in the art, one or more embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
One or more embodiments of the present description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. One or more embodiments of the present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment. In the description of the specification, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the specification. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
The above description is only an example of the present specification, and is not intended to limit the present specification. Various modifications and alterations to this description will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the scope of the claims of the present specification.
Claims (8)
1. A method for cleaning mirror image data is characterized by comprising the following steps:
triggering a mirror image cleaning operation instruction based on a pre-configured production time node; the production time node refers to a time point when the updated version of the application is released to the platform to replace the original version to run;
after a mirror image cleaning operation instruction is triggered, receiving the production information of the production application under the current production time node corresponding to the mirror image cleaning operation instruction, wherein the production information comprises the application name of the production application;
acquiring mirror image name data corresponding to the production application from a mirror image warehouse according to the application name, wherein the mirror image name data comprises an application name and a label, and the label comprises application version information;
sequencing the mirror image name data according to the application name and the application version information;
screening out mirror image names of the mirror images to be cleaned from the sorted mirror image name data according to a mirror image cleaning rule;
and sending the mirror name of the mirror image to be cleaned to a mirror image warehouse so that the mirror image warehouse can clean the mirror image according to the mirror name of the mirror image to be cleaned.
2. The method according to claim 1, wherein the mirror name of the mirror to be cleaned is screened out from the sorted mirror name data according to a mirror cleaning rule:
acquiring a mirror image cleaning rule corresponding to a target application, wherein the mirror image cleaning rule is determined according to application version information and/or version updating characteristics corresponding to the target application;
and screening out the mirror image name of the mirror image to be cleaned corresponding to the target application from the sorted mirror image name data corresponding to the target application according to the mirror image cleaning rule.
3. The method of claim 1, wherein the tag comprises a periodic version field, a patch version field, and a timestamp field arranged in sequence;
the period version field is determined according to a preset updating period and the number of updated period versions in the preset updating period, the patch version field is determined according to the number of patch versions added to the period version, and the timestamp field is determined according to the updating time of the application.
4. A mirrored data cleansing apparatus, the apparatus comprising:
the data acquisition module is used for triggering a mirror image cleaning operation instruction based on a pre-configured production time node; the production time node refers to a time point when the updated version of the application is released to the platform to replace the original version to run; after a mirror image cleaning operation instruction is triggered, receiving the production information of the production application under the current production time node corresponding to the mirror image cleaning operation instruction, wherein the production information comprises the application name of the production application; acquiring mirror image name data corresponding to the production application from a mirror image warehouse according to the application name, wherein the mirror image name data comprises an application name and a label, and the label comprises application version information;
the data sorting module is used for sorting the mirror image name data according to the application name and the application version information;
the data screening module is used for screening out mirror image names of the mirror images to be cleaned from the sorted mirror image name data according to the mirror image cleaning rule;
and the data sending module is used for sending the mirror name of the mirror image to be cleaned to the mirror image warehouse so as to enable the mirror image warehouse to carry out mirror image cleaning according to the mirror name of the mirror image to be cleaned.
5. The apparatus of claim 4, wherein the data filtering module comprises:
the system comprises a rule determining unit, a judging unit and a judging unit, wherein the rule determining unit is used for acquiring a mirror image cleaning rule corresponding to a target application, and the mirror image cleaning rule is determined according to application version information and/or version updating characteristics corresponding to the target application;
and the data screening unit is used for screening the mirror image name of the mirror image to be cleaned corresponding to the target application from the sorted mirror image name data corresponding to the target application according to the mirror image cleaning rule.
6. The apparatus of claim 4, wherein the tag comprises a periodic version field, a patch version field, and a timestamp field arranged in sequence;
the period version field is determined according to a preset updating period and the number of updated period versions in the preset updating period, the patch version field is determined according to the number of patch versions added to the period version, and the timestamp field is determined according to the updating time of the application.
7. An image data cleaning device, comprising a processor and a memory for storing processor-executable instructions, which when executed by the processor, implement steps comprising:
triggering a mirror image cleaning operation instruction based on a pre-configured production time node; the production time node refers to a time point when the updated version of the application is released to the platform to replace the original version to run;
after a mirror image cleaning operation instruction is triggered, receiving the production information of the production application under the current production time node corresponding to the mirror image cleaning operation instruction, wherein the production information comprises the application name of the production application;
acquiring mirror image name data corresponding to the production application from a mirror image warehouse according to the application name, wherein the mirror image name data comprises an application name and a label, and the label comprises application version information;
sequencing the mirror image name data according to the application name and the application version information;
screening out mirror image names of the mirror images to be cleaned from the sorted mirror image name data according to a mirror image cleaning rule;
and sending the mirror name of the mirror image to be cleaned to a mirror image warehouse so that the mirror image warehouse can clean the mirror image according to the mirror name of the mirror image to be cleaned.
8. A mirrored data cleansing system comprising at least one processor and memory storing computer-executable instructions which, when executed by the processor, implement the steps of the method of any of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910699440.3A CN110442480B (en) | 2019-07-31 | 2019-07-31 | Mirror image data cleaning method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910699440.3A CN110442480B (en) | 2019-07-31 | 2019-07-31 | Mirror image data cleaning method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110442480A CN110442480A (en) | 2019-11-12 |
| CN110442480B true CN110442480B (en) | 2022-04-15 |
Family
ID=68432547
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910699440.3A Active CN110442480B (en) | 2019-07-31 | 2019-07-31 | Mirror image data cleaning method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110442480B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111291017B (en) * | 2020-03-03 | 2024-04-05 | 中国工商银行股份有限公司 | Mirror image storage and extraction method and device of mirror image warehouse |
| CN112433964B (en) * | 2020-04-17 | 2022-06-14 | 上海哔哩哔哩科技有限公司 | Method and equipment for cleaning cache dirty data |
| CN111752676B (en) * | 2020-05-28 | 2022-06-21 | 苏州浪潮智能科技有限公司 | Docker-based homonymous mirror image removing method, system, terminal and storage medium |
| CN116842001A (en) * | 2023-08-29 | 2023-10-03 | 合肥中科类脑智能技术有限公司 | Mirror image data cleaning method and device, electronic equipment and artificial intelligent platform |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107066296A (en) * | 2017-03-31 | 2017-08-18 | 北京奇艺世纪科技有限公司 | The method for cleaning and device of mirror image in a kind of clustered node |
| CN107105054A (en) * | 2017-05-17 | 2017-08-29 | 郑州云海信息技术有限公司 | A kind of mirror image garbage-cleaning system and method towards docker mirror images warehouse |
| CN109343859A (en) * | 2018-08-17 | 2019-02-15 | 咪咕文化科技有限公司 | A kind of information processing method, device and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8606765B2 (en) * | 2007-11-30 | 2013-12-10 | Red Hat, Inc. | Systems and methods for updating software appliances |
| CN106484906B (en) * | 2016-10-21 | 2020-01-10 | 焦点科技股份有限公司 | Distributed object storage system flash-back method and device |
-
2019
- 2019-07-31 CN CN201910699440.3A patent/CN110442480B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107066296A (en) * | 2017-03-31 | 2017-08-18 | 北京奇艺世纪科技有限公司 | The method for cleaning and device of mirror image in a kind of clustered node |
| CN107105054A (en) * | 2017-05-17 | 2017-08-29 | 郑州云海信息技术有限公司 | A kind of mirror image garbage-cleaning system and method towards docker mirror images warehouse |
| CN109343859A (en) * | 2018-08-17 | 2019-02-15 | 咪咕文化科技有限公司 | A kind of information processing method, device and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| delete-docker-registry-image;kevin burnettk等;《https://github.com/burnettk/delete-docker-registry-image》;20171225;第1-21页 * |
| Docker registry仓库历史镜像批量清理;ywq935;《https://blog.csdn.net/ywq935/article/details/83828888》;20181107;第1-4页 * |
| Harbor镜像清理;breezey;《https://www.cnblogs.com/breezey/p/10615231.html》;20190328;第1-3页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110442480A (en) | 2019-11-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110442480B (en) | Mirror image data cleaning method, device and system | |
| CN110597616B (en) | Memory allocation method and device for neural network | |
| CN107870845B (en) | Management method and system for micro-service architecture application | |
| US7117294B1 (en) | Method and system for archiving and compacting data in a data storage array | |
| CN110471746B (en) | Distributed transaction callback method, device and system | |
| CN108196792B (en) | Method and device for clearing expired data | |
| CN106557307B (en) | Service data processing method and system | |
| CN106682017B (en) | Database updating method and device | |
| WO2021257263A1 (en) | Techniques for generating a consistent view of an eventually consistent database | |
| CN112379906A (en) | Service updating method, device, storage medium and electronic device | |
| CN112003821A (en) | Cloud platform security management method and system and security management server | |
| CN113961228A (en) | Change reminding method, device, equipment and storage medium | |
| CN105488165A (en) | Data retrieval method and system based on index database | |
| CN109684331A (en) | A kind of object storage meta data management device and method based on Kudu | |
| CN111046003B (en) | Soil erosion factor data updating method and device and electronic equipment | |
| CN110659125A (en) | Analysis task execution method, device and system and electronic equipment | |
| CN114089962A (en) | Component management method, device and medium | |
| CN110362575B (en) | Method and device for generating global index of data | |
| US20210042271A1 (en) | Distributed garbage collection for dedupe file system in cloud storage bucket | |
| CN112905386A (en) | Table data backup cleaning method and device based on life cycle | |
| CN112905306A (en) | Multi-cluster container management method and device, electronic equipment and storage medium | |
| WO2016176045A1 (en) | System, method, and apparatus for updating data in a distributed storage system | |
| CN117609257A (en) | Database data synchronous updating method, device, equipment and storage medium | |
| CN115544626B (en) | Sub-model extraction method, device, computer equipment and medium | |
| CN115687333B (en) | V2x big data life cycle management method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |