CN113934510A - Mirror image processing method and device, electronic equipment and computer readable storage medium - Google Patents
Mirror image processing method and device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN113934510A CN113934510A CN202111211973.6A CN202111211973A CN113934510A CN 113934510 A CN113934510 A CN 113934510A CN 202111211973 A CN202111211973 A CN 202111211973A CN 113934510 A CN113934510 A CN 113934510A
- Authority
- CN
- China
- Prior art keywords
- data file
- processed
- mirror image
- public
- mirror
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
According to the mirror image processing method, the mirror image processing device, the electronic equipment and the computer readable storage medium, the compression of mirror image data can be realized, and the storage space occupied by the mirror image is reduced. Specifically, in the process of processing the mirror image data, the method firstly determines a plurality of public data files corresponding to the mirror images to be processed, so as to determine a public processing layer according to the public data files, and the purpose of merging the public data files is realized, thereby achieving the purpose of compressing the mirror image data. When reconstruction is carried out, reconstruction of the mirror image is realized on the basis of reserving other data files except the public data file in each mirror image to be processed, so that the mirror image can share the same public mirror image layer with the mirror images of other applications on the server, and the storage space is saved by combining the public data files to avoid file redundancy.
Description
Technical Field
The invention relates to the technical field of containers, in particular to a mirror image processing method and device, electronic equipment and a computer readable storage medium.
Background
At present, with the development of internet technology and the improvement of application requirements, the requirements on the aspects of improving the utilization rate of network resources and storage resources, managing resources and the like are higher and higher. With this demand, Container technology (Docker, Container) has evolved and developed rapidly.
The container is a type of operating system virtualization, is created based on an image, is an instantiation of the image, and in an actual project deployment process, a plurality of images and containers are generally formed. The images may have the same data file, but in the process of constructing the images, each image is constructed separately, which causes the same data file to be reused, occupies the storage resources of the container warehouse, causes the storage cost to be increased, and increases the time consumption of service deployment and migration.
Disclosure of Invention
An object of the present invention is to provide a method and an apparatus for processing an image, an electronic device, and a computer-readable storage medium, which can save storage space and reduce storage cost and time consumption for deployment and migration.
Embodiments of the invention may be implemented as follows:
in a first aspect, the present invention provides a mirror image processing method, including: determining a public data file corresponding to a plurality of mirror images to be processed; the public data file is at least one of a plurality of data files in each mirror image to be processed; each data file corresponds to an application program; determining a public mirror image layer according to the public data file; and reconstructing the plurality of images to be processed according to the common image layer and other data files except the common data file in each image to be processed.
In a second aspect, the present invention provides an image processing apparatus, including: the determining module is used for determining public data files corresponding to the plurality of to-be-processed mirror images; the public data file is at least one of a plurality of data files to be installed of each to-be-processed mirror image; each data file corresponds to an application program; determining a public mirror image layer according to the public data file; and the construction module is used for reconstructing the plurality of images to be processed according to the public image layer and other data files except the public data file in each image to be processed.
In a third aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the mirroring processing method of the first aspect.
In a fourth aspect, the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the mirror processing method of the first aspect.
According to the mirror image processing method, the mirror image processing device, the electronic equipment and the computer readable storage medium, the compression of mirror image data can be realized, and the storage space occupied by mirror images is reduced. Specifically, the method comprises the following steps: determining a public data file corresponding to a plurality of mirror images to be processed; the public data file is at least one of a plurality of data files in each mirror image to be processed; each data file corresponds to an application program; determining a public mirror image layer according to the public data file; and reconstructing the plurality of images to be processed according to the common image layer and other data files except the common data file in each image to be processed. In the process of processing the mirror image data, the public data files corresponding to the plurality of mirror images to be processed are determined at first, so that the public mirror image layer is determined according to the public data files, the public data files are merged, and the purpose of compressing the mirror image data is achieved. When reconstruction is carried out, reconstruction of the mirror image is realized on the basis of reserving other data files except the public data file in each mirror image to be processed, so that the mirror image can share the same public mirror image layer with the mirror images of other applications on the server, and the storage space is saved by combining the public data files to avoid file redundancy.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a diagram of an exemplary configuration file according to an embodiment of the present invention;
fig. 2 is an application environment of the image processing method according to the embodiment of the present invention;
FIG. 3 is a schematic flow chart of a mirror processing method according to an embodiment of the present invention;
fig. 4 is a schematic flowchart of an implementation manner of step S301 provided in the embodiment of the present invention;
FIG. 5 is a schematic flow chart of an implementation of step S301-3 provided by an embodiment of the present invention;
fig. 6 is a schematic flowchart of an implementation manner of step S303 provided by the embodiment of the present invention;
FIG. 7 is a directed graph according to an embodiment of the present invention;
FIG. 8 is a functional block diagram of an image processing apparatus according to an embodiment of the present invention;
fig. 9 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. indicate an orientation or a positional relationship based on that shown in the drawings or that the product of the present invention is used as it is, this is only for convenience of description and simplification of the description, and it does not indicate or imply that the device or the element referred to must have a specific orientation, be constructed in a specific orientation, and be operated, and thus should not be construed as limiting the present invention.
Furthermore, the appearances of the terms "first," "second," and the like, if any, are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
It should be noted that the features of the embodiments of the present invention may be combined with each other without conflict.
The terms referred to in the examples of the present application are explained as follows:
mirroring: the file storage method is a file storage form, combines a plurality of files into one configuration file, and is convenient to distribute and use. The mirror image adopts a mirror image layered storage mode, and one mirror image can be composed of a plurality of mirror image layers. For example: an image may comprise a complete operating system environment. The mirror image may be used to create a container. Containers are runtime instances of mirror creation that can be started, stopped, deleted, each container being a separate and secure platform.
Configuration file (Dockerfile): is a text file used for constructing an image, and the text content comprises a piece of instruction and description required for constructing the image. The method comprises the steps that contents needed for constructing the mirror image are filled in a configuration file in a command mode, when the container mirror image is constructed by utilizing a container management tool (such as a docker command in a Linux operating system), the container management tool reads commands line by line, a single mirror image layer is created for each line of commands with characteristic characters (such as RUN), and after all the commands are executed, the container mirror image is generated.
For ease of understanding, the configuration file is described below in conjunction with FIG. 1. Fig. 1 shows an example of configuration files of two images, taking image 1 as an example, where "RUN" refers to a command executed when an image is constructed, and a command line includes data files (i.e. data packets) of an application: wget, curl, net-tools, vim, python, and opencv.
Data file: refers to a data packet corresponding to an application program, and may be, but is not limited to, a game application, social software, an audio-video application, office software, and the like.
A common mirror layer: the method refers to that a plurality of common data files are combined together to establish a mirror image layer, and different mirror images can share one common mirror image layer.
The application embodiment of the present application does not limit the applicable scenario of the image processing method in the embodiment of the present application, and may be applicable to any scenario in which the container image of the application needs to be compressed.
Referring to fig. 2, fig. 2 is an application environment of an image processing method according to an embodiment of the present invention, wherein the application environment includes a first server 210, a network 230, and a second server 220.
The first server 210 and the second server 220 may be independent physical servers, or may be a server cluster or a distributed system formed by a plurality of physical servers, and the network 230 may include but is not limited to: a wired network, a wireless network, wherein the wired network comprises: a local area network, a metropolitan area network, and a wide area network, the wireless network comprising: bluetooth, Wi-Fi, and other networks that enable wireless communication.
The mirror image processing method in the embodiment of the present invention may be applied to the first server 210, and the first server 210 compresses multiple container mirror images in the system by reconstructing multiple mirror images to be processed, thereby reducing the storage and time overhead of the system during the deployment, capacity expansion and migration processes, and improving the efficiency.
Continuing with fig. 1 and fig. 2 as an example, in the process of constructing the mirror image 1 and the mirror image 2 in the prior art, for the mirror image 1, 6 data files of wget, curl, net-tools, vim, python, and opencv are constructed in the same mirror image layer, and similarly, when constructing the mirror image 2, wget, curl, net-tools, vim, and node are constructed in the same mirror image layer. Since the data files are cached when the image is constructed, it can be seen that the same data files in the images 1 and 2 are repeatedly cached in the first server 210, that is, "wget, curl, net-tools, vim" causes the storage cost to increase.
Further, the above-mentioned construction method may also increase the storage and time overhead of the system during the deployment, capacity expansion and migration processes, for example, generally, a service system includes a plurality of micro services, each micro service implements a relatively independent function, and a container arrangement tool (e.g., kubernets) is responsible for the arrangement task of the container and the container start and capacity expansion of each service. If the service system needs to be migrated to another environment, images corresponding to the micro services are usually exported and then image files are imported, however, the images may have potentially the same data files, so that the volume of the deployment data package is increased in the migration process, and the deployment and migration efficiency is reduced.
In order to solve the above problem, an embodiment of the present invention provides a mirror processing method, where the mirror processing method may be applied to the first server 210 shown in fig. 2, please refer to fig. 3, and fig. 3 is a schematic flowchart of the mirror processing method provided in the embodiment of the present invention, and the method may include:
s301, determining public data files corresponding to the multiple to-be-processed mirror images.
The common data file is at least one of a plurality of data files in each mirror image to be processed; each data file corresponds to an application.
For example, with continued reference to FIG. 1, the common data file may be the data files "wget, curl, net-tools, vim" of the application program that both image 1 and image 2 have.
And S302, determining a public mirror image layer according to the public data file.
S303, reconstructing a plurality of images to be processed according to the common image layer and other data files except the common data file in each image to be processed.
It can be understood that the public data files are constructed on the same mirror image layer, the public data files only need to be cached once when the to-be-processed mirror images are reconstructed, and the public data files do not need to be cached every time one to-be-processed mirror image is constructed, so that the phenomena of file redundancy and layer redundancy can be prevented, and the sizes of a plurality of to-be-processed mirror images in a warehouse can be greatly reduced.
The mirror image processing method provided by the embodiment can realize compression of mirror image data and reduce storage space occupied by mirror images. Specifically, in the process of processing the mirror image data, the method firstly determines the public data files corresponding to a plurality of mirror images to be processed, so that a public processing layer is constructed according to the public data files, the public data files are merged, and the purpose of compressing the mirror image data is achieved. When reconstruction is carried out, reconstruction of the mirror image is realized on the basis of reserving other data files except the public data file in each mirror image to be processed, so that the mirror image can share the same public mirror image layer with the mirror images of other applications on the server, and the storage space is saved by combining the public data files to avoid file redundancy.
Optionally, an implementation manner of determining a common data file corresponding to a plurality of images to be processed is further provided below, please refer to fig. 4, where fig. 4 is a schematic flowchart of an implementation manner of step S301 provided in an embodiment of the present invention, where step S301 may include:
s301-1, acquiring identification information of each mirror image to be processed.
In this embodiment, the identification information may be a unique identifier of the image to be processed, and may be input to the server by a user, or may be read from the memory after the server is started, which is not limited herein.
S301-2, obtaining a configuration file corresponding to each mirror image to be processed according to the identification information.
It can be understood that, after an image is successfully constructed, the server generates a configuration file corresponding to the image, where the configuration file is shown in fig. 1 and is not described herein again.
In one scenario, when a configuration file exists in the server, the configuration file can be directly read according to the identification information.
In another scenario, when the configuration file does not exist in the server and the history of the mirror image construction exists, the construction record information corresponding to the identification information can be directly read according to each identification information through the identification information, and the construction record information is converted into the configuration file.
S301-3, determining a public data file according to a target command field for constructing the mirror image to be processed in each configuration file.
In this embodiment, the target command field may be an apt-get (apt) instruction in the RUN instruction shown in fig. 1, or may be an ADD command or a COPY command in the case of no directory conflict.
In the following, a possible implementation manner is given for how to determine the public data file in step S301-3, please refer to fig. 5, where fig. 5 is a schematic flowchart of an implementation manner of step S301-3 provided in an embodiment of the present invention, where step S301-3 may include:
s301-3-1, analyzing the target command field in each configuration file to obtain a data file set corresponding to each mirror image to be processed.
In this embodiment, a target command field in each configuration file is analyzed, and actually the target command field in the configuration file needs to be determined first, in one scenario, when a command field consistent with a predefined command field exists in the configuration file, the command field is determined as the target command field, for example, an apt-get (apt) install command in an RUN statement is the target command field, in another scenario, a determination rule may be further predefined to determine the target command field, for example, an ADD command and a COPY command under the condition that directories do not conflict may both be used as the target command field.
S301-3-2, taking the union of all data file sets as a target data file set.
S301-3-3, determining a public data file according to the target data file set.
It can be understood that, by analyzing the configuration file of each mirror image to be processed, a data file set Si can be obtained, and the target data file set Sa is obtained by summing these Si, so that the public data file can be determined according to the target data file set Sa.
For the convenience of understanding the above steps, the following description will be made by taking the mirror image 1 in fig. 1 as an example.
And (3) splitting the RUN statement aiming at the Dockerfile of the mirror image 1, and respectively analyzing each command and parameter in the statement. If a RUN statement consists of multiple commands, the commands are parsed in sequence and concatenated in sequence, preferably represented by an array. Each mirror image corresponds to an array, and then the installation package of apt-get install is split one by one to split the Dockerfile of mirror image 1 to form the arrays connected in series:
["FROM ubuntu:16.04","RUN apt-get install wget","RUN apt-get install curl","RUN apt-get install net-tools","RUN apt-get install vim","RUN apt-get install python","RUN apt-get install opencv",...,"CMD['/usr/bin/python','xxx.py']"]
because apt-get install is a swap command, these commands are abstracted as elements of a collection. For simplicity of illustration, some characters of the command are omitted here, and then the Dockerfile of image 1 corresponds to a set of data files that are:
S1＝{"wget","curl","net-tools","vim","python","opencv"}。
similarly, the data file set corresponding to the Dockerfile of the mirror image 2 is as follows:
S2＝{"wget","curl","net-tools","vim","node"}。
further, the union of S1 and S2 may determine Sa as:
Sa＝{"wget","curl","net-tools","vim","python","opencv","node"}。
further, the public data file may be determined according to Sa, and one possible implementation is given below, that is, the step S301-3-3 may include the following steps:
the first step is to determine all non-vacuum true subsets corresponding to the target data file set.
And secondly, screening out a target non-vacuum subset meeting a first preset condition and a second preset condition according to all the non-vacuum subsets.
Wherein the first preset condition is as follows: the intersection of the target non-vacuum true subset and any one data file set is a target non-vacuum true subset or a vacuum set; the second preset condition is as follows: and the intersection of the difference set between the target non-aerial-true subset and the target data file set and the target non-aerial-true subset does not meet the first preset condition.
And thirdly, determining all data files in the target non-aerial-truth subset with the largest number of data files as common data files.
That is, for Sa, all its non-vacuity proper subsets a ═ Aj are obtained, the set a is traversed, and the set Aj satisfying the first preset condition and the second preset condition at the same time is screened.
The first preset condition is as follows: aj satisfies, with any Si: aj & Si ═ a or Aj & Si ═ null. Where & represents the intersection operation of the set.
The second preset condition is as follows: aj is the maximum set, that is, Aj and any element in the difference set Ap between Sa and Aj do not satisfy the condition 1.
For ease of understanding, the explanation continues with the example of mirror 1 and mirror 2 shown in fig. 1.
By screening the non-empty subset of Sa { "wget", "curl", "net-tools", "vim", "python", "opencv", "node" }, the target non-empty subset of truth satisfying the first preset condition and the second preset condition is obtained as Sj [ { "wget", "curl", "net-tools", "vim" }, { "python", "opencv" }, { "node" } ], and it can be seen that the number of data files is the largest, and "wget", "curl", "net-tools", "vim" are all common data files.
Optionally, an implementation of reconstructing the mirror image to be processed is further provided below, please refer to fig. 6, where fig. 6 is a schematic flowchart of an implementation of step S303 provided in the embodiment of the present invention, where step S303 may include:
s303-1, constructing a directed graph according to the data files in all the target non-aerial subsets;
wherein each node in the directed graph corresponds to a data file; one public data file is a bifurcation node of the directed graph; the data file corresponding to each node before the forking node corresponds to a common mirror layer.
It is understood that the way to construct the above directed graph may be: restoring the command statements corresponding to the target non-vacuum true subset Aj, and constructing a directed graph by combining the command statements corresponding to Aj and the array corresponding to the mirror image to be processed, namely traversing the array corresponding to the original mirror image to be processed, and if the element is in the set Aj, selecting the set Aj as a node and connecting the nodes. Until it is finished.
S303-2, reconstructing a plurality of mirror images to be processed according to the data file set, the public mirror image layer and the directed graph corresponding to each mirror image to be processed.
Continuing with fig. 1 as an example, the target non-empty subset of truths corresponding to mirror 1 and mirror 2 is Sj [ { "wget", "curl", "net-tools", "vim" }, { "python", "opencv" }, { "node" }. In combination with S1 { "wget", "curl", "net-tools", "vim", "python", "opencv" }correspondingto the mirror image 1, and S2 { "wget", "curl", "net-tools", "vim", "node" }correspondingto the Dockerfile of the mirror image 2.
For example, according to the element sequence of Sj, data files in { "wget", "curl", "net-tools", "vim" } are sequentially connected as nodes, it should be noted that the sequence of the data elements in each set can be exchanged, and if "vim" is a cross node, any one of { "python", "opencv" } and any one of { "node" } are further sequentially connected as child nodes of the cross node; then, for each child node, the data files belonging to the same set as the child node are connected according to the same logic and until all the data files are connected as nodes, so as to obtain a final directed graph, as shown in fig. 7, fig. 7 is a directed graph provided in the embodiment of the present invention.
And traversing the directed graph by adopting a depth-first algorithm to obtain mirror image constructed statements, combining command statements before the bifurcation node of the graph 7 to be used as a common mirror image layer, regenerating a configuration file of each mirror image to be processed, and then reconstructing each mirror image to be processed according to the regenerated configuration file.
For ease of understanding, continuing with the example of FIG. 1, the configuration file for mirror 1 is regenerated from the directed graph, common mirror layer, and the data file collection for mirror 1 as follows:
FROM ubuntu:16.04
RUN apt-get install wget curl net-tools vim
RUN apt-get install python opencv
...
CMD['/usr/bin/python','xxx.py']
according to the directed graph, the common mirror layer and the data file set of the mirror image 2, the configuration file of the mirror image 2 is regenerated as follows:
ROM ubuntu:16.04
RUN apt-get install wget curl net-tools vim
RUN apt-get install node
...
CMD['/usr/bin/node','xxx.js']
from the above results, it can be seen that there is a common mirror layer in the configuration files regenerated by mirror 1 and mirror 2, namely: the wget curve net-tools vim, that is, the wget curve net-tools vim is built in the same layer, so that the mirror image 1 and the mirror image 2 can share the layer, which is compared with the configuration file in fig. 1, in the embodiment of the present application, it is not necessary to repeatedly build a mirror image layer for the wget curve net-tools vim, and it is also not necessary to repeatedly cache the same data, so that the storage space can be obviously saved.
In order to implement the steps in the foregoing embodiments to achieve the corresponding technical effects, the mirror image processing method provided in the embodiment of the present invention may be implemented in a hardware device or in a form of a software module, and when the mirror image processing method is implemented in a form of a software module, an image processing apparatus is further provided in the embodiment of the present invention, please refer to fig. 8, where fig. 8 is a functional block diagram of the mirror image processing apparatus provided in the embodiment of the present invention, and the mirror image processing apparatus 400 may include:
a determining module 410, configured to determine a common data file corresponding to a plurality of images to be processed; the public data file is at least one of a plurality of data files to be installed of each to-be-processed mirror image; each data file corresponds to an application program; and determining a public mirror image layer according to the public data file.
A constructing module 420, configured to reconstruct the multiple to-be-processed images according to the common image layer and other data files except the common data file in each to-be-processed image.
It is appreciated that the determining module 410, the constructing module 420 may cooperatively perform the steps of fig. 3 to achieve a corresponding technical effect.
In some possible embodiments, the determining module 410 is specifically configured to: acquiring identification information of each mirror image to be processed; acquiring a configuration file corresponding to each mirror image to be processed according to the identification information; and determining the public data file according to a target command field for constructing the mirror image to be processed in each configuration file.
In some possible embodiments, the determining module 410 is further specifically configured to parse the target command field in each of the configuration files to obtain a data file set corresponding to each of the to-be-processed images; taking the union of all the data file sets as a target data file set; and determining the public data file according to the target data file set.
In some possible embodiments, the determining module 410 is further specifically configured to determine all non-empty proper subsets corresponding to the target data file set; screening out a target non-vacuum subset which meets a first preset condition and a second preset condition according to all the non-vacuum subsets; wherein the first preset condition is as follows: the intersection of the target non-vacuum true subset and any one of the data file sets is the target non-vacuum true subset or a vacuum set; the second preset condition is as follows: and the intersection of the difference set between the target non-aerial-true subset and the target data file set and the target non-aerial-true subset does not meet the first preset condition. And determining all data files in the target non-vacuum proper subset with the largest number of data files as the common data file.
In some possible embodiments, the constructing module 420 is specifically configured to reconstruct the multiple images to be processed according to the common image layer and other data files except the common data file in each image to be processed, and includes: constructing a directed graph according to the data files in all the target non-vacuum subsets; wherein each node in the directed graph corresponds to a data file; one of the public data files is a bifurcation node of the directed graph; the data file corresponding to each node before the bifurcation node corresponds to the common mirror image layer; and reconstructing the plurality of mirror images to be processed according to the data file set corresponding to each mirror image to be processed, the public mirror image layer and the directed graph.
In some possible embodiments, the determining module 410 is further specifically configured to parse the target command field corresponding to each of the configuration files, including: determining the target command field in the configuration file according to a predefined command field or a preset judgment rule; and analyzing the configuration file according to the determined target command field.
In some possible embodiments, the determining module 410 is further specifically configured to, according to each piece of the identification information, read a configuration file corresponding to the identification information; or reading the construction record information corresponding to the identification information according to each identification information, and converting the construction record information into the configuration file.
The image processing device provided by the embodiment of the invention determines the public data files corresponding to a plurality of images to be processed through the determining module; the public data file is at least one of a plurality of data files to be installed of each to-be-processed mirror image; each data file corresponds to an application program; determining a public mirror image layer according to the public data file; and reconstructing the plurality of images to be processed according to the public image layer and other data files except the public data file in each image to be processed through a construction module. The image data can be compressed, and the storage space occupied by the image is reduced.
It should be noted that each functional module in the image processing apparatus 400 according to the embodiment of the present invention may be stored in a memory in the form of software or Firmware (Firmware) or be fixed in an Operating System (OS) of the electronic device, and may be executed by a processor in the electronic device. Meanwhile, data, codes of programs, and the like required to execute the above modules may be stored in the memory.
Therefore, an embodiment of the present invention further provides an electronic device, which may be the first server 210 shown in fig. 1, or other electronic devices with a data processing function, and the present invention is not limited thereto.
Referring to fig. 9, fig. 9 is a block diagram of an electronic device according to an embodiment of the present invention. The electronic device 500 comprises a communication interface 501, a processor 502 and a memory 503. The processor 502, memory 503 and communication interface 501 are electrically connected to each other, directly or indirectly, to enable the transfer or interaction of data. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory 503 may be used for storing software programs and modules, such as program instructions/modules corresponding to the image processing method provided by the embodiment of the present invention, and the processor 502 executes various functional applications and data processing by executing the software programs and modules stored in the memory 503. The communication interface 501 may be used for communicating signaling or data with other node devices. The electronic device 50 may have a plurality of communication interfaces 501 in the present invention.
The Memory 503 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.
The processor 502 may be an integrated circuit chip having signal processing capabilities. The Processor may be a general-purpose Processor including a Central Processing Unit (CPU), a Network Processor (NP), etc.; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the image processing method according to any one of the foregoing embodiments. The computer readable storage medium may be, but is not limited to, various media that can store program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a PROM, an EPROM, an EEPROM, a magnetic or optical disk, etc.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A method of image processing, the method comprising:
determining a public data file corresponding to a plurality of mirror images to be processed;
the public data file is at least one of a plurality of data files in each mirror image to be processed; each data file corresponds to an application program;
determining a public mirror image layer according to the public data file;
and reconstructing the plurality of images to be processed according to the common image layer and other data files except the common data file in each image to be processed.
2. The image processing method of claim 1, wherein determining the common data file corresponding to the plurality of images to be processed comprises:
acquiring identification information of each mirror image to be processed;
acquiring a configuration file corresponding to each mirror image to be processed according to the identification information;
and determining the public data file according to a target command field for constructing the mirror image to be processed in each configuration file.
3. The image processing method of claim 2, wherein obtaining the common data file according to a target command field in each of the configuration files for constructing the image to be processed comprises:
analyzing the target command field in each configuration file to obtain a data file set corresponding to each mirror image to be processed;
taking the union of all the data file sets as a target data file set;
and determining the public data file according to the target data file set.
4. An image processing method according to claim 3, wherein determining the common data file from the set of target data files comprises:
determining all non-vacuum true subsets corresponding to the target data file set;
screening out a target non-vacuum subset which meets a first preset condition and a second preset condition according to all the non-vacuum subsets;
wherein the first preset condition is as follows: the intersection of the target non-vacuum true subset and any one of the data file sets is the target non-vacuum true subset or a vacuum set; the second preset condition is as follows: the intersection of the difference set between the target non-aerial-true subset and the target data file set and the target non-aerial-true subset does not meet the first preset condition;
and determining all data files in the target non-vacuum proper subset with the largest number of data files as the common data file.
5. The image processing method according to claim 4, wherein reconstructing the plurality of images to be processed according to the common image layer and other data files except the common data file in each of the images to be processed comprises:
constructing a directed graph according to the data files in all the target non-vacuum subsets;
wherein each node in the directed graph corresponds to a data file; one of the public data files is a bifurcation node of the directed graph; the data file corresponding to each node before the bifurcation node corresponds to the common mirror image layer;
and reconstructing the plurality of mirror images to be processed according to the data file set corresponding to each mirror image to be processed, the public mirror image layer and the directed graph.
6. The image processing method according to claim 2 or 3, wherein parsing the target command field corresponding to each of the configuration files comprises:
determining the target command field in the configuration file according to a predefined command field or a preset judgment rule;
and analyzing the configuration file according to the determined target command field.
7. The image processing method according to claim 2, wherein obtaining the configuration file corresponding to each of the to-be-processed images according to the identification information comprises:
reading a configuration file corresponding to the identification information according to each identification information; alternatively, the first and second electrodes may be,
and reading the construction record information corresponding to the identification information according to each identification information, and converting the construction record information into the configuration file.
8. An image processing apparatus, comprising:
the determining module is used for determining public data files corresponding to the plurality of to-be-processed mirror images; the public data file is at least one of a plurality of data files to be installed of each to-be-processed mirror image; each data file corresponds to an application program; determining a public mirror image layer according to the public data file;
and the construction module is used for reconstructing the plurality of images to be processed according to the public image layer and other data files except the public data file in each image to be processed.
9. An electronic device comprising a processor and a memory, the memory storing a computer program executable by the processor, the processor being operable to execute the computer program to implement the image processing method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the image processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111211973.6A CN113934510A (en) | 2021-10-18 | 2021-10-18 | Mirror image processing method and device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111211973.6A CN113934510A (en) | 2021-10-18 | 2021-10-18 | Mirror image processing method and device, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113934510A true CN113934510A (en) | 2022-01-14 |
Family
ID=79280145
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111211973.6A Pending CN113934510A (en) | 2021-10-18 | 2021-10-18 | Mirror image processing method and device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113934510A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114579520A (en) * | 2022-03-04 | 2022-06-03 | 北京百度网讯科技有限公司 | Mirror image data processing method, mirror image data loading method and device |
-
2021
- 2021-10-18 CN CN202111211973.6A patent/CN113934510A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114579520A (en) * | 2022-03-04 | 2022-06-03 | 北京百度网讯科技有限公司 | Mirror image data processing method, mirror image data loading method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102682052B (en) | The data query that filtering data stores | |
| CN107391629B (en) | Method, system, server and computer storage medium for data migration between clusters | |
| WO2016180055A1 (en) | Method, device and system for storing and reading data | |
| CN111209090B (en) | Method and assembly for creating virtual machine in cloud platform and server | |
| CN106649676A (en) | Duplication eliminating method and device based on HDFS storage file | |
| CN112929401A (en) | Registration method and device | |
| US11347413B2 (en) | Opportunistic storage service | |
| CN112579319A (en) | Service calling method and device based on LRU Cache optimization | |
| CN110781159B (en) | Ceph directory file information reading method and device, server and storage medium | |
| US20200220910A1 (en) | Idempotent processing of data streams | |
| CN110413588B (en) | Distributed object storage method and device, computer equipment and storage medium | |
| CN113934510A (en) | Mirror image processing method and device, electronic equipment and computer readable storage medium | |
| CN113535068A (en) | Data reading method and system | |
| CN112860412B (en) | Service data processing method and device, electronic equipment and storage medium | |
| CN114489690A (en) | Distributed system deployment method and device, electronic equipment and storage medium | |
| US11625192B2 (en) | Peer storage compute sharing using memory buffer | |
| CN113077260A (en) | Data access method and device based on block chain and electronic equipment | |
| CN115361389A (en) | Cloud computing instance creation method and device | |
| CN113468182B (en) | Data storage method and system | |
| CN115426375A (en) | Data processing method and data processing system | |
| CN111694809B (en) | File filtering method and device based on block chain and computer equipment | |
| US20210141791A1 (en) | Method and system for generating a hybrid data model | |
| WO2024066904A1 (en) | Container creation method, system, and node | |
| CN113553329B (en) | Data integration system and method | |
| WO2024109066A1 (en) | Data processing method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |