JP6581155B2 - Unnecessary file detection device, unnecessary file detection method and unnecessary file detection program - Google Patents

Description

  The present invention relates to an unnecessary file detection device, an unnecessary file detection method, and an unnecessary file detection program.

  Conventionally, a container image that is a source of an application that operates on an edge terminal in edge computing is known. Distribute container images managed in the cloud to various edge servers. Such a container image is an image obtained by installing middleware and libraries necessary for application execution, an application execution file, and the like on a certain OS (Operating System).

  As a means for defining a container image, there is a method in which the container image is divided into layers. A layer becomes one layer for every procedure for creating an image defined in an image definition file, and there is a method of stacking these layers to form one container image (see, for example, Non-Patent Document 1). Further, for example, when transferring an image, there is a method in which transfer for each layer is possible, and when an image transferred to an edge server is changed, only a layer having a difference is transferred. On the other hand, there is a method of integrating image layer information and managing it as one large layer (see, for example, Non-Patent Document 2).

“Dockerfile reference”, [online], Docker, [searched July 18, 2017], Internet <https://docs.docker.com/engine/reference/builder/> “Squash an image's layers (-squash) Experimental Only”, [online], Docker, [Search July 18, 2017], Internet <https://docs.docker.com/engine/reference/commandline/build/ # squash-an-images-layers-squash-experimental-only>

  However, the conventional technique has a problem that the container image cannot be reduced in weight because unnecessary information may be included in the container image. For example, in the conventional technology, the container image is divided into multiple layers to create and distribute images more efficiently. However, there is a relationship between upper and lower layers, and lower layers do not depend on higher layers. The lower layer is not aware of the processing in the upper layer. For this reason, a file to be deleted in the upper layer may remain in the lower layer.

  For example, since the operation is independent for each layer, even if the file A is saved in the image in a certain layer 1 and the file A saved in the upper layer 2 is deleted, the layer 1 The information of file A remains, and the file A capacity is added to the image size even though the file A is deleted at the time of final execution.

  Incidentally, it is necessary to image creators to devise how to create a layer in order to prevent this, which files for the runtime file system file does not remain even if it remains in the intermediate layers file intermediate layer It was difficult to detect what remained.

  In order to solve the above-described problems and achieve the object, the unnecessary file detection apparatus of the present invention creates an image file using an image definition file that defines a container image that is a source of application operation for each layer, A creation unit that creates a runtime file system in which the layer information is deleted from the image file; a content of the runtime file system created by the creation unit; and a content of the file system of each layer in the image definition file; And a detection unit for detecting each file not included in the runtime file system in the file system of each layer.

  The unnecessary file detection method of the present invention is an unnecessary file detection method executed by an unnecessary file detection apparatus, and uses an image definition file that defines a container image that is a source of operation of an application for each layer. Creating a runtime file system in which the layer information is deleted from the image file, the contents of the runtime file system created by the creating step, and the file of each layer in the image definition file A detection step of comparing the contents of the system with each other and detecting a file not included in the runtime file system in the file system of each layer.

  In addition, the unnecessary file detection program of the present invention creates an image file using an image definition file that defines a container image that is a source of application operation for each layer, and deletes the layer information from the image file. A creation step for creating an hour file system, and the contents of the runtime file system created by the creation step are compared with the contents of the file system of each layer in the image definition file, and the file system of each layer Causing the computer to execute a detection step of detecting each file not included in the runtime file system.

  According to the present invention, there is an effect that unnecessary information can be deleted from the container image and the container image can be reduced in weight.

FIG. 1 is a diagram illustrating an example of a configuration of a distribution system according to the first embodiment. FIG. 2 is a block diagram illustrating a configuration example of the unnecessary file detection apparatus according to the first embodiment. FIG. 3 is a diagram illustrating an example of information stored in the unnecessary file information storage unit. FIG. 4 is a diagram illustrating an example of the unnecessary file detection process. FIG. 5 is a flowchart for explaining processing by the unnecessary file detection apparatus according to the first embodiment. FIG. 6 is a diagram illustrating a computer that executes an unnecessary file detection program.

  Embodiments of an unnecessary file detection device, an unnecessary file detection method, and an unnecessary file detection program according to the present application will be described below in detail with reference to the drawings. In addition, the unnecessary file detection apparatus, the unnecessary file detection method, and the unnecessary file detection program according to the present application are not limited by this embodiment.

[First embodiment]
In the following embodiments, the configuration of the distribution system according to the first embodiment, the configuration of the unnecessary file detection device, the processing flow in the unnecessary file detection device will be described in order, and finally the effect of the first embodiment Will be explained.

[Configuration of distribution system]
First, the distribution system according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a configuration of a distribution system according to the first embodiment. The distribution system according to the first embodiment includes an unnecessary file detection device 10, a distribution server 20, and a plurality of edge terminals 30 </ b> A and 30 </ b> B, and each device is connected via a communication network 40. Note that the edge terminals 30 </ b> A and 30 </ b> B are referred to as edge terminals 30 when they are described without distinction. Moreover, the number of each apparatus shown in FIG. 1 is an example to the last, and is not restricted to this.

  When receiving the input of the image definition file, the unnecessary file detection apparatus 10 creates an image using the image definition file and creates an image from which the layer information is deleted as a runtime file system. Thereafter, the unnecessary file detection apparatus 10 compares the contents of the file system in each layer of the image definition file with the contents of the runtime file system, and confirms whether a file not included in the runtime file system exists in the layer. To do. Then, when a file that is not included in the runtime file system exists in the layer, the unnecessary file detection apparatus 10 presents the file to the user as an unnecessary file.

  The distribution server 20 distributes to the edge terminal 30 a container image that is a source of an application that runs on the edge terminal 30 in edge computing. A container image is an image file that includes an application execution environment, and is created by an application developer using an image definition file, for example. The container image is composed of a plurality of layers. Each layer is defined by an image definition file.

  The edge terminals 30A and 30B are, for example, devices used as edge servers in the edge computing architecture, and are devices that operate applications. The edge terminals 30A and 30B receive the container image distributed from the distribution server 20, and operate the application based on the received container image. In the edge architecture, the edge terminals 30A and 30B often have poor resources (disk and network bandwidth) compared to the data center. Therefore, in order to effectively use the disk, it is necessary to make the container image small, and it is necessary to delete unnecessary files from the image.

  In the unnecessary file detection apparatus 10 according to the first embodiment, by comparing each layer with the runtime file system, it is automatically determined whether any files that are not in the runtime file system remain in all layers. To detect. For this reason, it is possible to reduce the size of the entire image file and prevent unnecessary files (for example, source code) that are not desired to be included in the container image from being mixed in the image.

[Configuration of unnecessary file detection device]
Next, the configuration of the unnecessary file detection apparatus 10 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a block diagram illustrating a configuration example of the unnecessary file detection apparatus according to the first embodiment. As illustrated in FIG. 2, the unnecessary file detection apparatus 10 includes a communication processing unit 11, an input unit 12, an output unit 13, a control unit 14, and a storage unit 15. The processing of each unit included in the unnecessary file detection device 10 will be described below. The distribution server 20 may hold some or all of the functions of the unnecessary file detection device 10.

  The communication processing unit 11 controls communication related to various types of information exchanged with a connected device. For example, the communication processing unit 11 transmits a container image from which unnecessary files are deleted to the distribution server 20.

  The input unit 12 is an input device such as a keyboard or a mouse, and receives input operations of various information from the user. The output unit 13 is an output device such as a display, and outputs various information.

  The storage unit 15 stores data and programs necessary for various processes performed by the control unit 14, and has an unnecessary file information storage unit 15a particularly closely related to the present invention. For example, the storage unit 15 is a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk.

  The unnecessary file information storage unit 15a stores an unnecessary file of each layer detected by the detection unit 14b described later. Here, the information stored in the unnecessary file information storage unit 15a will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of information stored in the unnecessary file information storage unit. As illustrated in FIG. 3, the unnecessary file information storage unit 15 a includes “layer” that is information for identifying a layer and “unnecessary file” that is information for identifying a file detected as an unnecessary file in each layer. Store in association with each other.

  As illustrated in FIG. 3, the unnecessary file information storage unit 15 a stores unnecessary files “file A, file B” in association with “layer 1”. This means that “file A, file B” is detected as an unnecessary file in “layer 1”.

  The control unit 14 has an internal memory for storing a program that defines various processing procedures and the necessary data, and performs various processes by using these programs. Particularly, as closely related to the present invention, A creation unit 14a, a detection unit 14b, and a presentation unit 14c are included. Here, the control unit 14 is an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

  The creation unit 14a creates an image file using an image definition file that defines a container image that is an operation source of an application for each layer, and creates a runtime file system in which layer information is deleted from the image file. For example, when receiving the input of the image definition file from the user via the input unit 12, the creation unit 14a creates an image file using the image definition file.

  Then, the creation unit 14a creates a runtime file system in which the layer information is deleted from the image file. Here, as a method for creating a runtime file system in which layer information is deleted from an image file, a known method (for example, see Non-Patent Document 2) may be used. Is deleted, and a runtime file system composed of one large layer is created. It is assumed that no unnecessary files remain in this runtime file system.

  The detection unit 14b compares the content of the runtime file system created by the creation unit 14a with the content of the file system of each layer in the image definition file, and is not included in the runtime file system in the file system of each layer. Detect each file.

  For example, the detection unit 14b sequentially compares the contents of the runtime file system and the contents of the file system of each layer from the lowest layer to the highest layer in the image definition file, and executes the execution in the file system of each layer. Each file not included in the file system is detected and stored in the unnecessary file information storage unit 15a.

  Here, an example of the unnecessary file detection process will be described with reference to FIG. FIG. 4 is a diagram illustrating an example of the unnecessary file detection process. As illustrated in FIG. 4, the detection unit 14 b selects the lowest layer among the layers in the image definition file as the target layer. In the example of FIG. 4, the lowest layer is “layer 1”, and is assumed to be an upper layer in the order of “layer 2”, “layer 3”. Then, the detection unit 14b compares the contents of the runtime file system with the contents of the “layer 1” file system, and identifies “files” as unnecessary files that are not included in the runtime file system in the “layer 1” file system. A ”and“ File B ”are detected.

  Next, the detection unit 14b selects “layer 2”, which is an upper layer, as a target layer. Then, the detection unit 14b compares the contents of the runtime file system with the contents of the “layer 2” file system, and identifies “files” as unnecessary files that are not included in the runtime file system in the “layer 2” file system. A ”is detected. Such unnecessary file detection processing is repeated up to the highest layer.

  The presentation unit 14c presents the file detected by the detection unit 14b as an unnecessary file. Specifically, the presentation unit 14c reads out unnecessary files of each layer stored in the unnecessary file information storage unit 15a and presents them to the user.

  In this way, the unnecessary file detection apparatus 10 can notify the user that files that are not in the system remain in the runtime file by presenting the unnecessary files in each layer to the user. For this reason, it is possible to prevent a file that is not desired to be included in the container image from being mixed in the image, and the container image from which unnecessary files are deleted can be transmitted to the distribution server 20.

[Processing flow of unnecessary file detection device]
Next, a processing flow of the unnecessary file detection apparatus 10 according to the first embodiment will be described with reference to FIG. FIG. 5 is a flowchart for explaining processing by the unnecessary file detection apparatus according to the first embodiment.

  As illustrated in FIG. 5, when the creation unit 14a of the unnecessary file detection apparatus 10 receives an input of an image definition file from the user via the input unit 12 (Yes in step S101), the image file is used using the image definition file. And a runtime file system is created from the image file (step S102).

  Then, the detection unit 14b selects the lowest layer among the layers in the image definition file as the target layer (step S103). Subsequently, the detection unit 14b expands the file system of the target layer (step S104), and compares the file system of the target layer with the runtime file system (step S105).

  Then, the detection unit 14b detects a file that does not exist in the runtime file system but exists in the file system of the target layer (step S106). Thereafter, the detection unit 14b determines whether the target layer is the highest layer (step S107). As a result, when the target layer is not the highest layer (No at Step S107), the detection unit 14b selects the upper layer as the target layer (Step S108), and returns to the process of Step S104. And the detection part 14b repeats the process of step S104-step S107 until the highest layer is made into an object layer.

  When the detection unit 14b determines that the target layer is the highest layer (Yes at Step S107), the presentation unit 14c presents the detected file as an unnecessary file (Step S109).

[Effect of the first embodiment]
As described above, the unnecessary file detection apparatus 10 according to the first embodiment creates an image file using an image definition file that defines a container image that is a source of operation of an application for each layer, and uses the image file from the image file. Create a runtime file system from which layer information has been deleted. The unnecessary file detection apparatus 10 compares the contents of the created runtime file system with the contents of the file system of each layer in the image definition file, and is not included in the runtime file system in the file system of each layer. Detect each file. For this reason, unnecessary information can be deleted from the container image to reduce the weight of the container image.

  In other words, the unnecessary file detection apparatus 10 automatically detects whether or not any files that are not in the runtime file system remain in all layers by comparing each layer with the runtime file system. For this reason, it is possible to reduce the size of the entire image file and prevent unnecessary files (for example, source code) that are not desired to be included in the container image from being mixed in the image.

[System configuration, etc.]
Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Furthermore, all or a part of each processing function performed in each device may be realized by a CPU and a program that is analyzed and executed by the CPU, or may be realized as hardware by wired logic.

  In addition, among the processes described in the present embodiment, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed All or a part of the above can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

[program]
FIG. 6 is a diagram illustrating a computer that executes an unnecessary file detection program. The computer 1000 includes a memory 1010 and a CPU 1020, for example. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1051 and a keyboard 1052, for example. The video adapter 1060 is connected to the display 1061, for example.

  The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the unnecessary file detection apparatus 10 is implemented as a program module 1093 in which a code executable by a computer is described. The program module 1093 is stored in the hard disk drive 1090, for example. For example, a program module 1093 for executing processing similar to the functional configuration in the apparatus is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

  Data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 and executes them as necessary.

  The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network or a WAN. Then, the program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.

DESCRIPTION OF SYMBOLS 10 Unnecessary file detection apparatus 11 Communication processing part 12 Input part 13 Output part 14 Control part 14a Creation part 14b Detection part 14c Presentation part 15 Storage part 15a Unnecessary file information storage part 20 Distribution server 30A, 30B Edge terminal

Claims (5)

An image file is created using an image definition file that defines a container image for application operation for each layer, and a creation unit that creates a runtime file system in which the layer information is deleted from the image file;
The contents of the runtime file system created by the creating unit and the contents of the file system of each layer in the image definition file are compared, and files not included in the runtime file system in the file system of each layer are compared. An unnecessary file detection apparatus comprising: a detection unit that detects each of the detection units.   The unnecessary file detection apparatus according to claim 1, further comprising a presentation unit that presents the file detected by the detection unit as an unnecessary file.   The detection unit sequentially compares the contents of the runtime file system with the contents of the file system of each layer from the lowest layer to the highest layer in the image definition file, and the file system of each layer The unnecessary file detection device according to claim 1, wherein each of the files not included in the runtime file system is detected. An unnecessary file detection method executed by an unnecessary file detection device,
Creating an image file using an image definition file that defines a container image for each operation of the application for each layer, and creating a runtime file system in which the layer information is deleted from the image file;
The contents of the runtime file system created by the creation step are compared with the contents of the file system of each layer in the image definition file, and files that are not included in the runtime file system in the file system of each layer An unnecessary file detection method comprising: a detection step for detecting each of them. Creating an image file using an image definition file that defines a container image that is a source of application operation for each layer, and creating a runtime file system in which the layer information is deleted from the image file;
The contents of the runtime file system created by the creating step are compared with the contents of the file system of each layer in the image definition file, and files that are not included in the runtime file system in the file system of each layer An unnecessary file detection program for causing a computer to execute a detection step for detecting each of them.

Images