CN110688201B - Log management method and related equipment - Google Patents
Log management method and related equipment Download PDFInfo
- Publication number
- CN110688201B CN110688201B CN201910937791.3A CN201910937791A CN110688201B CN 110688201 B CN110688201 B CN 110688201B CN 201910937791 A CN201910937791 A CN 201910937791A CN 110688201 B CN110688201 B CN 110688201B
- Authority
- CN
- China
- Prior art keywords
- container
- service
- log
- directory
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45566—Nested virtual machines
Abstract
The embodiment of the invention discloses a log management method and related equipment, wherein the method is applied to the technical field of computers and comprises the following steps: sending a container screening instruction to a service manager to instruct the service manager to determine the identification information of a target service container from the started service container, receiving the returned identification information, and sending an attribute query instruction to the service manager to instruct the service manager to query and return the attribute information of the target service container; and receiving attribute information of the target service container returned by the service manager, determining a physical machine directory mounted by a service log directory of the target service container from the attribute information based on a label of the target service container, determining a target subdirectory corresponding to the physical machine directory under a preset directory of the log management container, and further managing log files in the target subdirectory. By adopting the mode, the log file in the service container can be managed under the condition of random mounting.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a log management method and related equipment.
Background
The Docker container provides 6 isolation technologies, where Mount namespaces to isolate Mount points of a file system, so that different Mount namespaces have independent Mount point information, and different namespaces do not affect each other, which is very useful for constructing a file system directory of a user or a container.
Each Mount namespaces can see different file system hierarchies, and they cannot see the actual file directories in the host computer, nor the specific situations in other Mount namespaces. In general, if a service log of a service container needs to be managed, a specific directory of a host and a service container directory are bound, and the service log of the service container is directly written into the directory of the host, but the binding in this way has very strong limitation, for example, if a plurality of service containers on one host have the same working content (for example, all perform the same work of the same service), the service log directories of the service containers are bound to the same host directory inevitably, and if a plurality of service containers operate simultaneously, the service logs of the service containers are affected by each other to cause confusion, and the service log of which service container cannot be distinguished.
In order to solve the above problem, currently, a service manager may create a random directory on a host to mount a service log directory of a service container in an anonymous mounting manner. However, this method cannot uniformly manage the logs of the service containers directly on the host, because the user cannot know which specific directory the service log directory of the service container is mounted to the host. Therefore, how to manage the log files in the service container under the condition of random mount becomes an urgent problem to be solved.
Disclosure of Invention
The embodiment of the invention provides a log management method and related equipment, which can realize the management of log files in a service container under the condition of random mounting.
In a first aspect, an embodiment of the present invention provides a log management method, where the method is applied to a log management container running on a physical machine, where a service manager and at least one service container are also running on the physical machine, and the at least one service container is managed by the service manager in a unified manner, and the method includes:
sending a container screening instruction to the service manager, wherein the container screening instruction carries a label of a first service container to indicate the service manager to determine identification information of a target service container matched with the label from the started service container, and the first service container is a service container needing log management;
Receiving the identification information returned by the service manager, and sending an attribute query instruction carrying the identification information to the service manager to instruct the service manager to query and return the attribute information of the target service container;
receiving attribute information of the target service container returned by the service manager, and determining a physical machine directory of the service log directory of the target service container from the attribute information based on a label of the target service container;
and determining a target subdirectory corresponding to the physical machine directory under the preset directory of the log management container, and managing log files in the target subdirectory, wherein the preset directory of the log management container is pre-mounted with a root directory of the physical machine.
In an embodiment, the specific implementation manner of determining, from the attribute information, a physical machine directory on which the service log directory of the target service container is mounted based on the tag of the target service container is as follows: extracting a service log directory from the label of the target service container; screening out a physical machine directory corresponding to the service log directory from the attribute information; and determining the physical machine directory corresponding to the service log directory as the physical machine directory mounted by the service log directory of the target service container.
In an embodiment, the specific implementation manner of managing the log files in the target subdirectory is as follows: acquiring file attributes of log files in the target subdirectory, wherein the file attributes comprise modification time and log file names; and performing management operation on the log file based on the file attribute, wherein the management operation comprises compressing the log file or deleting the log file.
In an embodiment, the management operation includes compressing the log file, and the specific implementation of the management operation on the log file based on the file attribute is as follows: and if the time interval between the modification time of the first log file and the system time is detected to be greater than or equal to a first preset duration threshold value, and the log file name of the first log file is not ended by a first preset field and a second preset field, compressing the first log file so that the log file name of the compressed first log file is ended by the second preset field, wherein the first log file is one or more log files in the target subdirectory.
In an embodiment, the management operation includes deleting the log file, and the specific implementation of the management operation on the log file based on the file attribute is as follows: and if the time interval between the modification time and the system time of the first log file is detected to be greater than or equal to a second preset duration threshold and the log file name of the first log file ends in the second preset field, deleting the first log file, wherein the first log file is one or more log files in the target subdirectory.
In a second aspect, an embodiment of the present invention provides another log management method, where the method is applied to a service manager running on a physical machine, where a log management container and at least one service container are also running on the physical machine, and the at least one service container is managed by the service manager in a unified manner, and the method includes:
receiving a container screening instruction sent by the log manager, wherein the container screening instruction carries a label of a first service container, and the first service container is a service container needing log management;
determining the identification information of a target service container matched with the label from the started service container, and returning the identification information to the log management container;
and when receiving an attribute query instruction which is sent by the log management container and carries the identification information, querying and returning the attribute information of the target service container.
In an embodiment, the service manager may further mount a root directory of the physical machine to a preset directory of the log management container, so that the log management container determines, in the preset directory, a target sub-directory corresponding to the physical machine directory under the root directory.
In a third aspect, an embodiment of the present invention provides a log management apparatus, where the log management apparatus includes a module configured to execute the method in the first aspect.
In a fourth aspect, an embodiment of the present invention provides another log management apparatus, where the log management apparatus includes a module configured to execute the method of the second aspect.
In a fifth aspect, an embodiment of the present invention provides a server, where a log management container, a service manager, and at least one service container run on the server, where the at least one service container is managed by the service manager in a unified manner, and the log management container includes a processor and a memory, where the processor and the memory are connected to each other, where the memory is used to store a computer program, and the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method of the first aspect or the second aspect.
In a sixth aspect, embodiments of the present invention provide a computer-readable storage medium, in which a computer program is stored, the computer program comprising program instructions that, when executed by a processor, cause the processor to perform the method of the first or second aspect.
In the embodiment of the present invention, the log management container may send a container screening instruction to the service manager to instruct the service manager to determine the identification information of the target service container from the started service container, receive the identification information returned by the service manager, and send an attribute query instruction to the service manager to instruct the service manager to query and return the attribute information of the target service container. Further, the log management container may receive attribute information of the target service container returned by the service manager, determine, from the attribute information, a physical machine directory on which the service log directory of the target service container is mounted based on a tag of the target service container, determine a target sub-directory corresponding to the physical machine directory under a preset directory of the log management container, and further manage log files in the target sub-directory. By adopting the mode, the log file in the service container can be managed under the condition of random mounting.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a log management system according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a log management method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of attribute information provided by an embodiment of the invention;
FIG. 4 is a flowchart illustrating another log management method according to an embodiment of the present invention;
fig. 5 is a schematic block diagram of a log management apparatus according to an embodiment of the present invention;
FIG. 6 is a schematic block diagram of another log management apparatus provided by an embodiment of the present invention;
fig. 7 is a schematic block diagram of a server according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a log management method, which can be applied to a log management container running on a physical machine, wherein the physical machine also runs a service manager and at least one service container, the at least one service container is uniformly managed by the service manager, each physical machine runs one log management container, and the physical machine can be a server, for example. The log management container may send a container screening instruction to the service manager, where the container screening instruction carries a tag of a first service container (the first service container is a service container that needs log management), so as to instruct the service manager to determine, from the started service container, identification information of a target service container that matches the tag. Further, the log management container may receive the identifier information returned by the service manager, and send an attribute query instruction carrying the identifier information to the service manager, so as to instruct the service manager to query and return the attribute information of the target service container. Further, the log management container may determine, from the attribute information, a physical machine directory on which the service log directory of the target service container is mounted based on the tag of the target service container, determine a target subdirectory corresponding to the physical machine directory under a preset directory of the log management container, and manage log files in the target subdirectory, where the preset directory of the log management container has mounted a root directory of the physical machine in advance.
It can be seen that, in the log management method provided in the embodiment of the present invention, on one hand, each physical machine only needs to start one log management container, and the log management container can manage the log files of all the service containers, thereby reducing waste of a large amount of resources and being beneficial to saving resources; on the other hand, the physical machine directory where the service container needing log management really is located can be obtained by sending instructions (such as container screening instructions, attribute query instructions and the like) to the service manager, so that the situation that each service container Mount namespaces is isolated from each other is avoided, and log files in the service container are managed.
Referring to fig. 1, a journal management system includes a physical machine 10, a journal management container 101 running on the physical machine, a service manager 102 running on the physical machine, and at least one service container 103 running on the physical machine, where the at least one service container 103 is managed by the service manager 102. In one embodiment, the service manager 102 may mount the root directory "/" of the physical machine to the preset directory "/mnt" of the journal management container 101, and after the mount is completed, if the journal management container wants to access the directory "/sub/path" of the physical machine (i.e. the physical machine directory is "/sub/path"), only the directory of the physical machine "/mnt/sub/path" of the journal management container needs to be accessed (i.e. the target sub-directory corresponding to the physical machine directory "/sub/path" under the preset directory "/mnt" of the journal management container).
Further, the service manager 102 may map a socket file, which is a communication interface for communication between the log management container 101 and the service manager 102, to the log management container 101, so that the log management container 101 may communicate with the service manager 102 through the socket. In this case, the log management container 101 may send a container screening instruction carrying a label of a first service container to the service manager 102 through a docker. Further, after receiving the identification information, the log management container 101 may send an attribute query instruction carrying the identification information to the service manager 102, and the service manager 102 responds to the attribute query instruction, and queries and returns the attribute information of the target service container. Further, after receiving the attribute information of the target service container returned by the service manager 102, the log management container 101 may determine, from the attribute information, a physical machine directory on which the service log directory of the target service container is mounted based on a tag of the target service container, further determine a target subdirectory corresponding to the physical machine directory under the preset directory of the log management container, and manage log files in the target subdirectory, where the preset directory of the log management container has previously mounted a root directory of a physical machine.
Referring to fig. 2, fig. 2 is a schematic flowchart of a log management method according to an embodiment of the present invention, where this embodiment may be executed by a log management container running on a physical machine, where the physical machine further runs a service manager and at least one service container, and the at least one service container is uniformly managed by the service manager, as shown in the figure, the log management method may include:
s201: and sending a container screening instruction to the service manager, wherein the container screening instruction carries a label of a first service container to indicate the service manager to determine the identification information of a target service container matched with the label from the started service container, and the first service container is a service container needing log management. The identification information may manage the ID of the container for each target log, for example.
In one embodiment, before performing step S201, the log management container may determine, based on a log management confirmation operation input for a tag of one or more service containers of the at least one service container, a first service container from the at least one service container, and further generate a container filtering instruction including the tag of the first service container. The input mode of the log management confirmation operation may include voice, click, slide, press, and the like, which is not specifically limited in this embodiment of the present invention.
Exemplarily, it is assumed that the service container running on the physical machine includes a service container a and a service container B, where a tag added to the service container a in advance is "tomcat _ log ═ log/sub/path", and a tag added to the service container B is "project _ log ═ home/work/data". In this case, the labels corresponding to the service container a and the service container B may be displayed on the display device, and when the user wants to perform log management on the service container a, a log management confirmation operation may be input on the label "tomcat _ log ═ log/sub/path" corresponding to the service container a. Further, the log management container may determine the service container a as a first service container from among the service containers a and B based on the log management confirmation operation, and generate a container filtering instruction including a label "tomcat _ log ═ log/sub/path" (i.e., a label of the first service container).
In one embodiment, the log management container is mapped with a socket file of the traffic manager, which is a communication interface for communicating with the traffic manager. For this case, the log management container may send a container screening instruction carrying a label of the first service container to the service manager based on the socket file. Further, after receiving the container screening instruction, the service manager may determine, from the started service containers, identification information of a target service container that matches the tag, and return the identification information to the log management container.
S202: and receiving the identification information returned by the service manager, and sending an attribute query instruction carrying the identification information to the service manager to instruct the service manager to query and return the attribute information of the target service container. The attribute information includes a container name of the target service container, a network mode of the target service container, mounting information between the target service container and the physical machine, and the like, where the mounting information represents a service log directory in which the target service container and the physical machine are mounted, and a physical machine directory corresponding to the mounted service log directory.
In an embodiment, after receiving the identifier information returned by the service manager, the log management container may send an attribute query instruction carrying the identifier information to the service manager through the socket file, and the service manager may respond to the attribute query instruction to query and return the attribute information of the target service container.
S203: and receiving attribute information of the target service container returned by the service manager, and determining a physical machine directory mounted by the service log directory of the target service container from the attribute information based on the label of the target service container.
In one embodiment, when each service container is started, the user may label each service container through the service manager 102, where the label includes the service log directory of the corresponding service container, for example, the label corresponding to the service container a is "tomcat _ log ═ log/sub/path", where "/log/sub/path" represents the service log directory of the service container a. In this case, after the log management container receives the attribute information of the target service container returned by the service manager, the log management container may extract the service log directory from the tag of the target service container, and screen out the physical machine directory corresponding to the service log directory from the attribute information, thereby determining the physical machine directory corresponding to the service log directory as the physical machine directory mounted on the service log directory of the target service container.
Exemplarily, it is assumed that the tag of the target service container is "tomcat _ log ═ home/work/data", and the above attribute information is shown in fig. 3. In this case, the target service container may extract the corresponding service log directory "/home/work/data" from the tag of the target service container, and may query, from the attribute information shown in fig. 3, that the physical machine directory corresponding to the service log directory "/home/work/data" is "/home/docker/volumes/ecs-tasks-sd-files-data-9-data-b 2ddfda89fb9d3af9a01/_ data" (i.e., the physical machine directory mounted on the service log directory of the target service container).
S204: determining a target subdirectory corresponding to the physical machine directory under a preset directory of a log management container, and managing log files in the target subdirectory, wherein the preset directory of the log management container is provided with a root directory of the physical machine in advance. The preset directory is any one of the designated directories of the log management container and can be preset by a developer.
In one embodiment, the service manager may mount the root directory "/" of the physical machine to the preset directory "/mnt" of the journal management container in advance, and after the mount is completed, if the journal management container wants to access the directory "/sub/path" of the physical machine, the journal management container only needs to access the directory "/mnt/sub/path" of its own. Illustratively, assuming that the log management container determines that the physical machine directory "/home/docker/volume/ecs-tasks-sd-filesystem-9-data-b 2ddfda89fb9d3af9a01/_ data" of the service log directory mount of the target service container, since the root directory of the physical machine has been previously mounted by the preset directory of the log management container, the log management container may determine that the target subdirectory corresponding to the physical machine directory under the preset directory "/mnt" is "/mnt/home/docker/volume/ecs-tasks-sd-filesystem-9-data-b 2ddfda89fb9d3af9a01/_ data".
In one embodiment, the specific implementation of the log management container managing the log files in the target subdirectory may be as follows: and acquiring file attributes of the log files in the target subdirectory (the file attributes comprise modification time and log file names), and performing management operation on the log files in the target subdirectory based on the file attributes, wherein the management operation comprises log file compression or log file deletion.
In one embodiment, the managing operation includes compressing the log file, in which case, if the log management container detects that a time interval between a modification time and a system time of the first log file is greater than or equal to a first preset duration threshold and a log file name of the first log file does not end in a first preset field and a second preset field, the first log file is compressed so that the log file name of the compressed first log file ends in the second preset field, and the first log file is one or more log files in the target subdirectory.
In one embodiment, the managing operation includes deleting the log file, in which case the log management container deletes the first log file if it detects that the time interval between the modification time and the system time of the first log file is greater than or equal to the second preset duration threshold and the log file name of the first log file ends in the second preset field, the first log file being one or more log files in the target subdirectory.
The first preset field and the second preset field are preset by developers and can be subsequently adjusted according to requirements. Illustratively, assume that the first preset field is ". log", the second preset field is ". gz", the first preset duration threshold is 3 days, the second preset duration is 90 days, and the first log file is log file 01. In this case, if the log management container detects that the modification time of the log file 01 exceeds 3 days compared to the current system time, and the log file name ends other than ". log" and ". gz", the log file 01 may be compressed so that the log file name of the compressed log file 01 ends in ". gz".
Or if the log management container detects that the modification time of the log file 01 exceeds 90 days compared with the current system time and the log file name ends in the ". gz", deleting the log file 01.
In the embodiment of the present invention, the log management container may send a container screening instruction to the service manager to instruct the service manager to determine the identification information of the target service container from the started service container, receive the identification information returned by the service manager, and send an attribute query instruction to the service manager to instruct the service manager to query and return the attribute information of the target service container. Further, the log management container may receive attribute information of the target service container returned by the service manager, determine, from the attribute information, a physical machine directory on which the service log directory of the target service container is mounted based on a tag of the target service container, determine a target sub-directory corresponding to the physical machine directory under a preset directory of the log management container, and further manage log files in the target sub-directory. By adopting the mode, the log file in the service container can be managed under the condition of random mounting.
Referring to fig. 4, fig. 4 is a flowchart illustrating another log management method according to an embodiment of the present invention, where the embodiment may be executed by a service manager running on a physical machine, where the physical machine further runs a log management container and at least one service container, and the at least one service container is uniformly managed by the service manager, as shown in the figure, the log management method may include:
s401: receiving a container screening instruction sent by a log manager, wherein the container screening instruction carries a label of a first service container, and the first service container is a service container needing log management.
In one embodiment, the service manager may pre-map a socket file to the log management container, the socket file being a communication interface for communication between the service manager and the log management container.
S402: and determining the identification information of a target service container matched with the label from the started service containers, and returning the identification information to the log management container, wherein the target service container is one or more of the started service containers.
In one embodiment, the identification information may be, for example, an ID of the service container, since the service container is assigned a different ID at each start-up. For this situation, after receiving the container screening instruction carrying the label of the first service container, the service manager may determine the ID of the target service container matching the label from the currently started service container. By adopting the method, the validity of the ID of the determined target service container is ensured.
Exemplarily, it is assumed that the identification information is an ID of a service container, the first service container includes a service container a, a label previously allocated to the service container a is "tomcat _ log ═ log/sub/path", and a currently started service container includes the service container a and the service container B. For this situation, after receiving a container screening instruction carrying a label "tomcat _ log ═ log/sub/path" (i.e., a label of a first service container) sent by a log management container, the service manager may determine, from currently started service containers, a service container a having a label "tomcat _ log ═ log/sub/path" as a target service container, search for an ID of the service container a, and then return the ID of the service container a to the log management container.
S403: and when an attribute query instruction which is sent by the log management container and carries the identification information is received, querying and returning the attribute information of the target service container. The attribute information is used for indicating the log management container to determine a physical machine directory of the service log directory of the target service container from the attribute information based on the label of the target service container, and determining a target subdirectory corresponding to the physical machine directory under a preset directory of the log management container, so as to manage log files in the target subdirectory.
In an embodiment, the service manager may obtain attribute information of each service container in at least one service container, and in this case, when the service manager receives an attribute query instruction carrying the identification information and sent by the log management container, the service manager may query, from the attribute information of each service container, attribute information of a target service container matched with the identification information, and further return the attribute information of the target service container to the log management container.
In one embodiment, the service manager may mount a root directory of the physical machine in advance to a preset directory of the log management container, so that the log management container determines a target subdirectory of the physical machine directory under the root directory, which corresponds to the preset directory. For example, assuming that the root directory of the physical machine is "/", the preset directory of the log management container is "/mnt", and the log management container determines that the physical machine directory mounted by the service log directory of the target service container is:
"/home/docker/volumes/ecs-tasks-sd-files-9-data-b 2ddfda89fb9d3af9a01/_ data". In this case, since the service manager has previously mounted the root directory "/" of the physical machine to the preset directory "/mnt" of the log management container, the log management container may determine that the target subdirectory corresponding to the physical machine directory under the preset directory "/mnt" is:
“/mnt/home/docker/volumes/ecs-tasks-sd-filesystem-9-data-b2ddfda89fb9d3af9a01/_data”。
In the embodiment of the invention, a service manager can receive a container screening instruction which is sent by a log manager and carries a label of a first service container, determine identification information of a target service container matched with the label from a started service container, return the identification information to a log management container, query and return attribute information of the target service container when receiving an attribute query instruction which is sent by the log management container and carries the identification information, so as to indicate the log management container to determine a physical machine directory which is mounted on a service log directory of the target service container from the attribute information based on the label of the target service container, determine a target subdirectory which corresponds to the physical machine directory under a preset directory of the log management container, and manage log files in the target subdirectory. By adopting the mode, the auxiliary log management container can realize the management of the log files in the service container under the condition of random mounting.
Embodiments of the present invention further provide a computer storage medium, in which program instructions are stored, and when the program instructions are executed, the computer storage medium is configured to implement the corresponding method described in the above embodiments.
An embodiment of the present invention further provides a log management apparatus, where the apparatus includes a module configured to execute the method described in fig. 2, and a log management container configured to run on a physical machine, where a service manager and at least one service container are also run on the physical machine, and the at least one service container is managed by the service manager in a unified manner. Specifically, referring to fig. 5, it is a schematic block diagram of a log management apparatus according to an embodiment of the present invention. The log management device of the present embodiment includes:
a communication module 50, configured to send a container screening instruction to the service manager, where the container screening instruction carries a tag of a first service container, so as to instruct the service manager to determine, from a started service container, identification information of a target service container that matches the tag, where the first service container is a service container that needs to be log-managed;
the communication module 50 is further configured to receive the identifier information returned by the service manager, and send an attribute query instruction carrying the identifier information to the service manager, so as to instruct the service manager to query and return the attribute information of the target service container;
The communication module 50 is further configured to receive attribute information of the target service container returned by the service manager;
a processing module 51, configured to determine, from the attribute information, a physical machine directory on which a service log directory of the target service container is mounted based on the tag of the target service container;
the processing module 51 is further configured to determine a target subdirectory corresponding to the physical machine directory under the preset directory of the log management container, and manage log files in the target subdirectory, where the preset directory of the log management container has previously mounted the root directory of the physical machine.
In an embodiment, the processing module 51 is specifically configured to extract a service log directory from the tag of the target service container; screening out a physical machine directory corresponding to the service log directory from the attribute information;
and determining the physical machine directory corresponding to the service log directory as the physical machine directory mounted by the service log directory of the target service container.
In an embodiment, the processing module 51 is further specifically configured to obtain file attributes of the log file in the target subdirectory, where the file attributes include modification time and a log file name; and performing management operation on the log file based on the file attribute, wherein the management operation comprises compressing the log file or deleting the log file.
In an embodiment, the managing operation includes compressing the log files, and the processing module 51 is further configured to, if it is detected that a time interval between a modification time and a system time of a first log file is greater than or equal to a first preset duration threshold and a log file name of the first log file does not end in a first preset field and a second preset field, compress the first log file so that the log file name of the compressed first log file ends in the second preset field, where the first log file is one or more log files in the target subdirectory.
In an embodiment, the management operation includes deleting the log files, and the processing module 51 is further specifically configured to delete a first log file if it is detected that a time interval between the modification time and the system time of the first log file is greater than or equal to a second preset duration threshold and a log file name of the first log file ends in the second preset field, where the first log file is one or more log files in the target subdirectory.
It should be noted that the functions of the functional modules of the log management apparatus described in the embodiment of the present invention may be specifically implemented according to the method in the method embodiment described in the foregoing fig. 2, and a specific implementation process thereof may refer to the related description of the method embodiment of fig. 2, which is not described herein again.
An embodiment of the present invention further provides a log management apparatus, where the apparatus includes a module configured to execute the method described in fig. 4, and a service manager configured to run on a physical machine, where a log management container and at least one service container are also run on the physical machine, and the at least one service container is uniformly managed by the service manager. Specifically, referring to fig. 6, it is a schematic block diagram of another log management apparatus provided in the embodiment of the present invention. The log management device of the present embodiment includes:
a communication module 60, configured to receive a container screening instruction sent by the log manager, where the container screening instruction carries a tag of a first service container, and the first service container is a service container that needs log management;
a processing module 61, configured to determine, from the started service containers, identification information of a target service container that matches the tag, and return the identification information to the log management container;
the communication module 60 is configured to receive an attribute query instruction carrying the identification information and sent by the log management container;
the processing module 61 is further configured to query the attribute information of the target service container when the communication module receives an attribute query instruction which is sent by the log management container and carries the identification information;
The communication module 60 is further configured to return attribute information of the target service container, so as to instruct the log management container to determine, from the attribute information based on the tag of the target service container, a physical machine directory on which a service log directory of the target service container is mounted, determine a target subdirectory corresponding to the physical machine directory under a preset directory of the log management container, and manage log files in the target subdirectory.
In an embodiment, the processing module 61 is further configured to mount a root directory of a physical machine to a preset directory of the log management container, so that the log management container determines, in the preset directory, a target subdirectory corresponding to the physical machine directory under the root directory.
It should be noted that the functions of the functional modules of the log management apparatus described in the embodiment of the present invention may be specifically implemented according to the method in the method embodiment described in the foregoing fig. 4, and a specific implementation process of the functions may refer to the related description of the method embodiment of fig. 4, which is not described herein again.
Referring to fig. 7, fig. 7 is a schematic block diagram of a server provided in an embodiment of the present invention, where the server runs a log management container, a service manager, and at least one service container, where the at least one service container is uniformly managed by the service manager, and as shown in fig. 7, the log management container includes a processor 701, a memory 702, and a transceiver 703. The processor 701, the memory 702, and the transceiver 703 may be connected by a bus or other means, and the bus connection is taken as an example in fig. 7 in the embodiment of the present invention. Wherein the transceiver 703 is controlled by the processor for transceiving messages, the memory 702 is for storing a computer program comprising program instructions, and the processor 701 is for executing the program instructions stored by the memory 702.
The log management container calls the processor 701 to execute: sending a container screening instruction to the service manager through the transceiver 703, where the container screening instruction carries a tag of a first service container, so as to instruct the service manager to determine, from a started service container, identification information of a target service container matched with the tag, where the first service container is a service container requiring log management; receiving, by the transceiver 703, the identifier information returned by the service manager, and sending an attribute query instruction carrying the identifier information to the service manager, so as to instruct the service manager to query and return the attribute information of the target service container; receiving, by the transceiver 703, attribute information of the target service container returned by the service manager, and determining, from the attribute information, a physical machine directory on which a service log directory of the target service container is mounted based on a tag of the target service container; and determining a target subdirectory corresponding to the physical machine directory under the preset directory of the log management container, and managing log files in the target subdirectory, wherein the preset directory of the log management container is pre-mounted with a root directory of the physical machine.
In an embodiment, the processor 701 is specifically configured to extract a service log directory from the tag of the target service container; screening out a physical machine directory corresponding to the service log directory from the attribute information; and determining the physical machine directory corresponding to the service log directory as the physical machine directory mounted by the service log directory of the target service container.
In an embodiment, the processor 701 is further specifically configured to obtain a file attribute of a log file in the target subdirectory, where the file attribute includes modification time and a log file name; and performing management operation on the log file based on the file attribute, wherein the management operation comprises compressing the log file or deleting the log file.
In an embodiment, the management operation includes compressing the log files, and the processor 701 is further specifically configured to, if it is detected that a time interval between a modification time of the first log file and a system time is greater than or equal to a first preset duration threshold, and a log file name of the first log file does not end in a first preset field and a second preset field, compress the first log file so that the log file name of the compressed first log file ends in the second preset field, where the first log file is one or more log files in the target subdirectory.
In an embodiment, the managing operation includes deleting the log file, and the processor 701 is further specifically configured to delete the first log file if it is detected that a time interval between the modification time and the system time of the first log file is greater than or equal to a second preset duration threshold, and a log file name of the first log file ends in the second preset field, where the first log file is one or more log files in the target subdirectory.
Wherein, the service manager calls the processor 701 to execute: receiving, by the transceiver 703, a container screening instruction sent by the log manager, where the container screening instruction carries a label of a first service container, and the first service container is a service container that needs to perform log management; determining the identification information of the target service container matched with the label from the started service container, and returning the identification information to the log management container through the transceiver 703; when receiving an attribute query instruction carrying the identification information, sent by the log management container, through the transceiver 703, querying and returning the attribute information of the target service container through the transceiver 703, so as to instruct the log management container to determine, from the attribute information, a physical machine directory of the service log directory of the target service container based on a tag of the target service container, determine a target subdirectory corresponding to the physical machine directory under a preset directory of the log management container, and manage log files in the target subdirectory.
In an embodiment, the processor 701 is further configured to mount a root directory of the physical machine to a preset directory of the log management container, so that the log management container determines, in the preset directory, a target subdirectory corresponding to the physical machine directory under the root directory.
It should be understood that, in the embodiment of the present invention, the Processor 701 may be a Central Processing Unit (CPU), and the Processor 701 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 702 may include both read-only memory and random access memory and provides instructions and data to the processor 701. A portion of the memory 702 may also include non-volatile random access memory. For example, the memory 702 may also store device type information.
In a specific implementation, the processor 701, the memory 702, and the transceiver 703 described in this embodiment of the present invention may execute the implementation described in the method embodiment shown in fig. 2 or fig. 4 provided in this embodiment of the present invention, or may execute the implementation of the log management apparatus described in fig. 5 or fig. 6 provided in this embodiment of the present invention, which is not described herein again.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
While the present disclosure has been described with reference to a particular embodiment, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure.
Claims (10)
1. A method for managing logs, the method being applied to a log management container running on a physical machine, the physical machine further running a service manager and at least one service container, the at least one service container being uniformly managed by the service manager, the method comprising:
sending a container screening instruction to the service manager, wherein the container screening instruction carries a label of a first service container to indicate the service manager to determine identification information of a target service container matched with the label from a started service container, and the label comprises a service log directory of the target service container;
receiving the identification information returned by the service manager, and sending an attribute query instruction carrying the identification information to the service manager to instruct the service manager to query and return the attribute information of the target service container, wherein the attribute information comprises a container name of the target service container, a network mode of the target service container, and mounting information between the target service container and a physical machine;
receiving attribute information of the target service container returned by the service manager, and determining a physical machine directory mounted by the service log directory of the target service container from the attribute information based on a label of the target service container;
And determining a target subdirectory corresponding to the physical machine directory under the preset directory of the log management container, and managing log files in the target subdirectory, wherein the preset directory of the log management container is pre-mounted with a root directory of the physical machine.
2. The method according to claim 1, wherein the determining, from the attribute information, a physical machine directory on which the service log directory of the target service container is mounted based on the tag of the target service container comprises:
extracting a service log directory from the label of the target service container;
screening out a physical machine directory corresponding to the service log directory from the attribute information;
and determining the physical machine directory corresponding to the service log directory as the physical machine directory mounted by the service log directory of the target service container.
3. The method of claim 1 or 2, wherein managing the log files in the target subdirectory comprises:
acquiring file attributes of log files in the target subdirectory, wherein the file attributes comprise modification time and log file names;
and performing management operation on the log file based on the file attribute, wherein the management operation comprises compressing the log file or deleting the log file.
4. The method of claim 3, wherein the managing operation comprises compressing the log file, and wherein the managing operation on the log file based on the file attribute comprises:
and if the time interval between the modification time and the system time of the first log file is detected to be greater than or equal to a first preset time length threshold value and the log file name of the first log file is not ended in a first preset field and a second preset field, compressing the first log file so that the log file name of the compressed first log file is ended in the second preset field, wherein the first log file is one or more log files in the target subdirectory.
5. The method of claim 3, wherein the managing operation comprises deleting the log file, and wherein the managing operation on the log file based on the file attribute comprises:
and if the time interval between the modification time and the system time of the first log file is detected to be greater than or equal to a second preset time threshold and the log file name of the first log file is ended by the second preset field, deleting the first log file, wherein the first log file is one or more log files in the target subdirectory.
6. A method for managing logs, the method being applied to a service manager running on a physical machine, the physical machine further running a log management container and at least one service container, the at least one service container being uniformly managed by the service manager, the method comprising:
receiving a container screening instruction sent by the log manager, wherein the container screening instruction carries a label of a first service container;
determining identification information of a target service container matched with the label from the started service container, and returning the identification information to the log management container, wherein the label comprises a service log directory of the target service container;
when an attribute query instruction which is sent by the log management container and carries the identification information is received, the attribute information of the target service container is queried and returned, wherein the attribute information comprises a container name of the target service container, a network mode of the target service container and mounting information between the target service container and a physical machine, so that the log management container is instructed to determine a physical machine directory mounted with the service log directory of the target service container from the attribute information based on a label of the target service container, determine a target subdirectory corresponding to the physical machine directory under a preset directory of the log management container, and manage log files in the target subdirectory.
7. The method of claim 6, further comprising:
and mounting the root directory of the physical machine to a preset directory of the log management container so that the log management container determines a target subdirectory corresponding to the physical machine directory under the root directory under the preset directory.
8. A log management apparatus, wherein the apparatus is configured to a log management container running on a physical machine, the physical machine further runs a service manager and at least one service container, and the at least one service container is uniformly managed by the service manager, and the apparatus comprises:
a communication module, configured to send a container screening instruction to the service manager, where the container screening instruction carries a tag of a first service container to instruct the service manager to determine, from a started service container, identification information of a target service container that matches the tag, where the tag includes a service log directory of the target service container;
the communication module is further configured to receive the identifier information returned by the service manager, and send an attribute query instruction carrying the identifier information to the service manager, so as to instruct the service manager to query and return the attribute information of the target service container;
The communication module is further configured to receive attribute information of the target service container returned by the service manager, where the attribute information includes a container name of the target service container, a network mode of the target service container, and mount information between the target service container and a physical machine;
the processing module is used for determining a physical machine directory of the target service container from the attribute information based on the label of the target service container, wherein the physical machine directory is mounted on the service log directory of the target service container;
the processing module is further configured to determine a target subdirectory corresponding to the physical machine directory under the preset directory of the log management container, and manage log files in the target subdirectory, where the preset directory of the log management container has previously mounted the root directory of the physical machine.
9. A log management apparatus configured to a service manager running on a physical machine, the physical machine further running a log management container and at least one service container, the at least one service container being uniformly managed by the service manager, the apparatus comprising:
the communication module is used for receiving a container screening instruction sent by the log manager, wherein the container screening instruction carries a label of a first service container;
The processing module is used for determining the identification information of a target service container matched with the label from the started service container and returning the identification information to the log management container, wherein the label comprises a service log directory of the target service container;
the communication module is used for receiving an attribute query instruction which is sent by the log management container and carries the identification information;
the processing module is further configured to query attribute information of the target service container when the communication module receives an attribute query instruction which is sent by the log management container and carries the identification information, where the attribute information includes a container name of the target service container, a network mode of the target service container, and mount information between the target service container and a physical machine;
the communication module is further configured to return attribute information of the target service container to instruct the log management container to determine, based on the tag of the target service container, a physical machine directory on which the service log directory of the target service container is mounted from the attribute information, determine a target subdirectory corresponding to the physical machine directory under a preset directory of the log management container, and manage log files in the target subdirectory.
10. A server, characterized in that a log management container, a service manager and at least one service container are run on the server, the at least one service container is uniformly managed by the service manager, the log management container comprises a processor and a memory, the processor and the memory are connected with each other, wherein the memory is used for storing a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions, execute the method according to any one of claims 1 to 5 or execute the method according to any one of claims 6 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910937791.3A CN110688201B (en) | 2019-09-29 | 2019-09-29 | Log management method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910937791.3A CN110688201B (en) | 2019-09-29 | 2019-09-29 | Log management method and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110688201A CN110688201A (en) | 2020-01-14 |
| CN110688201B true CN110688201B (en) | 2022-06-28 |
Family
ID=69111079
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910937791.3A Active CN110688201B (en) | 2019-09-29 | 2019-09-29 | Log management method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110688201B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112905537B (en) * | 2021-02-20 | 2022-09-02 | 北京百度网讯科技有限公司 | File processing method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105893010A (en) * | 2015-11-17 | 2016-08-24 | 乐视云计算有限公司 | Host scheduling method and system used for container deployment |
| CN106383852A (en) * | 2016-08-30 | 2017-02-08 | 中国民生银行股份有限公司 | Docker container-based log acquisition method and apparatus |
| CN106789303A (en) * | 2016-12-29 | 2017-05-31 | 北京奇艺世纪科技有限公司 | A kind of container log collection method and device |
| CN108153649A (en) * | 2018-01-23 | 2018-06-12 | 北京粉笔蓝天科技有限公司 | Using the acquisition of daily record and inspection method and its device in docker containers |
| CN108845917A (en) * | 2018-04-09 | 2018-11-20 | 东峡大通（北京）管理咨询有限公司 | File journalization carry module, system and method in container |
| CN109542791A (en) * | 2018-11-27 | 2019-03-29 | 长沙智擎信息技术有限公司 | A kind of program large-scale concurrent evaluating method based on container technique |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10678447B2 (en) * | 2016-07-15 | 2020-06-09 | Red Hat, Inc. | Containerizing a block storage service |
-
2019
- 2019-09-29 CN CN201910937791.3A patent/CN110688201B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105893010A (en) * | 2015-11-17 | 2016-08-24 | 乐视云计算有限公司 | Host scheduling method and system used for container deployment |
| CN106383852A (en) * | 2016-08-30 | 2017-02-08 | 中国民生银行股份有限公司 | Docker container-based log acquisition method and apparatus |
| CN106789303A (en) * | 2016-12-29 | 2017-05-31 | 北京奇艺世纪科技有限公司 | A kind of container log collection method and device |
| CN108153649A (en) * | 2018-01-23 | 2018-06-12 | 北京粉笔蓝天科技有限公司 | Using the acquisition of daily record and inspection method and its device in docker containers |
| CN108845917A (en) * | 2018-04-09 | 2018-11-20 | 东峡大通（北京）管理咨询有限公司 | File journalization carry module, system and method in container |
| CN109542791A (en) * | 2018-11-27 | 2019-03-29 | 长沙智擎信息技术有限公司 | A kind of program large-scale concurrent evaluating method based on container technique |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110688201A (en) | 2020-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108427705B (en) | Electronic device, distributed system log query method and storage medium | |
| CN109299157B (en) | Data export method and device for distributed big single table | |
| CN111367925A (en) | Data dynamic real-time updating method, device and storage medium | |
| CN111339171B (en) | Data query method, device and equipment | |
| EP3026573A1 (en) | Data storage method and apparatus | |
| CN113961510B (en) | File processing method, device, equipment and storage medium | |
| CN112905537A (en) | File processing method and device, electronic equipment and storage medium | |
| CN110688201B (en) | Log management method and related equipment | |
| CN115039090A (en) | Data query method, device, server and storage medium | |
| CN113360519A (en) | Data processing method, device, equipment and storage medium | |
| CN111061802A (en) | Power data management processing method and device and storage medium | |
| CN113495765A (en) | Business data filling method and device, storage medium and electronic device | |
| CN111125226B (en) | Configuration data acquisition method and device | |
| CN115686497A (en) | Business development data management method, development engine, electronic device, and storage medium | |
| CN112000618B (en) | File change management method, device, equipment and storage medium for cluster nodes | |
| CN115004667B (en) | Information pushing method, device, electronic equipment and computer readable medium | |
| CN115794876A (en) | Fragment processing method, device, equipment and storage medium for service data packet | |
| CN102760155A (en) | Database-based transaction control method and device | |
| CN107832195B (en) | Host environment management method, device and system | |
| CN111339173A (en) | Data sharing method, server and readable storage medium | |
| CN111698109A (en) | Method and device for monitoring log | |
| CN110932896A (en) | Method, device and equipment for creating log inverted index and readable storage medium | |
| CN111782428B (en) | Data calling system and method | |
| CN114168417A (en) | K8S-based flink task log processing method and computer equipment | |
| CN115510204B (en) | Intelligent water service data resource catalog management method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |