WO2020019436A1

WO2020019436A1 - Log merging method, apparatus, electronic device, and medium

Info

Publication number: WO2020019436A1
Application number: PCT/CN2018/105540
Authority: WO
Inventors: 王倩
Original assignee: 平安科技（深圳）有限公司
Priority date: 2018-07-25
Filing date: 2018-09-13
Publication date: 2020-01-30
Also published as: CN109165201B; CN109165201A

Abstract

The present application is used in the field of data processing, and provides a log merging method, apparatus, electronic device, and medium; after determining a target container, a log directory is created and log files generated by the target container in each host machine are stored in said log directory; according to the generation time of the data in the log file, the file name of each of the log files is modified, and after sorting the log files on the basis of file name, the data in each log file is merged to generate a data merge file; the total time complexity and the average value of the differences in the amount of data between individual log files during the process of generating data merges are calculated; finally, the total time complexity and the average of the differences are added to the data merge file to generate a log merge file; thus the reasonable integration of a large number of log files is achieved, making it convenient for a user to browse and find log file information; also, during the process of integration, the relevant parameters of the log file are calculated, increasing data utilization.

Description

Method, device, electronic equipment and medium for merging logs

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on July 25, 2018, with application number 201810824218.7, and the invention name is "Merger of Logging and Terminal Equipment", the entire contents of which are incorporated herein by reference. .

Technical field

The present application belongs to the field of data processing, and particularly relates to a method, a device, an electronic device, and a medium for merging logs.

Background technique

Container-based virtualization technology represented by Docker is a hot topic of current virtualization technology. Container technology enables lightweight virtualization by isolating processes and resources. After the container is created and run, the application is deployed. Multiple Docker containers can be deployed on a physical machine. Each Docker container is isolated from each other. This physical machine is generally called the host machine. In the prior art, Docker containers can also run across hosts. When a Docker container runs across hosts, the resulting logs may also be scattered across the hosts. If multiple logs generated by a container are stored in multiple hosts, it is not conducive for the engineering staff to fully understand the progress of the project. The existing log merge schemes are generally simple to merge multiple log data, and then merge them. The log data is often disordered, and it is difficult for engineers to analyze it well. At the same time, there is a lack of research on the overall properties of log data in the process of log merging, which also causes the problem of low utilization of log data.

In summary, the current log merging scheme has the problems that the merged log data is messy and the data utilization rate is low.

technical problem

The current log merging scheme has the problems that the merged log data is messy and the data utilization rate is low.

Technical solutions

A first aspect of the embodiments of the present application provides a method for merging logs, including:

After the target container is determined, a log directory is created, and a log file generated by the target container in each host is stored in the log directory; according to the generation time of the data in the log file, the log file of each of the log files is modified. File name; sorting each of the log files according to the file name of the log file, and sequentially combining data in each of the log files according to the sorted order to generate a data merged file; calculating the total time complexity The total time complexity is used to indicate the number of unit times required to generate the data merge file; an average value of differences is calculated, and the average value of the differences is used to indicate the amount of data between each of the log files The difference in time; adding the total time complexity and the difference average to the data merge file to generate a log merge file.

A second aspect of the embodiments of the present application provides a log merging device, including:

A storage module configured to create a log directory after determining a target container, and store a log file generated by the target container in each host under the log directory; and a modification module configured to Generate time, modify the file names of each of the log files; a merge module for sorting each of the log files according to the file names of the log files, and sequentially sort the Data is merged to generate a data merged file; a first calculation module is configured to calculate a total time complexity, the total time complexity is used to represent a number of unit times required to generate the data merged file; a second calculation A module for calculating an average value of differences, where the average value of differences is used to indicate a difference in data amount between each of the log files; and a generating module is configured to convert the total time complexity and the difference The average value is added to the data merge file to generate a log merge file.

A third aspect of the embodiments of the present application provides a log merging electronic device, including a memory and a processor. The memory stores computer-readable instructions that can run on the processor, and the processor executes all When describing computer-readable instructions, the following steps are implemented:

After the target container is determined, a log directory is created, and a log file generated by the target container in each host is stored in the log directory; according to the generation time of the data in the log file, the File name; sorting each of the log files according to the file name of the log file, and sequentially combining data in each of the log files according to the sorted order to generate a data merged file; calculating the total time complexity The total time complexity is used to indicate the number of unit times required to generate the data merge file; an average value of differences is calculated, and the average value of the differences is used to indicate the amount of data between each of the log files The difference in time; adding the total time complexity and the difference average to the data merge file to generate a log merge file.

A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores computer-readable instructions, and is characterized in that the computer-readable instructions are implemented when executed by at least one processor. The following steps:

Beneficial effect

It is convenient for users to browse and find log file information, and to calculate related parameters of the log file during the integration process, thereby improving data utilization.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present application. For some embodiments, for those of ordinary skill in the art, other drawings can be obtained according to these drawings without paying creative labor.

FIG. 1 is an implementation flowchart of a method for merging logs provided by an embodiment of the present application; FIG.

FIG. 2 is a detailed implementation flowchart of a log merging method S103 provided in an embodiment of the present application;

FIG. 3 is a detailed implementation flowchart of a log merging method S104 according to an embodiment of the present application; FIG.

4 is a structural block diagram of a log merging device according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an electronic device according to an embodiment of the present application.

Embodiments of the invention

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are provided in order to thoroughly understand the embodiments of the present application. However, it should be clear to a person skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary details.

In order to explain the technical solution described in this application, specific examples are used for description below.

FIG. 1 shows an implementation flow of a log merging method provided by an embodiment of the present application, and the method flow includes steps S101 to S106. The specific implementation principle of each step is as follows.

S101: After the target container is determined, a log directory is created, and a log file generated by the target container in each host is stored in the log directory.

As container technology can realize the rapid deployment of programs, it has become a technology often used in people's lives. Container-based virtualization technology represented by Docker is a hot topic of current virtualization technology. Current Docker is an open source application container engine. The application container engine can control the operation of multiple Docker containers. Multiple Docker containers can be deployed in a host, and programs in a Docker container can also run in different hosts.

Understandably, when a program in a Docker container runs on different hosts, the related log files generated will also be stored in different hosts, so it is difficult for users to analyze the overall operation of the Docker container.

Optionally, in the embodiment of the present application, the target container selected by the user is determined by receiving the container IP input by the user. Then create a basedir directory, and according to the host list corresponding to the container ip of the target container, copy and store the log files stored in the host in the host list and generated by the target container under the basedir directory, so that each destination is The log files generated by the target container stored in the host are merged for the first time to provide data support for subsequent merging and analysis processes.

S102: Modify the file name of each log file according to the generation time of the data in the log file.

In the embodiment of the present application, since each row of data in each log file in the log directory needs to be sorted in the subsequent process, in order to reduce the time consumed during sorting, the log file needs to be stored in Modify the file name for each host.

Optionally, the modifying the file name of each log file according to the generation time of the data in the log file includes:

The time corresponding to the timestamp of the first line of data in the log file is taken as the first time of the log file, and the timestamp is used to record the generation time of the data; The time corresponding to the timestamp is taken as the second time of the log file; the first time and the second time are written to the end of the file name of the log file to modify the file name of the log file .

Exemplarily, assuming that the original file name of a log file is: A, if the time stamp corresponding to the time stamp of the first line of data in the log file is 20180101100333, and the time stamp corresponding to the time stamp of the first line of data in the log file Is 20180104090234, the file name of the modified log file is: A (20180101100333, 20180104090234).

S103: Sort each of the log files according to the file name of the log file, and sequentially combine the data in each of the log files according to the sorted order to generate a data merged file.

Understandably, in the embodiment of the present application, sorting is performed by the file names of the log files, and it is not necessary to analyze the timestamp of each row of data in each log file each time data is merged, which is beneficial to reducing the sorting and merging process. time consumed.

As an embodiment of the present application, as shown in FIG. 2, the above S103 includes:

S1031: Sort each of the log files according to the first time in the file name of the log file in ascending order.

Exemplarily, as described in the example above, the file name of one log file is: A (20180101100333, 20180104090234), and the file name of the other log file is: B (20180102000003, 20180105090001). In the embodiment of the present application, according to 20180101100333 And 20180102000003 sorts the two log files. Understandably, according to the sorting rules in the embodiment of the present application, log file A (20180101100333, 20180104090234) will be ranked before log file B (20180102000003, 20180105090001).

S1032: Mark the log files sorted at the first position as the first log file, and mark the log files sorted at the second position as the second log file.

It is worth noting that, in the embodiment of the present application, the second-ranked log file is the initial second-log file. Since this embodiment provides a circular execution process, the second-log file is variable. After each round of calculation, the next log file of the current second log file in the log directory is used as a new second log file, and a new round of execution process is repeated according to the new second log file.

S1033. Store data in the first log file into the data merge file.

Understandably, before the data in the first log file is stored, a data merge file has been created, but before the data in the first log file is stored, there is no data in the data merge file. The embodiment of the present application is actually a process of gradually adding data to a data merge file.

S1034: Determine whether the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last line of data in the data merge file.

Understandably, after the data in the first log file is stored in the data merge file, the data merge file contains multiple lines of data, and each line of data has its corresponding time stamp. Moreover, in the embodiment of the present application, each line of data contained in each log file is arranged from morning to night according to the time represented by the corresponding timestamp, and according to the following merge rules, each line of data in a data merge file is also The times represented by the corresponding timestamps are arranged from morning to night, so in the embodiment of the present application, it is determined whether the first time in the file name of the second log file is later than the data merged file. The time corresponding to the time stamp of the last row of data can determine whether the generation time of all the data in the current second log file is later than the entire data in the current data merge file.

S1035: If the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last row of data in the data merge file, all data in the second log file is unified After the last line of data is stored in the data merge file.

S1036, if the first time in the file name of the second log file is not later than the time corresponding to the timestamp of the last row of data in the data merge file, according to each piece of data in the second log file Inserting each piece of data in the second log file into the data merge file in sequence, so that the time corresponding to the timestamp of each line of data in the inserted data merge file is arranged from small to large.

In this step in the embodiment of the present application, according to the time corresponding to the timestamp of each line of data in the second log file, each line of data is sequentially used as the data to be inserted, and the data to be inserted is inserted into a specific area in the data merge file. This specific The time corresponding to the time stamp of the previous line of data in the region is not later than the time corresponding to the time stamp of the data to be inserted, and the time corresponding to the time stamp of the next line of data in the specific region is not earlier than the time corresponding to the time stamp of the data to be inserted.

S1037: Mark the next log file next to the current second log file as a new second log file according to the order in which the log files in the log directory are sorted, and return to execute and judge the second log file. Whether the first time in the file name is later than the time corresponding to the time stamp of the last row of data in the data merge file, until the data in each of the log files in the log directory is stored in Describe the data merge files.

As described above, this embodiment provides a circular execution process, so the second log file is variable. After each round of calculation, the next log of the current second log file in the log directory will be changed. The file serves as a new second log file, and a new round of execution process is repeated according to the new second log file.

Understandably, through the above-mentioned sorting and merging process, first, the number of times of time comparison of each line of data in the second log file when inserting the data merge file can be reduced; second, because the file name of each log file includes the first time and The second time, so the time of analyzing the timestamp can be shortened during the actual sorting, which is beneficial to improve the efficiency of the single time comparison. Finally, the rows of data in multiple log files are arranged and integrated in an orderly manner, which is beneficial for users to browse. And analyze log file data.

S104. Calculate a total time complexity, where the total time complexity is used to indicate a number of unit times required to generate the data merge file.

Understandably, although the data merge file mentioned above integrates the data originally stored in various log files in different hosts, no new data or parameters are generated. In the embodiment of the present application, in order to calculate each The data characteristics of the log file itself need to calculate the total time complexity, thereby improving the utilization of the data in the log file.

As an embodiment of the present application, as shown in FIG. 3, the above S104 includes:

S1041. Counting that in the process of sequentially combining data in each of the log files, the first time in the file name of the second log file that appears later than the last line of data in the data merge file The number of times corresponding to the timestamp, as the first number.

S1042, it is counted that in the process of sequentially combining data in each of the log files, the first time in the file name of the second log file appears no later than the last line in the data merge file The number of times corresponding to the time stamp of the data, as the second number.

S1043, when the first time in the file name of the second log file appears no later than the time corresponding to the time stamp of the last line of data in the data merge file, record the current time in the second log file The number of data lines is used as the first data line number, and the current number of data lines in the data merge file is recorded as the second data line number.

S1044. Calculate the total time complexity through Formula 1.

Optionally, the formula 1 is:

Calculate the total time complexity, where Ticom represents the total time complexity, the num1 is the first quantity, the num2 is the second quantity, and the linum1 _i represents the i-th occurrence of the The first time in the file name of the second log file is later than the number of the first data lines when the time corresponding to the time stamp of the last line of data in the data merge file, the linum2 _i represents the i-th occurrence The first time in the file name of the second log file is not later than the number of the second data lines when the time corresponding to the time stamp of the last line of data in the data merge file.

Understandably, in case 1, since the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last row of data in the data merge file, only one comparison is required. , All the data in the second log file can be uniformly stored in the last line of data in the data merge file, so the time complexity in this case is 1, so if the second log appears num1 times in total If the first time in the file name of the file is later than the time corresponding to the time stamp of the last row of data in the data merge file, then the total time complexity corresponding to this case is num1.

Understandably, in case 2, when the first time in the file name of the second log file is not later than the time corresponding to the time stamp of the last row of data in the data merge file, the current data merge file is Multiple ordered data are composed, and the current second log file is also composed of multiple ordered data. The calculation rule based on the time complexity of the ordered queue merge: if there are two ordered queues, the data amount of an ordered queue is m, and the data amount of an ordered queue is n, the time complexity of the merge of the two is O (n + m), so in the embodiment of the present application, all cases where the first time in the file name of the second log file is not later than the time corresponding to the time stamp of the last row of data in the data merge file , The total time complexity is

Understandably, the total time complexity calculated by combining the above two cases is

S105. Calculate a difference average value, where the difference average value is used to indicate a difference in data amount between the log files.

Optionally, by formula 2:

Calculate the average of the difference, the logsize _i represents the data amount of the i-th log file from top to bottom according to the order, lognum is the total amount of the log file, and logmean is the average of the difference.

S106. Add the total time complexity and the difference average to the data merge file to generate a log merge file.

Optionally, the total time complexity and the average value of the differences can be added to the first line of the data merge file to more clearly reflect the characteristics of the log file generated by the target container, and it is convenient for users to grasp the overall operation of the target container. Happening.

Understandably, after the target container is determined, a log directory is created, and the log files generated by the target container in each host are stored in the log directory, and each of the modifications is modified according to the generation time of the data in the log file. Log file file names, and after sorting each log file based on the file names, combine the data in each log file to generate a data merge file, calculate the total time complexity in generating the data merge process, and The average value of the difference between the amount of data, and finally add the total time complexity and the average value of the difference to the data merge file to generate a log merge file to achieve reasonable integration of a large number of log files, making it easy for users to browse and find logs File information, and in the process of integration, the relevant parameters of the log file are calculated to improve data utilization.

Corresponding to the log merging method described in the foregoing embodiment, FIG. 4 shows a structural block diagram of the log merging device provided in the embodiment of the present application. For convenience of explanation, only a part related to the embodiment of the application is shown. Referring to FIG. 4, the device includes:

The storage module 401 is configured to create a log directory after the target container is determined, and store a log file generated by the target container in each host under the log directory; and a modification module 402 is configured to store the log file according to the log file. When the data is generated, the file names of each of the log files are modified; a merging module 403 is configured to sort each of the log files according to the file names of the log files, and sequentially sort each of the logs according to the sorted order The data in the files are merged to generate a data merged file. The first calculation module 404 is configured to calculate a total time complexity, where the total time complexity is used to indicate the number of unit times required to generate the data merged file. A second calculation module 405, configured to calculate an average value of the difference, the average value of the difference is used to indicate a difference in the amount of data between each of the log files; and a generation module 406 is used to complicate the total time Degree and the difference average value are added to the data merge file to generate a log merge file.

The time corresponding to the timestamp of the first line of data in the log file is taken as the first time of the log file, and the timestamp is used to record the generation time of the data; The time corresponding to the timestamp is taken as the second time of the log file; the first time and the second time are written to the end of the file name of the log file to modify the file name of the log file.

Optionally, sorting each of the log files according to the file name of the log file, and sequentially combining the data in each of the log files according to the sorted order to generate a data merged file includes: Sorting each of the log files according to the first time in the file name of the log file in ascending order; marking the log file ranked first as the first log file, and The second-ranked log file is marked as a second log file; the data in the first log file is stored in the data merge file; and the second log file in the file name of the second log file is judged. Whether a time is later than the time corresponding to the time stamp of the last line of data in the data merge file;

If the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last row of data in the data merge file, all the data in the second log file are stored in a unified manner After the last line of data in the data merge file; if the first time in the file name of the second log file is no later than the time corresponding to the time stamp of the last line of data in the data merge file, The timestamp of each piece of data in the second log file is described, and each piece of data in the second log file is sequentially inserted into the data merge file, so that the timestamp of each line of data in the inserted data merge file The corresponding time is arranged from small to large; according to the sort order of each log file in the log directory, the next log file of the current second log file is marked as a new second log file, and returned to execute An operation of judging whether the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last line of data in the data merge file, until the log destination The data in each of the recorded log files is stored in the data merge file.

Optionally, the calculating the total time complexity includes:

In the process of sequentially merging data in each of the log files, the time when the first time in the file name of the second log file appears later than the last line of data in the data merge file is counted The number of times corresponding to the timestamp is taken as the first number; in the process of sequentially merging data in each of the log files, the first time in the file name of the second log file appears Number of times later than the time corresponding to the time stamp of the last line of data in the data merge file, as the second number; when the first time in the file name of the second log file appears, the data merge is not later than the data When the time corresponding to the time stamp of the last line of data in the file is recorded, the current number of data lines in the second log file is recorded as the first data line number, and the current number of data lines in the data merge file is recorded as Number of second data rows; by formula:

Optionally, the calculating the average of the difference includes:

By formula:

FIG. 5 is a schematic diagram of an electronic device according to an embodiment of the present application. As shown in FIG. 5, the electronic device 5 of this embodiment includes: a processor 50, a memory 51, and computer-readable instructions 52 stored in the memory 51 and executable on the processor 50, such as a merge of logs. program. When the processor 50 executes the computer-readable instructions 52, the steps in the embodiment of the method for merging the above logs are implemented, for example, steps 101 to 106 shown in FIG. Alternatively, when the processor 50 executes the computer-readable instructions 52, the functions of the modules / units in the foregoing device embodiments are implemented, for example, the functions of the units 401 to 406 shown in FIG. 4.

Exemplarily, the computer-readable instructions 52 may be divided into one or more modules / units, the one or more modules / units are stored in the memory 51 and executed by the processor 50, To complete this application. The one or more modules / units may be a series of computer-readable instruction instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer-readable instructions 52 in the electronic device 5.

The electronic device 5 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The electronic device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art can understand that FIG. 5 is only an example of the electronic device 5 and does not constitute a limitation on the electronic device 5. It may include more or fewer components than shown in the figure, or combine some components or different components. For example, the electronic device may further include an input / output device, a network access device, a bus, and the like.

The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 51 may be an internal storage unit of the electronic device 5, such as a hard disk or a memory of the electronic device 5. The memory 51 may also be an external storage device of the electronic device 5, such as a plug-in hard disk, a Smart Media Card (SMC), and a Secure Digital (SD) provided on the electronic device 5. Card, flash card, etc. Further, the memory 51 may further include both an internal storage unit of the electronic device 5 and an external storage device. The memory 51 is configured to store the computer-readable instructions and other programs and data required by the electronic device. The memory 51 may also be used to temporarily store data that has been output or is to be output.

In the above embodiments, the description of each embodiment has its own emphasis. For a part that is not detailed or recorded in an embodiment, reference may be made to related descriptions of other embodiments.

The above-mentioned embodiments are only used to describe the technical solution of the present application, but are not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still implement the foregoing implementations. The technical solutions described in the examples are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of this application.

Claims

A method for merging logs, including:

After the target container is determined, a log directory is created, and a log file generated by the target container in each host is stored in the log directory;

Modify the file name of each log file according to the generation time of the data in the log file;

Sorting each of the log files according to the file name of the log file, and sequentially combining data in each of the log files according to the sorted order to generate a data merged file;

Calculating a total time complexity, where the total time complexity is used to represent the number of unit times required to generate the data merge file;

Calculating a difference average value, where the difference average value is used to indicate a difference in data amount between each of the log files;

Adding the total time complexity and the difference average to the data merge file to generate a log merge file.
The method for merging logs according to claim 1, wherein the modifying the file name of each log file according to the generation time of the data in the log file comprises:

Taking the time corresponding to the time stamp of the first line of data in the log file as the first time of the log file, where the time stamp is used to record the generation time of the data;

Taking the time corresponding to the time stamp of the last line of data in the log file as the second time of the log file;

Writing the first time and the second time to the end of the file name of the log file to modify the file name of the log file.
The method for merging logs according to claim 2, wherein each of the log files is sorted according to the file name of the log file, and the log files in each of the log files are sequentially sorted according to the sorted order. Data is merged to generate a data merge file, including:

Sorting each of the log files according to the order of the first time in the file name of the log file from small to large;

Mark the log files sorted in the first position as the first log file, and mark the log files sorted in the second position as the second log file;

Storing data in the first log file into the data merge file;

Determining whether the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last line of data in the data merge file;

If the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last row of data in the data merge file, all the data in the second log file are stored in a unified manner After the data of the last line of the data merge file;

If the first time in the file name of the second log file is not later than the time corresponding to the time stamp of the last row of data in the data merge file, the time of each piece of data in the second log file is Stamp, insert each piece of data in the second log file into the data merge file in order, so that the time corresponding to the time stamp of each line of data in the inserted data merge file is arranged from small to large;

Mark the log file next to the current second log file as a new second log file according to the order of sorting of each log file in the log directory, and return to execute the judgment of the second log file Whether the first time in the name is later than the time corresponding to the time stamp of the last line of data in the data merge file, until the data in each of the log files in the log directory is stored in the data Merge files.
The method for merging logs according to claim 3, wherein the calculating the total time complexity comprises:

In the process of sequentially merging data in each of the log files, the time when the first time in the file name of the second log file appears later than the last line of data in the data merge file is counted The number of times corresponding to the time stamp, as the first number;

In the process of sequentially merging data in each of the log files, it appears that the first time in the file name of the second log file is no later than the last line of data in the data merge file. The number of times corresponding to the timestamp, as the second number;

When the first time in the file name of the second log file appears no later than the time corresponding to the time stamp of the last line of data in the data merge file, the current data in the second log file is recorded. Number of lines, as the number of first data lines, recording the current number of data lines in the data merge file, as the number of second data lines;

By formula:
Calculate the total time complexity, where Ticom represents the total time complexity, the num1 is the first quantity, the num2 is the second quantity, and the linum1 i represents the i-th occurrence of the The first time in the file name of the second log file is later than the number of the first data lines when the time corresponding to the time stamp of the last line of data in the data merge file, the linum2 i represents the i-th occurrence The first time in the file name of the second log file is not later than the number of the second data lines when the time corresponding to the time stamp of the last line of data in the data merge file.
The method for merging logs according to claim 1, wherein the calculating the average value of the differences comprises:

By formula:
Calculate the average of the difference, the logsize i represents the data amount of the i-th log file from top to bottom according to the order, lognum is the total amount of the log file, and logmean is the average of the difference.
A log merging device, comprising:

A storage module, configured to create a log directory after determining a target container, and store a log file generated by the target container in each host machine under the log directory;

A modification module, configured to modify a file name of each of the log files according to a generation time of data in the log files;

A merging module, configured to sort each of the log files according to the file name of the log file, and sequentially combine the data in each of the log files according to the sorted order to generate a data merge file;

A first calculation module, configured to calculate a total time complexity, where the total time complexity is used to represent a number of unit times required to generate the data merge file;

A second calculation module, configured to calculate an average value of differences, where the average value of differences is used to indicate a difference in data amount between the log files;

A generating module is configured to add the total time complexity and the average value of the differences to the data merge file to generate a log merge file.
The log merging device according to claim 6, wherein the modifying the file name of each of the log files according to the generation time of the data in the log file comprises:

The time corresponding to the timestamp of the first line of data in the log file is taken as the first time of the log file, and the timestamp is used to record the generation time of the data; The time corresponding to the timestamp is taken as the second time of the log file; the first time and the second time are written to the end of the file name of the log file to modify the file name of the log file.
The apparatus for merging logs according to claim 7, wherein the sorting of each of the log files according to the file name of the log file, and sequentially sorting the contents of each of the log files according to the sorting order. Data is merged to generate a data merge file, including:

Sorting each of the log files according to the order of the first time in the file name of the log file from small to large;

Mark the log files sorted in the first position as the first log file, and mark the log files sorted in the second position as the second log file;

Storing data in the first log file into the data merge file;

Determining whether the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last line of data in the data merge file;

If the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last row of data in the data merge file, all the data in the second log file are stored in a unified manner After the data of the last line of the data merge file;

If the first time in the file name of the second log file is not later than the time corresponding to the time stamp of the last row of data in the data merge file, the time of each piece of data in the second log file is Stamp, insert each piece of data in the second log file into the data merge file in order, so that the time corresponding to the time stamp of each line of data in the inserted data merge file is arranged from small to large;

Mark the log file next to the current second log file as a new second log file according to the order of sorting of each log file in the log directory, and return to execute the judgment of the second log file Whether the first time in the name is later than the time corresponding to the time stamp of the last line of data in the data merge file, until the data in each of the log files in the log directory is stored in the data Merge files.
The apparatus for merging logs according to claim 8, wherein the calculating the total time complexity comprises:

In the process of sequentially merging data in each of the log files, the time when the first time in the file name of the second log file appears later than the last line of data in the data merge file is counted The number of times corresponding to the time stamp, as the first number;

In the process of sequentially merging data in each of the log files, it appears that the first time in the file name of the second log file is no later than the last line of data in the data merge file. The number of times corresponding to the timestamp, as the second number;

When the first time in the file name of the second log file appears no later than the time corresponding to the time stamp of the last line of data in the data merge file, the current data in the second log file is recorded. Number of lines, as the number of first data lines, recording the current number of data lines in the data merge file, as the number of second data lines;

By formula:
Calculate the total time complexity, where Ticom represents the total time complexity, the num1 is the first quantity, the num2 is the second quantity, and the linum1 i represents the i-th occurrence of the The first time in the file name of the second log file is later than the number of the first data lines when the time corresponding to the time stamp of the last line of data in the data merge file, the linum2 i represents the i-th occurrence The first time in the file name of the second log file is not later than the number of the second data lines when the time corresponding to the time stamp of the last line of data in the data merge file.
The log merging device according to claim 6, wherein the calculating the average value of the differences comprises:

By formula:
Calculate the average of the difference, the logsize i represents the data amount of the i-th log file from top to bottom according to the order, lognum is the total amount of the log file, and logmean is the average of the difference.
An electronic device for merging logs, characterized in that the electronic device for merging logs includes a memory and a processor, and the memory stores computer-readable instructions that can be run on the processor, and the processor When the computer-readable instructions are executed, the following steps are implemented:

After the target container is determined, a log directory is created, and a log file generated by the target container in each host is stored in the log directory;

Modify the file name of each log file according to the generation time of the data in the log file;

Sorting each of the log files according to the file name of the log file, and sequentially combining data in each of the log files according to the sorted order to generate a data merged file;

Calculating a total time complexity, where the total time complexity is used to represent the number of unit times required to generate the data merge file;

Calculating a difference average value, where the difference average value is used to indicate a difference in data amount between each of the log files;

Adding the total time complexity and the difference average to the data merge file to generate a log merge file.
The electronic device for merging logs according to claim 11, wherein the modifying the file name of each log file according to the generation time of the data in the log file comprises:

Taking the time corresponding to the time stamp of the first line of data in the log file as the first time of the log file, where the time stamp is used to record the generation time of the data;

Taking the time corresponding to the time stamp of the last line of data in the log file as the second time of the log file;

Writing the first time and the second time to the end of the file name of the log file to modify the file name of the log file.
The electronic device for merging logs according to claim 12, wherein each of the log files is sorted according to a file name of the log file, and each of the log files is sequentially sorted according to a sorted order. Data to merge to generate a data merge file, including:

Sorting each of the log files according to the order of the first time in the file name of the log file from small to large;

Mark the log files sorted in the first position as the first log file, and mark the log files sorted in the second position as the second log file;

Storing data in the first log file into the data merge file;

Determining whether the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last line of data in the data merge file;

If the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last row of data in the data merge file, all the data in the second log file are stored in a unified manner After the data of the last line of the data merge file;

If the first time in the file name of the second log file is not later than the time corresponding to the time stamp of the last row of data in the data merge file, the time of each piece of data in the second log file is Stamp, insert each piece of data in the second log file into the data merge file in order, so that the time corresponding to the time stamp of each line of data in the inserted data merge file is arranged from small to large;

Mark the log file next to the current second log file as a new second log file according to the order of sorting of each log file in the log directory, and return to execute the judgment of the second log file Whether the first time in the name is later than the time corresponding to the time stamp of the last line of data in the data merge file, until the data in each of the log files in the log directory is stored in the data Merge files.
The electronic device for merging logs according to claim 13, wherein the calculating the total time complexity comprises:

In the process of sequentially merging data in each of the log files, the time when the first time in the file name of the second log file appears later than the last line of data in the data merge file is counted The number of times corresponding to the time stamp, as the first number;

In the process of sequentially merging data in each of the log files, it appears that the first time in the file name of the second log file is no later than the last line of data in the data merge file. The number of times corresponding to the timestamp, as the second number;

When the first time in the file name of the second log file appears no later than the time corresponding to the time stamp of the last line of data in the data merge file, the current data in the second log file is recorded. Number of lines, as the number of first data lines, recording the current number of data lines in the data merge file, as the number of second data lines;

By formula:
Calculate the total time complexity, where Ticom represents the total time complexity, the num1 is the first quantity, the num2 is the second quantity, and the linum1 i represents the i-th occurrence of the The first time in the file name of the second log file is later than the number of the first data lines when the time corresponding to the time stamp of the last line of data in the data merge file, the linum2 i represents the i-th occurrence The first time in the file name of the second log file is not later than the number of the second data lines when the time corresponding to the time stamp of the last line of data in the data merge file.
The electronic device for merging logs according to claim 11, wherein the calculating the average value of the differences comprises:

By formula:
Calculate the average of the difference, the logsize i represents the data amount of the i-th log file from top to bottom according to the order, lognum is the total amount of the log file, and logmean is the average of the difference.
A computer-readable storage medium storing computer-readable instructions, wherein the computer-readable instructions implement the following steps when executed by at least one processor:

After the target container is determined, a log directory is created, and a log file generated by the target container in each host is stored in the log directory;

Modify the file name of each log file according to the generation time of the data in the log file;

Sorting each of the log files according to the file name of the log file, and sequentially combining data in each of the log files according to the sorted order to generate a data merged file;

Calculating a total time complexity, where the total time complexity is used to represent the number of unit times required to generate the data merge file;

Calculating a difference average value, where the difference average value is used to indicate a difference in data amount between each of the log files;

Adding the total time complexity and the difference average to the data merge file to generate a log merge file.
The computer-readable storage medium according to claim 16, wherein the modifying the file name of each of the log files according to the generation time of the data in the log file comprises:

Taking the time corresponding to the time stamp of the first line of data in the log file as the first time of the log file, where the time stamp is used to record the generation time of the data;

Taking the time corresponding to the time stamp of the last line of data in the log file as the second time of the log file;

Writing the first time and the second time to the end of the file name of the log file to modify the file name of the log file.
The computer-readable storage medium according to claim 17, wherein each of the log files is sorted according to a file name of the log file, and each of the log files is sequentially sorted according to a sorted order. Data to merge to generate a data merge file, including:

Sorting each of the log files according to the order of the first time in the file name of the log file from small to large;

Mark the log files sorted in the first position as the first log file, and mark the log files sorted in the second position as the second log file;

Storing data in the first log file into the data merge file;

Determining whether the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last line of data in the data merge file;

If the first time in the file name of the second log file is later than the time corresponding to the time stamp of the last row of data in the data merge file, all the data in the second log file are stored in a unified manner After the data of the last line of the data merge file;

If the first time in the file name of the second log file is not later than the time corresponding to the time stamp of the last row of data in the data merge file, the time of each piece of data in the second log file is Stamp, insert each piece of data in the second log file into the data merge file in order, so that the time corresponding to the time stamp of each line of data in the inserted data merge file is arranged from small to large;

Mark the log file next to the current second log file as a new second log file according to the order of sorting of each log file in the log directory, and return to execute the judgment of the second log file Whether the first time in the name is later than the time corresponding to the time stamp of the last line of data in the data merge file, until the data in each of the log files in the log directory is stored in the data Merge files.
The computer-readable storage medium of claim 18, wherein the calculating the total time complexity comprises:

In the process of sequentially merging data in each of the log files, the time when the first time in the file name of the second log file appears later than the last line of data in the data merge file is counted The number of times corresponding to the time stamp, as the first number;

In the process of sequentially merging data in each of the log files, it appears that the first time in the file name of the second log file is no later than the last line of data in the data merge file. The number of times corresponding to the timestamp, as the second number;

When the first time in the file name of the second log file appears no later than the time corresponding to the time stamp of the last line of data in the data merge file, the current data in the second log file is recorded. Number of lines, as the number of first data lines, recording the current number of data lines in the data merge file, as the number of second data lines;

By formula:
Calculate the total time complexity, where Ticom represents the total time complexity, the num1 is the first quantity, the num2 is the second quantity, and the linum1 i represents the i-th occurrence of the The first time in the file name of the second log file is later than the number of the first data lines when the time corresponding to the time stamp of the last line of data in the data merge file, the linum2 i represents the i-th occurrence The first time in the file name of the second log file is not later than the number of the second data lines when the time corresponding to the time stamp of the last line of data in the data merge file.
The computer-readable storage medium of claim 16, wherein the calculating the average of the differences comprises:

By formula:
Calculate the average of the difference, the logsize i represents the data amount of the i-th log file from top to bottom according to the order, lognum is the total amount of the log file, and logmean is the average of the difference.