CN111258964A - Log processing method and device, storage medium and electronic equipment - Google Patents
Log processing method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN111258964A CN111258964A CN201811468637.8A CN201811468637A CN111258964A CN 111258964 A CN111258964 A CN 111258964A CN 201811468637 A CN201811468637 A CN 201811468637A CN 111258964 A CN111258964 A CN 111258964A
- Authority
- CN
- China
- Prior art keywords
- log
- newly added
- processed
- browser
- processing method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention relates to a log processing method and a device, belonging to the technical field of data processing, wherein the method comprises the following steps: receiving a log to be processed, and judging whether the log to be processed is a newly added log according to a file path of the log to be processed; if the log to be processed is a newly added log, arranging the newly added log according to a preset arrangement sequence to obtain a synchronous refreshing log; and synchronizing the synchronous refreshing log to a browser so that the browser displays the synchronous refreshing log. The method solves the problem of overlarge load pressure of the server caused by excessive logs in the prior art, reduces the number of logs in the server, and simultaneously improves the speed of sequencing the newly added logs.
Description
Technical Field
The embodiment of the invention relates to the technical field of data processing, in particular to a log processing method, a log processing device, a computer readable storage medium and electronic equipment.
Background
With the prevalence of internet technologies such as distributed systems and microservice architectures, log data management is more and more performed due to the distributed storage of log information.
Most of the existing log collection management methods adopt a centralized management method. For example, common log management methods include: using open-source framework ELK (elastic search + logstack + Kibana); the logstack (data collection engine 101) may be used to collect logs, collect the collected logs into an ElasticSearch (search server 102), retrieve the data by using the ElasticSearch, and display the retrieved log information by using Kibana (analysis and visualization platform 103), where a specific architecture of the ELK may be as shown in fig. 1.
However, the above ELK architecture has the following disadvantages: on one hand, the Logstash collects a large amount of logs and collects the logs into the ElasticSearch, so that the retrieval is time-consuming, and the load pressure of the server is increased due to excessive logs; on the other hand, for a light-weight scene, if the architecture is used for processing the logs, resources are wasted and the value is not large; furthermore, the configuration of the ELK is complicated, and thus more human resources are wasted.
Therefore, it is desirable to provide a new log processing method and apparatus.
It is to be noted that the information invented in the above background section is only for enhancing the understanding of the background of the present invention, and therefore, may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The present invention is directed to a log processing method, a log processing apparatus, a computer-readable storage medium, and an electronic device, which overcome, at least to some extent, the problem of excessive server-side load stress caused by the limitations and disadvantages of the related art.
According to an aspect of the present disclosure, there is provided a log processing method including:
receiving a log to be processed, and judging whether the log to be processed is a newly added log according to a file path of the log to be processed;
if the log to be processed is a newly added log, arranging the newly added log according to a preset arrangement sequence to obtain a synchronous refreshing log;
and synchronizing the synchronous refreshing log to a browser so that the browser displays the synchronous refreshing log.
In an exemplary embodiment of the disclosure, the preset arrangement order includes one or more of a descending order of update time of the newly added log, an ascending order of update time of the newly added log, and a magnitude of correlation between the newly added log and the history log.
In an exemplary embodiment of the present disclosure, the arranging the newly added logs according to a preset arrangement order to obtain a synchronous refresh log includes:
and arranging the newly added logs according to the updating time of the newly added logs in a descending order to obtain the synchronous refreshing log.
In an exemplary embodiment of the present disclosure, synchronizing the synchronized refresh log to a browser includes:
and synchronizing the synchronous refreshing log to the browser through a socket input and output interface.
In an exemplary embodiment of the present disclosure, receiving the pending log includes:
establishing a transmission control protocol with a log reaper;
and receiving the log to be processed sent by the log harvester through the transmission control protocol.
In an exemplary embodiment of the present disclosure, before synchronizing the synchronous refresh log to a browser, the log processing method further includes:
creating a log stream and a node corresponding to the log stream;
storing the synchronous refreshing log into a log stream corresponding to the file path;
and placing the log stream corresponding to the file path into a node corresponding to the log stream.
In an exemplary embodiment of the present disclosure, synchronizing the synchronization refresh log to a browser through a socket input output interface includes:
and synchronizing the synchronous refreshing log to a browser through a socket input and output interface according to the log stream to which the synchronous refreshing log belongs and the node corresponding to the log stream.
According to an aspect of the present disclosure, there is provided a log processing apparatus including:
the judging module is used for receiving the log to be processed and judging whether the log to be processed is a newly added log according to the file path of the log to be processed;
the arranging module is used for arranging the newly added logs according to a preset arranging sequence to obtain synchronous refreshing logs if the logs to be processed are newly added logs;
and the synchronization module is used for synchronizing the synchronous refreshing log to a browser so that the browser can display the synchronous refreshing log.
According to an aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a log processing method as described in any one of the above.
According to an aspect of the present disclosure, there is provided an electronic device including:
a processor; and
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform any one of the log processing methods described above via execution of the executable instructions.
The invention relates to a log processing method, which comprises the steps of judging whether a log to be processed is a newly added log according to a file path of the log to be processed; when the log to be processed is judged to be a newly added log, arranging the newly added log according to a preset arrangement sequence to obtain a synchronous refreshing log; finally, synchronizing the synchronous refreshing log to the browser so that the browser can display the synchronous refreshing log; on one hand, when the to-be-processed logs are judged to be newly added logs, the newly added logs are arranged according to the preset arrangement sequence to obtain the synchronous refreshing logs, so that the problem of overlarge load pressure of the server caused by excessive logs in the prior art is solved, the number of the logs in the server is reduced, and the sorting speed of the newly added logs is improved; on the other hand, by synchronizing the synchronous refreshing log to the browser, the browser can acquire the synchronous refreshing log without sending a request, so that the log synchronization and log display speed is improved, a user can check the log in time and solve the problems in the log, and the error rate of the system is reduced; on the other hand, the newly added logs are arranged according to the preset arrangement sequence to obtain the synchronous refreshing logs, so that the user can directly obtain the required logs without blind checking, the efficiency of checking the logs by the user is improved, and the user experience is further improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 schematically shows a structural framework diagram of a log processing.
Fig. 2 schematically shows a flow chart of a method of log processing.
Fig. 3 schematically illustrates an application scenario diagram of a log processing method.
Fig. 4 schematically shows a flow chart of another log processing method.
Fig. 5 schematically shows an example diagram of a log stream and nodes.
Fig. 6 schematically shows another example diagram of a log stream and nodes.
Fig. 7 schematically shows a flow chart of another log processing method.
Fig. 8 schematically shows a block diagram of a log processing apparatus.
Fig. 9 schematically shows an example block diagram of an electronic device for implementing the log processing method described above.
Fig. 10 schematically illustrates a computer-readable storage medium for implementing the log processing method described above.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the invention.
Furthermore, the drawings are merely schematic illustrations of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
Commercial versions of log collection tools are also widely used, most notably Splunk. Splunk, Inc. in the United states, established in 2004, was released to the market in 2012, Nasdake, the first big data release company, honored numerous awards and crimes. Headquarters is located in san francisco, usa, london is the international headquarters, hong kong is provided with a asia tai support center, and shanghai is provided with the first research and development center overseas. The Splunk data acquisition tool also adopts a distributed deployment mode, has complete functions, basically takes a data engine as a basis, can realize data acquisition and analysis, but is expensive and not suitable for light-weight scenes.
The example embodiment first provides a log processing method, which may be executed in a server, a server cluster, a cloud server, or the like; of course, those skilled in the art may also operate the method of the present invention on other platforms as needed, and this is not particularly limited in this exemplary embodiment. Referring to fig. 2, the log processing method may include the steps of:
step S210, receiving a log to be processed, and judging whether the log to be processed is a newly added log according to a file path of the log to be processed.
Step S220, if the log to be processed is a newly added log, arranging the newly added log according to a preset arrangement sequence to obtain a synchronous refreshing log.
And step S230, synchronizing the synchronous refreshing log to a browser so that the browser displays the synchronous refreshing log.
In the log processing method, on one hand, when the to-be-processed logs are judged to be the newly added logs, the newly added logs are arranged according to the preset arrangement sequence to obtain the synchronously refreshed logs, so that the problem of overlarge load pressure of the server caused by excessive logs in the prior art is solved, the number of logs in the server is reduced, and the sorting speed of the newly added logs is improved; on the other hand, by synchronizing the synchronous refreshing log to the browser, the browser can acquire the synchronous refreshing log without sending a request, so that the log synchronization and log display speed is improved, a user can check the log in time and solve the problems in the log, and the error rate of the system is reduced; on the other hand, the newly added logs are arranged according to the preset arrangement sequence to obtain the synchronous refreshing logs, so that the user can directly obtain the required logs without blind checking, the efficiency of checking the logs by the user is improved, and the user experience is further improved.
Hereinafter, each step in the above-described log processing method in the present exemplary embodiment will be explained and explained in detail with reference to the drawings.
In step S210, a log to be processed is received, and whether the log to be processed is a newly added log is determined according to a file path of the log to be processed.
In the present exemplary embodiment, first, a pending log sent by the log reaper is received. Receiving the to-be-processed log sent by the log reaper may include: establishing a transmission control protocol with a log reaper; and receiving the log to be processed sent by the log harvester through the transmission control protocol. In detail:
referring to fig. 3, first, a TCP (Transmission Control Protocol) is established between a Server (Server)301 and a log Harvester 302 (harvest); wherein, the transmission control protocol can be sent through Telnet (remote terminal protocol); further, the transmission process of Telnet (remote terminal protocol) may be as follows:
[root@symfred1～]#telnet localhost 28777
Trying 127.0.0.1...
Connected to localhost.
Escape character is'^]'.
then, after the TCP protocol is created, the log receiver 302 sends the collected to-be-processed log to the server 301 through the TCP protocol; after receiving the to-be-processed log, the server 301 determines whether the to-be-processed log is a newly added log according to the file path of the to-be-processed log. The determining whether the log to be processed is a newly added log according to the file path of the log to be processed may include: comparing the file path of the log to be processed with the file path of the historical log, and if the file path of the log to be processed exists in the file path of the historical log, judging that the log to be processed is the historical log; if the file path of the log to be processed does not exist in the file path of the historical log, the log to be processed can be judged to be a newly added log; the file path may include log generation time, and may also include other information, for example, the address of the terminal and the name of the application to which the file path belongs, which is not limited in this example. It should be further added here that, when the log to be processed is a history log, in order to reduce the load pressure of the server, the log to be processed may be deleted; for backup, the data may be stored in other databases, which is not limited in this example.
In step S220, if the to-be-processed log is a newly added log, the newly added log is arranged according to a preset arrangement order to obtain a synchronous refresh log.
In this exemplary embodiment, the preset arrangement order may include a descending order of the update time of the new journal, an ascending order of the update time of the new journal, a magnitude of a correlation between the new journal and the history journal, and the like, and may also include other arrangement orders, for example, an importance level of the journal (for example, may include debugging, information, warning, error, and major error, and the like), and the like, which is not limited in this example. The arranging the newly added logs according to the preset arrangement order to obtain the synchronous refresh log may include: and arranging the newly added logs according to the updating time of the newly added logs in a descending order to obtain the synchronous refreshing log. In detail:
the newly added logs may be sorted in a descending order or in an ascending order according to the time of log generation included in the file path of each newly added log, which is not limited in this example.
In step S230, the synchronous refresh log is synchronized to a browser, so that the browser displays the synchronous refresh log.
In the present exemplary embodiment, with continued reference to fig. 3, after obtaining the synchronous refresh log, the server 201 synchronizes the synchronous refresh log to the browser 303. Wherein synchronizing the synchronous refresh log to the browser may comprise: and synchronizing the synchronous refreshing log to the browser through a socket input and output interface. In detail:
firstly, establishing socket input and output interface (socket io) communication between the server 301 and the browser 303; after the socket io communication between the server 301 and the browser 303 is established, the server 301 synchronizes the synchronous refresh log to the browser 303 through the socket io. It should be added that, the synchronous refresh log is synchronized to the browser 303 by using the socket io, so that the synchronous refresh log is synchronized to the browser in time after being obtained, and then the synchronous refresh log can be displayed by the browser, thereby reducing the time for synchronizing the log, so that relevant personnel can check the log in time, and debug the system in time when the log has an error, thereby improving the performance of the system.
Fig. 4 schematically shows another log processing method. Referring to fig. 4, the log processing method may further include steps S410 to S430. Wherein:
in step S410, a log stream and a node corresponding to the log stream are created.
In the present exemplary embodiment, first, a log stream (stream) is created; each log stream (stream) can be used as a channel to transmit monitoring information of the log in real time, and can also be used as a broadcasting station to transmit broadcasting information in real time; then, a node (node) corresponding to the log stream (stream) is created; the node (node) may be a management container of a log stream (stream), and one node (node) may include one log stream (stream) or a plurality of log streams (streams). Further, when a node (node) is deleted, the log stream (stream) belonging to the node (node) will also be deleted. By adopting the mode, the method is convenient for viewing the log stream (stream) under each node (node), and saves the viewing time of the log stream (stream). Further, as shown in fig. 5, the nodes (nodes) may include test1, test2, and so on; the node (node) test2 may include log streams (streams) s1, s2, s3, and the like, and this example is not particularly limited thereto.
It should be further added that, when a node is created, an empty log stream (stream) is also created; for example, may be + node | test 1; further, a node is created, and two log streams s1 and s2 can be bound; for example, it may be + node | test2| s1, s 2; further, a stream s3 may be added to the node; for example, it may be + stream | s3| test 2; further, one stream is deleted, and here, a node to which the stream belongs must be specified; for example, it may be-stream | s3| test 2; finally, a log can be added on s2 of test2, for example, it can be + log | s2| test2| info | this is a message; specific examples of which fig. 6 may be referred to.
In step S420, the synchronous refresh log is stored in the log stream corresponding to the file path.
In this example embodiment, when the creation of the log stream and the creation of the node corresponding to the log stream are completed, the synchronous flush log may be stored in the log stream corresponding to the file path. For example:
for example, when the user a needs to monitor a certain log, the synchronous refresh log needed by the user may be stored in the log stream corresponding to the file path corresponding to the synchronous refresh log. It should be noted that, for the convenience of storage, the correspondence between the file path and the log stream and the correspondence between the log stream and the node may be stored.
In step S430, the log stream corresponding to the file path is placed into the node corresponding to the log stream.
In the present exemplary embodiment, after the synchronous flush log is stored in the log stream corresponding to the file path, the log stream corresponding to the file path is placed in the node corresponding to the log stream. Further, after the log stream is stored, synchronizing the synchronous refreshing log to a browser through a socket input and output interface according to the log stream to which the synchronous refreshing log belongs and the node corresponding to the log stream, and reminding a related user to check. By the method, the speed of checking the related synchronous refreshing logs by the user is greatly improved, and further the user experience is further improved.
In addition, it should be added that the default reaper implements the function of monitoring a single file. However, in daily situations, the logs are all placed in a file directory, if the change of the whole file directory is monitored, the default reaper cannot be used, but the known interface between the reaper and the server is TCP, and the method can be realized by writing a program by self.
When a file is created in the monitoring directory, stream is automatically established. When a file is created in the monitoring directory, stream is automatically established. When a file is deleted, stream is also deleted. When a file is renamed, the corresponding stream becomes a new file name. When the file has new information, the incremental part is output to the screen.
Next, a log processing method according to an exemplary embodiment of the present disclosure is further explained and explained with reference to fig. 7. Referring to fig. 7, the log processing method may include the steps of:
step S710, establishing a TCP (Transmission Control Protocol) between the Server (Server)301 and the log Harvester 302 (harvest);
step S720, the log reaper 302 sends the collected to-be-processed log to the server 301 through the TCP protocol; after receiving the log to be processed, the server 301 determines whether the log to be processed is a newly added log according to the file path of the log to be processed;
step 730, when the server 301 judges that the log to be processed is a newly added log, arranging the newly added log according to a preset arrangement sequence to obtain a synchronous refreshing log;
step S740, establishing socket input/output interface (socket io) communication between the server 301 and the browser 303;
step S750, after the socket i/o interface (socket io) communication between the server 301 and the browser 303 is established, the server 301 synchronizes the synchronous refresh log to the browser 303 through the socket io;
step S760, creating a log stream (stream)701 and a node (node)702 corresponding to the log stream (stream);
step S770, the browser 303 stores the synchronous refresh log in the log stream 701 corresponding to the file path; the browser 303 places the log stream corresponding to the file path into the node 702 corresponding to the log stream.
The disclosure also provides a log processing device. Referring to fig. 8, the log processing apparatus may include a determination module 810, an arrangement module 820, and a synchronization module 830. Wherein:
the determining module 810 may be configured to receive a log to be processed, and determine whether the log to be processed is a newly added log according to a file path of the log to be processed.
The arranging module 820 may be configured to, if the to-be-processed log is a newly added log, arrange the newly added log according to a preset arrangement order to obtain a synchronous refresh log.
The synchronization module 830 may be configured to synchronize the synchronous refresh log to a browser, so that the browser exposes the synchronous refresh log.
In an example embodiment of the present disclosure, the preset arrangement order includes one or more of a descending order of update time of the newly added log, an ascending order of update time of the newly added log, and a magnitude of correlation between the newly added log and the history log.
In an example embodiment of the present disclosure, the arranging the newly added logs according to a preset arrangement order to obtain a synchronous refresh log includes:
and arranging the newly added logs according to the updating time of the newly added logs in a descending order to obtain the synchronous refreshing log.
In an example embodiment of the present disclosure, synchronizing the synchronized refresh log to a browser comprises:
and synchronizing the synchronous refreshing log to the browser through a socket input and output interface.
In an example embodiment of the present disclosure, receiving a pending log includes:
establishing a transmission control protocol with a log reaper; and receiving the log to be processed sent by the log harvester through the transmission control protocol.
In an example embodiment of the present disclosure, the log processing apparatus further includes:
the system comprises a creating module, a sending module and a processing module, wherein the creating module can be used for creating a log stream and a node corresponding to the log stream;
the storage module may be configured to store the synchronous refresh log into a log stream corresponding to the file path;
a placing module, configured to place the log stream corresponding to the file path into a node corresponding to the log stream.
In an example embodiment of the present disclosure, synchronizing the synchronized refresh log to a browser through a socket input output interface comprises:
and synchronizing the synchronous refreshing log to a browser through a socket input and output interface according to the log stream to which the synchronous refreshing log belongs and the node corresponding to the log stream.
The specific details of each module in the log processing apparatus have been described in detail in the corresponding log processing method, and therefore are not described herein again.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the invention. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Moreover, although the steps of the methods of the present invention are depicted in the drawings in a particular order, this does not require or imply that the steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the method according to the embodiment of the present invention.
In an exemplary embodiment of the present invention, there is also provided an electronic device capable of implementing the above method.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 900 according to this embodiment of the invention is described below with reference to fig. 9. The electronic device 900 shown in fig. 9 is only an example and should not bring any limitations to the function and scope of use of the embodiments of the present invention.
As shown in fig. 9, the electronic device 900 is embodied in the form of a general purpose computing device. Components of electronic device 900 may include, but are not limited to: the at least one processing unit 910, the at least one memory unit 920, and a bus 930 that couples various system components including the memory unit 920 and the processing unit 910.
Wherein the storage unit stores program code that is executable by the processing unit 910 to cause the processing unit 910 to perform steps according to various exemplary embodiments of the present invention described in the above section "exemplary methods" of the present specification. For example, the processing unit 910 may perform step S210 as shown in fig. 2: receiving a log to be processed, and judging whether the log to be processed is a newly added log according to a file path of the log to be processed; step S220: if the log to be processed is a newly added log, arranging the newly added log according to a preset arrangement sequence to obtain a synchronous refreshing log; step S230: and synchronizing the synchronous refreshing log to a browser so that the browser displays the synchronous refreshing log.
The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM)9201 and/or a cache memory unit 9202, and may further include a read only memory unit (ROM) 9203.
The electronic device 900 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 900, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 900 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interface 950. Also, the electronic device 900 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet) via the network adapter 960. As shown, the network adapter 960 communicates with the other modules of the electronic device 900 via the bus 930. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiment of the present invention.
In an exemplary embodiment of the present invention, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above section "exemplary methods" of the present description, when said program product is run on the terminal device.
Referring to fig. 10, a program product 1010 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
Claims (10)
1. A log processing method, comprising:
receiving a log to be processed, and judging whether the log to be processed is a newly added log according to a file path of the log to be processed;
if the log to be processed is a newly added log, arranging the newly added log according to a preset arrangement sequence to obtain a synchronous refreshing log;
and synchronizing the synchronous refreshing log to a browser so that the browser displays the synchronous refreshing log.
2. The log processing method as claimed in claim 1, wherein the predetermined arrangement order includes one or more of a descending order of update time of the newly added log, an ascending order of update time of the newly added log, and a magnitude of correlation between the newly added log and the history log.
3. The log processing method of claim 2, wherein the arranging the newly added logs according to a preset arrangement order to obtain the synchronous refresh log comprises:
and arranging the newly added logs according to the updating time of the newly added logs in a descending order to obtain the synchronous refreshing log.
4. The log processing method of claim 1, wherein synchronizing the synchronized flush log to a browser comprises:
and synchronizing the synchronous refreshing log to the browser through a socket input and output interface.
5. The log processing method of claim 1, wherein receiving the log to be processed comprises:
establishing a transmission control protocol with a log reaper;
and receiving the log to be processed sent by the log harvester through the transmission control protocol.
6. The log processing method of claim 4, wherein prior to synchronizing the synchronized flush log to a browser, the log processing method further comprises:
creating a log stream and a node corresponding to the log stream;
storing the synchronous refreshing log into a log stream corresponding to the file path;
and placing the log stream corresponding to the file path into a node corresponding to the log stream.
7. The log processing method of claim 6, wherein synchronizing the synchronized refresh log to a browser via a socket input output interface comprises:
and synchronizing the synchronous refreshing log to a browser through a socket input and output interface according to the log stream to which the synchronous refreshing log belongs and the node corresponding to the log stream.
8. A log processing apparatus, comprising:
the judging module is used for receiving the log to be processed and judging whether the log to be processed is a newly added log according to the file path of the log to be processed;
the arranging module is used for arranging the newly added logs according to a preset arranging sequence to obtain synchronous refreshing logs if the logs to be processed are newly added logs;
and the synchronization module is used for synchronizing the synchronous refreshing log to a browser so that the browser can display the synchronous refreshing log.
9. A computer-readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing the log processing method of any one of claims 1 to 7.
10. An electronic device, comprising:
a processor; and
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the log processing method of any of claims 1-7 via execution of the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811468637.8A CN111258964A (en) | 2018-12-03 | 2018-12-03 | Log processing method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811468637.8A CN111258964A (en) | 2018-12-03 | 2018-12-03 | Log processing method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111258964A true CN111258964A (en) | 2020-06-09 |
Family
ID=70952073
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811468637.8A Pending CN111258964A (en) | 2018-12-03 | 2018-12-03 | Log processing method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111258964A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113254460A (en) * | 2021-07-07 | 2021-08-13 | 阿里云计算有限公司 | Data processing method, system, electronic device and computer program product |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120272240A1 (en) * | 2011-04-25 | 2012-10-25 | Microsoft Corporation | Virtual Disk Storage Techniques |
| CN105426292A (en) * | 2015-10-29 | 2016-03-23 | 网易（杭州）网络有限公司 | Game log real-time processing system and method |
| CN106101264A (en) * | 2016-07-20 | 2016-11-09 | 腾讯科技（深圳）有限公司 | Content distributing network daily record method for pushing, device and system |
| CN106886483A (en) * | 2015-12-16 | 2017-06-23 | 阿里巴巴集团控股有限公司 | The method and apparatus for providing, obtaining terminal applies daily record |
| CN107347062A (en) * | 2017-06-19 | 2017-11-14 | 北京开数科技有限公司 | A kind of method, electronic equipment and the readable storage medium storing program for executing of daily record data processing |
-
2018
- 2018-12-03 CN CN201811468637.8A patent/CN111258964A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120272240A1 (en) * | 2011-04-25 | 2012-10-25 | Microsoft Corporation | Virtual Disk Storage Techniques |
| CN105426292A (en) * | 2015-10-29 | 2016-03-23 | 网易（杭州）网络有限公司 | Game log real-time processing system and method |
| CN106886483A (en) * | 2015-12-16 | 2017-06-23 | 阿里巴巴集团控股有限公司 | The method and apparatus for providing, obtaining terminal applies daily record |
| CN106101264A (en) * | 2016-07-20 | 2016-11-09 | 腾讯科技（深圳）有限公司 | Content distributing network daily record method for pushing, device and system |
| CN107347062A (en) * | 2017-06-19 | 2017-11-14 | 北京开数科技有限公司 | A kind of method, electronic equipment and the readable storage medium storing program for executing of daily record data processing |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113254460A (en) * | 2021-07-07 | 2021-08-13 | 阿里云计算有限公司 | Data processing method, system, electronic device and computer program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109660397B (en) | System, method and device for collecting logs | |
| CN110262807B (en) | Cluster creation progress log acquisition system, method and device | |
| US10180893B2 (en) | System and method for providing additional functionality to developer side application in an integrated development environment | |
| CN107797887B (en) | Data backup and recovery method and device, storage medium and electronic equipment | |
| CN109672722B (en) | Data deployment method and device, computer storage medium and electronic equipment | |
| CN110955578A (en) | Log collection method and device based on host machine, computer equipment and storage medium | |
| CN114625597A (en) | Monitoring operation and maintenance system, method and device, electronic equipment and storage medium | |
| CN111737207A (en) | Method and device for displaying and collecting logs of service nodes in distributed system | |
| CN112130781A (en) | Log printing method and device, electronic equipment and storage medium | |
| CN113138900B (en) | Data acquisition processing method and device and electronic equipment | |
| CN111258964A (en) | Log processing method and device, storage medium and electronic equipment | |
| CN113312321A (en) | Abnormal monitoring method for traffic and related equipment | |
| CN109725913A (en) | The method and apparatus that data update | |
| CN112417050A (en) | Data synchronization method and device, system, storage medium and electronic device | |
| CN111162938A (en) | Data processing system and method | |
| CN105446707B (en) | Data conversion method | |
| CN112685486B (en) | Data management method and device for database cluster, electronic equipment and storage medium | |
| KR102365743B1 (en) | Apparatus and method for monitoring and controlling equipment status of multiple ground stations | |
| CN111698109A (en) | Method and device for monitoring log | |
| CN115809119A (en) | Monitoring method, system and device for container arrangement engine | |
| CN110750424B (en) | Resource inspection method and device | |
| CN114327563A (en) | Data synchronization method, device, system, storage medium and computer system | |
| CN114721674A (en) | Model deployment method, device, equipment and storage medium | |
| CN113760856B (en) | Database management method and device, computer readable storage medium and electronic equipment | |
| CN109756393B (en) | Information processing method, system, medium, and computing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |