CN114356862A - Data statistical method, device, electronic equipment, storage medium and product - Google Patents

Data statistical method, device, electronic equipment, storage medium and product Download PDF

Info

Publication number
CN114356862A
CN114356862A CN202111596920.0A CN202111596920A CN114356862A CN 114356862 A CN114356862 A CN 114356862A CN 202111596920 A CN202111596920 A CN 202111596920A CN 114356862 A CN114356862 A CN 114356862A
Authority
CN
China
Prior art keywords
log
module
output
mode
log data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202111596920.0A
Other languages
Chinese (zh)
Inventor
郭俊峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202111596920.0A priority Critical patent/CN114356862A/en
Publication of CN114356862A publication Critical patent/CN114356862A/en
Withdrawn legal-status Critical Current

Links

Images

Landscapes

  • Debugging And Monitoring (AREA)

Abstract

The disclosure relates to the technical field of system optimization, in particular to the technical field of data processing. The specific implementation scheme is as follows: acquiring log data generated during the operation of a system module; inputting the log data into a visualization tool to obtain an analysis result of the output log quantity; and determining a system module to be optimized based on the analysis result. The method and the device can determine a large number of modules for outputting the logs, so that the modules to be optimized are determined, and the phenomenon that some modules output the logs in a large number and flush important information is avoided.

Description

Data statistical method, device, electronic equipment, storage medium and product
Technical Field
The present disclosure relates to the field of system optimization technologies, and in particular, to a data statistical method, apparatus, electronic device, storage medium, and product.
Background
Relatively complex systems include a large number of operational modules, each of which generates a log during operation. The logs generated by a certain module in a large amount are generated continuously, the logs of other modules can be flushed, and even important information can be flushed when the logs are not checked in time.
Disclosure of Invention
The disclosure provides a data statistical method, a device, an electronic device, a storage medium and a product.
According to a first aspect of the present disclosure, there is provided a data statistics method, the method comprising:
acquiring log data generated during the operation of a system module; inputting the log data into a visualization tool to obtain an analysis result of the output log quantity; and determining a system module to be optimized based on the analysis result.
According to a second aspect of the present disclosure, there is provided a data statistics apparatus, the apparatus comprising:
the acquisition module is used for acquiring log data generated when the plurality of system modules run; the visualization module is used for inputting the log data into a visualization tool to obtain an analysis result of the output log quantity; and the determining module is used for determining the system module to be optimized based on the analysis result.
According to a third aspect of the present disclosure, there is provided an electronic device comprising:
at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method according to the first aspect.
According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method according to the first aspect.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
fig. 1 is a schematic flow chart illustrating a data statistics method provided by an embodiment of the present disclosure;
FIG. 2 is a flow chart diagram illustrating a data analysis method provided by an embodiment of the present disclosure;
FIG. 3 is a flow chart diagram illustrating a method for generating a visual graph according to an embodiment of the present disclosure;
FIG. 4 is a schematic structural diagram illustrating a visualization graph in a data statistics method according to an embodiment of the present disclosure;
FIG. 5 is a schematic structural diagram illustrating a visualization graph in still another data statistics method provided by the embodiment of the present disclosure;
FIG. 6 is a schematic structural diagram of a data statistics apparatus provided by an embodiment of the present disclosure;
FIG. 7 illustrates a schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
With the development of technology, systems including modules that can implement different functions are increasing, especially relatively complex systems. For example, in an automatic driving system, the system is generally operated on an embedded board under the requirement of reducing the cost, thereby bringing higher requirements to the use of a Central Processing Unit (CPU), a memory, a hard disk and the like.
In general, for a relatively complicated system such as an automatic driving system, logs of a plurality of modules are printed in a single log file. Therefore, a large amount of output logs of some modules may cause inconvenience in debugging of other modules, and in addition, due to the limitation of the number of log files, if there is no analysis on the output logs of the modules, the output logs of other modules may be flushed due to a large amount of output logs of a certain module, so that important information is lost.
Based on this, the application provides a data statistical method and system, which are used for analyzing output log data, and optimizing a module in time when a large amount of logs are output by the module, so that important information of logs output by other modules is prevented from being flushed.
Fig. 1 shows a schematic flowchart of a data statistics method provided by an embodiment of the present disclosure, and as shown in fig. 1, the method may include:
in step S110, log data generated while the system module is running is acquired.
In the embodiment of the disclosure, when the system runs, a plurality of modules output the logs in time order. The output log can comprise the module to which the log belongs and the log output time.
And setting storage time intervals, and storing the output logs to a specified storage position every other time interval to obtain all log information of the output logs. Wherein, the designated position can be a hard disk, a network disk and the like.
In the designated storage location, log data generated during the operation of the system module is acquired.
In step S120, the log data is input into the visualization tool, and the analysis result of the output log number is obtained.
In the embodiment of the present disclosure, after the output log data is obtained, the obtained log file needs to be input into a visualization tool, and the visualization tool processes the log data to obtain an analysis result of the output number of the logs in the obtained log data.
In step S130, based on the analysis result, a system module to be optimized is determined.
In the embodiment of the disclosure, a module which outputs a large amount of logs may be determined based on the obtained analysis result, and a module which does not need to be optimized may be determined from the module which outputs a large amount of logs. And pushing the module support to be optimized to the corresponding user so as to optimize the module to be optimized.
In the data statistical method provided by the embodiment of the disclosure, a large number of modules outputting logs can be determined by performing statistics on the logs output by the system module, so that the modules to be optimized are determined, and the phenomenon that some modules output logs in a large amount and flush important information is avoided.
In the embodiment of the disclosure, the log data includes module information of all output logs and a corresponding number of output logs of each module; and/or outputting the high-frequency sentences appearing in the log and the number of log outputs corresponding to the high-frequency sentences. The log data can be acquired in different modes.
Further, in the present disclosure, the log data may be obtained by obtaining a log file output by the system module from a printed log file, and the log data may be obtained by obtaining a log file output by the system module through a log interface. Wherein, different data structures are added in the log interface in advance. The data interface added in the log interface comprises a module data structure and a high-frequency statement data structure.
The module data structure may be understood as that, for each module, the module data structure is refreshed once every time each module outputs one log, so as to count the number of logs output by each module. The log data output by all modules is obtained at every certain time, for example, every ten minutes, which is only illustrative of the distance of the present disclosure and is not a specific limitation of the present disclosure.
The high-frequency sentence data structure can be understood as counting log data output by all modules, and screening out high-frequency sentences in output logs in all log data based on a preset counting threshold value. For each high frequency statement, the count of the high frequency statement is incremented by one each time the high frequency statement is output.
Fig. 2 shows a schematic flow chart of data analysis provided by an embodiment of the present disclosure, and as shown in fig. 2, the method may include:
in step S210, a mode of a visualization tool for generating a log analysis result is determined according to an acquisition manner of the log data.
In the embodiment of the present disclosure, the visualization tool includes a plurality of modes, the modes of acquiring log data are different, and the modes of generating the log analysis result using the visualization tool are different.
For example, log data is obtained based on the data structure, and log analysis results are generated using a first mode of a visualization tool; log data is obtained based on the log file, and a log analysis result is generated using a second mode of the visualization tool.
In step S220, log data is loaded based on the pattern, and a visualization graph of the analysis result of the number of output logs is obtained.
In the embodiment of the present disclosure, the visualization tool loads log data based on a mode corresponding to a log data acquisition manner, and obtains a visualization graph of an analysis result of the output log number.
The visual graph can be a histogram, a sector graph and the like.
According to the method and the device, log data are displayed in a graph mode, the running state of the module outputting the log can be visually represented, and an accurate analysis result is provided for subsequently judging the module to be optimized.
In the method, a mode corresponding to a visualization tool for loading log data is determined, the log data is loaded in the mode, and an analysis result of the number of logs output by each module is obtained, and/or an analysis result of the number of high-frequency statements output is obtained. The following embodiments will further illustrate the analysis of the input of log data into the visualization tool to obtain the output log number.
Fig. 3 illustrates a flowchart of a method for generating a visual graph according to an embodiment of the present disclosure, and as shown in fig. 3, the method may include:
in step S310, a mode of a visualization tool for generating a log analysis result is determined according to an acquisition manner of the log data.
In the disclosed embodiments, the log data obtained based on the log file is determined, or the log file obtained based on the data structure is determined. According to the mode of obtaining the log, the mode of the adopted visualization tool is determined, and then the visualization tool can be used for generating a visualization log analysis result in the mode.
In step S320, log data is loaded based on the pattern, and a visualization graph of the analysis result of the number of output logs is obtained.
In the embodiment of the present disclosure, after the mode of the visualization tool is determined, the log data is loaded in the mode, and the log data is statistically calculated, and the statistical log data is displayed in the form of a visualization graph.
In an exemplary embodiment of the present disclosure, if module information of output logs is loaded based on a corresponding mode, a corresponding number of output logs of each module also needs to be loaded. And counting the number of the logs output by each module, and representing the information of each module, the level of each log output by each module and the number corresponding to the log level output logs by a visual graph. For the convenience of description, the visualization graph for characterizing each module information, the level of output logs in each module, and the number corresponding to the log level output logs is referred to as a first visualization graph.
For example, fig. 4 shows a schematic structural diagram of a visualization graph in a data statistics method provided by an embodiment of the present disclosure, as shown in fig. 4, the visualization graph includes a Map Engine module (Map Engine), a Localization module (Localization), a cognitive module (Perception), a prediction module (prediction), a Planning module (Planning), a Control module (Control), a Human Machine Interface (HIM), and an adaptation module (Adapter). The levels of the log may include program DEBUG tool detail information (DEBUG), Instructions (INFO), WARNINGs (WARNING), ERRORs (ERROR), and critical ERRORs (FATAL). Therein, it is understood that the DEBUG is used for characterization, which may be used in any way that is deemed advantageous for more detailed understanding of the system operating state at DEBUG. INFO may be used to characterize the validation procedure as expected. WARNING may be used to characterize an accident (e.g., insufficient disk space) that has occurred or is about to occur. ERROR may characterize that some functions of a program have failed to perform properly due to a serious problem. FATAL can be used to characterize the module as having a FATAL error. It should be noted that the present disclosure is only described in terms of the related modules and the distance at the log level, and is not a specific limitation of the present disclosure.
As shown in fig. 4, the visualization graph may be a histogram, and the number of log levels respectively output by each module may be represented by a bar, taking the histogram as an example. This embodiment may be implemented alone or in combination with other embodiments.
In another exemplary embodiment of the present disclosure, the relationship between the total output of all modules and time may be displayed by using a visual graph. Fig. 5 is a schematic diagram illustrating a visual graph in a data statistics method provided by an embodiment of the present disclosure, and as shown in fig. 5, by taking an interval of one hour as an example, the total number of log outputs of all modules in several time periods is screened out, where the total number of log outputs includes 1: 00-2: 00,3: 00-4: 00,5: 00-6: 00,7: 00-8: 00,9: 00-10: 00, 11: 00-12: 00, 13: 00-14: 00, 15: 00-16: 00, 17: 00-18: 00, 19: 00-20: 00, 21: 00-22: 00, 23: 00-24: 00. this embodiment may be implemented alone or in combination with other embodiments.
In yet another exemplary embodiment of the present disclosure, high frequency statements in the output log may also be screened out. If the high-frequency sentences appearing in the output log are loaded based on the corresponding mode, the number of log outputs corresponding to the high-frequency sentences is loaded, the number of the outputs of each high-frequency sentence is counted, and the number of the outputs of each high-frequency sentence and the information of the high-frequency sentence are represented by a visual graph. For the convenience of description, the visualization graph for characterizing the relationship between the high-frequency sentence, the number of log outputs corresponding to the high-frequency sentence, and time is referred to as a second visualization graph.
In the present disclosure, a sentence in which the occurrence probability in the output log exceeds a preset probability threshold is referred to as a high-frequency sentence. The high-frequency sentence can include a high-frequency word, a high-frequency sentence and the like. This embodiment may be implemented alone or in combination with other embodiments.
In the above embodiment, the manner of acquiring log data is different, and the mode of using the visualization tool is also different, and for convenience of description in the present disclosure, the mode of the visualization tool corresponding to acquiring log data based on the data structure is referred to as a first mode, and the mode of the visualization tool corresponding to acquiring log data based on the log file is referred to as a second mode. In combination with the above embodiments, in the present disclosure, based on the first mode of the visualization tool, a first visualization graph for characterizing a relationship between the module information, the number of module output logs, and time is generated. And generating a second visualization graph for representing the relationship among the high-frequency sentences, the log output quantity corresponding to the high-frequency sentences and time based on a second mode of the visualization tool.
In embodiments of the present disclosure, the first mode and the second model of the visualization tool may be the same or different.
In the disclosure, the log level and the number output by each module can be analyzed through the visual graph generated by the visualization tool, and the module to be optimized is determined. For example, as shown in fig. 5, 7: 00-8: 00, the total number of output logs is relatively large, the number of logs output by each module in the time period and the number of logs corresponding to different log levels output by each module can be further analyzed, as shown in fig. 4. If, as in fig. 5, 7: 00-8: 00 in this time period, if the number of logs of each module output in different levels is shown in fig. 4, each module in fig. 4 may be further analyzed to determine a module to be optimized.
Based on the same principle as the method shown in fig. 1, fig. 6 shows a schematic structural diagram of a data statistics apparatus provided by an embodiment of the present disclosure, as shown in fig. 6, the data statistics apparatus 100 may include:
an obtaining module 101, configured to obtain log data generated when multiple system modules run; the visualization module 102 is configured to input the log data into a visualization tool to obtain an analysis result of the output log quantity; a determining module 103, configured to determine a system module to be optimized based on the analysis result.
In this embodiment of the present disclosure, the obtaining module 101 is configured to obtain log data based on a data structure, or obtain log data based on a log file; the log data comprises module information of all output logs and the number corresponding to the output logs of each module; and/or outputting the high-frequency sentences appearing in the log and the number of log outputs corresponding to the high-frequency sentences.
In an embodiment of the present disclosure, the visualization module 102 is configured to determine, according to an obtaining manner of the log data, a mode of a visualization tool used for generating a log analysis result; and loading the log data based on the mode to obtain a visual graph of the analysis result of the output log quantity.
In this disclosure, the visualization module 102 is configured to load module information of the output logs based on the mode, and load a quantity corresponding to each module output log, and generate a first visualization graph, where the first visualization graph is used to represent a relationship between the module information, the quantity corresponding to the module output logs, and time;
and/or
And loading high-frequency sentences appearing in the output log based on the mode, loading the number of log outputs corresponding to the high-frequency sentences, and generating a second visual graph, wherein the second visual graph is used for representing the relationship among the high-frequency sentences, the number of the log outputs corresponding to the high-frequency sentences and time.
In the embodiment of the present disclosure, the determining module 103 is configured to determine, in response to obtaining log data based on a data structure, a first mode of a visualization tool corresponding to the data structure; or
In response to obtaining log data based on the log file, a second mode of the visualization tool corresponding to the data structure is determined.
In the disclosed embodiments, the first mode is the same as the second mode.
In the embodiment of the present disclosure, the determining module 103 is configured to determine, as a system module to be optimized, a system module in which the number of output logs exceeds a threshold in a preset time period length.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
FIG. 7 shows a schematic block diagram of an example electronic device 200 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 7, the apparatus 200 includes a computing unit 201 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)202 or a computer program loaded from a storage unit 208 into a Random Access Memory (RAM) 203. In the RAM 203, various programs and data required for the operation of the device 200 can also be stored. The computing unit 201, the ROM 202, and the RAM 203 are connected to each other via a bus 204. An input/output (I/O) interface 205 is also connected to bus 204.
Various components in the device 200 are connected to the I/O interface 205, including: an input unit 206 such as a keyboard, a mouse, or the like; an output unit 207 such as various types of displays, speakers, and the like; a storage unit 208, such as a magnetic disk, optical disk, or the like; and a communication unit 209 such as a network card, modem, wireless communication transceiver, etc. The communication unit 209 allows the device 200 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 201 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 201 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 201 performs the respective methods and processes described above, such as the method data statistical method. For example, in some embodiments, the method data statistics method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 208. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 200 via the ROM 202 and/or the communication unit 209. When the computer program is loaded into the RAM 203 and executed by the computing unit 201, one or more steps of the method data statistics method described above may be performed. Alternatively, in other embodiments, the computing unit 201 may be configured to perform the method data statistics method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel or sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (17)

1. A method of data statistics, the method comprising:
acquiring log data generated during the operation of a system module;
inputting the log data into a visualization tool to obtain an analysis result of the output log quantity;
and determining a system module to be optimized based on the analysis result.
2. The method of claim 1, wherein the obtaining log data generated by a system module runtime comprises:
acquiring log data based on a log interface, or acquiring log data based on a log file;
the log data comprises module information of all output logs and the number corresponding to the output logs of each module; and/or outputting the high-frequency sentences appearing in the log and the number of log outputs corresponding to the high-frequency sentences.
3. The method of claim 1, wherein the inputting the log data into a visualization tool, resulting in an analysis of the output log quantity, comprises:
determining a mode of a visualization tool for generating a log analysis result according to the acquisition mode of the log data;
and loading the log data based on the mode to obtain a visual graph of the analysis result of the output log quantity.
4. The method of claim 3, wherein said loading the log data based on the pattern, resulting in a visual graph of the results of the analysis outputting the log quantity, comprises:
loading module information of the output logs based on the mode, loading the number corresponding to each module output log, and generating a first visual graph, wherein the first visual graph is used for representing the relationship among the module information, the number corresponding to the module output logs and time;
and/or
And loading high-frequency sentences appearing in the output log based on the mode, loading the number of log outputs corresponding to the high-frequency sentences, and generating a second visual graph, wherein the second visual graph is used for representing the relationship among the high-frequency sentences, the number of the log outputs corresponding to the high-frequency sentences and time.
5. The method of claim 3, wherein determining a mode of a visualization tool for generating log analysis results according to the log data acquisition manner comprises:
in response to obtaining log data based on a data interface, determining a first mode of a visualization tool corresponding to the data structure; or
In response to obtaining log data based on the log file, a second mode of the visualization tool corresponding to the data structure is determined.
6. The method of claim 5, wherein the first mode is the same as the second mode.
7. The method of claim 1, wherein the determining a system module to be optimized based on the analysis results comprises:
and in the preset time period length, determining the system module with the output log number exceeding a threshold value as the system module to be optimized.
8. A data statistics apparatus, the apparatus comprising:
the acquisition module is used for acquiring log data generated when the plurality of system modules run;
the visualization module is used for inputting the log data into a visualization tool to obtain an analysis result of the output log quantity;
and the determining module is used for determining the system module to be optimized based on the analysis result.
9. The apparatus of claim 8, wherein the means for obtaining is configured to:
acquiring log data based on a log interface, or acquiring log data based on a log file;
the log data comprises module information of all output logs and the number corresponding to the output logs of each module; and/or outputting the high-frequency sentences appearing in the log and the number of log outputs corresponding to the high-frequency sentences.
10. The apparatus of claim 8, wherein the visualization module is to:
determining a mode of a visualization tool for generating a log analysis result according to the acquisition mode of the log data;
and loading the log data based on the mode to obtain a visual graph of the analysis result of the output log quantity.
11. The apparatus of claim 10, wherein the visualization module is to:
loading module information of the output logs based on the mode, loading the number corresponding to each module output log, and generating a first visual graph, wherein the first visual graph is used for representing the relationship among the module information, the number corresponding to the module output logs and time;
and/or
And loading high-frequency sentences appearing in the output log based on the mode, loading the number of log outputs corresponding to the high-frequency sentences, and generating a second visual graph, wherein the second visual graph is used for representing the relationship among the high-frequency sentences, the number of the log outputs corresponding to the high-frequency sentences and time.
12. The apparatus of claim 10, wherein the means for determining is configured to:
in response to obtaining log data based on a data interface, determining a first mode of a visualization tool corresponding to the data structure; or
In response to obtaining log data based on the log file, a second mode of the visualization tool corresponding to the data structure is determined.
13. The apparatus of claim 12, wherein the first mode is the same as the second mode.
14. The apparatus of claim 13, wherein the means for determining is configured to:
and in the preset time period length, determining the system module with the output log number exceeding a threshold value as the system module to be optimized.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.
17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.
CN202111596920.0A 2021-12-24 2021-12-24 Data statistical method, device, electronic equipment, storage medium and product Withdrawn CN114356862A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111596920.0A CN114356862A (en) 2021-12-24 2021-12-24 Data statistical method, device, electronic equipment, storage medium and product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111596920.0A CN114356862A (en) 2021-12-24 2021-12-24 Data statistical method, device, electronic equipment, storage medium and product

Publications (1)

Publication Number Publication Date
CN114356862A true CN114356862A (en) 2022-04-15

Family

ID=81102214

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111596920.0A Withdrawn CN114356862A (en) 2021-12-24 2021-12-24 Data statistical method, device, electronic equipment, storage medium and product

Country Status (1)

Country Link
CN (1) CN114356862A (en)

Similar Documents

Publication Publication Date Title
CN113778849A (en) Method, apparatus, device and storage medium for testing code
CN109471787B (en) Software quality evaluation method and device, storage medium and electronic equipment
CN111538656B (en) Monitoring method, device and equipment for gradient inspection and storage medium
CN114356862A (en) Data statistical method, device, electronic equipment, storage medium and product
CN115269431A (en) Interface testing method and device, electronic equipment and storage medium
CN107229487B (en) Code processing method and device
CN115061921A (en) Automatic test method, device, electronic equipment and readable storage medium
CN113887101A (en) Visualization method and device of network model, electronic equipment and storage medium
CN114218313A (en) Data management method, device, electronic equipment, storage medium and product
CN114881112A (en) System anomaly detection method, device, equipment and medium
CN114546799A (en) Point burying log checking method and device, electronic equipment, storage medium and product
CN114327577A (en) Code change determining method and device, electronic equipment and storage medium
CN113126881A (en) System configuration method, device, equipment, readable storage medium and distributed storage system
CN112764957A (en) Application fault delimiting method and device
CN112162528A (en) Fault diagnosis method, device, equipment and storage medium of numerical control machine tool
CN115098405B (en) Software product evaluation method and device, electronic equipment and storage medium
CN114816758B (en) Resource allocation method and device
KR20220025148A (en) Methods, devices and electronic devices for monitoring wake-up indicators
CN114064411A (en) Component detection method, device, equipment and storage medium
CN115292179A (en) Test data generation method and device, electronic equipment and storage medium
CN115374010A (en) Function testing method, device, equipment and storage medium
CN115629690A (en) Method, device, equipment and storage medium for generating FIO working file
CN115729561A (en) Abnormality detection method, abnormality detection device, electronic apparatus, and storage medium
CN114328234A (en) Application program detection method and device, equipment and storage medium
CN115878362A (en) Operating system abnormity positioning method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20220415

WW01 Invention patent application withdrawn after publication