CN107870921B - Log data processing method and device - Google Patents
Log data processing method and device Download PDFInfo
- Publication number
- CN107870921B CN107870921B CN201610849308.2A CN201610849308A CN107870921B CN 107870921 B CN107870921 B CN 107870921B CN 201610849308 A CN201610849308 A CN 201610849308A CN 107870921 B CN107870921 B CN 107870921B
- Authority
- CN
- China
- Prior art keywords
- data
- log
- service
- processing mode
- log data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title abstract description 12
- 238000000034 method Methods 0.000 claims description 26
- 238000004458 analytical method Methods 0.000 claims description 24
- 230000015654 memory Effects 0.000 claims description 17
- 238000012163 sequencing technique Methods 0.000 claims description 13
- 238000004891 communication Methods 0.000 claims description 10
- 238000007405 data analysis Methods 0.000 abstract description 13
- 238000007781 pre-processing Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 101100261000 Caenorhabditis elegans top-3 gene Proteins 0.000 description 4
- 101150041570 TOP1 gene Proteins 0.000 description 4
- 101150107801 Top2a gene Proteins 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a log data processing method, which comprises the following steps: acquiring service log data of application operation from an application server, and acquiring interface sorting data from a data platform, wherein the interface sorting data comprises the calling times of each log processing mode in a plurality of log processing modes; selecting a target log processing mode from the multiple log processing modes according to the calling times of each log processing mode; and processing the service log data by using the target log processing mode. The embodiment of the invention also discloses a log data processing device. By adopting the embodiment of the invention, the output of the service log data can be reduced, the efficiency of log data analysis is improved, and the accuracy of the log data analysis is ensured.
Description
Technical Field
The invention relates to the technical field of electronics, in particular to a log data processing method and device.
Background
In order to mine Data value, various products and platforms are analyzed based on log Data in the IT (Information Technology) field at present, Data analysis is gradually the mainstream from IT to DT (Data Technology) era transition, application scenes comprise user behavior analysis, security intrusion analysis, traffic flow prediction, Internet of things, shopping recommendation systems and the like, services of the type are firstly output by dotting based on log Data of a large Data platform, then log Data are collected, then the log Data are stored, and finally a model for analyzing large Data gradually becomes a factual standard.
In the prior art, the log data analysis process is as follows, firstly, applying log dotting and generating a log; secondly, log data are captured in a distributed and parallel mode; thirdly, transmitting log data to be integrated, stored and summarized; fourthly, analyzing log data; fifth, the interface presents the analysis and statistics. In addition, in order to balance the performance loss of application performance and logging, a high-performance CPU (Central Processing Unit) may be configured to handle a large amount of logging during the peak period of the business, or turn on a sampling switch on the application service to reduce the logging output. However, the above solution still has the following disadvantages: firstly, an application server generates a large amount of log data (logging is performed), meanwhile, the log data is transmitted to a data platform through a network, the CPU performance of an application node is seriously consumed (the output quantity of the log is in direct proportion to the CPU consumption), although the problem can be solved by configuring a high-performance CPU, the CPU is basically idle in the low-ebb period, and resources are seriously wasted; secondly, the log data output by the application server contains a large amount of useless information and occupies a large amount of storage space; third, although the sampling switch is turned on to reduce log output, the incompleteness of log data affects the accuracy of data analysis results.
Disclosure of Invention
The embodiment of the invention provides a log data processing method and device. The technical problems of low efficiency and low accuracy of log data analysis in the prior art can be solved.
In a first aspect, the present application provides a log data processing method, including:
firstly, acquiring service log data of application operation from an application server, and acquiring interface sorting data from a data platform, wherein the interface sorting data comprises the calling times of each log processing mode in a plurality of log processing modes; then selecting a target log processing mode from the multiple log processing modes according to the calling times of each log processing mode; and finally, processing the service log data by using a target log processing mode, determining the log processing mode by acquiring analysis hotspot requirements counted by a data platform, and preprocessing the log data by adopting the determined log processing mode before analyzing the log data, so that the number of the service log data is reduced, and the log data analysis efficiency and the log analysis accuracy are improved.
In another possible design, the multiple log processing modes at least include retaining fields required in the log analysis stage in the service log data, counting contents matched with preset keywords in the service log data, counting and time-stamping the same contents in the service log data within a preset time range, and establishing indexes of designated fields in the service log data.
In another possible design, the priority of each log processing mode may be determined according to the number of calls of each log processing mode; and selecting the log processing mode with the highest priority from the plurality of log processing modes as the target log processing mode, so that the log processing mode is selected by sensing the analysis hotspot requirement of a data platform, and the real-time performance of log data analysis is met.
In another possible design, after business log data of application operation is acquired from an application server, a read-write operation mode of the business log data can be determined; when the read-write operation mode of the service log data is data write-in operation, executing the operation step of acquiring interface sequencing data from the data platform; and when the read-write operation mode of the service log data is data read operation, the following operation steps are not executed.
In another possible design, the processed service log data is stored in the data platform, so that the local log storage space of the application server is saved, and the problem that the storage and transmission of the local log consume a CPU is solved.
In a second aspect, the present application provides a log data processing apparatus configured to implement the method and the functions performed by the log data processing apparatus in the first aspect, and implemented by hardware/software, where the hardware/software includes units corresponding to the functions.
In a third aspect, the present application provides a log data processing apparatus, including: the processing method comprises a processor, a memory and a communication bus, wherein the communication bus is used for realizing connection communication between the processor and the memory, and the processor executes a program stored in the memory for realizing the steps in the processing method for processing log data provided by the first aspect.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a log processing system according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a log processing method according to a first embodiment of the present invention;
fig. 3 is a schematic flowchart of a log processing method according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a log processing apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a log processing device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a log processing system according to an embodiment of the present invention. As shown in the figure, the system in the embodiment of the present invention includes an application server, a coprocessor and a data platform, where the application server may include a service module and a log module, the service module is used to execute various service logics, and the log module is used to print log data when the service module executes the service logics; the coprocessor, such as an ARM64(Advanced RISC processors) or an FPGA (Field Programmable Gate Array), may include a log pass-through module, a log preprocessing module, and an interface sorting module, where the interface sorting module is configured to obtain interface sorting data from a data platform and transmit the interface sorting data to the log preprocessing module, the log preprocessing module is configured to select a log processing mode according to the interface sorting data and preprocess the log data using the selected log processing mode, and the log pass-through module is configured to transmit the preprocessed log data to the data platform; the data platform can comprise a log storage module and a log analysis module, wherein the log storage module is used for storing the preprocessed service log data, and the log analysis module is used for counting the call data of the log processing mode used by the coprocessor.
Referring to fig. 2, fig. 2 is a flowchart illustrating a log processing method according to a first embodiment of the present invention. As shown in the figure, the method in the embodiment of the present invention includes:
s201, acquiring service log data of application operation from an application server, and acquiring interface sequencing data from a data platform, wherein the interface sequencing data comprises the calling times of each log processing mode in a plurality of log processing modes.
In the specific implementation, the interface sequencing data is the calling times of a log processing mode adopted by all processors associated with the data platform to process the service log data, after the application server receives a service request, the application server runs various applications and prints the service log data, and outputs the service log data to the coprocessor; the data platform stores a large number of times of calling of a log processing mode adopted by the processor for processing the service log data, and after the coprocessor acquires the service log data transmitted by the application server, the statistical data can be acquired from the data platform.
Optionally, after acquiring service log data of application operation from the application server, a read-write operation mode of the service log data may be determined; when the read-write operation mode of the service log data is data write-in operation, executing the operation step of acquiring interface sequencing data from the data platform; and when the read-write operation mode of the service log data is data read operation, the following operation steps are not executed.
S202, selecting a target log processing mode from the multiple log processing modes according to the calling times of each log processing mode.
In a specific implementation, the priority of each log processing mode can be determined according to the calling times of each log processing mode; and selecting the log processing mode with the highest priority from the plurality of log processing modes as the target log processing mode. Furthermore, if the number of times of calling a certain log processing mode is the largest, the priority of the log processing mode can be set to be the highest priority, so that the log processing mode is selected by sensing the analysis hotspot requirement of a data platform, and the real-time performance of log data analysis is met.
Optionally, the priorities of multiple log processing modes may be sorted, and a log processing mode with the top N bits of priorities is selected from the multiple log processing modes, where N is an integer greater than or equal to 1.
S203, the service log data is processed by using the target log processing mode.
In a specific implementation, the multiple log processing modes at least include retaining fields required in a log analysis stage in the service log data, counting contents matched with preset keywords in the service log data, counting and time-stamping the same contents in the service log data within a preset time range, and establishing an index of a designated field in the service log data. At least one log processing mode can be selected, and the selected log processing mode is adopted to preprocess the service log data, so that the output of the log data is reduced, and the indexing speed of the log data is accelerated.
For example: as shown in fig. 1, the log preprocessing module includes a plurality of log processing interfaces (Top1 interface, Top2 interface, Top3 interface, … …), each of which uses different log processing modes to process the service log data, and may select a Top1 interface, a Top2 interface, and a Top3 interface to process the service log data at the same time, and obtain a result after each of the log processing interfaces processes the log service data, and finally transmit the result to all data platforms.
The method can be applied to a user behavior analysis system, a security intrusion analysis system, a traffic flow prediction system and a shopping recommendation system, and the service log data output by the user behavior analysis system, the security intrusion analysis system, the traffic flow prediction system and the shopping recommendation system are preprocessed by the method, so that the log output quantity and the data quantity are reduced, the performance loss of log dotting of an application server is reduced, the log storage quantity is reduced, and the data analysis performance is accelerated.
Referring to fig. 3, fig. 3 is a flowchart illustrating a log processing method according to a second embodiment of the present invention. As shown in the figure, the method in the embodiment of the present invention includes:
s301, the application server sends the service log data of the application operation to the coprocessor. The same steps as those in the above embodiments are performed, and the description of this embodiment is omitted.
S302, the data platform sends interface sequencing data to the co-processing. The same steps as in the previous embodiment are performed and the present embodiment is no longer cumbersome.
And S303, the coprocessor selects a target log processing mode from the multiple log processing modes according to the calling times of each log processing mode. The same steps as those in the above embodiments are performed, and the description of this embodiment is omitted.
S304, the coprocessor uses the target log processing mode to process the service log data. The same steps as those in the above embodiments are performed, and the description of this embodiment is omitted.
S305, the coprocessor stores the processed service log data to the data platform, so that the local log storage space of the application server is saved, and the problem that the local log storage and transmission consume a CPU is solved.
In the embodiment of the invention, firstly, business log data of application operation is obtained from an application server, interface sequencing data is obtained from a data platform, and then a target log processing mode is selected from a plurality of log processing modes according to the calling times of each log processing mode; and finally, processing the service log data by using a target log processing mode, determining the log processing mode by acquiring analysis hotspot requirements counted by a data platform, and preprocessing the log data by adopting the determined log processing mode before analyzing the log data, so that the number of the service log data is reduced, and the log data analysis efficiency and the log analysis accuracy are improved.
As shown in fig. 4, fig. 4 is a schematic structural diagram of a log data processing apparatus according to an embodiment of the present invention. As shown in the figures, the apparatus in the embodiment of the present invention includes:
the data obtaining module 401 is configured to obtain service log data of application operation from an application server, and obtain interface sorting data from a data platform, where the interface sorting data includes the number of times of calling each log processing mode in multiple log processing modes.
In the specific implementation, the interface sequencing data is the calling times of a log processing mode adopted by all processors associated with the data platform to process the service log data, after the application server receives a service request, the application server runs various applications and prints the service log data, and outputs the service log data to the coprocessor; the data platform stores a large number of times of calling of a log processing mode adopted by the processor for processing the service log data, and after the coprocessor acquires the service log data transmitted by the application server, the statistical data can be acquired from the data platform.
Optionally, after acquiring service log data of application operation from the application server, a read-write operation mode of the service log data may be determined; and when the read-write operation mode of the service log data is data write-in operation, executing the operation step of acquiring interface sequencing data from the data platform.
A mode selecting module 402, configured to select a target log processing mode from the multiple log processing modes according to the number of calls of each log processing mode.
In a specific implementation, the priority of each log processing mode can be determined according to the calling times of each log processing mode; and selecting the log processing mode with the highest priority from the plurality of log processing modes as the target log processing mode. Furthermore, if the number of times of calling a certain log processing mode is the largest, the priority of the log processing mode can be set to be the highest priority, so that the log processing mode is selected by sensing the analysis hotspot requirement of a data platform, and the real-time performance of log data analysis is met.
Optionally, the priorities of multiple log processing modes may be sorted, and a log processing mode with the top N bits of priorities is selected from the multiple log processing modes, where N is an integer greater than or equal to 1.
A data processing module 403, configured to process the service log data by using the target log processing manner.
In a specific implementation, the multiple log processing modes at least include retaining fields required in a log analysis stage in the service log data, counting contents matched with preset keywords in the service log data, counting and time-stamping the same contents in the service log data within a preset time range, and establishing an index of a designated field in the service log data. At least one log processing mode can be selected, and the selected log processing mode is adopted to preprocess the service log data, so that the output of the log data is reduced, and the indexing speed of the log data is accelerated.
For example: as shown in fig. 1, the log preprocessing module includes a plurality of log processing interfaces (Top1 interface, Top2 interface, Top3 interface, … …), each of which uses different log processing modes to process the service log data, and may select a Top1 interface, a Top2 interface, and a Top3 interface to process the service log data at the same time, and obtain a result after each of the log processing interfaces processes the log service data, and finally transmit the result to all data platforms.
Optionally, as shown in fig. 4, the apparatus in the embodiment of the present invention may further include:
and a data sending module 404, configured to store the processed service log data in the data platform, so as to save a local log storage space of the application server, and solve a problem that the storage and transmission of the local log consume a CPU.
In the embodiment of the invention, firstly, business log data of application operation is obtained from an application server, interface sequencing data is obtained from a data platform, and then a target log processing mode is selected from a plurality of log processing modes according to the calling times of each log processing mode; and finally, processing the service log data by using a target log processing mode, determining the log processing mode by acquiring analysis hotspot requirements counted by a data platform, and preprocessing the log data by adopting the determined log processing mode before analyzing the log data, so that the number of the service log data is reduced, and the log data analysis efficiency and the log analysis accuracy are improved.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a log data processing apparatus according to the present invention. As shown, the apparatus may include: at least one processor 501, such as a CPU, at least one communication interface 502, at least one memory 503, and at least one communication bus 504. Wherein a communication bus 504 is used to enable the connection communication between these components. The communication interface 502 of the device in the embodiment of the present invention is used for performing signaling or data communication with other node devices. The memory 503 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 503 may optionally be at least one storage device located remotely from the processor 501. A set of program codes is stored in the memory 503 and the processor 501 executes the program stored in the memory 503, performs the method performed by the log data device or implements the function implemented by the log data device.
It should be noted that, for simplicity of description, the above-mentioned embodiments of the method are described as a series of acts or combinations, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: flash Memory disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The content downloading method, the related device and the system provided by the embodiment of the present invention are described in detail above, and a specific example is applied in the text to explain the principle and the embodiment of the present invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (12)
1. A method of processing log data, the method comprising:
acquiring service log data of application operation from an application server, and acquiring interface sorting data from a data platform, wherein the interface sorting data comprises the calling times of each log processing mode in a plurality of log processing modes;
determining the priority of each log processing mode according to the calling times of each log processing mode, and selecting the log processing mode with the highest priority from the multiple log processing modes as a target log processing mode, wherein the log processing mode with the highest calling times has the highest priority;
and processing the service log data by using the target log processing mode.
2. The method of claim 1, wherein the plurality of log processing manners at least include retaining fields required for a log analysis stage in the service log data, counting contents in the service log data that match a preset keyword, counting and time-stamping the same contents in the service log data within a preset time range, and establishing an index of a designated field in the service log data.
3. The method of claim 1, wherein after obtaining the service log data of the application running from the application server, further comprising:
determining a read-write operation mode of the service log data;
and when the read-write operation mode of the service log data is data write-in operation, executing the operation step of acquiring interface sequencing data from the data platform.
4. The method of any of claims 1-3, wherein after processing the traffic log data using the target log processing mode, further comprising:
and storing the processed service log data to the data platform.
5. An apparatus for processing log data, the apparatus comprising:
the data acquisition module is used for acquiring service log data of application operation from an application server and acquiring interface sorting data from a data platform, wherein the interface sorting data comprises the calling times of each log processing mode in a plurality of log processing modes;
a mode selection module, configured to determine a priority of each log processing mode according to the number of calls of each log processing mode, and select a log processing mode with a highest priority from the multiple log processing modes as a target log processing mode, where the log processing mode with the highest number of calls in the multiple log processing modes has the highest priority;
and the data processing module is used for processing the service log data by using the target log processing mode.
6. The apparatus of claim 5, wherein the plurality of log processing manners at least include retaining fields required for a log analysis stage in the service log data, counting contents in the service log data that match a preset keyword, counting and time-stamping the same contents in the service log data within a preset time range, and establishing an index of a designated field in the service log data.
7. The apparatus of claim 5,
the data acquisition module is also used for determining a read-write operation mode of the service log data; and when the read-write operation mode of the service log data is data write-in operation, executing the operation step of acquiring interface sequencing data from the data platform.
8. The apparatus of any of claims 5-7, wherein the apparatus further comprises:
and the data sending module is used for storing the processed service log data to the data platform.
9. A log data processing apparatus characterized by comprising: the device comprises a memory, a communication bus and a processor, wherein the memory is used for storing program codes, and the processor is used for calling the program codes and executing the following operations:
acquiring service log data of application operation from an application server, and acquiring interface sorting data from a data platform, wherein the interface sorting data comprises the calling times of each log processing mode in a plurality of log processing modes;
determining the priority of each log processing mode according to the calling times of each log processing mode, and selecting the log processing mode with the highest priority from the multiple log processing modes as a target log processing mode, wherein the log processing mode with the highest calling times has the highest priority;
and processing the service log data by using the target log processing mode.
10. The apparatus of claim 9, wherein the plurality of log processing manners at least include retaining fields required for a log analysis stage in the service log data, counting contents in the service log data that match a preset keyword, counting and time-stamping the same contents in the service log data within a preset time range, and establishing an index of a designated field in the service log data.
11. The apparatus of claim 9, wherein the processor is further configured to perform the operational steps of:
determining a read-write operation mode of the service log data;
and when the read-write operation mode of the service log data is data write-in operation, executing the operation step of acquiring interface sequencing data from the data platform.
12. The apparatus of any of claims 9-11, wherein the processor is further configured to perform the operational steps of:
and storing the processed service log data to the data platform.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610849308.2A CN107870921B (en) | 2016-09-26 | 2016-09-26 | Log data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610849308.2A CN107870921B (en) | 2016-09-26 | 2016-09-26 | Log data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107870921A CN107870921A (en) | 2018-04-03 |
| CN107870921B true CN107870921B (en) | 2021-10-15 |
Family
ID=61751796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610849308.2A Active CN107870921B (en) | 2016-09-26 | 2016-09-26 | Log data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107870921B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109656988B (en) * | 2018-10-16 | 2024-05-03 | 平安科技（深圳）有限公司 | Cloud computing-based service operation and maintenance method, device, equipment and readable storage medium |
| CN110347443B (en) * | 2019-06-28 | 2021-02-19 | 联想(北京)有限公司 | Log processing method and log processing device |
| CN111145382A (en) * | 2019-11-27 | 2020-05-12 | 苏州智加科技有限公司 | Log data processing method and device of automatic driving system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499068A (en) * | 2008-02-01 | 2009-08-05 | 中兴通讯股份有限公司 | Data base call coverage statistics and analysis method based on log |
| US8452786B2 (en) * | 2011-05-06 | 2013-05-28 | Sap Ag | Systems and methods for business process logging |
| CN103414758A (en) * | 2013-07-19 | 2013-11-27 | 北京奇虎科技有限公司 | Method and device for processing logs |
| CN104391781A (en) * | 2014-10-24 | 2015-03-04 | 苏州阔地网络科技有限公司 | Processing method and system for log information |
| CN104978256A (en) * | 2014-04-10 | 2015-10-14 | 阿里巴巴集团控股有限公司 | Log output method and equipment |
| CN105068804A (en) * | 2015-08-07 | 2015-11-18 | 北京思特奇信息技术股份有限公司 | Data-driven App function module layout method, system and apparatus |
| CN105262812A (en) * | 2015-10-16 | 2016-01-20 | 浪潮(北京)电子信息产业有限公司 | Log data processing method based on cloud computing platform, log data processing device and log data processing system |
-
2016
- 2016-09-26 CN CN201610849308.2A patent/CN107870921B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499068A (en) * | 2008-02-01 | 2009-08-05 | 中兴通讯股份有限公司 | Data base call coverage statistics and analysis method based on log |
| US8452786B2 (en) * | 2011-05-06 | 2013-05-28 | Sap Ag | Systems and methods for business process logging |
| CN103414758A (en) * | 2013-07-19 | 2013-11-27 | 北京奇虎科技有限公司 | Method and device for processing logs |
| CN104978256A (en) * | 2014-04-10 | 2015-10-14 | 阿里巴巴集团控股有限公司 | Log output method and equipment |
| CN104391781A (en) * | 2014-10-24 | 2015-03-04 | 苏州阔地网络科技有限公司 | Processing method and system for log information |
| CN105068804A (en) * | 2015-08-07 | 2015-11-18 | 北京思特奇信息技术股份有限公司 | Data-driven App function module layout method, system and apparatus |
| CN105262812A (en) * | 2015-10-16 | 2016-01-20 | 浪潮(北京)电子信息产业有限公司 | Log data processing method based on cloud computing platform, log data processing device and log data processing system |
Non-Patent Citations (2)
| Title |
|---|
| Web日志挖掘技术的应用研究;谭文武;《无线互联科技》;20121215(第12期);第158页 * |
| 基于Hadoop平台的日志分析模型;于兆良等;《计算机工程与设计》;20160216(第02期);第338-344页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107870921A (en) | 2018-04-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107870921B (en) | Log data processing method and device | |
| CN106919438B (en) | Workflow processing method and framework device in virtualization environment | |
| CN110781180B (en) | Data screening method and data screening device | |
| CN111190727B (en) | Asynchronous memory destructor method and device, computer equipment and storage medium | |
| CN107589990B (en) | Data communication method and system based on thread pool | |
| CN110321364B (en) | Transaction data query method, device and terminal of credit card management system | |
| CN109815092B (en) | Automatic cloud platform expansion method and system | |
| CN111163018A (en) | Network equipment and method for reducing transmission delay thereof | |
| CN110928905A (en) | Data processing method and device | |
| CN114223189A (en) | Duration statistical method and device, electronic equipment and computer readable medium | |
| CN104462420A (en) | Method and device for executing query tasks on database | |
| CN112925634A (en) | Heterogeneous resource scheduling method and system | |
| CN107291743B (en) | Data calling method and device | |
| CN110442439B (en) | Task process processing method and device and computer equipment | |
| CN116126719A (en) | Interface testing method and device, electronic equipment and storage medium | |
| CN110188258B (en) | Method and device for acquiring external data by using crawler | |
| CN110865877A (en) | Task request response method and device | |
| CN110020166A (en) | A kind of data analysing method and relevant device | |
| CN113722141A (en) | Method and device for determining delay reason of data task, electronic equipment and medium | |
| CN113656731A (en) | Advertisement page processing method and device, electronic equipment and storage medium | |
| CN111782479A (en) | Log processing method and device, electronic equipment and computer readable storage medium | |
| CN113992493A (en) | Video processing method, system, device and storage medium | |
| CN111641959A (en) | Method and device for detecting network environment information | |
| CN113126779A (en) | Man-machine interaction method, device, equipment and system | |
| CN110365342A (en) | Waveform decoder method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200420 Address after: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Applicant after: HUAWEI TECHNOLOGIES Co.,Ltd. Address before: 301, A building, room 3, building 301, foreshore Road, No. 310053, Binjiang District, Zhejiang, Hangzhou Applicant before: Hangzhou Huawei Digital Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20211228 Address after: 450046 Floor 9, building 1, Zhengshang Boya Plaza, Longzihu wisdom Island, Zhengdong New Area, Zhengzhou City, Henan Province Patentee after: xFusion Digital Technologies Co., Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd. |