CN103678092A - Log analysis method and system - Google Patents
Log analysis method and system Download PDFInfo
- Publication number
- CN103678092A CN103678092A CN201310743923.1A CN201310743923A CN103678092A CN 103678092 A CN103678092 A CN 103678092A CN 201310743923 A CN201310743923 A CN 201310743923A CN 103678092 A CN103678092 A CN 103678092A
- Authority
- CN
- China
- Prior art keywords
- analysis
- data
- processing unit
- data processing
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to a log analysis method and system. The log analysis method includes the following steps that a data processing unit receives a user request sent by a client end; the data processing unit processes the user request as a task message and sends the task message to a data analysis unit; the data analysis unit carries out log analysis according to the received task message, generates an analysis result and sends the analysis result to the data processing unit; the data processing unit processes the analysis result as a reply message and sends the reply message to the client end. According to the log analysis method and system, real-time performance is achieved, log information analysis can be carried out on original data at any dimensionalities, and the requirements of users for multidimensional data query can be met.
Description
Technical field
The present invention relates to computer realm, relate in particular to a kind of log analysis method and system.
Background technology
Along with the develop rapidly of computer technology, data volume increases sharply, and the accumulation of data is also increasing, and this also just means that the mass data epoch arrive.In carrying out transmission, exchange and the processing procedure of data, security is a very important factor, and therefore, the much equipment relevant to information processing (as fire wall, system for monitoring intrusion, router and server etc.) all can produce daily record.Log recording on equipment and the various things that on network, occur every day, we can be by understanding the situation of each equipment and whole network to methods such as the inquiry of daily record, statistics.
Yet, in the face of the daily record data of magnanimity, how could add up more accurately sooner, to analyze and want the information of obtaining to become the difficult problem that we will face.For this reason, the people such as the Ariel Rabkin of Princeton University have proposed a kind of " distributed data analyzing method ", be about to original log data and carry out the gathering in early stage, gather, put in storage, analyze, thereby reduced data scale, thereby reached the requirement of " real-time ".But, in existing log analysis technology, major part all take " daily record data centralized stores " be prerequisite, and log analysis does not have real-time; Especially under the environment of wide area network, during due to " log information transmission " and analysis, " each equipment room information interaction " more can cause real-time significantly to decline.And in prior art, not only when data gather in earlier stage, increased the transmission scale of data, and paid the cost of " losing other dimension data information ", cannot meet the demand of various dimensions data query; And prior art also exists that cost is high, efficiency is low and the shortcoming such as consuming time.
Summary of the invention
The object of the invention is for the problems referred to above, a kind of have high, the real-time log analysis method of efficiency and system are provided.
For achieving the above object, the invention provides a kind of log analysis method, comprise the following steps:
Data processing unit receives user's request that client sends;
Described data processing unit is treated to mission bit stream by described user's request, and described mission bit stream is sent to described data analysis unit;
Described data analysis unit is carried out log analysis according to the described mission bit stream receiving and is generated analysis result, and described analysis result is sent to data processing unit;
Described data processing unit is treated to return information to described analysis result, and described return information is sent to described client.
Preferably, described data processing unit is treated to mission bit stream by described user request and specifically comprises:
Data processing unit is converted to data call parameter by described user's request, described call parameters is generated to set of tasks, and mission bit stream corresponding to described set of tasks sent to data analysis unit.
Preferably, described data processing unit is treated to mission bit stream by described user request and specifically also comprises:
Judge whether described set of tasks is stored in local cache;
If described set of tasks is stored in the middle of local cache, by described set of tasks at local Cache associativity to this task; If described set of tasks is not stored in the middle of local cache, mission bit stream corresponding to described set of tasks sends to described data analysis unit.
Preferably, described data processing unit is treated to mission bit stream by described user request and specifically also comprises:
Judge whether described user's request is converted to data call parameter;
If described user's request is converted to data call parameter, directly return; If described user's request is not converted to data call parameter, user's request is not converted to data call parameter, and in monitoring period, monitors in real time the executing state of this log analysis.
Preferably, the described executing state of monitoring in real time this log analysis in monitoring period specifically comprises:
If in described monitoring period, described data analysis unit sends to described data processing unit by analysis result, and described data processing unit has been updated to current state;
If in described monitoring period, described data analysis unit sends to several data processing units by error message, and described data processing unit is updated to failure by current state, and described data processing unit records described error message;
If in described monitoring period, described data analysis unit does not send information to described data processing unit, and described data processing unit is updated to current state overtime, and described data processing unit records described overtime.
Preferably, described data processing list specifically comprises described analysis result is treated to described return information:
Described data processing unit receives the described analysis result that described data analysis unit sends, and described data processing unit carries out secondary Macro or mass analysis to described analysis result, and is return information by the result treatment of described secondary Macro or mass analysis.
A Log Analysis System, described system comprises:
Data processing unit, for collection, the analysis of user's request, is converted to mission bit stream by described user's request, and receives the analysis result that data analysis unit sends, and described analysis result is sent to client;
Data analysis unit, the mission bit stream sending for receiving described data processing unit, analyzes and generates described analysis result, and described analysis result is sent to described data processing unit to daily record data according to described mission bit stream.
Preferably, described data processing unit comprises: data access interface, and Data Collection scheduler module,
Data analysis module, the first memory module, the second memory module, data collection module;
Described data access interface, for receiving described user's request, is converted to the call parameters of described collection scheduler module and data analysis module by user's request, and receives the analysis result of institute's data analysis module;
Described Data Collection scheduler module, receive the call parameters of described data access interface, call parameters from described data access interface is converted to set of tasks, and issues all tasks in task-set, initialization and the cleaning of the conversion of asking for described user, monitoring, environment;
Described data analysis module, carries out Macro or mass analysis to the combined data in described the second memory module, and the result of combined data analysis is sent to described data access interface, for all data to described data processing unit, carries out secondary Macro or mass analysis;
Described data collection module, receives the task that described Data Collection scheduler module issues, and described mission bit stream is handed down to described data analysis module, and receives the analysis result of described data analysis module, for the collection with data that issues of task;
Described the first memory module, merges storage for receiving the described task-set of described Data Collection calling module;
Described the second memory module, stores for the analysis result that described data analysis module is sent.
The beneficial effect that the present invention brings is: log analysis method of the present invention and system have real-time, can in any dimension, carry out log information analysis to raw data, can meet the demand of user's various dimensions data query.
Accompanying drawing explanation
Fig. 1 is the theory diagram of one embodiment of the invention;
Fig. 2 is the process flow diagram of log analysis method in the embodiment of the present invention;
Fig. 3 is the schematic diagram that in one embodiment of the invention, Log Analysis System is disposed;
Fig. 4 is the system principle schematic diagram in Fig. 1 of the present invention.
Embodiment
Below by drawings and Examples, technical scheme of the present invention is described in further detail.
Fig. 1 is the theory diagram of one embodiment of the invention.
As shown in Figure 1, adopt this law in an embodiment of log analysis, user sends user by client 100 and asks 11, and client 100 asks 11 to send to data processing unit 200 user.In the present embodiment, user asks 11 can be the accurate inquiry of daily record, can be also the statistics (as server ip address, flow etc.) of data, can also be user's self-defined inquiry, carries out the inquiry of full dimension according to user's needs.Above user asks 11 to include but not limited to this.
In Fig. 1, data processing phrase 200 receives client 100 users and asks 11, and data processing unit 200 can ask user 11 to change, store and the operation such as analysis, makes user ask 11 to be treated to mission bit stream 12 and to send to data analysis unit 300.Data analysis unit 300 is analyzed according to the 12 pairs of local daily record datas of mission bit stream that receive, and log analysis is treated to analysis result 13 sends to data processing unit 200.Data processing unit 200 is processed according to receiving the analysis result 13 sending from data analysis unit 300, and analysis result 300 is treated to return information 14 sends to client 100.User returns to return information 14 by client 100 and obtains the data message that will inquire about.
Fig. 2 is the process flow diagram of one embodiment of the invention.
As shown in Figure 2, in step 401, the user that data processing unit 200 receives client 100 transmissions asks 11.User crosses client 100 by required information exchange and sends to data processing unit with user's form of 11 of asking.User's solicited message can be various, as described above, repeats no more herein.
In Fig. 2, step 402 asks 11 to be treated to mission bit stream 12 user for data processing unit 200, and mission bit stream 12 is sent to described data analysis unit 300.
When receiving the user who sends from client 100, data processing unit 200 asks after 11, first data processing unit 200 judges and whether analysis user request results is present in the middle of local cache, if data processing unit 200 finds user's request results Already in the middle of local cache, directly return.If data processing unit 200 does not exist in the middle of local cache through this user's request results of discriminatory analysis, data processing unit 200 asks 11 to be converted to call parameters user, and t analysis time is set, the executing state of this analysis of monitoring in real time in time t.
If in monitoring period t, data processing unit 200 receives the analysis result 13 sending from data analysis unit 300, and data processing Unit 200 have been updated to current state.
If in monitoring period t, data processing unit 200 receives the error message sending from data analysis unit 300, and data processing unit 200 is updated to failure by current state, and data processing unit 200 misregistration information.
If in monitoring period t, data processing unit 200 does not receive the analysis result 13 sending from data analysis unit 300, and data processing unit is updated to current state overtime, and data processing unit records timeout mode.
Data processing unit 200 is converted to set of tasks by call parameters, and store tasks set.When a user asks 11 to send to data processing unit 200, data processing unit 200 can ask 11 set of tasks that are finally converted to carry out discriminatory analysis to this user, in the time of in the middle of this set of tasks is stored in local cache, 200 of data processing units are associated with set of tasks in the middle of this subtask; If data processing unit 200 does not find that through discriminatory analysises this set of tasks stores in local cache, data processing unit 200 sends to data analysis unit 300 by this set of tasks with the form of mission bit stream 12.
In step 403, data analysis unit 300 is carried out log analysis according to the described mission bit stream 12 receiving and is generated analysis result 13, and analysis result 13 is sent to data processing unit 200.
In step 404,200 pairs of analysis results 13 of data processing unit are treated to return information 14, and return information 14 is sent to client 100.Data processing unit 200 receives the analysis result 13 sending from data analysis unit 300, and analysis result 13 is carried out to secondary Macro or mass analysis.Data processing unit is by the result store of secondary Macro or mass analysis and be treated to return information 14 and send to client 100.
The schematic diagram that in one embodiment of the invention, Log Analysis System is disposed as shown in Figure 3, the system preferred distribution formula of log analysis storage in the embodiment of the present invention, data processing unit 200 is deployed in data center server 310, data analysis unit 300 is deployed in the log server 320 in wide area network, and the log information on each equipment is stored in the log server 320 in network.Data analysis unit 300 can carry out being stored in original log data on log server in wide area network early stage gathering, gather, the processing such as warehouse-in and analysis, can reduce the scale of daily record data, can reach the effect of real-time.
Above-mentioned steps is the preferred steps of log analysis method of the present invention, and concrete protection domain be take claims as foundation.Wherein, step 404 and step 403 be the ordinal relation of " successively " not necessarily, and, when data analysis unit 300 returning part analysis result 13, step 404 also can be carried out, and can be once carrying out repeatedly in " analytic process ", to meet the real-time demand of user data query.
Fig. 4 is the systematic schematic diagram of an embodiment in Fig. 1 of the present invention.
As shown in Figure 4, data processing unit 200 comprises:
Data access interface 201, connects client 100, Data Collection scheduler module 202 and data analysis module 203.Data access interface 201 is accepted to ask 11 from the user of client, and asks 11 to resolve to the request of collection user, for Data Collection scheduler module 202 and data analysis module are called, and for return information 14 is returned to client 100.
Data Collection scheduler module 202, connection data access interface, the first memory module and data collection module 204, the collection request that it sends for receiving data access interface 201, and request task is changed, monitored, and for to the initialization of environment and cleaning.Data Collection scheduler module 202 receives the collection request that data access interfaces 201 send and also the request of collecting is converted into a series of mission bit streams that can carry out on distinct device, and mission bit stream is converted to set of tasks is stored in the first memory module 205.
Data analysis module 203, connection data access interface 201 and the second memory module, it carries out secondary Macro or mass analysis for all data to data center server 310.
Data collection module 204, connection data access interface 201, the first memory module 205, the second memory module 506 and data analysis unit 300, it issues and Data Collection for set of tasks.The connection of each equipment of data collection module 204 management, and the contextual information of record and each equipment connection, and data collection module 204 is issued to each the corresponding mission bit stream in set of tasks in the middle of corresponding equipment.Data collection module 204 also sends to the second memory module for the analysis result 13 that data analysis unit 300 is sent.
The first memory module 205, connection data is collected scheduler module 202 and data collection module 204, the set of tasks generating for storing Data Collection scheduler module 202.
The second memory module 206, connection data is collected scheduler module 202, data analysis module 203 and data collection module 204, the data that it sends for storing data analysis unit 300, and the second memory module can be used for the ephemeral data structure that Data Collection scheduler module 202 creates log analysis use.The analysis result 13 that data storing received data collection module 204 sends is also stored.
As shown in Fig. 4 and other accompanying drawing of instructions of the present invention, principle of work of the present invention is as follows:
User inputs the information (as the information such as accurate inquiry, statistical information and self-defined inquiry of daily record) of needs inquiry by client 100, client 100 asks 11 to send to data access interface 201 by receiving from user's Query Information and being converted to user, data access interface 201 asks 11 to resolve to the request of collection the user who receives, for Data Collection scheduler module 202 and data analysis module are called.First data access interface 201 is analyzed and is judged that user asks 11 whether to be stored in the middle of local cache, if user asks 11 to be present in the middle of local cache, data access interface 201 can directly return so; If user ask 11 do not exist with local cache in the middle of, data access interface 201, asks 11 to be converted to the request of collecting, the both call parameters of Data Collection scheduler module 202 and data analysis module 203 user.Data access interface 201 first calling data is collected scheduler module 202, and monitoring period t is set.If in monitoring period t, data processing unit 200 receives the analysis result 13 sending from data analysis unit 300, and data processing Unit 200 have been updated to current state.If in monitoring period t, data processing unit 200 receives the error message sending from data analysis unit 300, and data processing unit 200 is updated to failure by current state, and data processing unit 200 misregistration information.If in monitoring period t, data processing unit 200 does not receive the analysis result 13 sending from data analysis unit 300, and data processing unit is updated to current state overtime, and data processing unit records timeout mode.
Data access interface 201 is monitored the executing state of this log analysis in monitoring period.Data Collection scheduler module 202 receives the collection request that data access interfaces 201 send and also the request of collecting is converted into a series of mission bit streams that can carry out on distinct device, and mission bit stream is converted to set of tasks is stored in the first memory module 205.Data collection module 204 judges whether each mission bit stream in set of tasks has been buffered in the middle of the second memory module, if each mission bit stream in set of tasks has been buffered in the middle of the second memory module, this task requests is associated with this task so; Otherwise if the mission bit stream in set of tasks is not buffered in the middle of the second memory module, data collection module sends to data analysis unit 300 by mission bit stream corresponding in set of tasks so.
Data analysis unit 300 is deployed in log server 320, log information on the equipment that data analysis unit 300 gathers log server 320 carries out data analysis, and mission bit stream is carried out to log analysis generation analysis result 13, data analysis unit 300 sends to data collection module 204 by analysis result.Send user and ask 11 o'clock carrying out first client, data processing unit 200 can ask the script module of 11 correspondences to send to relevant log server 320 this user.Wherein, it is that user is asked to 11 character string parsings is internal representation structure that user asks 11 parsing, and according to the classification of log server 320, for relevant log server 320 respectively generates a mission bit stream, forms set of tasks.
Data collection module 204 receives the analysis result that data analysis unit 300 send, and data collection module 204 has been updated to current task status, and analysis result 13 is stored in the middle of the second memory module 206.The combined data that 203 pairs of data analysis modules are stored in the middle of the second memory module 206 is analyzed, and the result of analysis is sent to data access interface 201, and data access interface 201 is that return information sends to client 100 by the result treatment of analysis.
In sum, adopt log analysis method of the present invention and system for realize real-time information needed is inquired about in the log information of magnanimity, to there is the advantages such as real-time, quick and efficient.
Professional should further recognize, unit and the algorithm steps of each example of describing in conjunction with embodiment disclosed herein, can realize with electronic hardware, computer software or the combination of the two, for the interchangeability of hardware and software is clearly described, composition and the step of each example described according to function in the above description in general manner.These functions are carried out with hardware or software mode actually, depend on application-specific and the design constraint of technical scheme.Professional and technical personnel can specifically should be used for realizing described function with distinct methods to each, but this realization should not thought and exceeds scope of the present invention.
The software module that the method for describing in conjunction with embodiment disclosed herein or the step of algorithm can use hardware, processor to carry out, or the combination of the two is implemented.Software module can be placed in the storage medium of any other form known in random access memory (RAM), internal memory, ROM (read-only memory) (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technical field.
Above-described embodiment; object of the present invention, technical scheme and beneficial effect are further described; institute is understood that; the foregoing is only the specific embodiment of the present invention; the protection domain being not intended to limit the present invention; within the spirit and principles in the present invention all, any modification of making, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.
Claims (8)
1. a log analysis method, is characterized in that, comprises the following steps:
Data processing unit receives user's request that client sends;
Described data processing unit is treated to mission bit stream by described user's request, and described mission bit stream is sent to described data analysis unit;
Described data analysis unit is carried out log analysis according to the described mission bit stream receiving and is generated analysis result, and described analysis result is sent to data processing unit;
Described data processing unit is treated to return information to described analysis result, and described return information is sent to described client.
2. log analysis method as claimed in claim 1, is characterized in that, described data processing unit is treated to mission bit stream by described user's request and specifically comprises:
Data processing unit is converted to data call parameter by described user's request, described call parameters is generated to set of tasks, and mission bit stream corresponding to described set of tasks sent to data analysis unit.
3. log analysis method as claimed in claim 2, is characterized in that, described data processing unit is treated to mission bit stream by described user's request and specifically also comprises:
Judge whether described set of tasks is stored in local cache;
If described set of tasks is stored in the middle of local cache, by described set of tasks at local Cache associativity to this task; If described set of tasks is not stored in the middle of local cache, mission bit stream corresponding to described set of tasks sends to described data analysis unit.
4. log analysis method as claimed in claim 2, is characterized in that, described data processing unit is treated to mission bit stream by described user's request and specifically also comprises:
Judge whether described user's request is converted to data call parameter;
If described user's request is converted to data call parameter, directly return; If described user's request is not converted to data call parameter, user's request is not converted to data call parameter, and in monitoring period, monitors in real time the executing state of this log analysis.
5. log analysis method as claimed in claim 4, is characterized in that, the described executing state of monitoring in real time this log analysis in monitoring period specifically comprises:
If in described monitoring period, described data analysis unit sends to described data processing unit by analysis result, and described data processing unit has been updated to current state;
If in described monitoring period, described data analysis unit sends to several data processing units by error message, and described data processing unit is updated to failure by current state, and described data processing unit records described error message;
If in described monitoring period, described data analysis unit does not send information to described data processing unit, and described data processing unit is updated to current state overtime, and described data processing unit records described overtime.
6. log analysis method as claimed in claim 1, is characterized in that, described data processing list specifically comprises described analysis result is treated to described return information:
Described data processing unit receives the described analysis result that described data analysis unit sends, and described data processing unit carries out secondary Macro or mass analysis to described analysis result, and is return information by the result treatment of described secondary Macro or mass analysis.
7. a Log Analysis System, is characterized in that, described system comprises:
Data processing unit, for collection, the analysis of user's request, is converted to mission bit stream by described user's request, and receives the analysis result that data analysis unit sends, and described analysis result is sent to client;
Data analysis unit, the mission bit stream sending for receiving described data processing unit, analyzes and generates described analysis result, and described analysis result is sent to described data processing unit to daily record data according to described mission bit stream.
8. Log Analysis System as claimed in claim 7, is characterized in that, described data processing unit comprises: data access interface, Data Collection scheduler module, data analysis module, the first memory module, the second memory module, data collection module;
Described data access interface, for receiving described user's request, is converted to the call parameters of described collection scheduler module and data analysis module by user's request, and receives the analysis result of institute's data analysis module;
Described Data Collection scheduler module, receive the call parameters of described data access interface, call parameters from described data access interface is converted to set of tasks, and issues all tasks in task-set, initialization and the cleaning of the conversion of asking for described user, monitoring, environment;
Described data analysis module, carries out Macro or mass analysis to the combined data in described the second memory module, and the result of combined data analysis is sent to described data access interface, for all data to described data processing unit, carries out secondary Macro or mass analysis;
Described data collection module, receives the task that described Data Collection scheduler module issues, and described mission bit stream is handed down to described data analysis module, and receives the analysis result of described data analysis module, for the collection with data that issues of task;
Described the first memory module, merges storage for receiving the described task-set of described Data Collection calling module;
Described the second memory module, stores for the analysis result that described data analysis module is sent.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310743923.1A CN103678092B (en) | 2013-12-30 | 2013-12-30 | log analysis method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310743923.1A CN103678092B (en) | 2013-12-30 | 2013-12-30 | log analysis method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103678092A true CN103678092A (en) | 2014-03-26 |
| CN103678092B CN103678092B (en) | 2017-01-25 |
Family
ID=50315733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310743923.1A Active CN103678092B (en) | 2013-12-30 | 2013-12-30 | log analysis method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103678092B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108737467A (en) * | 2017-04-19 | 2018-11-02 | 腾讯科技（深圳）有限公司 | A kind of server log inspection method, device and system |
| CN109189993A (en) * | 2018-08-16 | 2019-01-11 | 深圳云安宝科技有限公司 | Big data processing method, device, server and storage medium |
| CN113868100A (en) * | 2021-10-27 | 2021-12-31 | 北京值得买科技股份有限公司 | Automatic dispatching and collecting system for E-commerce field data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102053983A (en) * | 2009-11-02 | 2011-05-11 | 阿里巴巴集团控股有限公司 | Method, system and device for querying vertical search |
| CN102915269A (en) * | 2012-09-20 | 2013-02-06 | 山东浪潮齐鲁软件产业股份有限公司 | Method for analyzing common logs of B/S (browser/server) software system |
| KR101252471B1 (en) * | 2012-10-05 | 2013-04-09 | 주식회사 시큐브 | Analysis of chain based integrated log correlation analysis system and method |
| US20130282788A1 (en) * | 2012-04-18 | 2013-10-24 | International Business Machines Corporation | File System Optimization by Log/Metadata Analysis |
-
2013
- 2013-12-30 CN CN201310743923.1A patent/CN103678092B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102053983A (en) * | 2009-11-02 | 2011-05-11 | 阿里巴巴集团控股有限公司 | Method, system and device for querying vertical search |
| US20130282788A1 (en) * | 2012-04-18 | 2013-10-24 | International Business Machines Corporation | File System Optimization by Log/Metadata Analysis |
| CN102915269A (en) * | 2012-09-20 | 2013-02-06 | 山东浪潮齐鲁软件产业股份有限公司 | Method for analyzing common logs of B/S (browser/server) software system |
| KR101252471B1 (en) * | 2012-10-05 | 2013-04-09 | 주식회사 시큐브 | Analysis of chain based integrated log correlation analysis system and method |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108737467A (en) * | 2017-04-19 | 2018-11-02 | 腾讯科技（深圳）有限公司 | A kind of server log inspection method, device and system |
| CN108737467B (en) * | 2017-04-19 | 2021-03-26 | 腾讯科技（深圳）有限公司 | Server log viewing method, device and system |
| CN109189993A (en) * | 2018-08-16 | 2019-01-11 | 深圳云安宝科技有限公司 | Big data processing method, device, server and storage medium |
| CN113868100A (en) * | 2021-10-27 | 2021-12-31 | 北京值得买科技股份有限公司 | Automatic dispatching and collecting system for E-commerce field data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103678092B (en) | 2017-01-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102404126B (en) | Charging method of cloud computing during application process | |
| Bolot et al. | Performance engineering of the World Wide Web: Application to dimensioning and cache design | |
| CN104639374B (en) | A kind of application deployment management system | |
| CN106709003A (en) | Hadoop-based mass log data processing method | |
| US8560684B2 (en) | Network transaction discovery | |
| CN106612199B (en) | A kind of network monitoring data is collected and analysis system and method | |
| CN103502990A (en) | Systems and methods for in-memory processing of events | |
| CN1835014A (en) | Method and system of monitoring on-line service risk | |
| CN106201754A (en) | Mission bit stream analyzes method and device | |
| Bumgardner et al. | Scalable hybrid stream and hadoop network analysis system | |
| CN110147470B (en) | Cross-machine-room data comparison system and method | |
| CN104699529B (en) | A kind of information acquisition method and device | |
| CN113676400B (en) | Cloud gateway system for Internet of things access of environment-friendly data acquisition equipment | |
| CN109510721A (en) | A kind of network log management method and system based on Syslog | |
| CN113347170A (en) | Intelligent analysis platform design method based on big data framework | |
| CN103678092A (en) | Log analysis method and system | |
| WO2009100528A1 (en) | System and method for estimating combined workloads of systems with uncorrelated and non-deterministic workload patterns | |
| CN208046653U (en) | A kind of electric power monitoring system network security monitoring main website plateform system | |
| CN108713310A (en) | Method and system for information security data in online and transmission to be compressed and optimized | |
| CN109460829A (en) | Based on the intelligent monitoring method and platform under big data processing and cloud transmission | |
| CN103916463A (en) | Network access statistical analysis method and system | |
| CN101471846A (en) | Network event analysis and management system | |
| CN116126415A (en) | Scene-based cloud component configuration recommendation method and system | |
| Gupta et al. | Fractional difference based hybrid model for resource prediction in cloud network | |
| CN107948280A (en) | The monitoring system of point and mirror image spectral fluxes is visited in a kind of combination |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |