CN103853743A - Distributed system and log query method thereof - Google Patents
Distributed system and log query method thereof Download PDFInfo
- Publication number
- CN103853743A CN103853743A CN201210501969.8A CN201210501969A CN103853743A CN 103853743 A CN103853743 A CN 103853743A CN 201210501969 A CN201210501969 A CN 201210501969A CN 103853743 A CN103853743 A CN 103853743A
- Authority
- CN
- China
- Prior art keywords
- log
- query
- file
- machine
- storing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000003860 storage Methods 0.000 claims abstract description 28
- 230000003993 interaction Effects 0.000 claims description 13
- 238000004458 analytical method Methods 0.000 claims description 9
- 230000002452 interceptive effect Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 description 6
- 230000002159 abnormal effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Fuzzy Systems (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a distributed system and a log query method thereof. The log query method comprises the steps that a control unit inputs predetermined keywords and time information as parameter values into a predetermined log query foot script template, generates a log query foot script and sends the generated log query foot script to a machine cluster, and a log query unit of a machine in the machine cluster queries a log in a storage unit according to the log query foot script and returns the found log to the control unit. According to the adopted technical scheme, real-time query of logs in machines can be realized.
Description
[ technical field ] A method for producing a semiconductor device
The invention relates to a search technology in the field of internet application, in particular to a distributed system and a log query method thereof.
[ background of the invention ]
At present, hundreds of machines are often deployed in a large distributed cluster, the size of a log file of a program deployed on each machine is as high as hundreds of G or more, and when system debugging and program problem analysis are performed, operation and maintenance personnel need to input keywords to check the log file of the program on the machine. In the prior art, a method for querying a log file of a program on a machine is as follows: and copying the log files on each machine in the machine cluster to a built storage system, and building an inquiry index in the storage system for operation and maintenance personnel to inquire.
The current query mode of log files has the following problems:
1. a storage system needs to be built, and indexes are built for massive log files, so that the query cost is high.
2. Copying the log files on each machine in the machine cluster to the built storage system occupies a large amount of transmission time, and wastes a large amount of time resources and storage resources.
3. When the logs are inquired in the storage system, the logs are inquired by running the index algorithm, the index algorithm has many log files to inquire, the operation time is long, and the inquiry result can be obtained in hours, so the inquiry efficiency is low.
[ summary of the invention ]
The invention provides a distributed system and a log query method thereof, which can realize real-time query of logs in a machine.
The specific technical scheme of the invention is as follows:
according to a preferred embodiment of the present invention, a distributed system comprises: a control unit and a machine cluster; the machine cluster comprises more than one machine, and the machine comprises a log query unit and a storage unit; wherein,
the control unit is used for inputting preset keywords and time information as parameter values into a preset log query script template, generating a log query script and sending the log query script to the machine cluster;
and the log query unit is used for querying the logs in the storage unit according to the log query script and returning the found logs to the control unit.
In the above system, the system further includes:
and the interaction unit is used for receiving the keywords input by the user and the time information of the log, carrying the keywords and the time information in the query request and sending the query request to the control unit.
In the above system, the log query script template includes the following contents: the operation executed by the machine in the machine cluster, the file name prefix of the file for storing the query result, the directory for storing the log file in the machine and the time information.
In the above system, the log query unit specifically includes, when querying the log:
analyzing the log query script, and finding a directory for storing a log file in a storage unit of the machine according to the parameters obtained by analysis;
according to the name suffix of the log file, classifying the log file under the directory for storing the log file;
extracting a log file corresponding to the time information from each type of log file according to the time information;
and inquiring the corresponding log in the extracted log file according to the keyword according to the log inquiry operation required to be executed.
In the above system, the log query unit specifically includes, when returning the found log to the control unit:
and creating a file for storing the query result according to a preset file name prefix of the file for storing the query result, storing the queried log as the query result into the file for storing the query result, and reporting the file for storing the query result to the control unit.
In the above-mentioned system, the first and second sensors are arranged in a single circuit,
the control unit is also used for receiving the log reported by the log query unit of the machine; when the preset waiting overtime time is up, displaying the query result to the user through the interaction unit; the query result comprises: machine identification, log time, log classification, and specific content of the log.
A log query method of a distributed system comprises the following steps:
the control unit inputs preset keywords and time information as parameter values into a preset log query script template, generates a log query script and sends the log query script to the machine cluster;
and the log query unit of the machine in the machine cluster queries the log in the storage unit according to the log query script and returns the found log to the control unit.
In the above method, the method further comprises:
the interactive unit receives keywords input by a user and time information of a log, carries the keywords and the time information in a query request and sends the query request to the control unit.
In the above method, the log query script template includes the following contents: the operation executed by the machine in the machine cluster, the file name prefix of the file for storing the query result, the directory for storing the log file in the machine and the time information.
In the above method, the method for querying the log by the log querying unit comprises:
analyzing the log query script, and finding a directory for storing a log file in a storage unit of the machine according to the parameters obtained by analysis;
according to the name suffix of the log file, classifying the log file under the directory for storing the log file;
extracting a log file corresponding to the time information from each type of log file according to the time information;
and inquiring the corresponding log in the extracted log file according to the keyword according to the log inquiry operation required to be executed.
In the above method, the method for returning the found log to the control unit by the log query unit is as follows:
the log query unit creates a file for storing the query result according to a preset file name prefix of the file for storing the query result, stores the queried log as the query result into the file for storing the query result, and reports the file for storing the query result to the control unit.
In the above method, the method further comprises:
the control unit receives the log reported by the log query unit of the machine;
after the preset waiting timeout time is up, the control unit displays the query result to the user through the interaction unit; the query result comprises: machine identification, log time, log classification, and specific content of the log.
According to the technical scheme, the technical scheme provided by the invention has the following beneficial effects:
the control unit adopts multi-process concurrent control and management for the distributed machine cluster, and starts a single process for each machine to complete interactive control, so that real-time log query can be realized, the log real-time query completion time of the machine cluster is independent of the scale of the machine cluster, and the query time depends on the machine with the longest query time no matter the scale of the machine cluster, thereby having good expansibility.
[ description of the drawings ]
FIG. 1 is a schematic block diagram of a preferred embodiment of the present invention for implementing a distributed system;
fig. 2 is a flow chart of a preferred embodiment of the log query method for implementing a distributed system according to the present invention.
[ detailed description ] embodiments
The basic idea of the invention is: the control unit inputs preset keywords and time information as parameter values into a preset log query script template, generates a log query script and sends the log query script to the machine cluster; and the log query unit of the machine in the machine cluster queries the log in the storage unit according to the log query script and returns the found log to the control unit.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.
The present invention provides a distributed system, fig. 1 is a schematic structural diagram of a preferred embodiment of implementing the distributed system of the present invention, as shown in fig. 1, the distributed system includes: the system comprises an interaction unit 10, a control unit 11 and a machine cluster; the machine cluster comprises more than one machine, the machine further comprising a log query unit 12 and a storage unit 13.
Based on the above system, the present invention provides a log query method for a distributed system, fig. 2 is a schematic flow chart of a preferred embodiment of the log query method for a distributed system, which is implemented by the present invention, and as shown in fig. 2, the preferred embodiment includes the following steps:
step 201, the interactive unit receives the keyword and time information input by the user and forwards the keyword and time information to the control unit.
Specifically, the interaction unit provides an input box and a time option for the user, so as to receive keywords input by the user according to the query requirement and time information of logs needing to be queried, wherein the time information may include a date and a specific time period.
The interactive unit carries the keyword and the time information in the query request and sends the query request to the control unit; in practical application, the interaction unit may be a web page, through which a user may input keywords and select time, and may also view query results corresponding to the keywords; the web page may be implemented using a Hypertext Preprocessor (PHP).
Here, the date and the time period of the log needing to be queried are selected in the interaction unit, so that the query range can be narrowed, the query time can be shortened, and the response speed of the webpage can be improved.
Step 202, the control unit inputs the keywords and the time information as parameter values into a preset log query script template, generates a log query script and sends the log query script to the machine cluster.
Specifically, the control unit can adopt a multi-process concurrent control and management mechanism to perform log query control and management on more than one machine in the machine cluster; when receiving the inquiry request from the interactive unit, the keyword and the time information are obtained from the inquiry request.
More than one script template is preset in the control unit, such as a log query script template, a flow statistics script template, a disk cleaning script template and the like; in the preferred embodiment, the log query is taken as an example, and the log query script template may be mcontrol (list, cmd, outfile, errfile, directory, time); the parameter value of the list parameter is an identifier of a machine in the machine cluster, such as a machine name, a domain name or an IP address, and the control unit can read the identifier of each machine in the machine cluster from an hlist file preset by a user; in the preferred embodiment, the parameter value of the cmd parameter represents the operation of querying the log, the cmd parameter serves as a specific log query command, and the directory parameter and the time parameter can serve as the input of the log query command when the log query unit queries the log; the parameter value of the outfile parameter is the file name prefix of the file for storing the query result, the parameter value of the errfile parameter is the prefix of the file name of the file for storing the error result, the parameter value of the directory parameter is the directory for storing the log file in the machine, and the parameter value of the time parameter is the time information input by the user.
The key words input by the user are fields, and the control unit preprocesses the fields; for example, whether a keyword is an illegal input or a legal input is determined; if the keyword is a URL, judging whether the URL is a legal website or a non-legal website, or when the length of the URL is longer, reducing the URL, and deleting fields such as http,/, and the like.
The control unit takes the preprocessed keywords, the received time information and the identification of the machine as parameter values to input into a log query script template to generate a log query script, other parameter values in the log query script template are fixed and can be uniformly set, and only the three parameter values are variable; the control unit sends the generated log query script to the machines corresponding to the identifiers of the machines in the log query script in the machine cluster, so that the log query script is automatically distributed and deployed to all the machines in the distributed cluster, and thus, a plurality of processes and links can be established between the control unit and the machines in the machine cluster to control the machines to start a log query task and realize concurrent query operation of logs on the machines in the machine cluster; the machine cluster comprises more than one machine.
Step 203, the log query unit of the machine queries the log in the storage unit according to the log query script, and returns the found log to the control unit.
Specifically, an entity for executing an operation according to a script issued by the control unit is deployed in the machine cluster, and in the preferred embodiment, log query is taken as an example, and the entity is a log query unit; storage units are also deployed on the machines in the machine cluster.
After receiving the log query script sent by the control unit, the log query script is analyzed, the parameters obtained by analysis are used as the input parameters of the log query script, the operation to be executed is determined to be the log query operation according to the cmd parameters, then a directory for storing log files is found in a storage unit of the machine according to the parameter values of the directory parameters, and the storage unit stores massive log files, wherein the logs to be queried can be quickly found only according to the configured directory for storing the log files; the log query unit classifies the found log files in the directory for storing the log files according to name suffixes of the log files, for example, the log files are classified into construction types, integration types and the like; the log query unit extracts log files corresponding to dates and time periods in the time parameters from each type of log files according to the time parameters obtained through analysis, and queries corresponding logs from the extracted log files according to keywords obtained through analysis; the log query unit creates a file for storing the query result according to the parameter value of the outfile parameter, then stores the queried log as the query result into the file, and reports the file for storing the query result to the control unit; the log query unit also creates a file for storing an error result according to the parameter value of the errfile parameter, the file is empty under normal conditions, the log query unit records error information of log query only under abnormal conditions, such as failure in reading the log file, and the like, and the log query unit can report the abnormal conditions to the control unit through the file for storing the error result.
In the preferred embodiment, in the process of querying the log by the log querying unit, the control unit obtains and displays the querying progress of the log querying unit on each machine, for example, whether the querying is completed or not, whether the querying is abnormal or not, by monitoring the execution state of the process established with each machine.
And step 204, the control unit displays the found log to the user through the interaction unit.
Specifically, the control unit receives files for storing query results reported by log query units of all machines, summarizes the query results and displays the summarized query results to a user through the interaction unit; and the summarized query result comprises the identification of the machine, the log time, the log classification and the specific content of the log.
The control unit can also store the received query results to the local, and the query results of each machine can be stored in a file separately.
Here, the waiting timeout time may be configured in the control unit in advance according to the query requirement, and even if there is a file for storing the query result that is not reported by the log query unit of the machine within the waiting timeout time, the control unit displays the query result reported by the log query unit of another machine to the user after the waiting timeout time expires.
To implement the above method, as shown in fig. 1, in a distributed system:
the interactive unit 10 is configured to receive a keyword and time information of a log input by a user, carry the keyword and the time information in a query request, and send the query request to the control unit 11.
And the control unit 11 is configured to input preset keywords and time information as parameter values to a preset log query script template, generate a log query script, and send the log query script to the machine cluster.
And the log query unit 12 is configured to query a log in the storage unit 13 according to the log query script, and return the found log to the control unit 11.
Wherein the log query script template comprises the following contents: the operation executed by the machine in the machine cluster, the file name prefix of the file for storing the query result, the directory for storing the log file in the machine and the time information.
The log query unit 12 specifically includes, when querying the log:
analyzing the log query script, and finding a directory for storing a log file in a storage unit of the machine according to the parameters obtained by analysis;
according to the name suffix of the log file, classifying the log file under the directory for storing the log file;
extracting a log file corresponding to the time information from each type of log file according to the time information;
and inquiring the corresponding log in the extracted log file according to the keyword according to the log inquiry operation required to be executed.
The log query unit 12 specifically includes, when returning the found log to the control unit:
and creating a file for storing the query result according to a preset file name prefix of the file for storing the query result, storing the queried log as the query result into the file for storing the query result, and reporting the file for storing the query result to the control unit.
The control unit 11 is further configured to receive a log reported by a log query unit 12 of the machine; when the preset waiting overtime time is up, displaying the query result to the user through the interaction unit 10; the query result comprises: machine identification, log time, log classification, and specific content of the log.
The technical scheme of the invention has the following beneficial technical effects:
1. the control unit adopts multi-process concurrent control and management for the distributed machine cluster, and starts a single process for each machine to complete interactive control, so that real-time log query can be realized, the log real-time query completion time of the machine cluster is independent of the scale of the machine cluster, and the query time depends on the machine with the longest query time no matter the scale of the machine cluster, thereby having good expansibility. For example, for a machine cluster with 30 machines, after the technical scheme of the invention is applied to query the URL, the query time of a single URL is about 4s, and the previous process of saving the log file to the storage system and establishing the index needs at least one hour, so that the requirement of real-time query cannot be met.
2. The distributed system can be applied to any distributed cluster, such as query of other files, flow statistics, disk cleaning and the like, and only an operation execution entity (such as a log query unit) needs to be upgraded aiming at different distributed systems.
3. And a special storage system is not required to be created and an index is not required to be established, so that bandwidth resources and storage space can be saved, the query cost is reduced, and the query efficiency is improved.
4. The control unit can flexibly configure the log query unit to meet different query requirements.
5. The distributed system has a simple and clear structure, is uniformly controlled by the control unit, and has high feasibility.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (12)
1. A distributed system, the system comprising: a control unit and a machine cluster; the machine cluster comprises more than one machine, and the machine comprises a log query unit and a storage unit; wherein,
the control unit is used for inputting preset keywords and time information as parameter values into a preset log query script template, generating a log query script and sending the log query script to the machine cluster;
and the log query unit is used for querying the logs in the storage unit according to the log query script and returning the found logs to the control unit.
2. The system of claim 1, further comprising:
and the interaction unit is used for receiving the keywords input by the user and the time information of the log, carrying the keywords and the time information in the query request and sending the query request to the control unit.
3. The system of claim 1, wherein the log query script template comprises the following: the operation executed by the machine in the machine cluster, the file name prefix of the file for storing the query result, the directory for storing the log file in the machine and the time information.
4. The system according to claim 1, wherein the log query unit specifically includes, when querying the log:
analyzing the log query script, and finding a directory for storing a log file in a storage unit of the machine according to the parameters obtained by analysis;
according to the name suffix of the log file, classifying the log file under the directory for storing the log file;
extracting a log file corresponding to the time information from each type of log file according to the time information;
and inquiring the corresponding log in the extracted log file according to the keyword according to the log inquiry operation required to be executed.
5. The system according to claim 1, wherein the log query unit, when returning the found log to the control unit, specifically comprises:
and creating a file for storing the query result according to a preset file name prefix of the file for storing the query result, storing the queried log as the query result into the file for storing the query result, and reporting the file for storing the query result to the control unit.
6. The system of claim 1,
the control unit is also used for receiving the log reported by the log query unit of the machine; when the preset waiting overtime time is up, displaying the query result to the user through the interaction unit; the query result comprises: machine identification, log time, log classification, and specific content of the log.
7. A log query method of a distributed system is characterized by comprising the following steps:
the control unit inputs preset keywords and time information as parameter values into a preset log query script template, generates a log query script and sends the log query script to the machine cluster;
and the log query unit of the machine in the machine cluster queries the log in the storage unit according to the log query script and returns the found log to the control unit.
8. The method of claim 7, further comprising:
the interactive unit receives keywords input by a user and time information of a log, carries the keywords and the time information in a query request and sends the query request to the control unit.
9. The method of claim 7, wherein the log query script template comprises the following: the operation executed by the machine in the machine cluster, the file name prefix of the file for storing the query result, the directory for storing the log file in the machine and the time information.
10. The method of claim 7, wherein the log querying unit queries the log by:
analyzing the log query script, and finding a directory for storing a log file in a storage unit of the machine according to the parameters obtained by analysis;
according to the name suffix of the log file, classifying the log file under the directory for storing the log file;
extracting a log file corresponding to the time information from each type of log file according to the time information;
and inquiring the corresponding log in the extracted log file according to the keyword according to the log inquiry operation required to be executed.
11. The method of claim 7, wherein the method for the log query unit to return the found log to the control unit is as follows:
the log query unit creates a file for storing the query result according to a preset file name prefix of the file for storing the query result, stores the queried log as the query result into the file for storing the query result, and reports the file for storing the query result to the control unit.
12. The method of claim 7, further comprising:
the control unit receives the log reported by the log query unit of the machine;
after the preset waiting timeout time is up, the control unit displays the query result to the user through the interaction unit; the query result comprises: machine identification, log time, log classification, and specific content of the log.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210501969.8A CN103853743A (en) | 2012-11-29 | 2012-11-29 | Distributed system and log query method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210501969.8A CN103853743A (en) | 2012-11-29 | 2012-11-29 | Distributed system and log query method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103853743A true CN103853743A (en) | 2014-06-11 |
Family
ID=50861410
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210501969.8A Pending CN103853743A (en) | 2012-11-29 | 2012-11-29 | Distributed system and log query method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103853743A (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105099764A (en) * | 2015-06-29 | 2015-11-25 | 百度在线网络技术（北京）有限公司 | Log processing method and log processing device |
| CN105117411A (en) * | 2015-07-29 | 2015-12-02 | 上海斐讯数据通信技术有限公司 | Server log file management system and method |
| CN105740348A (en) * | 2016-01-25 | 2016-07-06 | 武汉精伦电气有限公司 | Key log information extraction method and system based on distributed terminal system |
| CN105824958A (en) * | 2016-03-31 | 2016-08-03 | 广州华多网络科技有限公司 | Method, device and system for inquiring log |
| CN105897834A (en) * | 2015-12-04 | 2016-08-24 | 乐视网信息技术（北京）股份有限公司 | Hive client, Hive server and Hive execution log remote monitoring system and method |
| CN106227727A (en) * | 2016-06-30 | 2016-12-14 | 乐视控股（北京）有限公司 | Daily record update method, device and the system of a kind of distributed system |
| CN106682868A (en) * | 2015-11-10 | 2017-05-17 | 北京计算机技术及应用研究所 | Online data extracting device and method for office automation equipment storage component |
| CN106908258A (en) * | 2015-12-23 | 2017-06-30 | 阿里巴巴集团控股有限公司 | The method and device of performance test |
| CN107465549A (en) * | 2017-08-18 | 2017-12-12 | 北京奇艺世纪科技有限公司 | A kind of distributed message processing method and system |
| CN107644017A (en) * | 2016-07-20 | 2018-01-30 | 平安科技（深圳）有限公司 | The querying method and device of journal file |
| WO2018035799A1 (en) * | 2016-08-25 | 2018-03-01 | 阿里巴巴集团控股有限公司 | Data query method, application and database servers, middleware, and system |
| CN107784050A (en) * | 2016-12-14 | 2018-03-09 | 平安科技（深圳）有限公司 | Log information lookup method and device |
| CN107943591A (en) * | 2017-12-05 | 2018-04-20 | 郑州云海信息技术有限公司 | A kind of method of more host processes management based on linux |
| CN108427705A (en) * | 2018-01-17 | 2018-08-21 | 平安科技（深圳）有限公司 | Electronic device, distributed system journal querying method and storage medium |
| CN108737467A (en) * | 2017-04-19 | 2018-11-02 | 腾讯科技（深圳）有限公司 | A kind of server log inspection method, device and system |
| CN109885548A (en) * | 2019-02-22 | 2019-06-14 | 网易（杭州）网络有限公司 | Log inquiring method, device, storage medium and electronic device |
| CN112463738A (en) * | 2020-11-20 | 2021-03-09 | 中国建设银行股份有限公司 | Log retrieval method, system, computer device and storage medium |
| CN112579009A (en) * | 2019-09-30 | 2021-03-30 | 北京国双科技有限公司 | Error log printing method and device, storage medium and electronic equipment |
| CN113590372A (en) * | 2021-08-10 | 2021-11-02 | 平安普惠企业管理有限公司 | Log-based link tracking method and device, computer equipment and storage medium |
| CN113765694A (en) * | 2021-03-24 | 2021-12-07 | 北京京东拓先科技有限公司 | Log information display method and device, electronic equipment and computer readable medium |
| WO2023123397A1 (en) * | 2021-12-31 | 2023-07-06 | 深圳晶泰科技有限公司 | Multi-cloud log processing method, storage method, and computer storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101582087A (en) * | 2009-06-16 | 2009-11-18 | 恒生电子股份有限公司 | Log extracting method and device |
| US20100228784A1 (en) * | 2009-02-23 | 2010-09-09 | Iron Mountain Incorporated | Methods and Systems for Single Instance Storage of Asset Parts |
| CN101902505A (en) * | 2009-05-31 | 2010-12-01 | 中国科学院计算机网络信息中心 | Distributed DNS inquiry log real-time statistic device and method thereof |
| US20110179013A1 (en) * | 2010-01-21 | 2011-07-21 | Microsoft Corporation | Search Log Online Analytic Processing |
| US20120197928A1 (en) * | 2011-01-31 | 2012-08-02 | Splunk Inc. | Real time searching and reporting |
-
2012
- 2012-11-29 CN CN201210501969.8A patent/CN103853743A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100228784A1 (en) * | 2009-02-23 | 2010-09-09 | Iron Mountain Incorporated | Methods and Systems for Single Instance Storage of Asset Parts |
| CN101902505A (en) * | 2009-05-31 | 2010-12-01 | 中国科学院计算机网络信息中心 | Distributed DNS inquiry log real-time statistic device and method thereof |
| CN101582087A (en) * | 2009-06-16 | 2009-11-18 | 恒生电子股份有限公司 | Log extracting method and device |
| US20110179013A1 (en) * | 2010-01-21 | 2011-07-21 | Microsoft Corporation | Search Log Online Analytic Processing |
| US20120197928A1 (en) * | 2011-01-31 | 2012-08-02 | Splunk Inc. | Real time searching and reporting |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105099764A (en) * | 2015-06-29 | 2015-11-25 | 百度在线网络技术（北京）有限公司 | Log processing method and log processing device |
| CN105099764B (en) * | 2015-06-29 | 2019-01-18 | 北京音之邦文化科技有限公司 | Log processing method and device |
| CN105117411A (en) * | 2015-07-29 | 2015-12-02 | 上海斐讯数据通信技术有限公司 | Server log file management system and method |
| CN106682868B (en) * | 2015-11-10 | 2021-01-08 | 北京计算机技术及应用研究所 | On-line data extraction device and method for office automation equipment storage component |
| CN106682868A (en) * | 2015-11-10 | 2017-05-17 | 北京计算机技术及应用研究所 | Online data extracting device and method for office automation equipment storage component |
| CN105897834A (en) * | 2015-12-04 | 2016-08-24 | 乐视网信息技术（北京）股份有限公司 | Hive client, Hive server and Hive execution log remote monitoring system and method |
| CN106908258A (en) * | 2015-12-23 | 2017-06-30 | 阿里巴巴集团控股有限公司 | The method and device of performance test |
| CN105740348A (en) * | 2016-01-25 | 2016-07-06 | 武汉精伦电气有限公司 | Key log information extraction method and system based on distributed terminal system |
| CN105824958A (en) * | 2016-03-31 | 2016-08-03 | 广州华多网络科技有限公司 | Method, device and system for inquiring log |
| CN105824958B (en) * | 2016-03-31 | 2019-06-18 | 广州华多网络科技有限公司 | A kind of methods, devices and systems of inquiry log |
| CN106227727A (en) * | 2016-06-30 | 2016-12-14 | 乐视控股（北京）有限公司 | Daily record update method, device and the system of a kind of distributed system |
| CN107644017A (en) * | 2016-07-20 | 2018-01-30 | 平安科技（深圳）有限公司 | The querying method and device of journal file |
| WO2018035799A1 (en) * | 2016-08-25 | 2018-03-01 | 阿里巴巴集团控股有限公司 | Data query method, application and database servers, middleware, and system |
| CN107784050A (en) * | 2016-12-14 | 2018-03-09 | 平安科技（深圳）有限公司 | Log information lookup method and device |
| CN108737467A (en) * | 2017-04-19 | 2018-11-02 | 腾讯科技（深圳）有限公司 | A kind of server log inspection method, device and system |
| CN108737467B (en) * | 2017-04-19 | 2021-03-26 | 腾讯科技（深圳）有限公司 | Server log viewing method, device and system |
| CN107465549A (en) * | 2017-08-18 | 2017-12-12 | 北京奇艺世纪科技有限公司 | A kind of distributed message processing method and system |
| CN107465549B (en) * | 2017-08-18 | 2021-03-02 | 北京奇艺世纪科技有限公司 | Distributed message processing method and system |
| CN107943591A (en) * | 2017-12-05 | 2018-04-20 | 郑州云海信息技术有限公司 | A kind of method of more host processes management based on linux |
| CN108427705B (en) * | 2018-01-17 | 2022-04-12 | 平安科技（深圳）有限公司 | Electronic device, distributed system log query method and storage medium |
| WO2019140828A1 (en) * | 2018-01-17 | 2019-07-25 | 平安科技（深圳）有限公司 | Electronic apparatus, method for querying logs in distributed system, and storage medium |
| CN108427705A (en) * | 2018-01-17 | 2018-08-21 | 平安科技（深圳）有限公司 | Electronic device, distributed system journal querying method and storage medium |
| CN109885548A (en) * | 2019-02-22 | 2019-06-14 | 网易（杭州）网络有限公司 | Log inquiring method, device, storage medium and electronic device |
| CN112579009A (en) * | 2019-09-30 | 2021-03-30 | 北京国双科技有限公司 | Error log printing method and device, storage medium and electronic equipment |
| CN112463738A (en) * | 2020-11-20 | 2021-03-09 | 中国建设银行股份有限公司 | Log retrieval method, system, computer device and storage medium |
| CN113765694A (en) * | 2021-03-24 | 2021-12-07 | 北京京东拓先科技有限公司 | Log information display method and device, electronic equipment and computer readable medium |
| CN113765694B (en) * | 2021-03-24 | 2024-04-19 | 北京京东拓先科技有限公司 | Log information display method, device, electronic equipment and computer readable medium |
| CN113590372A (en) * | 2021-08-10 | 2021-11-02 | 平安普惠企业管理有限公司 | Log-based link tracking method and device, computer equipment and storage medium |
| WO2023123397A1 (en) * | 2021-12-31 | 2023-07-06 | 深圳晶泰科技有限公司 | Multi-cloud log processing method, storage method, and computer storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103853743A (en) | Distributed system and log query method thereof | |
| CN107087001B (en) | distributed internet important address space retrieval system | |
| CN111522922B (en) | Log information query method and device, storage medium and computer equipment | |
| CN101651707B (en) | Method for automatically acquiring user behavior log of network | |
| CN107273409B (en) | Network data acquisition, storage and processing method and system | |
| CN107145489B (en) | Information statistics method and device for client application based on cloud platform | |
| CN109800207B (en) | Log analysis method, device and equipment and computer readable storage medium | |
| CN102710795B (en) | Hotspot collecting method and device | |
| US20100088197A1 (en) | Systems and methods for generating remote system inventory capable of differential update reports | |
| CN104067281A (en) | Clustering event data by multiple time dimensions | |
| CN101902505A (en) | Distributed DNS inquiry log real-time statistic device and method thereof | |
| CN104268241B (en) | Attribute display method in a kind of configuration database | |
| CN105743730A (en) | Method and system used for providing real-time monitoring for webpage service of mobile terminal | |
| CN112491602A (en) | Behavior data monitoring method and device, computer equipment and medium | |
| CN108052358B (en) | Distributed deployment system and method | |
| CN101441629A (en) | Automatic acquiring method of non-structured web page information | |
| CN114528457B (en) | Web fingerprint detection method and related equipment | |
| CN111898036B (en) | Behavior data collecting and processing system and method | |
| CN110737634A (en) | Log searching method, client, server and computer readable storage medium | |
| CN107491463B (en) | Optimization method and system for data query | |
| CN102253948B (en) | Method and device for searching information in multi-source information system | |
| CN110941530A (en) | Method and device for acquiring monitoring data, computer equipment and storage medium | |
| CN103532737A (en) | Method, device and system for processing various types of alarms | |
| CN111078975B (en) | Multi-node incremental data acquisition system and acquisition method | |
| CN114937316B (en) | Software fault detection method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140611 |