CN109308329A - A kind of log collecting method and device based on cloud platform - Google Patents
A kind of log collecting method and device based on cloud platform Download PDFInfo
- Publication number
- CN109308329A CN109308329A CN201811129135.2A CN201811129135A CN109308329A CN 109308329 A CN109308329 A CN 109308329A CN 201811129135 A CN201811129135 A CN 201811129135A CN 109308329 A CN109308329 A CN 109308329A
- Authority
- CN
- China
- Prior art keywords
- log
- file
- data
- information
- message queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The present invention provides a kind of log collecting method and system based on cloud platform, this method comprises: the log information that S1, acquisition journal file generate, and according to the classification information of the log information, the log information is published in message queue in theme corresponding with log information classification；S2, Service tracing data are obtained from the corresponding theme of message queue, and the Service tracing data are parsed into the data into library format；S3, to it is described enter library format data carry out storage processing.The present invention can reduce the time of operation maintenance personnel investigation problem, and unified is centrally stored, can satisfy complicated inquiry and analysis, and the demand of user is understood by log.
Description
Technical field
The present invention relates to O&M technical field more particularly to a kind of log collecting methods and device based on cloud platform.
Background technique
With the development of cloud computing, large-scale data center has become the standard configuration of most of companies.These clusters
On run distributed basic software and service, provide a user data service and computing capability.In order to preferably service visitor
Log when family, user behaviors log when all data centers can all record user using Internet application and operation, then divides
On log each node also in the cluster that cloth application program generates, the generation of journal file be also it is distributed, dividing
The journal file Collection and conservation dissipated in the cluster is the first step on unified cloud storage platform.
Hadoop platform be Apache foundation support distributed software platform, including just go any management software,
A series of distributed softwares such as storage platform, computing platform.The log being collected into cluster is saved in often through network
In file system HDFS in Hadoop platform, HDFS is the distributed file system in Hadoop platform, for super large file
It is very good to support, its main feature is that write-once is supported repeatedly to read.During actual verification, find HDFS for Log Types
File support and it is bad.
Firstly, the distinguishing feature that log recording generates in cluster is: the log production being distributed on each node in the cluster
The continual generation daily record data of person.And primary hdfs does not support concurrent operations and additional operation, another party
In production environment, it is also very common to need to implement the case where handling log information for face, and the file of write state is not supported in Hdfs
Read operation.
Secondly, cluster log collector will obtain different stage for various SaaS application, cloud middleware
Log (info, warn, error, audio, debug, fatal) log etc., original collection terminal can not be to these on cloud
SaaS application, cloud middleware are directly taken.
Log is packaged into event, is sent to message-oriented middleware by last transmitter according to collected log；In this process
In, all logs are sent in different message-oriented middleware channels without classification, cause clogged with messages.
Summary of the invention
Technical problem to be solved by the present invention lies in, a kind of log collecting method and device based on cloud platform is provided,
This method and device realize that the log information for generating different journal files is exported to different message queue classification schemes, drop
The time of low operation maintenance personnel investigation problem, unified is centrally stored, can satisfy complicated inquiry and analysis, is understood by log
The demand of user.
In order to solve the above technical problem, the present invention provides a kind of log collecting method based on cloud platform, this method packets
Include following steps:
S1, the log information that journal file generates is obtained, and according to the classification information of the log information, by the log information
It is published in message queue in theme corresponding with log information classification；
S2, Service tracing data are obtained from the corresponding theme of message queue, and the Service tracing data are parsed into storage
The data of format；
S3, to it is described enter library format data carry out storage processing.
Wherein, the step S1 is specifically included:
Config directory of the timing scan carry on host obtains listed files information, it is undesirable to filter filename
File reads file content line by line, complete using row data as one when the beginning character of row data is the character of configuration
Record is published on the corresponding theme of message queue.
Wherein, the naming rule of the journal file are as follows:<systematic name>.log.yyyyMMddHH, yyyy expression of years,
MM indicates month, and dd indicates the date, and HH indicates the integral point time.
Wherein, when row data, which are published to the corresponding theme of message queue as a complete record, to fail, by the row
Data write-in cache corresponding catalogue in the following, cache file write it is full after, suffix will be added in order on old file name.
Wherein, the step S1 further include:
While log information is published to message queue corresponding theme, the state of record file acquisition, wherein state letter
The format of breath are as follows: file name: the last modification time of file: whether file acquisition is completed: the record number of file acquisition, record are adopted
The file designation of collection state are as follows: storage path+file theme of file.
Wherein, in the step S3:
To it is described enter library format data carry out storage processing include two ways, be respectively as follows:
(1) data parsed are written in file one by one, when file size reaches designated recorder number or reaches from last time
When the maximum time of storage, then order is imported by executing, file content is imported in database；
(2) one log of every parsing then passes through an insertion operation.
Wherein, the step S3 further include:
When parsing storage failure, write data into local wrong file catalogue.
Wherein, the log includes running log and audit log.
The present invention also provides a kind of log collection device based on cloud platform, including log collection sending module, message team
Column module, log parse memory module, wherein
The log collection sending module is used to obtain the log information of journal file generation, and according to point of the log information
The log information is published in message queue in theme corresponding with log information classification by category information；
Message queue module, for storing the log information；
The log parsing memory module is used to obtain the Service tracing from message queue from the corresponding theme of message queue
Data, and the Service tracing data are parsed into the data into library format, and storage processing is carried out to the data after parsing.
Wherein, the log collection sending module includes log collector and log transmission unit,
The log collector obtains listed files for scanning config directory, and obtains journal file according to listed files and generate
Log information, the log information is packaged to obtain log event, the log event includes classification information；
It is provided with classification information in the log sending module, when the classification information and the log thing of the log sending module
When the classification information matching of part, then the log event information is sent to corresponding in message queue by the log sending module
In classification scheme.
The beneficial effect of the embodiment of the present invention is: the present invention passes through the log information of different journal file generations is defeated
Out to different message queue classification schemes, in order to which the log information generated to different journal files carries out Life cycle
Management and use, the present invention can reduce the time of operation maintenance personnel investigation problem, and unified is centrally stored, can satisfy complexity
Inquiry and analysis, the demand of user is understood by log.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of flow diagram of log collecting method based on cloud platform of the invention.
Fig. 2 is a kind of structural schematic diagram of result collection system based on cloud platform of the invention.
Specific embodiment
The explanation of following embodiment be with reference to attached drawing, can be to the specific embodiment implemented to the example present invention.
It is illustrated referring to Fig. 1, the embodiment of the present invention one provides a kind of log collecting method, specifically includes following step
It is rapid:
S1, the log information that journal file generates is obtained, and according to the classification information of the log information, by the log information
It is published in message queue in theme corresponding with log information classification.
S2, the Service tracing data from message queue are obtained from the corresponding theme of message queue, and by the business
Tracking data is parsed into the data into library format.
S3, to it is described enter library format data carry out storage processing.
It is specifically described individually below from the storage of journal file, acquisition, parsing and storage.
One, the storage of journal file
In order to realize the acquisition of journal file, when log is stored, the data of write-in should be become be retrieved and polymerize
Index content each storage log is generated one and is fallen by the way that the data newly received are write inside new index file
Row's index, is called a section.Then it is presented a paper using another, all sections in recording indexes.And generate the data of section
Source is then a part in memory.After the data newly received enter memory, internal storage data brushes disk, generates one newly
Section, synchronized update of presenting a paper.While we write data into memory, an affairs day is also in addition had recorded in fact
Will.When refreshing, transaction log file is still kept intact.It, can be from submission position if be abnormal in the meantime
Beginning is set, the record in entire transaction log file is restored, guarantees data consistency.Until section is really brushed disk, and mention
When handing over file to be updated, transaction file is just emptied.
As distributed information log storage system, data copy is a standard configuration.Process is written in daily record data, also relates to pair
This.In the case where there is copy configuration, data are returned from host node is sent to host node response is connected to；More copies feelings
Under condition, the concurrent reading performance of log searching can be improved.
Two, the acquisition of journal file
If having write log in a receptacle, file is located inside container, can not access from host, can basis
A file inside it, realization of this mode with Docker is accessed in devicemapper, overlayfs of Docker
Mechanism is related；In addition, container operation very dynamic, log collection program is difficult to arrange, if there is a log collection
Program, which file configuration will collect on machine, its format be it is what kind of, send where because one
Surface vessel is to become in dynamic always on machine, it may increase by one or deletion one at any time, is not aware that this in advance
Can run how many a containers on platform machine, their configuration be how appearance, their log write on where, so not having
This capture program of face handle prepares method on a machine in advance, and here it is two more difficult places of file collection.
Log Directory by the way that container is arranged is mounted in the catalogue of host, is multiple days by the log cutting that should be transmitted
Will block is configured output according to following log rank.The priority of log information output is at least divided into five grades from high to low, point
It is not Fatal, ERROR, WARN, INFO, DEBUG.These ranks are used to specify the significance level of this log information.It is testing
Stage can open the log of all ranks, only allow to export INFO or more rank after online implementing (containing INFO).Diary information
It must support to be output to console and text file, journal file output is the catalogue of host, this catalogue passes through environmental variance
Write-in.
Wherein, the step S1 is specifically included: config directory of the timing scan carry on host, obtains listed files
Information filters the undesirable file of filename, reads file content line by line, when the beginning character of row data is configuration
When recordStartCharacter character, the corresponding theme of message queue is published to using row data as a complete record
On.
Specifically, the naming rule of the journal file are as follows:<systematic name>.log.yyyyMMddHH, yyyy indicate year
Part, MM indicates month, and dd indicates the date, and HH indicates the integral point time.More specifically, if the time be on September 12,12 2018 when,
Then yyyyMMddHH are as follows: 2018091212.
Wherein, the step S1 further include: when record publication failure, the row data write-in is cached under corresponding catalogue
Face, when cache file is write it is full after, old file name will be changed, wherein file write full finger file reached configuration cache file it is big
It is small.
Specifically, file size is limited to most 20Mb, i.e., when journal file is more than 20Mb, old journal file renaming
It is numbered for<systematic name>.log.yyyyMMddHH.n, n from 1 and is incremented by with step-length 1,<systematic name>is written in new log
.log.yyyyMMddHH.Here<systematic name>indicates that the english abbreviation of each actual items, english abbreviation must comply with list
Uniform provisions of the position to system english abbreviation.By way of changing old file name, simultaneously convenient for the identification of data cached processing routine
It is handled.
Wherein, the step S1 further include: while giving out information, the state of record file acquisition, wherein state letter
The format of breath are as follows: file name: the last modification time of file: whether file acquisition is completed: the record number of file acquisition, record are adopted
The file designation of collection state are as follows: path+theme.By while giving out information, recording the acquisition state of file, accident is avoided
In the case of, repeated acquisition log information.
It should be strongly noted that high level log alarm needs priority processing, by collection terminal by high level log write
Enter specific theme, then gone to subscribe to the theme with the role of consumer by monitoring system, it is handled to obtain log.
Three, log file data caches
Collector timing scan local cache catalogue obtains listed files, and reads the corresponding file content of listed files, to disappearing
It ceases queue and sends corresponding data, when data send failure, the data for sending failure are uploaded in CACHE DIRECTORY, are removed
The data of upload.
Four, the parsing of journal file, storage
From in the corresponding theme of message queue obtain message queue in Service tracing data, by Service tracing data be parsed into
The data of library format are respectively as follows: wherein the format being put in storage is divided into two kinds
A) files in batch is put in storage: in the data parsed write-in file one by one, when file size reach designated recorder/
When reaching the maximum time from last time storage, then order is imported by executing, file content is imported in database；
B) be put in storage by single log recording: one log of every parsing then passes through an insertion operation.
When parsing storage failure, write data into local wrong file catalogue.
Wherein Service tracing flow chart of data processing includes: to send Service tracing daily record data, receive Service tracing log number
According to, parsing Service tracing daily record data, save Service tracing daily record data and record save failure diary data.
Wherein, the parsing storage of journal file includes that the parsing of the parsing storage and audit log of running log is put in storage.
A kind of log collecting method based on cloud platform of the embodiment of the present invention is realized different journal file generations
Log information is exported to different message queue classification schemes, in order to which the log information generated to different journal files carries out entirely
The management and use of life cycle, the present invention can reduce the time of operation maintenance personnel investigation problem, and unified is centrally stored, can be with
Meet complicated inquiry and analysis, the demand of user is understood by log.
Based on the embodiment of the present invention one, second embodiment of the present invention provides a kind of log collection device based on cloud platform, such as
Shown in Fig. 2, which includes log collection sending module 1, message queue module 2, log parsing memory module 3, wherein
The log collection sending module 1 is used to obtain the log information of journal file generation, and according to the log information
The log information is published in message queue in theme corresponding with log information classification by classification information；
Message queue module 2, for storing the log information；
The log parsing memory module 3 is used to obtain the Service tracing from message queue from the corresponding theme of message queue
Data, and the Service tracing data are parsed into the data into library format, and storage processing is carried out to the data after parsing.
Wherein, the log collection sending module 1 includes log collector 11 and log transmission unit 12,
The log collector 11 obtains listed files for scanning config directory, and obtains journal file according to listed files and produce
The log information is packaged to obtain log event by raw log information, and the log event includes classification information；
It is provided with classification information in the log sending module 12, when the classification information and the log of the log sending module
When the classification information matching of event, then the log event information is sent in message queue corresponding by the log sending module
Classification scheme in.
Specifically, for the case where machine crashes, the process due to generating log can similarly die, so will not
New log is generated again, and there is no do not provide the case where service.For for the collector process dies the case where, it can drop really
The availability of low system.We have following three mode to improve the availability of system.
Firstly, all collectors start under the mode of supervise, if process dies can be weighed immediately by system
It opens, to provide service.
Secondly, carrying out survival monitoring to all collectors, discovery Agent, which dies, to alarm immediately and passes through Container Management
System carries out pull-up.
Finally, for very important log, using directly by log write distributed file system.
For log acquisition module, all events are all saved in local caching, then by transmission message team
In theme in column.Firstly, log sending module is exchanged with the affairs between message queue.Log sending module is done using affairs
Method guarantees the reliable delivery of event.Event and News Category are respectively encapsulated in affairs, these affairs are by preservation event
Message queue guarantees.It is reliable that this guarantees events in the point-to-point transmission of data flow.In multi-stage data stream, upper one
The event of grade and the event of next stage are contained in affairs, guarantee data reliably from a module to another module
Transfer.
The above disclosure is only the preferred embodiments of the present invention, cannot limit the right model of the present invention with this certainly
It encloses, therefore equivalent changes made in accordance with the claims of the present invention, is still within the scope of the present invention.
Claims (10)
1. a kind of log collecting method based on cloud platform, which comprises the steps of:
S1, the log information that journal file generates is obtained, and according to the classification information of the log information, by the log information
It is published in message queue in theme corresponding with log information classification；
S2, Service tracing data are obtained from the corresponding theme of message queue, and the Service tracing data are parsed into storage
The data of format；
S3, to it is described enter library format data carry out storage processing.
2. the method according to claim 1, wherein the step S1 is specifically included:
Config directory of the timing scan carry on host obtains listed files information, it is undesirable to filter filename
File reads file content line by line, complete using row data as one when the beginning character of row data is the setting character of configuration
Whole record is published on the corresponding theme of message queue.
3. according to the method described in claim 2, it is characterized by:
The naming rule of the journal file are as follows:<systematic name>.log.yyyyMMddHH, yyyy expression of years, MM indicate the moon
Part, dd indicates the date, and HH indicates the integral point time.
4. according to the method described in claim 3, it is characterized by:
When row data, which are published to the corresponding theme of message queue as a complete record, to fail, the row data are written slow
Deposit corresponding catalogue in the following, cache file write it is full after, suffix will be added in order on old file name.
5. according to the method described in claim 4, it is characterized in that, the step S1 further include:
While log information is published to message queue corresponding theme, the state of record file acquisition, wherein state letter
The format of breath are as follows: file name: the last modification time of file: whether file acquisition is completed: the record number of file acquisition, record are adopted
The file designation of collection state are as follows: storage path+file theme of file.
6. according to the method described in claim 5, it is characterized in that, in the step S3:
To it is described enter library format data carry out storage processing include two ways, be respectively as follows:
(1) data parsed are written in file one by one, when file size reaches designated recorder number or reaches from last time
When the maximum time of storage, then order is imported by executing, file content is imported in database；
(2) one log of every parsing then passes through an insertion operation.
7. according to the method described in claim 6, it is characterized in that, the step S3 further include:
It, will be in the local wrong file catalogue of the data that parsed write-in when parsing storage failure.
8. according to the method described in claim 7, it is characterized by:
The log includes running log and audit log.
9. a kind of log collection device based on cloud platform, which is characterized in that including log collection sending module, message queue mould
Block, log parse memory module, wherein
The log collection sending module is used to obtain the log information of journal file generation, and according to point of the log information
The log information is published in message queue in theme corresponding with log information classification by category information；
Message queue module, for storing the log information；
The log parsing memory module is used to obtain the Service tracing from message queue from the corresponding theme of message queue
Data, and the Service tracing data are parsed into the data into library format, and storage processing is carried out to the data after parsing.
10. collection device according to claim 9, which is characterized in that the log collection sending module includes that log is adopted
Storage and log transmission unit,
The log collector obtains listed files for scanning config directory, and obtains journal file according to listed files and generate
Log information, the log information is packaged to obtain log event, the log event includes classification information；
It is provided with classification information in the log sending module, when the classification information and the log thing of the log sending module
When the classification information matching of part, then the log event information is sent to corresponding in message queue by the log sending module
In classification scheme.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811129135.2A CN109308329A (en) | 2018-09-27 | 2018-09-27 | A kind of log collecting method and device based on cloud platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811129135.2A CN109308329A (en) | 2018-09-27 | 2018-09-27 | A kind of log collecting method and device based on cloud platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109308329A true CN109308329A (en) | 2019-02-05 |
Family
ID=65224150
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811129135.2A Pending CN109308329A (en) | 2018-09-27 | 2018-09-27 | A kind of log collecting method and device based on cloud platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109308329A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110262807A (en) * | 2019-06-20 | 2019-09-20 | 北京百度网讯科技有限公司 | Cluster creates Progress Log acquisition system, method and apparatus |
| CN110781136A (en) * | 2019-10-15 | 2020-02-11 | 上海御渡半导体科技有限公司 | Log-based message classification display structure and display method thereof |
| CN110879813A (en) * | 2019-11-20 | 2020-03-13 | 浪潮软件股份有限公司 | Binary log analysis-based MySQL database increment synchronization implementation method |
| CN111209170A (en) * | 2019-10-21 | 2020-05-29 | 江苏卓易信息科技股份有限公司 | Rule-based user behavior log acquisition and analysis method and system |
| CN111625412A (en) * | 2020-04-15 | 2020-09-04 | 中国建设银行股份有限公司 | Flume-based data acquisition method, system, device and storage medium |
| CN111708685A (en) * | 2020-05-18 | 2020-09-25 | 福建天晴在线互动科技有限公司 | Log acquisition monitoring method and system for distributed server |
| CN112214290A (en) * | 2019-07-11 | 2021-01-12 | 中移(苏州)软件技术有限公司 | Log information processing method, edge node, center node and system |
| CN112449750A (en) * | 2019-06-28 | 2021-03-05 | 京东方科技集团股份有限公司 | Log data collection method, log data collection device, storage medium, and log data collection system |
| CN112527844A (en) * | 2020-12-22 | 2021-03-19 | 北京明朝万达科技股份有限公司 | Data processing method and device and database architecture |
| CN112783953A (en) * | 2019-11-06 | 2021-05-11 | 广州凡科互联网科技股份有限公司 | Data acquisition method |
| CN113179302A (en) * | 2021-04-19 | 2021-07-27 | 杭州海康威视系统技术有限公司 | Log system, and method and device for collecting log data |
| CN113391973A (en) * | 2021-05-24 | 2021-09-14 | 青岛海信智慧生活科技股份有限公司 | Internet of things cloud container log collection method and device |
| CN114629786A (en) * | 2022-03-22 | 2022-06-14 | 康键信息技术（深圳）有限公司 | Log real-time analysis method, device, storage medium and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080155564A1 (en) * | 2006-12-01 | 2008-06-26 | International Business Machines Corporation | Event correlation based trouble ticket resolution system incorporating adaptive rules optimization |
| CN105589791A (en) * | 2015-12-28 | 2016-05-18 | 江苏省电力公司信息通信分公司 | Method for application system log monitoring management in cloud computing environment |
| CN106354434A (en) * | 2016-08-31 | 2017-01-25 | 中国人民大学 | Log data storing method and system |
| CN106817262A (en) * | 2017-03-01 | 2017-06-09 | 郑州云海信息技术有限公司 | A kind of log analysis device |
| CN107426023A (en) * | 2017-07-21 | 2017-12-01 | 携程旅游信息技术（上海）有限公司 | Cloud platform log collection and retransmission method, system, equipment and storage medium |
| CN107995169A (en) * | 2017-11-21 | 2018-05-04 | 厦门点触科技股份有限公司 | A kind of mobile phone games skeleton system based on micro services framework |
-
2018
- 2018-09-27 CN CN201811129135.2A patent/CN109308329A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080155564A1 (en) * | 2006-12-01 | 2008-06-26 | International Business Machines Corporation | Event correlation based trouble ticket resolution system incorporating adaptive rules optimization |
| CN105589791A (en) * | 2015-12-28 | 2016-05-18 | 江苏省电力公司信息通信分公司 | Method for application system log monitoring management in cloud computing environment |
| CN106354434A (en) * | 2016-08-31 | 2017-01-25 | 中国人民大学 | Log data storing method and system |
| CN106817262A (en) * | 2017-03-01 | 2017-06-09 | 郑州云海信息技术有限公司 | A kind of log analysis device |
| CN107426023A (en) * | 2017-07-21 | 2017-12-01 | 携程旅游信息技术（上海）有限公司 | Cloud platform log collection and retransmission method, system, equipment and storage medium |
| CN107995169A (en) * | 2017-11-21 | 2018-05-04 | 厦门点触科技股份有限公司 | A kind of mobile phone games skeleton system based on micro services framework |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110262807B (en) * | 2019-06-20 | 2023-12-26 | 北京百度网讯科技有限公司 | Cluster creation progress log acquisition system, method and device |
| CN110262807A (en) * | 2019-06-20 | 2019-09-20 | 北京百度网讯科技有限公司 | Cluster creates Progress Log acquisition system, method and apparatus |
| US11755452B2 (en) | 2019-06-28 | 2023-09-12 | Boe Technology Group Co., Ltd. | Log data collection method based on log data generated by container in application container environment, log data collection device, storage medium, and log data collection system |
| CN112449750A (en) * | 2019-06-28 | 2021-03-05 | 京东方科技集团股份有限公司 | Log data collection method, log data collection device, storage medium, and log data collection system |
| CN112214290B (en) * | 2019-07-11 | 2023-04-11 | 中移(苏州)软件技术有限公司 | Log information processing method, edge node, center node and system |
| CN112214290A (en) * | 2019-07-11 | 2021-01-12 | 中移(苏州)软件技术有限公司 | Log information processing method, edge node, center node and system |
| CN110781136A (en) * | 2019-10-15 | 2020-02-11 | 上海御渡半导体科技有限公司 | Log-based message classification display structure and display method thereof |
| CN110781136B (en) * | 2019-10-15 | 2023-05-16 | 上海御渡半导体科技有限公司 | Log-based message classification display device and method |
| CN111209170A (en) * | 2019-10-21 | 2020-05-29 | 江苏卓易信息科技股份有限公司 | Rule-based user behavior log acquisition and analysis method and system |
| CN111209170B (en) * | 2019-10-21 | 2023-09-08 | 江苏卓易信息科技股份有限公司 | Rule-based user behavior log acquisition and analysis method and system |
| CN112783953A (en) * | 2019-11-06 | 2021-05-11 | 广州凡科互联网科技股份有限公司 | Data acquisition method |
| CN110879813B (en) * | 2019-11-20 | 2024-04-12 | 浪潮软件股份有限公司 | Binary log analysis-based MySQL database increment synchronization implementation method |
| CN110879813A (en) * | 2019-11-20 | 2020-03-13 | 浪潮软件股份有限公司 | Binary log analysis-based MySQL database increment synchronization implementation method |
| CN111625412A (en) * | 2020-04-15 | 2020-09-04 | 中国建设银行股份有限公司 | Flume-based data acquisition method, system, device and storage medium |
| CN111708685A (en) * | 2020-05-18 | 2020-09-25 | 福建天晴在线互动科技有限公司 | Log acquisition monitoring method and system for distributed server |
| CN111708685B (en) * | 2020-05-18 | 2022-08-09 | 福建天晴在线互动科技有限公司 | Log acquisition monitoring method and system for distributed server |
| CN112527844A (en) * | 2020-12-22 | 2021-03-19 | 北京明朝万达科技股份有限公司 | Data processing method and device and database architecture |
| CN113179302A (en) * | 2021-04-19 | 2021-07-27 | 杭州海康威视系统技术有限公司 | Log system, and method and device for collecting log data |
| CN113391973B (en) * | 2021-05-24 | 2022-11-25 | 青岛海信智慧生活科技股份有限公司 | Internet of things cloud container log collection method and device |
| CN113391973A (en) * | 2021-05-24 | 2021-09-14 | 青岛海信智慧生活科技股份有限公司 | Internet of things cloud container log collection method and device |
| CN114629786A (en) * | 2022-03-22 | 2022-06-14 | 康键信息技术（深圳）有限公司 | Log real-time analysis method, device, storage medium and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109308329A (en) | A kind of log collecting method and device based on cloud platform | |
| Makris et al. | A classification of NoSQL data stores based on key design characteristics | |
| Tiwari | Professional nosql | |
| CN110413599A (en) | Generating date and storage system and method | |
| CN103793479A (en) | Log management method and log management system | |
| CN108399199A (en) | A kind of collection of the application software running log based on Spark and service processing system and method | |
| Barberis et al. | The ATLAS EventIndex: architecture, design choices, deployment and first operation experience | |
| CN108595664A (en) | A kind of agricultural data monitoring method under hadoop environment | |
| CN108681556A (en) | The access method and its system of distributed instruction numeric field data | |
| CN112860777B (en) | Data processing method, device and equipment | |
| CN108228755A (en) | The data of MySQL database based on daily record analytic technique to Hadoop platform synchronize clone method | |
| CN112163017B (en) | Knowledge mining system and method | |
| WO2022165168A1 (en) | Configuring an instance of a software program using machine learning | |
| CN113312376B (en) | Method and terminal for real-time processing and analysis of Nginx logs | |
| CN111753015B (en) | Data query method and device of payment clearing system | |
| CN114218218A (en) | Data processing method, device and equipment based on data warehouse and storage medium | |
| CN117149873A (en) | Data lake service platform construction method based on flow batch integration | |
| CN108763323A (en) | Meteorological lattice point file application process based on resource set and big data technology | |
| CN114090529A (en) | Log management method, device, system and storage medium | |
| CN112579567B (en) | MinIO-based industrial quality inspection file distributed storage method | |
| CN111414355A (en) | Offshore wind farm data monitoring and storing system, method and device | |
| CN116594982A (en) | Flow number bin construction method based on rule engine and Clickhouse | |
| CN115766527A (en) | Business analysis system and method based on API gateway inlet and outlet flow under trusted environment | |
| CN105761164A (en) | Hadoop-based electric power system accident recall method | |
| CN110311817A (en) | Container log processing system for Kubernetes cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190205 |
|
| RJ01 | Rejection of invention patent application after publication |