CN109766206A - A kind of log collection method and system - Google Patents
A kind of log collection method and system Download PDFInfo
- Publication number
- CN109766206A CN109766206A CN201811640957.7A CN201811640957A CN109766206A CN 109766206 A CN109766206 A CN 109766206A CN 201811640957 A CN201811640957 A CN 201811640957A CN 109766206 A CN109766206 A CN 109766206A
- Authority
- CN
- China
- Prior art keywords
- rule
- collection
- data
- log
- acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Debugging And Monitoring (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a kind of log collection method and systems, this method comprises: configuration log collection rule, create-rule file；Client is controlled according to the rule file, carries out log collection, obtains acquisition data；The acquisition data are sent to central server, and controls the central server and the acquisition data is parsed according to the rule file, obtain daily record data.The present invention is by the way that the log collection rule of configuration to be written in rule file, the cumbersome problem that bring program is changed in capture program can be directly write to avoid by collection rule, acquisition and data parsing due to log are all based on rule file progress, the simplification of log collection is realized, and improves the accuracy of log collection.
Description
Technical field
The present invention relates to technical field of information processing, more particularly to a kind of log collection method and system.
Background technique
With the rise of enterprise information systems integration and business finance automation platform construction, in power industry use information
Change software and has become common recognition to improve the efficiency of management of interior business.Utilities Electric Co. employee submits an expense account management business system using collection
Middle deployment way needs a large number of users for supporting the more branch companies, provinces and cities in the whole nation to use, and conventionally employed scheme is that F5 is disposed in front end
Hardware load is balanced, and more service servers are disposed to support high concurrent, a large number of users access in rear end.Due to asking under this mode
It asks business to be distributed at random on a certain service server to handle, when there is traffic failure, need to log in each server
It goes to check log, use process is relatively complicated, is easy error and development efficiency is low.
Summary of the invention
It is directed to the above problem, the present invention provides a kind of log collection method and system, realizes log collection process
Simplify, and improves the purpose of accuracy rate and development efficiency.
To achieve the goals above, the present invention provides the following technical scheme that
A kind of log collection method, this method comprises:
Configuration log collection rule, create-rule file；
Client is controlled according to the rule file, carries out log collection, obtains acquisition data；
The acquisition data are sent to central server, and control the central server according to the rule file pair
The acquisition data are parsed, and daily record data is obtained.
Optionally, the configuration log collection rule, create-rule file, comprising:
According to log collection routing information, the regular expression of acquisition format information and configuration, collection rule is generated；
The collection rule is written in rule file.
Optionally, the regular expression includes log lines regular expression, excludes log lines regular expression and multirow
Merge regular expression.
Optionally, the control client carries out log collection according to the rule file, obtains acquisition data, comprising:
It controls the client and loads the rule file, and read the rule file, obtain collection rule；
According to the collection rule, journal file, and the log according to the regular expression, to reading are read line by line
File is handled, and is obtained and is read data；
All collection rules are successively traversed, until being acquired after carrying out data acquisition according to all collection rules
Data.
Optionally, described that the acquisition data are sent to central server, and the central server is controlled according to institute
It states rule file to parse the acquisition data, obtains daily record data, comprising:
It controls the central server and loads the rule file；
The acquisition data that client is sent are received, and the acquisition data are parsed, obtain parsing content；
The collection rule traversed in the rule file handles the parsing content, obtains daily record data, and will
The daily record data is stored.
A kind of Log Collect System, the system include:
Configuration unit is used for configuration log collection rule, create-rule file；
Acquisition unit carries out log collection, obtains acquisition data for controlling client according to the rule file；
Resolution unit, for the acquisition data to be sent to central server, and control the central server according to
The rule file parses the acquisition data, obtains daily record data.
Optionally, the configuration unit includes:
Subelement is generated, it is raw for the regular expression according to log collection routing information, acquisition format information and configuration
At collection rule；
Subelement is written, for the collection rule to be written in rule file.
Optionally, the regular expression includes log lines regular expression, excludes log lines regular expression and multirow
Merge regular expression.
Optionally, the acquisition unit includes:
First load subelement loads the rule file for controlling the client, and reads the rule file,
Obtain collection rule；
First processing subelement, for reading journal file line by line, and according to the rule list according to the collection rule
Up to formula, the journal file read is handled, obtains and reads data；
Subelement is acquired, for successively traversing all collection rules, until carrying out data according to all collection rules
After acquisition, acquisition data are obtained.
Optionally, the resolution unit includes:
Second load subelement loads the rule file for controlling the central server；
Parsing subunit for receiving the acquisition data of client transmission, and parses the acquisition data, obtains
Parse content；
Second processing subelement, for traverse the collection rule in the rule file to the parsing content at
Reason obtains daily record data, and the daily record data is stored.
Compared to the prior art, the present invention provides a kind of log collection method and systems, by adopting the log of configuration
Collection rule is written in rule file, can directly write to what bring program in capture program was changed to avoid by collection rule
Cumbersome problem, improves development efficiency；Then control client carries out log collection according to the rule file, and will acquire data
It is sent to central server, while control centre's server parses acquisition data according to rule file, obtains log number
According to.Acquisition and data parsing due to log are all based on rule file progress, realize the simplification of log collection, and mention
The high accuracy of log collection.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow diagram of log collection method provided in an embodiment of the present invention；
Fig. 2 is the method flow schematic diagram that a kind of client provided in an embodiment of the present invention parses collection rule file；
Fig. 3 is the method flow flow chart that a kind of central server provided in an embodiment of the present invention receives log；
Fig. 4 is a kind of structural schematic diagram of Log Collect System provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Term " first " and " second " in description and claims of this specification and above-mentioned attached drawing etc. are for area
Not different objects, rather than for describing specific sequence.Furthermore term " includes " and " having " and their any deformations,
It is intended to cover and non-exclusive includes.Such as it contains the process, method of a series of steps or units, system, product or sets
It is standby not to be set in listed step or unit, but may include the step of not listing or unit.
A kind of log collection method is provided in embodiments of the present invention, referring to Fig. 1, this method comprises:
S101, configuration log collection rule, create-rule file.
The rule file is set the following steps are included:
According to log collection routing information, the regular expression of acquisition format information and configuration, collection rule is generated；
The collection rule is written in rule file.
Specifically include:
Newly-built rule file；
The IP address and port of an acquisition server are configured, i.e. configuration log acquires routing information；
It increases a collection rule newly, that is, configures each collection rule in the rule file；
Configure the log collection directory path of an application, which is for have will in the case where multiple client
All acquisition paths are stored in the form of catalogue, that is, after configuring the catalogue, All Files are as log processing under this catalogue.
The acquisition format for configuring the log successively increases each field name in log, data type, truncation length attribute；
Configuring includes log lines regular expression in collection rule；
It configures and excludes log lines regular expression in collection rule, for example, system prompt or slightly alerting this be not required to
The symbolic animal of the birth year information of the log to be paid close attention to can be configured by excluding log lines regular expression；
It configures multirow in collection rule and merges regular expression, for example, needing to merge when a mistake divides multirow to export
For a line.
Above-mentioned regular configuration process is repeated, until the process log for configuring the acquisition in need of cost host institute is regular.
S102, control client carry out log collection according to the rule file, obtain acquisition data.
When client carries out log collection according to rule file, may include:
It controls the client and loads the rule file, and read the rule file, obtain collection rule；
According to the collection rule, journal file, and the log according to the regular expression, to reading are read line by line
File is handled, and is obtained and is read data；
All collection rules are successively traversed, until being acquired after carrying out data acquisition according to all collection rules
Data.
Specifically, referring to fig. 2, for a kind of method stream of client parsing collection rule file provided in an embodiment of the present invention
Cheng Tu, comprising:
S201, loading rule file to memory；
S202, a collection rule is read；
One S203, starting new thread, read the journal file in respective directories according to the acquisition catalogue in this rule；
S204, a line log is read；
S205, judge whether the row meets multirow and merge regular expression, if it is, executing S206, otherwise, execute
S207；
S206, the row and previous row merge into a line；
S207, judge whether to meet exclusion line discipline expression formula, if it is, executing S208, otherwise, execute S209；
S208, ignore this and capable do not handle；
S209, judge whether to be configured with comprising line discipline expression formula, if it is, executing S210, otherwise execute S211；
S210, whether meet comprising line discipline expression formula, if it is, executing S211；
S211, satisfaction is needed to be combined into json message comprising line discipline expression formula under configuring condition, if no configuration does not include day
Aspirations and conduct regular expression is also combined into json message, has collection rule id, host name, timestamp, log content to be sent in message
Central server；
S212, record current log record processing line number, filename into memory, next time judge it is processed not
Reprocessing；
S213, judge whether to have handled all collection rules, if it is not, then step S202 to S212 is repeated, until traversal
All acquisition configuration rules in complete rule file, if it is, being parsed.
S103, the acquisition data are sent to central server, and control the central server according to the rule
File parses the acquisition data, obtains daily record data.
Central server carry out data acquisition process may include:
It controls the central server and loads the rule file；
The acquisition data that client is sent are received, and the acquisition data are parsed, obtain parsing content；
The collection rule traversed in the rule file handles the parsing content, obtains daily record data, and will
The daily record data is stored.
Referring to Fig. 3, for a kind of method flow flow chart of central server reception log provided in an embodiment of the present invention, packet
Include following steps:
S301, collection rule file is loaded into memory；
S302, judge whether to receive the acquisition data of client transmission, if it is not, then S303 is executed, if it is, holding
Row S304；
S303, etc. it is to be received；
S304, the acquisition data that client is sent are received, parsing data obtain collection rule ID, host name, timestamp, day
Will file path, log content；
S305, each field name, data type in acquisition data are read according to collection rule ID, assign to corresponding field name
Value, writes in database；
S306, next log collection data processing is carried out, repeats S302-S305.
It is accomplished that acquisition method is unified in a kind of log of variable format in the above-described embodiments, the branch in a manner of configurable
The log output format of Utilities Electric Co.'s multiple business support system is held, the content format field of various types of logs can lead to
Configuration flexible setting is crossed, the path of log can also configure, and need to log in the numerous of host when reducing audit log and log audit
Trivial step and workload.
The present invention provides a kind of log collection methods, by the way that the log collection rule of configuration is written to rule file
In, the cumbersome problem that bring program is changed in capture program can be directly write to avoid by collection rule, improve exploitation
Efficiency；Then control client carries out log collection according to the rule file, and acquisition data are sent to central server, together
When control central server according to rule file to acquisition data parse, obtain daily record data.Due to log acquisition and
Data parsing is all based on rule file progress, realizes the simplification of log collection, and improve the accurate of log collection
Property.
A kind of Log Collect System is additionally provided in embodiments of the present invention, and referring to fig. 4, which includes:
A kind of Log Collect System, the system include:
Configuration unit 401 is used for configuration log collection rule, create-rule file；
Acquisition unit 402 carries out log collection, obtains acquisition data for controlling client according to the rule file；
Resolution unit 403 for the acquisition data to be sent to central server, and controls the central server and presses
The acquisition data are parsed according to the rule file, obtain daily record data.
On the basis of the above embodiments, the configuration unit includes:
Subelement is generated, it is raw for the regular expression according to log collection routing information, acquisition format information and configuration
At collection rule；
Subelement is written, for the collection rule to be written in rule file.
On the basis of the above embodiments, the regular expression includes log lines regular expression, excludes log professional etiquette
Then expression formula and multirow merge regular expression.
On the basis of the above embodiments, the acquisition unit includes:
First load subelement loads the rule file for controlling the client, and reads the rule file,
Obtain collection rule；
First processing subelement, for reading journal file line by line, and according to the rule list according to the collection rule
Up to formula, the journal file read is handled, obtains and reads data；
Subelement is acquired, for successively traversing all collection rules, until carrying out data according to all collection rules
After acquisition, acquisition data are obtained.
On the basis of the above embodiments, the resolution unit includes:
Second load subelement loads the rule file for controlling the central server；
Parsing subunit for receiving the acquisition data of client transmission, and parses the acquisition data, obtains
Parse content；
Second processing subelement, for traverse the collection rule in the rule file to the parsing content at
Reason obtains daily record data, and the daily record data is stored.
The present invention provides a kind of Log Collect Systems, and the log collection rule of configuration is written to rule by configuration unit
Then in file, the cumbersome problem that bring program is changed in capture program can be directly write to avoid by collection rule, improved
Development efficiency；Then client is controlled in acquisition unit and resolution unit carries out log collection according to the rule file, and
Acquisition data are sent to central server, while control centre's server parses acquisition data according to rule file,
Obtain daily record data.Acquisition and data parsing due to log are all based on rule file progress, realize log collection
Simplify, and improves the accuracy of log collection.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with other
The difference of embodiment, the same or similar parts in each embodiment may refer to each other.For device disclosed in embodiment
For, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is said referring to method part
It is bright.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be readily apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest scope of cause.
Claims (10)
1. a kind of log collection method, which is characterized in that this method comprises:
Configuration log collection rule, create-rule file；
Client is controlled according to the rule file, carries out log collection, obtains acquisition data；
The acquisition data are sent to central server, and control the central server according to the rule file to described
Acquisition data are parsed, and daily record data is obtained.
2. the method according to claim 1, wherein the configuration log collection rule, create-rule file, packet
It includes:
According to log collection routing information, the regular expression of acquisition format information and configuration, collection rule is generated；
The collection rule is written in rule file.
3. according to the method described in claim 2, it is characterized in that, the regular expression include log lines regular expression,
It excludes log lines regular expression and multirow merges regular expression.
4. according to the method described in claim 3, it is characterized in that, the control client is carried out according to the rule file
Log collection obtains acquisition data, comprising:
It controls the client and loads the rule file, and read the rule file, obtain collection rule；
According to the collection rule, journal file is read line by line, and according to the regular expression, to the journal file read
It is handled, obtains and read data；
All collection rules are successively traversed, until obtaining acquisition data after carrying out data acquisition according to all collection rules.
5. according to the method described in claim 4, it is characterized in that, described be sent to central server for the acquisition data,
And control the central server and the acquisition data are parsed according to the rule file, obtain daily record data, comprising:
It controls the central server and loads the rule file；
The acquisition data that client is sent are received, and the acquisition data are parsed, obtain parsing content；
The collection rule traversed in the rule file handles the parsing content, obtains daily record data, and will be described
Daily record data is stored.
6. a kind of Log Collect System, which is characterized in that the system includes:
Configuration unit is used for configuration log collection rule, create-rule file；
Acquisition unit carries out log collection, obtains acquisition data for controlling client according to the rule file；
Resolution unit for the acquisition data to be sent to central server, and controls the central server according to described
Rule file parses the acquisition data, obtains daily record data.
7. system according to claim 6, which is characterized in that the configuration unit includes:
Subelement is generated, for the regular expression according to log collection routing information, acquisition format information and configuration, generation is adopted
Collection rule；
Subelement is written, for the collection rule to be written in rule file.
8. system according to claim 7, which is characterized in that the regular expression include log lines regular expression,
It excludes log lines regular expression and multirow merges regular expression.
9. system according to claim 8, which is characterized in that the acquisition unit includes:
First load subelement loads the rule file for controlling the client, and reads the rule file, obtains
Collection rule；
First processing subelement, for reading journal file line by line, and according to the regular expression according to the collection rule
Formula handles the journal file read, obtains and reads data；
Subelement is acquired, for successively traversing all collection rules, until carrying out data acquisition according to all collection rules
Afterwards, acquisition data are obtained.
10. system according to claim 9, which is characterized in that the resolution unit includes:
Second load subelement loads the rule file for controlling the central server；
Parsing subunit for receiving the acquisition data of client transmission, and parses the acquisition data, is parsed
Content；
Second processing subelement handles the parsing content for traversing the collection rule in the rule file, obtains
Daily record data is obtained, and the daily record data is stored.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811640957.7A CN109766206A (en) | 2018-12-29 | 2018-12-29 | A kind of log collection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811640957.7A CN109766206A (en) | 2018-12-29 | 2018-12-29 | A kind of log collection method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109766206A true CN109766206A (en) | 2019-05-17 |
Family
ID=66453125
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811640957.7A Pending CN109766206A (en) | 2018-12-29 | 2018-12-29 | A kind of log collection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109766206A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110635951A (en) * | 2019-10-10 | 2019-12-31 | 北京科东电力控制系统有限责任公司 | Electric power safety monitoring standardized communication method, safety monitoring device and master station management platform |
| CN110888789A (en) * | 2019-11-21 | 2020-03-17 | 北京中电普华信息技术有限公司 | Data transmission interface monitoring method and system |
| CN110912782A (en) * | 2019-12-17 | 2020-03-24 | 锐捷网络股份有限公司 | Data acquisition method, device and storage medium |
| CN111125004A (en) * | 2019-12-03 | 2020-05-08 | 中盈优创资讯科技有限公司 | File collection method and device |
| CN112100133A (en) * | 2020-11-04 | 2020-12-18 | 广州市玄武无线科技股份有限公司 | Distributed log processing system |
| CN112445937A (en) * | 2020-11-30 | 2021-03-05 | 成都新潮传媒集团有限公司 | Json log generation method and device and computer readable storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102739428A (en) * | 2011-04-15 | 2012-10-17 | 中兴通讯股份有限公司 | Method and device for data acquisition, and system |
| CN103092817A (en) * | 2013-01-18 | 2013-05-08 | 五八同城信息技术有限公司 | Data collection method and data collection device based on script engine |
| CN103345235A (en) * | 2013-07-18 | 2013-10-09 | 上海华力微电子有限公司 | Method and module for establishing data acquisition rule of statistical process control system |
| CN103702354A (en) * | 2013-12-17 | 2014-04-02 | 重庆重邮汇测通信技术有限公司 | Base station engineering parameter collection and analysis method |
| CN105610648A (en) * | 2016-01-11 | 2016-05-25 | 飞天诚信科技股份有限公司 | Operation and maintenance monitoring data collection method and server |
| CN106599120A (en) * | 2016-12-01 | 2017-04-26 | 中国联合网络通信集团有限公司 | Stream processing framework-based data processing method and apparatus |
| CN107451034A (en) * | 2017-08-17 | 2017-12-08 | 浪潮软件股份有限公司 | A kind of big data cluster log management apparatus, method and system |
| CN107547589A (en) * | 2016-06-27 | 2018-01-05 | 腾讯科技（深圳）有限公司 | A kind of data acquisition treatment method and device |
| CN107643932A (en) * | 2017-08-31 | 2018-01-30 | 深圳天珑无线科技有限公司 | A kind of collecting method, harvester and storage device |
-
2018
- 2018-12-29 CN CN201811640957.7A patent/CN109766206A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102739428A (en) * | 2011-04-15 | 2012-10-17 | 中兴通讯股份有限公司 | Method and device for data acquisition, and system |
| CN103092817A (en) * | 2013-01-18 | 2013-05-08 | 五八同城信息技术有限公司 | Data collection method and data collection device based on script engine |
| CN103345235A (en) * | 2013-07-18 | 2013-10-09 | 上海华力微电子有限公司 | Method and module for establishing data acquisition rule of statistical process control system |
| CN103702354A (en) * | 2013-12-17 | 2014-04-02 | 重庆重邮汇测通信技术有限公司 | Base station engineering parameter collection and analysis method |
| CN105610648A (en) * | 2016-01-11 | 2016-05-25 | 飞天诚信科技股份有限公司 | Operation and maintenance monitoring data collection method and server |
| CN107547589A (en) * | 2016-06-27 | 2018-01-05 | 腾讯科技（深圳）有限公司 | A kind of data acquisition treatment method and device |
| CN106599120A (en) * | 2016-12-01 | 2017-04-26 | 中国联合网络通信集团有限公司 | Stream processing framework-based data processing method and apparatus |
| CN107451034A (en) * | 2017-08-17 | 2017-12-08 | 浪潮软件股份有限公司 | A kind of big data cluster log management apparatus, method and system |
| CN107643932A (en) * | 2017-08-31 | 2018-01-30 | 深圳天珑无线科技有限公司 | A kind of collecting method, harvester and storage device |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110635951A (en) * | 2019-10-10 | 2019-12-31 | 北京科东电力控制系统有限责任公司 | Electric power safety monitoring standardized communication method, safety monitoring device and master station management platform |
| CN110888789A (en) * | 2019-11-21 | 2020-03-17 | 北京中电普华信息技术有限公司 | Data transmission interface monitoring method and system |
| CN111125004A (en) * | 2019-12-03 | 2020-05-08 | 中盈优创资讯科技有限公司 | File collection method and device |
| CN110912782A (en) * | 2019-12-17 | 2020-03-24 | 锐捷网络股份有限公司 | Data acquisition method, device and storage medium |
| CN110912782B (en) * | 2019-12-17 | 2022-08-16 | 锐捷网络股份有限公司 | Data acquisition method, device and storage medium |
| CN112100133A (en) * | 2020-11-04 | 2020-12-18 | 广州市玄武无线科技股份有限公司 | Distributed log processing system |
| CN112445937A (en) * | 2020-11-30 | 2021-03-05 | 成都新潮传媒集团有限公司 | Json log generation method and device and computer readable storage medium |
| CN112445937B (en) * | 2020-11-30 | 2023-11-14 | 成都新潮传媒集团有限公司 | Json log generation method and device and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109766206A (en) | A kind of log collection method and system | |
| CN101694626B (en) | Script execution system and method | |
| CA2991150C (en) | Multi-stage network discovery | |
| US20130297557A1 (en) | Column based data transfer in extract, transform and load (etl) systems | |
| CN102890682A (en) | Method for creating index, searching method, device and system | |
| US20120290626A1 (en) | Converting two-tier resource mapping to one-tier resource mapping | |
| CN111818175B (en) | Enterprise service bus configuration file generation method, device, equipment and storage medium | |
| CN103634361A (en) | Method and apparatus for downloading file | |
| CN104156381A (en) | Copy access method and device for Hadoop distributed file system and Hadoop distributed file system | |
| CN105183605A (en) | Server performance test method and device | |
| CN104679500A (en) | Automatic generation realizing method and device for entity classes | |
| CN109697020A (en) | A kind of date storage method, server and system | |
| US20210176138A1 (en) | Data center cartography bootstrapping from process table data | |
| CN110502472A (en) | A kind of the cloud storage optimization method and its system of large amount of small documents | |
| US9176974B1 (en) | Low priority, multi-pass, server file discovery and management | |
| CN106649847A (en) | A large data real-time processing system based on Hadoop | |
| CN110888840A (en) | File query method, device, equipment and medium in distributed file system | |
| CN105653566B (en) | A kind of method and device for realizing database write access | |
| US11061736B2 (en) | Multiple parallel reducer types in a single map-reduce job | |
| US8856152B2 (en) | Apparatus and method for visualizing data | |
| CN105069108B (en) | Based on PaaS system big data querying method and device | |
| CN110737636B (en) | Data import method, device and equipment | |
| CN111475291A (en) | Data processing method, system, server and medium | |
| CN113553329B (en) | Data integration system and method | |
| US11514016B2 (en) | Paging row-based data stored as objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190517 |