CN104750870B - A kind of log storage system and implementation method based on HBase - Google Patents
A kind of log storage system and implementation method based on HBase Download PDFInfo
- Publication number
- CN104750870B CN104750870B CN201510187185.6A CN201510187185A CN104750870B CN 104750870 B CN104750870 B CN 104750870B CN 201510187185 A CN201510187185 A CN 201510187185A CN 104750870 B CN104750870 B CN 104750870B
- Authority
- CN
- China
- Prior art keywords
- log
- user
- service
- storage
- hbase
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Computer And Data Communications (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The present invention discloses a kind of log storage system and implementation method based on HBase, belongs to big data technical field of memory.The system includes four big functional modules：User management module is distributed memory space for user's registration, signal auditing, administrator and is managed to the basic condition of user；Log management module, inquiry and downloading for application system account, management system account, customized Log Types, management Log Types and log；Remote service calling module, for the Web Service service of custom system invoking server publication, including authentication service and the long-range storage service of log；System queries module, the interior letter connection administrator of inquiry and displaying and station for remaining space amount, provides the diagrammatic representation of diversified forms.The present invention, as log storage container, can satisfy the storage demand of mass data and make full use of that Hadoop cluster is cheap, metastable storage characteristics using the HBase database of distribution Hadoop.
Description
Technical field
The present invention relates to one kind to be based on HBase（HBase is the PostgreSQL database based on Hadoop distributed storage.With
HDFS, Mapreduce belong to the sub-project of Hadoop）Log storage system and implementation method, belong to big data storage skill
Art field.
Background technique
Log is a kind of resource to be excavated constantly accumulated, its quantity is in explosive growth over time.Water conservancy
Department possesses multiple information systems, and each system is generating log recording all the time.How these log recordings to be carried out
The management of concentration, so that carrying out further data mining is the urgent problem to be solved that hydraulic department is faced.
In order to solve massive logs problem of management, software developer has started constantly to probe into.At present in log management
There have been some researchs in field：Hu Wei was realized in 2013 aims at centrally stored in magnetic disk media, the storage shape of log day
Formula is file and database, and this storage mode is not flexible in terms of log query and analysis（Hu Wei enterprise's log centralized management
Systematic research and design Beijing [D]:Beijing University of Post & Telecommunication, 2013.）；Wang Zhaoyong, which devised one in 2011, to expire
The key value database of sufficient big data storage, log pass through external importing form storage to key value database, are not able to satisfy log reality
When store, while the performance of database also needs to be considered（Wang Zhao research of the towards extensive batch daily record data storage method forever
The Xi'an [D]:University of Electronic Science and Technology, 2011）；Song in 2012 likes that blueness applies the Web type log storage system of Hadoop, day
The collection of will collects user journal using CDN content distribution network nearby, and the log collection of this mode is cumbersome, needs more
Node overheads（Song loves design and Beijing realization [D] of Log Analysis System of the blueness based on Hadoop:China University of Geosciences（North
Capital）,2012）.System designed by above-mentioned document solves massive logs storage problem using distinct methods, but these systems do not have
Have consider flexibility that log stores, real-time and in terms of demand.
Summary of the invention
Aiming at the problems existing in the prior art, the present invention proposes a kind of log storage system and realization based on HBase
Distributed information log is stored and introduces Water Resources Domain by method, to improve the centrally stored and data mining to Water Resources Domain to log
Ability.
The present invention is to solve its technical problem to adopt the following technical scheme that：
A kind of log storage system based on HBase, including user management module, for user's registration, registration audit, pipe
Reason person distributes memory space and is managed to the basic condition of user；Log management module, for system account, log class
The management of type and log；The Web clothes of publication are called in remote service calling module, publication and custom system for Web service
Business carries out the storage of log, the long-range real-time storage service including authentication service and log；System queries module, for remaining
Inquiry, displaying and the interior letter connection administrator in station of complementary space amount；Wherein user management module and the two-way company of distributed data base
It connects, distributed data base is bi-directionally connected with log management module, and log management module is connect with remote service calling module, remotely
Service calling module is connect by distributed data base with system queries module；
The user management module audits unit, storage space allocation unit and subscriber information management unit comprising user；
The log management module, including system account application unit, Log Types administrative unit and log query unit；
The remote service calling module, including system configuration unit, service release unit and service call unit；
Administrative unit is believed in the system queries module, including remaining space query unit, connection administrator's unit and station.
A kind of implementation method of the log storage system based on HBase, comprises the following steps：
S-1：One specifically used person of this system, hereinafter referred to as user, register this system by registration interface；Administrator
Audit registration user stores number by power distribution memory space and maximum system to satisfactory user；
S-2：The user's application system account to succeed in registration, according to current registion time generation system account, according to random
Password generates function and generates Unique System password, and same registration user can apply for that multiple system accounts, system default are maximum simultaneously
System application number is 10, and user can contact as needed administrator and change；
S-3：The meaning of the customized each attribute section of user and the identifier of attribute section, system default ' # ' are used as attribute section
Between separator, formed a new Log Types；
S-4：It is called to Web server Web Services Publishing for custom system after system starting；
S-5：The log of HBase is written in custom system request and called Web service, the log that custom system generates in real time
In table, at the same return log storage whether successful prompt information.
The step S-2 specifically includes that step is implemented as follows：
Step S-2-1：Judge whether system application number has reached the system account application upper limit, if having reached system account
Apply for the upper limit, then application is prompted to fail；Otherwise continue step S-2-2；
Step S-2-2：User applies for creating new system account, and system is by present system time to 1970-00-00 00:
00:00 time is converted into millisecond number and generates system account；System generates a string by big according to customized key derivation function
The random train of lowercase and number composition is as system password；
Step S-2-3：Major key key of one system account as a custom system, the letter such as system password, user's major key
Breath is used as value, then information is stored into the system account table of HBase in the form of key-value key-value pair, system account
Apply successfully；Otherwise, return step S-2-2；
The step S-3 specifically includes that step is implemented as follows：
Step S-3-1：Determine the meaning of each attribute section of Log Types, and the identifier of customized each attribute section and
Log Types title；
Step S-3-2：The sequence of attribute section by definition is spliced into character string with ' # ', forms the content of Log Types；
Step S-3-3：With present system time to 1970-00-00 00:00:00 millisecond number as major key key, with
Householder's key and Log Types information are as value；
Step S-3-4：By key-value pair key-value deposit HBase Log Types table in, if store successfully, return at
Function information；Otherwise, storage failure, return step S-3-1.
The step S-5 specifically includes that step is implemented as follows：
Step S-5-1：System end is configured using CXF mode, as the server end of Web service, for issuing Web
Service；Equally, it is configured using the client that CXF mode carries out Web service to custom system, custom system is made to become Web service
Client；
Step S-5-2：After running this system, Web service is published by network；Run custom system；When going to day
When will stores, step S-5-3 is executed；
Step S-5-3：Call authentication service：Custom system provides system account and system password, and the two is as ginseng
Number passes to the authentication service verifying of entitled " IsCorrect "；If authentication success, executes step S-5-4；It is no
Then, authentication fails, and returns to log and stores failure information；
Step S-5-4：The service of log real-time storage is called, newly generated log string calls the day of entitled " AddLogs "
Will real-time storage service；
Step S-5-5：It will be converted into the time using the function of time of Java from the current time in system to 1970-00-00
00:00:Parameter in 00 millisecond number the major key key as a log information, step S-5-4 is as value, deposit
In the log sheet of HBase, log information is stored successfully；Otherwise, log storage failure, goes to step S-5-2.
Beneficial effects of the present invention are as follows：
Using the HBase database of distributed Hadoop as log storage container, the storage of mass data can satisfy
Demand simultaneously makes full use of that Hadoop cluster is cheap, metastable storage characteristics；Remote journal storage side based on Web service
Formula will improve local text storage mode and give Web log mining bring inconvenience, be conducive to the data under the conditions of aiming at day concentrating and dig
Pick, while reducing the space expense that custom system stores log.Log Types are customized to expand Log Types, dashes forward
The personalized storage of log out, lays the foundation for the log analysis and data mining of subsequent personalization.
Detailed description of the invention
Fig. 1 is the log storage system functionality module map based on HBase of the embodiment of the present invention.
Fig. 2 is the Web service call flow chart of the embodiment of the present invention.
Fig. 3 is the log storage system flow chart based on HBase of the embodiment of the present invention.
Fig. 4 is the Hadoop aggregated pattern sketch map of the embodiment of the present invention.
Specific embodiment
With reference to the accompanying drawing, the present invention is furture elucidated, it should be understood that these embodiments are merely to illustrate the present invention and do not have to
In limiting the scope of the invention, after the present invention has been read, those skilled in the art are to various equivalent forms of the invention
Modification falls within the application range as defined in the appended claims.
For log centralized management the problems in, the present invention propose and realize one it is based on B/S framework, can deposit in real time
Store up the log storage service platform of massive logs.Custom system is stored log into HBase distribution number by called Web service
According to library；User carries out log management by browser access platform, into log management interface.
Log storage system based on HBase by user management module, log management module, remote service calling module and
System queries module composition, as shown in Figure 1.User management module distributes storage for user's registration, registration audit, administrator
Space and the basic condition of user is managed；Log management module, for system account, Log Types and the pipe of log
Reason；Remote service calling module, publication and custom system for Web service call the Web service of publication, including identity to test
The long-range real-time storage service of card service and log；System queries module, inquiry and displaying and station for remaining space amount
Interior letter contacts administrator, provides a variety of diagrammatic representation forms.
User management module mainly includes user's audit, by power distribution memory space, user basic information administrative unit.
Wherein user audits unit and audits for administrator to user's registration information, filters out effective registration user, avoids invading
Entering property data cause the waste of memory space；Storage space allocation unit is carried out the user that audit passes through by power for administrator
Space size distribution, space distribution have the characteristics that flexible；Subscriber information management unit includes user information inquiry, user information
Batch downloading, modification maximum system storage number and memory space, details show and delete unqualified user's operation；This system
It sets under initial situation, the memory space under each user's original state is 0, and each applied system account number of user is
10, if exceeding maximum system account number, user will be unable to complete the application of system account.Each user can only be in memory space
In the case where satisfaction carry out log storage, one log of every deposit, system all calculate memory space whether meet demand.
Table 1 is several example informations of user's table in HBase.Front two row respectively indicates two of same data in table 1
Different editions.On the basis of 1 data, the 2nd data is modified password, it is worth noting that in HBase table
The middle missing for allowing item, such as " Realname " column of the first row, but the additional space Hadoop will not be occupied.It is closed with tradition
It is that database is compared, can greatly avoids the waste of unnecessary memory space in terms of storing discrete data using HBase.
Table 1.HBase user's table example
The calculating of memory space, including two aspects：The calculating in space and administrator are distributed when user journal stores empty
Between when calculating to space.Because system account and the occupied memory space of system password is very small and its growth rate with
Log is compared to being almost 0, therefore when calculating user storage space, our spaces Hadoop occupied to user journal into
Row calculates, and is defined as：
Wherein, I indicates user I,Indicate storage size shared by i-th of system of I.Indicate user I's
Shared all storage sizes.
Administrator's allocation space calculate definition be：
M indicates total system user number, and n indicates the system number of each user, and p indicates system user and administrator
Summation；L indicates Log Types sum；Q indicates letter sum in station；Indicate the total of j-th of system of i-th of user
Log amount of storage；Indicate the memory space that user's table occupies；Indicate the memory space that system table occupies；The memory space that Log Types table occupies；The memory space that letter occupies in standing.
Log management module, including system account application unit, Log Types administrative unit and log query unit；System
Account application unit is used for application system account and management system account, and system account and system password are two string character strings, uses
In identify a custom system and custom system request access database when as authentication information.Each system account
Corresponding Unique System password, improves the safety of system access；Log Types administrative unit is used for traditional Log Types
Be extended with Log Types management, support the customized of Log Types, improve log storage diversity；Log query list
Member is downloaded for the multi-condition inquiry of log and log batch, and in order to avoid deleting the mistake of log, system does not support log
By a delete operation.Wherein Log Types administrative unit is mainly the management to customized Log Types, and table 2 lists system reality
A customization type in example.Wherein, typonym and attribute definition are all customized by the user completion, and type instance is user
An example of the system to the customization type.
The customized example of 2 Log Types of table
| Typonym | Operation log |
| Attribute definition | Login name # password #IP address # operates the system account of custom system described in # |
| Type instance | User's 1#11111#10.192.80.20# error in operation #14000294561 |
Log query unit uses the multi-condition inquiry based on HBase, compares full table scan energy using message-filter mechanism
Enough significantly improve search efficiency.The key-value of Hadoop（Key-value pair）Major key sorts according to dictionary in mechanism and HBase table
Design, make in the design process of HBase table by Rowkey design raising table search efficiency.
Remote service calling module is the nucleus module of log storage system, including system configuration unit, service publication list
Member and service call unit.System configuration unit includes both sides configuration, first service of the configuration this system as Web service
End, secondly configuring custom system becomes the client of Web service；Service release unit passes through Web Publishing for server end
Web service, the real-time storage service including authentication service and log；Service call unit is used for custom system far call
The Web service that success is issued.Service call passes to entitled " IsCorrect " using system account, system password as parameter
Authentication service；The Web service that entitled " AddLogs " is passed to using log string as a parameter is serviced by " AddLogs "
Final log is carried out remotely to store.Specific execution flow chart is shown in Fig. 2.The same service can be adjusted by multiple custom systems
With；The type and quantity of service can be modified and be added by system engineer, and custom system can be called after primary modification
This characteristic of newest service, Web service greatly improves portability and the flexibility of system.
Administrative unit is believed in system queries module, including remaining space query unit, connection administrator's unit and station.It is remaining
Space querying unit, the space for inquiring Hadoop cluster for user query Spatial Residual amount and usage amount, administrator use and
Surplus；Administrator's unit is contacted for searching and managing person's information and connection administrator, connection administrator supports letter and postal in station
Two kinds of contact methods of case；Letter administrative unit is for letter information in the station of account executive in standing, including has collected mail and part and sent letter
The operation such as the deletion of part and by date interval query.
Log storing method based on HBase, is implemented as follows：
S-1：User registers this system for example by registration interface：User name is respectively two use of user 1 and user 2
Family.Administrator audits the information of user 1 and user 2, if the license using the system has been obtained in user 1 before registration, and
User 2 belongs to illegal registration user, then administrator can carry out audit by operating and distributing storage for user 1 to user 1
Space, if license user 1 allow using maximum memory space be 100G, then administrator be user 1 distribute 100G it is big
Small space, while administrator can carry out delete operation to the information of user 2；
S-2：After succeeding in registration user 1 can with application system account, according to current registion time generation system account, according to
Random cipher generates function and generates Unique System password, and same registration user can apply for multiple system accounts, system default simultaneously
The system account application upper limit is 10, and user can contact as needed administrator and change；
Step S-2-1：Judge whether the system application number of user 1 has reached the system account application upper limit 10.If
Up to the system account application upper limit, then application is prompted to fail；Otherwise continue step S-2-2；
Step S-2-2：User 1 applies for creating new system account, and system is by present system time to 1970-00-00
00:00:00 time is converted into millisecond number and generates system account for example：1400001294561；System is according to customized password
Generating function generates a string by upper and lower case letter and the digital random train formed as system password, in system testing stage definitions
System password length is 12, such as：a7TwM2fkYnQ0；
Step S-2-3：1400001294561 major key as the custom system of user 1, system password：
A7TwM2fkYnQ0, user 1 the information such as major key 1339192226632 as the value in key-value pair, information is with key-value key
The form of value pair is stored into the system account Table A ppidTable of HBase database, so far, system account application success；It is no
Then, return step S-2-2；
S-3：The customized Log Types of user 1.Define the meaning of each attribute section and the identifier of attribute section, system default
' # ' forms a new Log Types as the separator between attribute section；
Step S-3-1：Determine the meaning of each attribute section of Log Types, and the identifier of customized each attribute section and
Log Types title, such as example in table 2, Log Types name：Log is logged in, log is made of 5 attribute sections, is to log in respectively
Name, logs in the address ip, operation, system account major key at login password；
Step S-3-2：The sequence of attribute section by definition is spliced into character string with ' # ', forms the content of Log Types；
Step S-3-3：With present system time to 1970-00-00 00:00:00 millisecond number is as major key key, herein
Key is in example：1400001327490, user's major key is 1339192226632：Made with user's major key and Log Types information
For value value；
Step S-3-4：Key-value pair key-value is stored in the Log Types table LogTypeTable of HBase, if storage
Success, returns success；Otherwise, storage failure, return step S-3-1.
S-4：It is called to Web server Web Services Publishing for custom system after system starting；
S-5：After Web service is issued successfully, 1400001294561 system requests and called Web service of user 1, user
System generate log by real time be written HBase log sheet in, while return log storage whether successful prompt information.
Step S-5-1：Log storage system based on HBase uses CXF（It is the Services frame of an open source）Side
Formula is configured, and as the server end of Web service, is used for Web Services Publishing；Equally, using CXF mode pair
1400001294561 systems carry out the client configuration of Web service, so that custom system is become the client of Web service, can adjust
With the Web service of publication；
Step S-5-2：After running the log storage system based on HBase, authentication service " IsCorrect " and log
Storage service " AddLogs " is published by network；Run 1400001294561 systems of user 1；It is stored when going to log
When, execute step S-5-3；
Step S-5-3：It calls authentication service " IsCorrect ".User 1 provides system account：1400001294561
And system password：A7TwM2fkYnQ0, the two pass to authentication service verifying " IsCorrect " as parameter.If identity
It is proved to be successful, executes step S-5-4；Otherwise, authentication fails, and returns to log and stores failure information；
Step S-5-4：Call log storage service AddLogs.Using newly generated log string as a parameter, name is called
For the log real-time storage service of AddLogs, store to HBase.HBase is the sub-project of Hadoop, and operation is based on
Hadoop distributed storage architecture, therefore before carrying out data storage using HBase, installation configuration Hadoop is first had to,
The concrete configuration that Hadoop configured in this system test phase is as shown in table 3.
The IP address of table 3.Hadoop cluster is distributed
Step S-5-5：Utilize the function of time of JavaSystem.currentTimeMill()It will be converted into time from being
Current time of uniting is to 1970-00-00 00:00:00 millisecond number, here, millisecond number is 14000132474504：As one
Parameter in the major key key of log information, step S-5-4 is stored in the log sheet LogsTable of HBase database as value
In table, log information is stored successfully.Otherwise, log storage failure, goes to step S-5-2.
Claims (4)
1. a kind of implementation method of the log storage system based on HBase, the log storage system that this method uses, including user
Management module is distributed memory space for user's registration, registration audit, administrator and is managed to the basic condition of user；
Log management module, for system account, Log Types and the management of log；Remote service calling module is used for Web service
Publication and custom system call the Web service of publication to carry out the storage of log, it is remote including authentication service and log
Journey real-time storage service；System queries module, for letter connection administrator in the inquiry of remaining space amount, displaying and station；Its
Middle user management module is bi-directionally connected with distributed data base, and distributed data base is bi-directionally connected with log management module, log
Management module is connect with remote service calling module, and remote service calling module passes through distributed data base and system queries module
Connection；
The user management module audits unit, storage space allocation unit and subscriber information management unit comprising user；
The log management module, including system account application unit, Log Types administrative unit and log query unit；
The remote service calling module, including system configuration unit, service release unit and service call unit；
Administrative unit is believed in the system queries module, including remaining space query unit, connection administrator's unit and station；
It is characterized in that, this method comprises the following steps：
S-1：One specifically used person of this system, hereinafter referred to as user, register this system by registration interface；Administrator's audit
User is registered, number is stored by power distribution memory space and maximum system to satisfactory user；
S-2：The user's application system account to succeed in registration, according to current registion time generation system account, according to random cipher
It generates function and generates Unique System password, same registration user can apply for multiple system accounts, system default maximum system simultaneously
Apply for that number is 10, user can contact as needed administrator and change；
S-3：The meaning of the customized each attribute section of user and the identifier of attribute section, system default ' # ' is as between attribute section
Separator, formed a new Log Types；
S-4：It is called to Web server Web Services Publishing for custom system after system starting；
S-5：The log sheet of HBase is written in custom system request and called Web service, the log that custom system generates in real time
In, at the same return log storage whether successful prompt information.
2. a kind of implementation method of log storage system based on HBase according to claim 1, it is characterised in that：
The step S-2 specifically includes that step is implemented as follows：
Step S-2-1：Judge whether system application number has reached the system account application upper limit, if having reached system account application
The upper limit then prompts application to fail；Otherwise continue step S-2-2；
Step S-2-2：User applies for creating new system account, and system is by present system time to 1970-00-00 00:00:
00 time is converted into millisecond number and generates system account；System generates a string by capital and small letter according to customized key derivation function
The random train of letter and number composition is as system password；
Step S-2-3：Major key key of one system account as a custom system, the information such as system password, user's major key are made
For value, then information is stored into the system account table of HBase in the form of key-value key-value pair, system account application
Success；Otherwise, return step S-2-2.
3. a kind of implementation method of log storage system based on HBase according to claim 1, it is characterised in that：
The step S-3 specifically includes that step is implemented as follows：
Step S-3-1：Determine the meaning of each attribute section of Log Types, and the identifier and log of customized each attribute section
Typonym；
Step S-3-2：The sequence of attribute section by definition is spliced into character string with ' # ', forms the content of Log Types；
Step S-3-3：With present system time to 1970-00-00 00:00:00 millisecond number is as major key key, with householder
Key and Log Types information are as value；
Step S-3-4：By in the Log Types table of key-value pair key-value deposit HBase, if storing successfully, returns and successfully believe
Breath；Otherwise, storage failure, return step S-3-1.
4. a kind of implementation method of log storage system based on HBase according to claim 1, it is characterised in that：
The step S-5 specifically includes that step is implemented as follows：
Step S-5-1：System end is configured using CXF mode, as the server end of Web service, for issuing Web clothes
Business；Equally, it is configured using the client that CXF mode carries out Web service to custom system, custom system is made to become Web service
Client；
Step S-5-2：After running this system, Web service is published by network；Run custom system；It is deposited when going to log
Chu Shi executes step S-5-3；
Step S-5-3：Call authentication service：Custom system provides system account and system password, and the two is passed as parameter
Pass the authentication service verifying of entitled " IsCorrect "；If authentication success, executes step S-5-4；Otherwise, body
Part authentication failed returns to log and stores failure information；
Step S-5-4：The service of log real-time storage is called, newly generated log string calls the log of entitled " AddLogs " real
When storage service；
Step S-5-5：It will be converted into the time using the function of time of Java from the current time in system to 1970-00-00 00:00:
Parameter in 00 millisecond number the major key key as a log information, step S-5-4 is stored in the day of HBase as value
In will table, log information is stored successfully；Otherwise, log storage failure, goes to step S-5-2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510187185.6A CN104750870B (en) | 2015-04-20 | 2015-04-20 | A kind of log storage system and implementation method based on HBase |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510187185.6A CN104750870B (en) | 2015-04-20 | 2015-04-20 | A kind of log storage system and implementation method based on HBase |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104750870A CN104750870A (en) | 2015-07-01 |
| CN104750870B true CN104750870B (en) | 2018-11-27 |
Family
ID=53590554
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510187185.6A Active CN104750870B (en) | 2015-04-20 | 2015-04-20 | A kind of log storage system and implementation method based on HBase |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104750870B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105956099A (en) * | 2016-04-28 | 2016-09-21 | 安徽四创电子股份有限公司 | HBase high table-based primary key design method |
| CN109933484A (en) * | 2017-12-15 | 2019-06-25 | 北京京东尚科信息技术有限公司 | Big data cluster quasi real time container resource allocation monitoring analysis method |
| CN108156230B (en) * | 2017-12-19 | 2020-09-04 | 杭州有赞科技有限公司 | Real-time data synchronization method, system and framework |
| CN109978291A (en) * | 2017-12-27 | 2019-07-05 | 广东电网有限责任公司电力调度控制中心 | A kind of Multifunctional power network dispatching management information system |
| CN108804606B (en) * | 2018-05-29 | 2021-08-31 | 上海欣能信息科技发展有限公司 | Method and system for migrating power measurement data to HBase |
| CN109446196A (en) * | 2018-09-26 | 2019-03-08 | 努比亚技术有限公司 | Major key generates processing and determines that method, major key generates and application server |
| CN109447468A (en) * | 2018-10-30 | 2019-03-08 | 福建南威软件有限公司 | A kind of Log Administration System for supporting multisystem, polymorphic type log unified management |
| CN110516434B (en) * | 2019-08-12 | 2021-12-10 | 广州海颐信息安全技术有限公司 | Privileged account scanning system |
| CN115225345B (en) * | 2022-06-29 | 2024-06-14 | 郑州浪潮数据技术有限公司 | Log downloading method, device and medium thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101635651A (en) * | 2009-08-31 | 2010-01-27 | 杭州华三通信技术有限公司 | Method, system and device for managing network log data |
| CN103391185A (en) * | 2013-08-12 | 2013-11-13 | 北京泰乐德信息技术有限公司 | Cloud security storage and processing method and system for rail transit monitoring data |
| CN103701851A (en) * | 2013-11-27 | 2014-04-02 | 中国农业科学院农业信息研究所 | Agricultural technology promotion cloud log management publishing system and method |
| CN104036025A (en) * | 2014-06-27 | 2014-09-10 | 蓝盾信息安全技术有限公司 | Distribution-base mass log collection system |
| CN104252536A (en) * | 2014-09-16 | 2014-12-31 | 福建新大陆软件工程有限公司 | Hbase-based internet log data inquiring method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100484021C (en) * | 2005-04-30 | 2009-04-29 | 华为技术有限公司 | Method and device for realizing household network management |
-
2015
- 2015-04-20 CN CN201510187185.6A patent/CN104750870B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101635651A (en) * | 2009-08-31 | 2010-01-27 | 杭州华三通信技术有限公司 | Method, system and device for managing network log data |
| CN103391185A (en) * | 2013-08-12 | 2013-11-13 | 北京泰乐德信息技术有限公司 | Cloud security storage and processing method and system for rail transit monitoring data |
| CN103701851A (en) * | 2013-11-27 | 2014-04-02 | 中国农业科学院农业信息研究所 | Agricultural technology promotion cloud log management publishing system and method |
| CN104036025A (en) * | 2014-06-27 | 2014-09-10 | 蓝盾信息安全技术有限公司 | Distribution-base mass log collection system |
| CN104252536A (en) * | 2014-09-16 | 2014-12-31 | 福建新大陆软件工程有限公司 | Hbase-based internet log data inquiring method and device |
Non-Patent Citations (1)
| Title |
|---|
| 基于Hadoop架构的大数据量日志存储和检索优化;陈耀兵 等;《信息网络安全》;20131231(第6期);第40-45页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104750870A (en) | 2015-07-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104750870B (en) | A kind of log storage system and implementation method based on HBase | |
| CN108259622B (en) | Cross-region sharing method for electronic license data | |
| CN107613041B (en) | Domain name management system, domain name management method and domain name resolution method based on block chain | |
| CN102880956B (en) | Paying server and payment channel integration method | |
| CN102164186B (en) | Method and system for realizing cloud search service | |
| CN107656981A (en) | A kind of data sharing and management method and system based on identification technology | |
| CN102254022A (en) | Method for sharing metadata of information resources of various data types | |
| CN101442558B (en) | Method and system for providing index service for P2SP network | |
| CN105989426A (en) | Customer relationship management system facing standard information service industry | |
| CN104717284A (en) | Menu management method, device and system | |
| CN103533023B (en) | Cloud service application cluster based on cloud service feature synchronizes system and synchronous method | |
| CN106648903A (en) | Method and system for calling distributed file system | |
| CN104050276A (en) | Cache processing method and system of distributed database | |
| WO2015154119A1 (en) | Sending bills | |
| US9374475B1 (en) | System for processing customer records | |
| US10140347B2 (en) | Exposing data to query generating applications using usage profiles | |
| CN111651525A (en) | Block chain data storage method and device | |
| CN106886592A (en) | A kind of city space-time data collecting system and method based on data source service | |
| US9824227B2 (en) | Simulated control of a third-party database | |
| CN100396002C (en) | System and method for identifying authority using relative inquire | |
| WO2017000592A1 (en) | Data processing method, apparatus and system | |
| CN102932443A (en) | HDFS (hadoop distributed file system) cluster based distributed cloud storage system | |
| CN110532313A (en) | DEU data exchange unit | |
| CN106789147A (en) | A kind of flow analysis method and device | |
| CN101924982A (en) | Method and system for realizing data services |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |