CN108319608A - The method, apparatus and system of access log storage inquiry - Google Patents
The method, apparatus and system of access log storage inquiry Download PDFInfo
- Publication number
- CN108319608A CN108319608A CN201710033494.7A CN201710033494A CN108319608A CN 108319608 A CN108319608 A CN 108319608A CN 201710033494 A CN201710033494 A CN 201710033494A CN 108319608 A CN108319608 A CN 108319608A
- Authority
- CN
- China
- Prior art keywords
- keyword
- key
- access log
- inquiry
- access
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention discloses a kind of method, apparatus and system based on nosql databases and the storage inquiry of the access log of file system, can reduce the pressure of relevant database, not influence the processing of regular traffic.This method includes：Key word of the inquiry input by user is obtained, generates keyword key using the key word of the inquiry, wherein the keyword key includes key word of the inquiry and keyword identification；By matching the keyword key with the keyword key stored in the nosql databases pre-established in preset log server, the corresponding value value of the keyword key are obtained；Corresponding journal file is obtained according to described value value, and corresponding access log is obtained from the journal file according to the key word of the inquiry, shows the access log.
Description
Technical field
The present invention relates to technical field of Internet information, and in particular to a kind of based on nosql databases and file system
The method, apparatus and system of access log storage inquiry.
Background technology
As social informatization degree is continuously improved, the channel for accessing system is on the increase, and the channel that system is recorded is visited
Ask that daily record will also increase significantly, system module is on the increase in addition, is also on the increase to relevant database access, this is to relationship
Database proposes more requirements, especially when database connects the resource constraints such as number, storage.
The storage of access log is all based on relevant database, and by taking the BOSS of Hunan as an example, daily daily record amount is about
3000W, it is about 90G to occupy table space, and preserving calculating in 45 days by daily record needs table space 4050G, when daily record stores and inquires
Occupy a large amount of other database resources such as memories, CPU.
It follows that mass data base resource can be occupied using relational data library storage and retrieval access log, and
In the peak traffic phase, memory, CPU that daily record is stored and consumed when inquiring can influence the efficiency that regular traffic accesses database, drop
Low user evaluates system service.
Invention content
In view of the shortcomings of the prior art and defect, the present invention provide a kind of based on nosql databases and file system
Access log storage inquiry method, apparatus and system
On the one hand, the embodiment of the present invention proposes that a kind of access log storage based on nosql databases and file system is looked into
The method of inquiry, including：
Key word of the inquiry input by user is obtained, generates keyword key using the key word of the inquiry, wherein the pass
Key word key includes key word of the inquiry and keyword identification；
By will store in the nosql databases pre-established in the keyword key and preset log server
Keyword key matchings, obtain the corresponding value value of the keyword key, wherein be stored with key in the nosql databases
Word key and its corresponding value value, each keyword key stored in the nosql databases include being used for canal from one
The keyword keyword identification corresponding with the keyword extracted in the access log of road access system, it is described to be used for channel
The access log of access system is stored in the file system of the log server, the key in the form of multiple journal files
The corresponding value value of word key are the filename of the journal file where the corresponding access log of the keyword；
Corresponding journal file is obtained according to described value value, and according to the key word of the inquiry from the journal file
It is middle to obtain corresponding access log, show the access log.
Optionally, described that corresponding access log is obtained from the journal file according to the key word of the inquiry, including：
Each access log in the journal file is traversed, the keyword of the access log is extracted, by the inquiry
Keyword is compared with the keyword extracted, if the key word of the inquiry is identical as the keyword that one is extracted, really
The fixed access log is the corresponding access log.
Optionally, the form of the keyword key is keyword identification _ keyword.
Optionally, if the keyword is to access serial number, the corresponding keyword identification of the keyword is TRANSID；
And/or
If the keyword is cell-phone number, the corresponding keyword identification of the keyword is serialNumber.
On the other hand, the embodiment of the present invention proposes a kind of based on nosql databases and the storage of the access log of file system
The device of inquiry, including：
Generation unit generates keyword for obtaining key word of the inquiry input by user using the key word of the inquiry
Key, wherein the keyword key includes key word of the inquiry and keyword identification；
Matching unit, for the nosql numbers by will be pre-established in the keyword key and preset log server
According to the keyword key matchings stored in library, the corresponding value value of the keyword key are obtained, wherein the nosql databases
In be stored with keyword key and its corresponding value value, each keyword key stored in the nosql databases includes
The keyword keyword identification corresponding with the keyword extracted in the access log for accessing system for channel from one,
The access log of the channel access system is stored in the file system of the log server in the form of multiple journal files,
The corresponding value value of the keyword key is the filename of the journal file where the corresponding access log of the keyword；
Acquiring unit, for obtaining corresponding journal file according to described value value, and according to the key word of the inquiry from
Corresponding access log is obtained in the journal file, shows the access log.
Optionally, the acquiring unit, is specifically used for：
Each access log in the journal file is traversed, the keyword of the access log is extracted, by the inquiry
Keyword is compared with the keyword extracted, if the key word of the inquiry is identical as the keyword that one is extracted, really
The fixed access log is the corresponding access log.
Optionally, the form of the keyword key is keyword identification _ keyword.
Optionally, if the keyword is to access serial number, the corresponding keyword identification of the keyword is TRANSID；
And/or
If the keyword is cell-phone number, the corresponding keyword identification of the keyword is serialNumber.
On the other hand, the embodiment of the present invention proposes a kind of based on nosql databases and the storage of the access log of file system
The system of inquiry, including：
Aforementioned any one of them device, log server and the AM access module that system is accessed for channel；Wherein,
The AM access module is recorded in multiple days for channel to be accessed the access log for accessing system for channel
In will file；
The multiple journal file is stored in file by the log server for obtaining the multiple journal file
System extracts the keyword of every access log in the multiple journal file, identifies the corresponding keyword identification of the keyword,
Keyword key is generated according to the keyword and the keyword identification, by the filename of the journal file where this access log
As the corresponding value value of keyword key, keyword key and its corresponding value value are stored in nosql databases
In.
Optionally, the AM access module is specifically used for generating the multiple journal file according to the preset period.
It is provided in an embodiment of the present invention based on nosql databases and the access log of file system storage inquiry method,
Apparatus and system, renunciation type database, access log is stored in the file system of log server, with the pass of every daily record
Key word and corresponding keyword identification are key, and the entitled value of file where more detailed logging is stored in log server
In nosql databases, the file where access log is found by inquiry nosql databases, then detailed day is found by file
Will simultaneously shows that entire scheme stores inquiry mechanism by the daily record of nosql databases and file system, reduces relational data
The pressure in library, the processing for no longer influencing regular traffic, in addition, this programme extension is easier, is smooth：File system only needs disk to include
Hold, and nosql databases such as redis clusters, only it need to increase node newly.
Description of the drawings
Fig. 1 is that the present invention is based on one embodiments of method of nosql databases and the storage inquiry of the access log of file system
Flow diagram；
Fig. 2 is that the present invention is based on one embodiments of device of nosql databases and the storage inquiry of the access log of file system
Structural schematic diagram；
Fig. 3 is the structural schematic diagram of one embodiment of electronic equipment of the present invention；
Fig. 4 is that the present invention is based on one embodiments of system of nosql databases and the storage inquiry of the access log of file system
Structural schematic diagram.
Specific implementation mode
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical solution in the embodiment of the present invention is explicitly described, it is clear that described embodiment is the present invention
A part of the embodiment, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not having
The every other embodiment obtained under the premise of creative work is made, shall fall within the protection scope of the present invention.
Referring to Fig. 1, the present embodiment disclose it is a kind of based on nosql databases and the access log of file system storage inquiry
Method, including：
S1, key word of the inquiry input by user is obtained, generates keyword key using the key word of the inquiry, wherein described
Keyword key includes key word of the inquiry and keyword identification；
It should be noted that if access log to be checked can provide the key of a variety of energy unique mark access logs
Word can then carry out a variety of condition queries, you can with the class for the unique mark keyword for being included according to access log to be checked
Key word of the inquiry input frame of the type in the corresponding each unique mark key word type of user input interface setting.For example, to Mr. Yu
One accesses system for channel, if its access log contains the serial number of the energy unique mark access log, if with serial number
As keyword, can serial number input frame be set in user input interface, so that user input flow water number, and it is defeated according to user
The serial number queried access daily record entered.It can the unique mark access day if the access log for accessing system for channel certainly contains
Then cell-phone number input frame can be arranged in user input interface at this time using cell-phone number as keyword in the cell-phone number of will, with
Make user's input handset number, and according to cell-phone number queried access daily record input by user.
S2, by will be stored in the keyword key and the nosql databases that are pre-established in preset log server
Keyword key matching, obtain the corresponding value value of the keyword key, wherein stored in the nosql databases related
Key word key and its corresponding value value, each keyword key stored in the nosql databases includes being used for from one
The keyword keyword identification corresponding with the keyword extracted in the access log of channel access system, it is described to be used for canal
The access log of road access system is stored in the file system of the log server, the pass in the form of multiple journal files
The corresponding value value of key word key are the filename of the journal file where the corresponding access log of the keyword；
It is understood that being stored with the energy unique mark visit extracted from access log in the nosql databases
Ask at least one keyword key of all keywords of daily record keyword identification composition corresponding with the keyword.Keyword key
Form can be keyword identification _ keyword, naturally it is also possible to for other forms, such as keyword-keyword identification etc., this
Place repeats no more.The keyword identification is used to identify the type of the keyword.For example, if keyword is serial number, close
Key word mark can be TRANSID；If keyword is cell-phone number, keyword identification can be serialNumber.
S3, corresponding journal file is obtained according to described value value, and literary from the daily record according to the key word of the inquiry
Corresponding access log is obtained in part, shows the access log.
Method provided in an embodiment of the present invention based on nosql databases and the storage inquiry of the access log of file system,
Renunciation type database, access log are stored in the file system of log server, with the keyword and correspondence of every daily record
Keyword identification be key, the entitled value of file where more detailed logging is stored in the nosql databases of log server,
The file where access log is found by inquiry nosql databases, then more detailed logging is found by file and is shown, entire side
Case stores inquiry mechanism by the daily record of nosql databases and file system, the pressure that reduces relevant database, no longer shadow
The processing of regular traffic is rung, in addition, this programme extension is easier, is smooth：File system only needs disk to include appearance, and
Nosql databases such as redis clusters only need to increase node newly.
It is described to be obtained from the journal file according to the key word of the inquiry pair on the basis of preceding method embodiment
The access log answered may include following the step of being not shown in figure：
Each access log in the journal file is traversed, the keyword of the access log is extracted, by the inquiry
Keyword is compared with the keyword extracted, if the key word of the inquiry is identical as the keyword that one is extracted, really
The fixed access log is the corresponding access log.
It is understood that can ensure access log by controlling the size for the journal file that access log stores
Inquiry velocity is within the acceptable range.
Referring to Fig. 2, the present embodiment disclose it is a kind of based on nosql databases and the access log of file system storage inquiry
Device, including：
Generation unit 1 generates keyword for obtaining key word of the inquiry input by user using the key word of the inquiry
Key, wherein the keyword key includes key word of the inquiry and keyword identification；
Matching unit 2, for the nosql by will be pre-established in the keyword key and preset log server
The keyword key matchings stored in database, obtain the corresponding value value of the keyword key, wherein the nosql data
Keyword key and its corresponding value value, each keyword key packets stored in the nosql databases are stored in library
Include the keyword keyword mark corresponding with the keyword extracted in the access log for accessing system for channel from one
Know, the access log of the channel access system is stored in the file system of the log server in the form of multiple journal files
System, the filename of journal files of the corresponding value value of the keyword key where the corresponding access log of the keyword；
In the present embodiment, the form of the keyword key can be keyword identification _ keyword.If the keyword is
Serial number is accessed, the corresponding keyword identification of the keyword can be TRANSID；If the keyword is cell-phone number, described
The corresponding keyword identification of keyword can be serialNumber.
Acquiring unit 3, for obtaining corresponding journal file according to described value value, and according to the key word of the inquiry
Corresponding access log is obtained from the journal file, shows the access log.
In the present embodiment, the acquiring unit specifically can be used for：
Each access log in the journal file is traversed, the keyword of the access log is extracted, by the inquiry
Keyword is compared with the keyword extracted, if the key word of the inquiry is identical as the keyword that one is extracted, really
The fixed access log is the corresponding access log.
Above-mentioned apparatus can execute any one aforementioned access log storage based on nosql databases and file system and look into
The method of inquiry, thus identical technical problem can be solved, reach same technique effect.
To sum up, provided in an embodiment of the present invention that inquiry is stored based on nosql databases and the access log of file system
Device, renunciation type database, access log are stored in the file system of log server, with the keyword of every daily record and
Corresponding keyword identification is key, and the entitled value of file where more detailed logging is stored in the nosql data of log server
In library, the file where access log is found by inquiry nosql databases, then more detailed logging is found by file and is shown,
Entire scheme stores inquiry mechanism by the daily record of nosql databases and file system, reduce relevant database pressure,
The processing of regular traffic is no longer influenced, in addition, this programme extension is easier, is smooth：File system only needs disk to include appearance,
And nosql databases such as redis clusters, only it need to increase node newly.
Fig. 3 shows the entity structure schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention, as shown in figure 3, should
Electronic equipment may include：At least one processor 101, at least one processor 102 and bus 103；Wherein,
The processor 101, memory 102 complete mutual communication by the bus 103；
The memory 102 is stored with the program instruction that can be executed by the processor 101, and the processor 101 can be with
Described program instruction is called, to execute the method that above method embodiment is provided, such as including：Obtain inquiry input by user
Keyword generates keyword key, wherein the keyword key includes key word of the inquiry and key using the key word of the inquiry
Word identifies；Pass through the pass that will be stored in the nosql databases pre-established in the keyword key and preset log server
The key matchings of key word, obtain the corresponding value value of the keyword key, wherein be stored with keyword in the nosql databases
Key and its corresponding value value, each keyword key stored in the nosql databases include being used for channel from one
The keyword keyword identification corresponding with the keyword extracted in the access log of access system, it is described to be visited for channel
Ask that the access log of system is stored in the file system of the log server, the keyword in the form of multiple journal files
The corresponding value value of key are the filename of the journal file where the corresponding access log of the keyword；According to described value
Value obtains corresponding journal file, and obtains corresponding access day from the journal file according to the key word of the inquiry
Will shows the access log.
The embodiment of the present invention discloses a kind of computer program product, and the computer program product is non-transient including being stored in
Computer program on computer readable storage medium, the computer program include program instruction, when described program instructs quilt
When computer executes, computer is able to carry out the method that above method embodiment is provided, such as including：It obtains input by user
Key word of the inquiry utilizes the key word of the inquiry to generate keyword key, wherein the keyword key include key word of the inquiry and
Keyword identification；By will be stored in the keyword key and the nosql databases that are pre-established in preset log server
Keyword key matching, obtain the corresponding value value of the keyword key, wherein stored in the nosql databases related
Key word key and its corresponding value value, each keyword key stored in the nosql databases includes being used for from one
The keyword keyword identification corresponding with the keyword extracted in the access log of channel access system, it is described to be used for canal
The access log of road access system is stored in the file system of the log server, the pass in the form of multiple journal files
The corresponding value value of key word key are the filename of the journal file where the corresponding access log of the keyword；According to described value
Value obtains corresponding journal file, and obtains corresponding access day from the journal file according to the key word of the inquiry
Will shows the access log.
The embodiment of the present invention provides a kind of non-transient computer readable storage medium, the non-transient computer readable storage
Medium storing computer instructs, and the computer instruction makes the computer execute the method that above method embodiment is provided,
Such as including：Key word of the inquiry input by user is obtained, generates keyword key using the key word of the inquiry, wherein the pass
Key word key includes key word of the inquiry and keyword identification；By will be in the keyword key and preset log server in advance
The keyword key matchings stored in the nosql databases of foundation, obtain the corresponding value value of the keyword key, wherein institute
It states and is stored with keyword key and its corresponding value value in nosql databases, each stored in the nosql databases
Keyword key include in the access log for accessed for channel system from one keyword extracting it is corresponding with the keyword
Keyword identification, it is described for channel access system access log the daily record is stored in the form of multiple journal files
The file system of server, the corresponding value value of the keyword key are the daily record where the corresponding access log of the keyword
The filename of file；Corresponding journal file is obtained according to described value value, and according to the key word of the inquiry from the daily record
Corresponding access log is obtained in file, shows the access log.
Referring to Fig. 4, the present embodiment disclose it is a kind of based on nosql databases and the access log of file system storage inquiry
System, including：
It device (i.e. query engine in Fig. 4), log server 5 described in aforementioned any embodiment and is accessed for channel
The AM access module 6 of system；Wherein,
The AM access module 6, it is multiple for the channel access access log for accessing system for channel to be recorded in
In journal file；
As shown in figure 4, when channel accesses and accesses system for channel, the AM access module that system is accessed for channel can handle
Access log is recorded in journal file, specifically can temporally period generation journal file, such as one daily record of generation per minute
Access log is stored in smaller journal file by file, convenient for ensureing the efficiency of subsequent access log query.
The multiple journal file is stored in file by the log server 5 for obtaining the multiple journal file
System extracts the keyword of every access log in the multiple journal file, identifies the corresponding keyword identification of the keyword,
Keyword key is generated according to the keyword and the keyword identification, by the filename of the journal file where this access log
As the corresponding value value of keyword key, keyword key and its corresponding value value are stored in nosql databases
In.
In the present embodiment, log server, can resolution file after obtaining the journal file where AM access module on host
In content, extract the unique mark keyword of every access log, with " keyword identification _ keyword " for key, file is entitled
Value is stored in nosql databases.User can then close to queried access daily record in user input interface input inquiry
Key word generates keyword key according to key word of the inquiry by query engine, is got from nosql databases by keyword key
Corresponding log file name then gets corresponding journal file, parses the journal file, is obtained further according to key word of the inquiry
The details of corresponding daily record, are finally illustrated in more detailed logging on query interface.
System provided in an embodiment of the present invention based on nosql databases and the storage inquiry of the access log of file system,
Renunciation type database, access log are stored in the file system of log server, with the keyword and correspondence of every daily record
Keyword identification be key, the entitled value of file where more detailed logging is stored in the nosql databases of log server,
The file where access log is found by inquiry nosql databases, then more detailed logging is found by file and is shown, entire side
Case stores inquiry mechanism by the daily record of nosql databases and file system, the pressure that reduces relevant database, no longer shadow
The processing of regular traffic is rung, in addition, this programme extension is easier, is smooth：File system only needs disk to include appearance, and
Nosql databases such as redis clusters only need to increase node newly.
It should be understood by those skilled in the art that, embodiments herein can be provided as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the application
Apply the form of example.Moreover, the application can be used in one or more wherein include computer usable program code computer
The computer program production implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of product.
The application is with reference to method, the flow of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that can be realized by computer program instructions every first-class in flowchart and/or the block diagram
The combination of flow and/or box in journey and/or box and flowchart and/or the block diagram.These computer programs can be provided
Instruct the processor of all-purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine so that the instruction executed by computer or the processor of other programmable data processing devices is generated for real
The device for the function of being specified in present one flow of flow chart or one box of multiple flows and/or block diagram or multiple boxes.
These computer program instructions, which may also be stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that instruction generation stored in the computer readable memory includes referring to
Enable the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one box of block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device so that count
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, in computer or
The instruction executed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in a box or multiple boxes.
It should be noted that herein, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also include other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including the element.The fingers such as term "upper", "lower"
The orientation or positional relationship shown is to be based on the orientation or positional relationship shown in the drawings, and is merely for convenience of the description present invention and simplifies
Description, does not indicate or imply the indicated device or element must have a particular orientation, with specific azimuth configuration and behaviour
Make, therefore is not considered as limiting the invention.Unless otherwise clearly defined and limited, term " installation ", " connected ",
" connection " shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or be integrally connected；Can be
Mechanical connection can also be electrical connection；It can be directly connected, can also can be indirectly connected through an intermediary two
Connection inside element.For the ordinary skill in the art, above-mentioned term can be understood at this as the case may be
Concrete meaning in invention.
In the specification of the present invention, numerous specific details are set forth.Although it is understood that the embodiment of the present invention can
To put into practice without these specific details.In some instances, well known method, structure and skill is not been shown in detail
Art, so as not to obscure the understanding of this description.Similarly, it should be understood that disclose in order to simplify the present invention and helps to understand respectively
One or more of a inventive aspect, in the above description of the exemplary embodiment of the present invention, each spy of the invention
Sign is grouped together into sometimes in single embodiment, figure or descriptions thereof.However, should not be by the method solution of the disclosure
It releases and is intended in reflection is following：The feature that i.e. the claimed invention requirement ratio is expressly recited in each claim is more
More features.More precisely, as the following claims reflect, inventive aspect is to be less than single reality disclosed above
Apply all features of example.Therefore, it then follows thus claims of specific implementation mode are expressly incorporated in the specific implementation mode,
Wherein each claim itself is as a separate embodiment of the present invention.It should be noted that in the absence of conflict, this
The feature in embodiment and embodiment in application can be combined with each other.The invention is not limited in any single aspect,
It is not limited to any single embodiment, is also not limited to the arbitrary combination and/or displacement of these aspects and/or embodiment.And
And can be used alone of the invention each aspect and/or embodiment or with other one or more aspects and/or its implement
Example is used in combination.
Finally it should be noted that：The above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Present invention has been described in detail with reference to the aforementioned embodiments for pipe, it will be understood by those of ordinary skill in the art that：Its according to
So can with technical scheme described in the above embodiments is modified, either to which part or all technical features into
Row equivalent replacement；And these modifications or replacements, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme should all cover in the claim of the present invention and the range of specification.
Claims (10)
1. a kind of method based on nosql databases and the storage inquiry of the access log of file system, which is characterized in that including：
Key word of the inquiry input by user is obtained, generates keyword key using the key word of the inquiry, wherein the keyword
Key includes key word of the inquiry and keyword identification；
Pass through the key that will be stored in the nosql databases pre-established in the keyword key and preset log server
Word key matchings, obtain the corresponding value value of the keyword key, wherein be stored with keyword in the nosql databases
Key and its corresponding value value, each keyword key stored in the nosql databases include being used for channel from one
The keyword keyword identification corresponding with the keyword extracted in the access log of access system, it is described to be visited for channel
Ask that the access log of system is stored in the file system of the log server, the keyword in the form of multiple journal files
The corresponding value value of key are the filename of the journal file where the corresponding access log of the keyword；
Corresponding journal file is obtained according to described value value, and is obtained from the journal file according to the key word of the inquiry
Corresponding access log is taken, shows the access log.
2. according to the method described in claim 1, it is characterized in that, it is described according to the key word of the inquiry from the journal file
It is middle to obtain corresponding access log, including：
Each access log in the journal file is traversed, the keyword of the access log is extracted, by inquiry key
Word is compared with the keyword extracted, if the key word of the inquiry is identical as the keyword that one is extracted, it is determined that should
Access log is the corresponding access log.
3. according to the method described in claim 1, it is characterized in that, the form of the keyword key is keyword identification _ key
Word.
4. if according to the method described in claim 1, it is characterized in that, the keyword is to access serial number, the keyword
Corresponding keyword identification is TRANSID；And/or
If the keyword is cell-phone number, the corresponding keyword identification of the keyword is serialNumber.
5. a kind of device based on nosql databases and the storage inquiry of the access log of file system, which is characterized in that including：
Generation unit generates keyword key for obtaining key word of the inquiry input by user using the key word of the inquiry,
In, the keyword key includes key word of the inquiry and keyword identification；
Matching unit, for the nosql databases by will be pre-established in the keyword key and preset log server
The keyword key matchings of middle storage, obtain the corresponding value value of the keyword key, wherein deposited in the nosql databases
Keyword key and its corresponding value value is contained, each keyword key stored in the nosql databases includes from one
The keyword keyword identification corresponding with the keyword that item extracts in the access log for channel access system, it is described
The access log of channel access system is stored in the file system of the log server in the form of multiple journal files, described
The corresponding value value of keyword key are the filename of the journal file where the corresponding access log of the keyword；
Acquiring unit, for obtaining corresponding journal file according to described value value, and according to the key word of the inquiry from described
Corresponding access log is obtained in journal file, shows the access log.
6. device according to claim 5, which is characterized in that the acquiring unit is specifically used for：
Each access log in the journal file is traversed, the keyword of the access log is extracted, by inquiry key
Word is compared with the keyword extracted, if the key word of the inquiry is identical as the keyword that one is extracted, it is determined that should
Access log is the corresponding access log.
7. device according to claim 5, which is characterized in that the form of the keyword key is keyword identification _ key
Word.
8. device according to claim 5, which is characterized in that if the keyword is to access serial number, the keyword
Corresponding keyword identification is TRANSID；And/or
If the keyword is cell-phone number, the corresponding keyword identification of the keyword is serialNumber.
9. a kind of system based on nosql databases and the storage inquiry of the access log of file system, which is characterized in that including：
Claim 5 to 8 any one of them device, log server and the AM access module that system is accessed for channel；Wherein,
The AM access module is recorded in multiple daily record texts for channel to be accessed the access log for accessing system for channel
In part；
The multiple journal file is stored in file system by the log server for obtaining the multiple journal file,
The keyword for extracting every access log in the multiple journal file identifies the corresponding keyword identification of the keyword, according to
The keyword and the keyword identification generate keyword key, using the filename of the journal file where this access log as
The corresponding value value of keyword key, keyword key and its corresponding value value are stored in nosql databases.
10. system according to claim 9, which is characterized in that the AM access module was specifically used for according to the preset period
Generate the multiple journal file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710033494.7A CN108319608A (en) | 2017-01-16 | 2017-01-16 | The method, apparatus and system of access log storage inquiry |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710033494.7A CN108319608A (en) | 2017-01-16 | 2017-01-16 | The method, apparatus and system of access log storage inquiry |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108319608A true CN108319608A (en) | 2018-07-24 |
Family
ID=62892029
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710033494.7A Pending CN108319608A (en) | 2017-01-16 | 2017-01-16 | The method, apparatus and system of access log storage inquiry |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108319608A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110309110A (en) * | 2019-05-24 | 2019-10-08 | 深圳壹账通智能科技有限公司 | A kind of big data log monitoring method and device, storage medium and computer equipment |
| CN110389989A (en) * | 2019-07-15 | 2019-10-29 | 阿里巴巴集团控股有限公司 | A kind of data processing method, device and equipment |
| CN111008183A (en) * | 2019-11-19 | 2020-04-14 | 武汉极意网络科技有限公司 | Storage method and system for business wind control log data |
| CN111488386A (en) * | 2020-04-14 | 2020-08-04 | 北京易数科技有限公司 | Data query method and device |
| CN111966677A (en) * | 2020-06-28 | 2020-11-20 | 北京百度网讯科技有限公司 | Data report processing method and device, electronic equipment and storage medium |
| CN112084249A (en) * | 2020-09-11 | 2020-12-15 | 浙江立元科技有限公司 | Access record extraction method and device |
| WO2021057383A1 (en) * | 2019-09-23 | 2021-04-01 | 深圳前海微众银行股份有限公司 | Log query method, apparatus, device, and computer-readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102193917A (en) * | 2010-03-01 | 2011-09-21 | 中国移动通信集团公司 | Method and device for processing and querying data |
| CN104252536A (en) * | 2014-09-16 | 2014-12-31 | 福建新大陆软件工程有限公司 | Hbase-based internet log data inquiring method and device |
| CN104281672A (en) * | 2014-09-28 | 2015-01-14 | 网神信息技术（北京）股份有限公司 | Log data processing method and device |
| CN105138592A (en) * | 2015-07-31 | 2015-12-09 | 武汉虹信技术服务有限责任公司 | Distributed framework-based log data storing and retrieving method |
-
2017
- 2017-01-16 CN CN201710033494.7A patent/CN108319608A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102193917A (en) * | 2010-03-01 | 2011-09-21 | 中国移动通信集团公司 | Method and device for processing and querying data |
| CN104252536A (en) * | 2014-09-16 | 2014-12-31 | 福建新大陆软件工程有限公司 | Hbase-based internet log data inquiring method and device |
| CN104281672A (en) * | 2014-09-28 | 2015-01-14 | 网神信息技术（北京）股份有限公司 | Log data processing method and device |
| CN105138592A (en) * | 2015-07-31 | 2015-12-09 | 武汉虹信技术服务有限责任公司 | Distributed framework-based log data storing and retrieving method |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110309110A (en) * | 2019-05-24 | 2019-10-08 | 深圳壹账通智能科技有限公司 | A kind of big data log monitoring method and device, storage medium and computer equipment |
| CN110389989A (en) * | 2019-07-15 | 2019-10-29 | 阿里巴巴集团控股有限公司 | A kind of data processing method, device and equipment |
| CN110389989B (en) * | 2019-07-15 | 2023-08-01 | 创新先进技术有限公司 | Data processing method, device and equipment |
| WO2021057383A1 (en) * | 2019-09-23 | 2021-04-01 | 深圳前海微众银行股份有限公司 | Log query method, apparatus, device, and computer-readable storage medium |
| CN111008183A (en) * | 2019-11-19 | 2020-04-14 | 武汉极意网络科技有限公司 | Storage method and system for business wind control log data |
| CN111008183B (en) * | 2019-11-19 | 2023-09-15 | 武汉极意网络科技有限公司 | Storage method and system for business wind control log data |
| CN111488386A (en) * | 2020-04-14 | 2020-08-04 | 北京易数科技有限公司 | Data query method and device |
| CN111488386B (en) * | 2020-04-14 | 2023-09-29 | 北京易数科技有限公司 | Data query method and device |
| CN111966677A (en) * | 2020-06-28 | 2020-11-20 | 北京百度网讯科技有限公司 | Data report processing method and device, electronic equipment and storage medium |
| CN111966677B (en) * | 2020-06-28 | 2024-04-19 | 北京百度网讯科技有限公司 | Data report processing method and device, electronic equipment and storage medium |
| CN112084249A (en) * | 2020-09-11 | 2020-12-15 | 浙江立元科技有限公司 | Access record extraction method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108319608A (en) | The method, apparatus and system of access log storage inquiry | |
| CN108874971B (en) | Tool and method applied to mass tagged entity data storage | |
| CN105183735B (en) | The querying method and inquiry unit of data | |
| US9158843B1 (en) | Addressing mechanism for data at world wide scale | |
| CN107798038B (en) | Data response method and data response equipment | |
| CN106326361B (en) | Data query method and device based on HBase database | |
| US20210303537A1 (en) | Log record identification using aggregated log indexes | |
| CN106407303A (en) | Data storage method and apparatus, and data query method and apparatus | |
| US10289707B2 (en) | Data skipping and compression through partitioning of data | |
| CN106528847A (en) | Multi-dimensional processing method and system for massive data | |
| CN101727502A (en) | Data query method, data query device and data query system | |
| US20210357461A1 (en) | Method, apparatus and storage medium for searching blockchain data | |
| WO2020024903A1 (en) | Method and device for searching for blockchain data, and computer readable storage medium | |
| CN105760380A (en) | Database query method, device and system | |
| US10152239B1 (en) | Migration to a multi-tiered data store | |
| US9323798B2 (en) | Storing a key value to a deleted row based on key range density | |
| CN104598550B (en) | A kind of update method and device of Internet video index | |
| CN106095698A (en) | OO caching write, read method and device | |
| CN109271449A (en) | A kind of distributed storage inquiry system file-based and querying method | |
| CN111258978A (en) | Data storage method | |
| CN106095964A (en) | A kind of method that data are carried out visualization filing and search | |
| US11074266B2 (en) | Semantic concept discovery over event databases | |
| CN110008197A (en) | A kind of data processing method, system and electronic equipment and storage medium | |
| EP2965492A1 (en) | Selection of data storage settings for an application | |
| CN102968456B (en) | A kind of raster data reading and processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180724 |