CN110175152A - A kind of log inquiring method, transfer server cluster and log query system - Google Patents
A kind of log inquiring method, transfer server cluster and log query system Download PDFInfo
- Publication number
- CN110175152A CN110175152A CN201910461187.8A CN201910461187A CN110175152A CN 110175152 A CN110175152 A CN 110175152A CN 201910461187 A CN201910461187 A CN 201910461187A CN 110175152 A CN110175152 A CN 110175152A
- Authority
- CN
- China
- Prior art keywords
- log
- imported
- server cluster
- message
- oriented middleware
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
- G06F16/134—Distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
Abstract
The embodiment of the invention provides a kind of log inquiring method, transfer server cluster and log query systems, it is related to techno-financial technical field, this method comprises: transfer server is when receiving the importing task of user terminal transmission, log to be imported is obtained from message-oriented middleware according to the attribute information of log to be imported, message-oriented middleware saves the log acquired from application server.Log to be imported is sent to search server cluster by transfer server, therefore search server cluster only needs to segment log to be imported, generate index file, without saving all logs acquired from application server and being segmented to log, to save the machine resources of search server cluster.Secondly, the frequency of user query log is lower, the log for inquiring all acquisitions is not needed, therefore just triggers importing task when user needs to inquire, log to be imported is imported into search server cluster, to improve the utilization rate of machine resources in search server cluster.
Description
Technical field
The present embodiments relate to techno-financial technical field more particularly to a kind of log inquiring methods, transfer server
Cluster and log query system.
Background technique
With the development of computer technology, more and more technical applications are in financial field, and traditional financial industry is gradually
Change to financial technology (Fintech), but due to the safety of financial industry, requirement of real-time, also technology is proposed higher
Requirement.Currently, when being collected to the log on applied host machine, by the various days of log collection collect components applied host machine
Will, compresses and concentrates and be sent to message-oriented middleware, and search server is written in the log that data collection engine collects message-oriented middleware
Cluster, the data that Web platform reads search server cluster are shown in Web page, and operation maintenance personnel is examined by Web page full text
Look into log content.Operation maintenance personnel can also submit search key to search for corresponding log in Web page.Due to various systems
Log amount it is very big, and search server cluster support log full-text search need to carry out word segmentation processing, establish a large amount of rope
Quotation part, therefore cause demand of the log storage to the machine resources of search server cluster very big, to reduce machine
The utilization rate of resource.
Summary of the invention
Since search server cluster, search server is all written in the log that data collection engine collects message-oriented middleware
Cluster carries out participle to log and generates a large amount of index files, causes log storage excessive to the resource requirement of search server cluster
And the problem for causing the utilization rate of machine resources too low, the embodiment of the invention provides a kind of log inquiring methods, transfer service
Device cluster and log query system.
On the one hand, the embodiment of the invention provides a kind of log inquiring methods, comprising:
Transfer server cluster receives the importing task that user terminal is sent, and the importing task includes the category of log to be imported
Property information；
Described in the transfer server cluster is obtained from message-oriented middleware according to the attribute information of the log to be imported
Log to be imported, the message-oriented middleware is for storing the log acquired from application server；
The log to be imported is sent to search server cluster by the transfer server cluster, so that described search takes
Business device cluster segments the log to be imported, generates the corresponding index file of the log to be imported, the index text
Part is used to obtain corresponding target day from the log to be imported when receiving the search key that the user terminal is sent
Will.
Optionally, carry distributed file system on the message-oriented middleware, the distributed file system are used for as institute
It states message-oriented middleware storage log and memory space is provided.
Optionally, the message-oriented middleware is for storing the log acquired from application server, comprising:
The message-oriented middleware constructs the index tab of the log of the acquisition according to the attribute information of the log of acquisition, and
The log of the acquisition is stored according to the index tab.
Optionally, the transfer server cluster is obtained from message-oriented middleware according to the attribute information of the log to be imported
Take the log to be imported, comprising:
The transfer server cluster determines that the log to be imported is corresponding according to the attribute information of the log to be imported
Target index tab；
The transfer server cluster obtains from the message-oriented middleware described wait lead according to the target index tab
Enter log.
Optionally, the importing task is that user needs to trigger when inquiry log in the user terminal.
On the one hand, the embodiment of the invention provides a kind of transfer server clusters, comprising:
Receiving module, for receiving the importing task of user terminal transmission, the importing task includes the category of log to be imported
Property information；
Processing module, it is described to be imported for being obtained from message-oriented middleware according to the attribute information of the log to be imported
Log, the message-oriented middleware is for storing the log acquired from application server；
Sending module, for the log to be imported to be sent to search server cluster, so that described search server
Cluster segments the log to be imported, generates the corresponding index file of the log to be imported, and the index file is used
In when receiving the search key that the user terminal is sent, corresponding target journaling is obtained from the log to be imported.
Optionally, the processing module is specifically used for:
The corresponding target index tab of the log to be imported is determined according to the attribute information of the log to be imported；
The log to be imported is obtained from the message-oriented middleware according to the target index tab.
On the one hand, the embodiment of the invention provides a kind of log query systems, comprising:
Message-oriented middleware, transfer server cluster, search server cluster；
The message-oriented middleware, for storing the log acquired from application server；
The transfer server cluster, for receiving the importing task of user terminal transmission, the importing task includes wait lead
Enter the attribute information of log；It is obtained from the message-oriented middleware according to the attribute information of the log to be imported described to be imported
Log；The log to be imported is sent to described search server cluster；
Described search server cluster generates the log pair to be imported for segmenting to the log to be imported
The index file answered；The search key that the user terminal is sent is received, according to described search keyword and the index file
Corresponding target journaling is obtained from the log to be imported, and the target journaling is sent to the user terminal.
Optionally, carry distributed file system on the message-oriented middleware, the distributed file system are used for as institute
It states message-oriented middleware storage log and memory space is provided.
Optionally, the message-oriented middleware is specifically used for:
The index tab of the log of the acquisition is constructed according to the attribute information of the log of acquisition, and is marked according to the index
Label store the log of the acquisition.
Optionally, the transfer server cluster is specifically used for:
The corresponding target index tab of the log to be imported is determined according to the attribute information of the log to be imported；
The log to be imported is obtained from the message-oriented middleware according to the target index tab.
On the one hand, the embodiment of the invention provides a kind of computer equipment, including memory, processor and it is stored in storage
On device and the computer program that can run on a processor, the processor realize log inquiring method when executing described program
Step.
On the one hand, the embodiment of the invention provides a kind of computer readable storage medium, being stored with can be set by computer
The standby computer program executed, when described program is run on a computing device, so that the computer equipment execution journal
The step of querying method.
In the embodiment of the present invention, transfer server is when receiving the importing task of user terminal transmission, according to day to be imported
The attribute information of will obtains log to be imported from message-oriented middleware, and message-oriented middleware saves the day acquired from application server
Will.Log to be imported is sent to search server cluster by transfer server, therefore search server cluster is only needed to be imported
Log is segmented, and index file is generated, without save all logs acquired from application server and to log into
Row participle, to save the machine resources of search server cluster.Secondly, the frequency of user query log is lower, and it is not required to
The log of all acquisitions is inquired, therefore log to be imported is just imported search service when user terminal triggers importing task by user
Device cluster, so that search server cluster generates the corresponding index file of log to be imported and is supplied to user query, to improve
The utilization rate of machine resources in search server cluster.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly introduced, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill in field, without any creative labor, it can also be obtained according to these attached drawings
His attached drawing.
Fig. 1 is a kind of schematic diagram of application scenarios provided in an embodiment of the present invention；
Fig. 2 is a kind of flow diagram of log inquiring method provided in an embodiment of the present invention；
Fig. 3 is a kind of flow diagram of log inquiring method provided in an embodiment of the present invention；
Fig. 4 is a kind of structural schematic diagram of transfer server cluster provided in an embodiment of the present invention；
Fig. 5 is a kind of structural schematic diagram of log query system provided in an embodiment of the present invention；
Fig. 6 is a kind of structural schematic diagram of computer equipment provided in an embodiment of the present invention.
Specific embodiment
In order to which the purpose of the present invention, technical solution and beneficial effect is more clearly understood, below in conjunction with attached drawing and implementation
Example, the present invention will be described in further detail.It should be appreciated that specific embodiment described herein is only used to explain this hair
It is bright, it is not intended to limit the present invention.
In order to facilitate understanding, noun involved in the embodiment of the present invention is explained below.
ElasticSearch: being the search server based on Lucene, it provides a distributed multi-user energy
The full-text search engine of power is based on RESTful web interface.
Log inquiring method in the embodiment of the present invention can be applied to carry out the server in distributed system architecture
The scene of log analysis, the server in distributed system architecture can be the service server of bank and other financial mechanism.Example
Property, in application scenarios shown in Fig. 1, including application server 101, message-oriented middleware 102, transfer server cluster 103,
Search server cluster 104, user terminal 105.Wherein, application server 101 can be the business service of bank and other financial mechanism
Device.Log collection component, all kinds of days on log collection component persistent collection application server are disposed on application server 101
Will.Log collection component classifies to log and stamps attribute tags in collector journal.It, can be according to day in classification
Type, the rank of log, the format of log, the IP address of application server, period, subsystem of will etc. are classified.Day
The log of collection is sent to message-oriented middleware 102 by will collection assembly.Message-oriented middleware 102 is determined according to the attribute tags of log
Then the index tab of log saves log according to index tab.102 carry distributed file system of message-oriented middleware, point
Cloth file system, which is used to store log for message-oriented middleware 102, provides memory space.User, can be with when needing inquiry log
Importing task is triggered in user terminal 105, user terminal 105 sends importing task to transfer server cluster 103, wraps in importing task
Include the attribute information of log to be imported.Transfer server cluster 103 determines day to be imported according to the attribute information of log to be imported
Then the target index tab of will obtains log to be imported according to target index tab from message-oriented middleware 102, and will be wait lead
Enter log and is sent to search server cluster 104.Search server cluster 104 segments log to be imported, generates wait lead
Enter the corresponding index file of log.User inputs search key in user terminal 105, and user terminal 105 sends search key
To search server cluster 104.Search server cluster 104 is true from log to be imported according to search key and index file
Set the goal log, and target journaling is sent to user terminal 105.
Based on application scenario diagram shown in FIG. 1, the embodiment of the invention provides a kind of process of log inquiring method, the party
The process of method can be executed by message-oriented middleware, transfer server cluster, search server cluster, user terminal interaction, such as Fig. 2 institute
Show, comprising the following steps:
Step S201, transfer server cluster receive the importing task that user terminal is sent.
Specifically, importing task can be when user needs inquiry log in user terminal triggering, and importing task can also be with
It is that user's timing is triggered in user terminal, in addition, the task of importing can also be user terminal clocked flip, in this regard, the present invention is real
Example is applied to be not specifically limited.Importing task includes the attribute information of log to be imported, wherein the attribute information of log to be imported can
To be type, the rank of log, the format of log, the IP address of application server, period, the subsystem etc. of log.It needs
Illustrate, existing inquiry mode is usually the log for inquiring all acquisitions, causes machine resources utilization rate lower.The present invention
In embodiment, the log for inquiring all acquisitions is not needed, therefore just triggers importing task when user needs to inquire, it will be to be imported
Log imports search server cluster, so that search server cluster generates the corresponding index file of log to be imported and is supplied to use
Family inquiry, to improve the utilization rate of machine resources in search server cluster.
Illustratively, setting user needs to inquire application server A in the log of 9:00-10:00, can be in user terminal
Trigger importing task, wherein the attribute information of log to be imported in importing task are as follows: the IP address of application server A, period
9:00-10:00。
Step S202, transfer server cluster are obtained from message-oriented middleware wait lead according to the attribute information of log to be imported
Enter log.
Specifically, message-oriented middleware is for storing the log acquired from application server.On each application server
Log collection component is disposed, the attribute letter of all kinds of logs and log on log collection component persistent collection application server
Breath.Log collection component sends the attribute information of log and log to message-oriented middleware in real time.Message-oriented middleware is according to log
Attribute information constructs the index tab of log, and according to the log of index tab storage acquisition, message-oriented middleware can use suitable
Sequence storage saves log.Optionally, in order to improve the storage capacity of message-oriented middleware, message-oriented middleware carry distributed field system
System, which, which is used to store log for message-oriented middleware, provides memory space.
It, can be with when transfer server cluster obtains log to be imported from message-oriented middleware in a kind of possible embodiment
The corresponding target index tab of log to be imported is first determined according to the attribute information of log to be imported, is then indexed and is marked according to target
Label obtain log to be imported from message-oriented middleware.
Illustratively, the attribute information of log to be imported is set are as follows: the IP address of application server A, period 9:00-
10:00, message-oriented middleware when saving log, using the period as level-one index tab, application server IP address as
Secondary index label, then can determine the corresponding primary target index tab of log to be imported be period 9:00-10:00, two
The IP address that grade target index tab is application server A.First according in period 9:00-10:00 query messages middleware
Then log further screens log using the IP address of application server A, obtains log to be imported.
Log to be imported is sent to search server cluster by step S203, transfer server cluster.
Step S204, search server cluster segment log to be imported, generate the corresponding index of log to be imported
File.
Specifically, search server cluster can be ElasticSearch cluster, and search server cluster is using the row's of falling rope
Draw and log to be imported is segmented, generates index file.Pair between search key and log is saved in index file
It should be related to, a search key may correspond to one or more log.Index file is used to receive user terminal transmission
When search key, corresponding target journaling is obtained from log to be imported, target journaling can be one or more.
Step S205, search server cluster receive the search key that user terminal is sent.
Step S206, search server cluster is according to search key and index file, the acquisition pair from log to be imported
The target journaling answered.
Step S207, search server collection pocket transmission target journaling to user terminal.
Illustratively, setting index file is as shown in table 1:
Table 1.
When the search key that search server cluster receives user terminal transmission is keyword 1, according to as shown in table 1
Index file can determine target journaling be log A, then by log A return user terminal.When search server cluster is receiving
When the search key sent to user terminal is keyword 2, it can determine that target journaling is according to index file as shown in Table 1
Log A and log B is then returned to user terminal by log A and log B.
In the embodiment of the present invention, after acquiring log in application server, log is stored in message-oriented middleware.Transfer
Server is obtained from message-oriented middleware when receiving the importing task of user terminal transmission according to the attribute information of log to be imported
Log to be imported is taken, and log to be imported is sent to search server cluster, therefore search server cluster only needs to treat to lead
Enter log to be segmented, generate corresponding index file, without saving all logs acquired from application server simultaneously
Log is segmented, to save the machine resources of search server cluster.Secondly, the frequency of user query log compared with
It is low, the log for inquiring all acquisitions is not needed, therefore importing task is just triggered when user needs to inquire, log to be imported is led
Enter search server cluster, so that the corresponding index file of search server cluster generation log to be imported is supplied to user and looks into
It askes, to improve the utilization rate of machine resources in search server cluster.
Embodiment in order to preferably explain the present invention describes the embodiment of the present invention below with reference to specific implement scene and provides
A kind of log inquiring method, this method is by log collection component, message-oriented middleware, transfer server cluster, search server
Cluster, user terminal interaction execute, as shown in figure 3, method includes the following steps:
Step S301, the attribute information of log and log on log collection collect components application server.
In the application server, application server can be the business clothes of bank and other financial mechanism to log collection deployment of components
Business device, the attribute information of log includes the IP address of application server, period, subsystem etc..
The attribute information of log and log is sent to message-oriented middleware by step S302, log collection component.
Step S303, message-oriented middleware construct the index tab of log according to the attribute information of log.
Step S304, message-oriented middleware store log according to index tab.
Step S305, user terminal send importing task to transfer server cluster.
User triggers importing task when needing inquiry log, in user terminal, and configures log to be imported in importing task
Attribute information.
Step S306, transfer server cluster determine the corresponding mesh of log to be imported according to the attribute information of log to be imported
Mark index tab.
Step S307, transfer server cluster obtain log to be imported according to target index tab from message-oriented middleware.
The attribute information of log to be imported is set as the IP address of application server B, period 7:00-8:00, subsystem
M.Message-oriented middleware when saving log, using the period as level-one index tab, application server IP address as second level
Index tab, subsystem can then determine that the primary target index tab of log to be imported is the time as three level list label
Section 7:00-8:00, Secondary objective index tab are the IP address of application server B, and three-level target index tab is subsystem M.
Then according to the log in target index tab step by step query messages middleware, log to be imported is obtained.
Log to be imported is sent to search server cluster by step S308, transfer server cluster.
Step S309, search server cluster segment log to be imported, generate the corresponding index of log to be imported
File.
The corresponding relationship between log to be imported and search key is saved in index file.
Step S310, user terminal send search key to search server cluster.
Step S311, search server cluster is according to search key and index file, the acquisition pair from log to be imported
The target journaling answered.
Search server cluster uses search key search index file, determines corresponding target journaling, target journaling
It can be one or more.
Target journaling is sent to user terminal by step S312, search server cluster.
Since the frequency of user query log is lower, the log for inquiring all acquisitions is not needed, therefore needs to look into user
Importing task is just triggered when inquiry, log to be imported is imported into search server cluster, so that search server cluster is generated wait lead
Enter the corresponding index file of log and be supplied to user query, without all logs of acquisition are sent to search server
Cluster to both save the machine resources of search server cluster, while improving the utilization rate of machine resources.
Based on the same technical idea, the embodiment of the invention provides a kind of transfer server clusters, as shown in figure 4, should
Device 400 includes:
Receiving module 401, for receiving the importing task of user terminal transmission, the importing task includes log to be imported
Attribute information；
Processing module 402, for according to the attribute information of the log to be imported obtained from message-oriented middleware it is described to
Log is imported, the message-oriented middleware is for storing the log acquired from application server；
Sending module 403, for the log to be imported to be sent to search server cluster, so that described search service
Device cluster segments the log to be imported, generates the corresponding index file of the log to be imported, the index file
For obtaining corresponding target day from the log to be imported when receiving the search key that the user terminal is sent
Will.
Optionally, the processing module 402 is specifically used for:
The corresponding target index tab of the log to be imported is determined according to the attribute information of the log to be imported；
The log to be imported is obtained from the message-oriented middleware according to the target index tab.
Based on the same technical idea, the embodiment of the invention provides a kind of log query systems, as shown in figure 5, this is
System 500 includes:
Message-oriented middleware 501, transfer server cluster 502, search server cluster 503；
The message-oriented middleware 501, for storing the log acquired from application server；
The transfer server cluster 502, for receive user terminal transmission importing task, the importing task include to
Import the attribute information of log；Described in being obtained from the message-oriented middleware 501 according to the attribute information of the log to be imported
Log to be imported；The log to be imported is sent to described search server cluster 503；
Described search server cluster 503 generates the log to be imported for segmenting to the log to be imported
Corresponding index file；The search key that the user terminal is sent is received, according to described search keyword and index text
Part obtains corresponding target journaling from the log to be imported, and the target journaling is sent to the user terminal.
Optionally, carry distributed file system on the message-oriented middleware 501, the distributed file system is for being
The message-oriented middleware storage log provides memory space.
Optionally, the message-oriented middleware 501 is specifically used for:
The index tab of the log of the acquisition is constructed according to the attribute information of the log of acquisition, and is marked according to the index
Label store the log of the acquisition.
Optionally, the transfer server cluster 503 is specifically used for:
The corresponding target index tab of the log to be imported is determined according to the attribute information of the log to be imported；
The log to be imported is obtained from the message-oriented middleware according to the target index tab.
Based on the same technical idea, the embodiment of the invention provides a kind of computer equipments, as shown in fig. 6, including extremely
Lack a processor 601, and the memory 602 connecting at least one processor, does not limit processing in the embodiment of the present invention
Specific connection medium between device 601 and memory 602 passes through bus between processor 601 and memory 602 in Fig. 6 and connects
For.Bus can be divided into address bus, data/address bus, control bus etc..
In embodiments of the present invention, memory 602 is stored with the instruction that can be executed by least one processor 601, at least
The instruction that one processor 601 is stored by executing memory 602 can execute included in log inquiring method above-mentioned
Step.
Wherein, processor 601 is the control centre of computer equipment, can use various interfaces and connection computer
The various pieces of equipment are stored in memory 602 by running or executing the instruction being stored in memory 602 and calling
Data, to acquire log and inquiry log.Optionally, processor 601 may include one or more processing units, processor
601 can integrate application processor and modem processor, wherein the main processing operation system of application processor, user interface
With application program etc., modem processor mainly handles wireless communication.It is understood that above-mentioned modem processor
It can not be integrated into processor 601.In some embodiments, processor 601 and memory 602 can be real on the same chip
Existing, in some embodiments, they can also be realized respectively on independent chip.
Processor 601 can be general processor, such as central processing unit (CPU), digital signal processor, dedicated integrated
Circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array or other can
Perhaps transistor logic, discrete hardware components may be implemented or execute present invention implementation for programmed logic device, discrete gate
Each method, step and logic diagram disclosed in example.General processor can be microprocessor or any conventional processor
Deng.The step of method in conjunction with disclosed in the embodiment of the present invention, can be embodied directly in hardware processor and execute completion, Huo Zheyong
Hardware and software module combination in processor execute completion.
Memory 602 is used as a kind of non-volatile computer readable storage medium storing program for executing, can be used for storing non-volatile software journey
Sequence, non-volatile computer executable program and module.Memory 602 may include the storage medium of at least one type,
It such as may include flash memory, hard disk, multimedia card, card-type memory, random access storage device (Random Access
Memory, RAM), static random-access memory (Static Random Access Memory, SRAM), may be programmed read-only deposit
Reservoir (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), band
Electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory,
EEPROM), magnetic storage, disk, CD etc..Memory 602 can be used for carrying or storing have instruction or data
The desired program code of structure type and can by any other medium of computer access, but not limited to this.The present invention is real
Applying the memory 602 in example can also be circuit or other devices that arbitrarily can be realized store function, for storing program
Instruction and/or data.
Based on the same technical idea, it the embodiment of the invention provides a kind of computer readable storage medium, is stored with
The computer program that can be executed by computer equipment, when described program is run on a computing device, so that the computer
The step of equipment execution journal querying method.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the present invention
Form.It is deposited moreover, the present invention can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (10)
1. a kind of log inquiring method characterized by comprising
Transfer server cluster receives the importing task that user terminal is sent, and the importing task includes the attribute letter of log to be imported
Breath；
The transfer server cluster obtains from message-oriented middleware described wait lead according to the attribute information of the log to be imported
Enter log, the message-oriented middleware is for storing the log acquired from application server；
The log to be imported is sent to search server cluster by the transfer server cluster, so that described search server
Cluster segments the log to be imported, generates the corresponding index file of the log to be imported, and the index file is used
In when receiving the search key that the user terminal is sent, corresponding target journaling is obtained from the log to be imported.
2. the method as described in claim 1, which is characterized in that carry distributed file system on the message-oriented middleware, institute
It states distributed file system and provides memory space for storing log for the message-oriented middleware.
3. the method as described in claim 1, which is characterized in that the message-oriented middleware is adopted from application server for storing
The log of collection, comprising:
The message-oriented middleware constructs the index tab of the log of the acquisition according to the attribute information of the log of acquisition, and according to
The index tab stores the log of the acquisition.
4. method as claimed in claim 3, which is characterized in that the transfer server cluster is according to the log to be imported
Attribute information obtains the log to be imported from message-oriented middleware, comprising:
The transfer server cluster determines the corresponding mesh of the log to be imported according to the attribute information of the log to be imported
Mark index tab；
The transfer server cluster obtains the day to be imported according to the target index tab from the message-oriented middleware
Will.
5. method according to any of claims 1-4, which is characterized in that the importing task is that user needs inquiry log
When the user terminal triggering.
6. a kind of transfer server cluster characterized by comprising
Receiving module, for receiving the importing task of user terminal transmission, the importing task includes the attribute letter of log to be imported
Breath；
Processing module, for obtaining the day to be imported from message-oriented middleware according to the attribute information of the log to be imported
Will, the message-oriented middleware is for storing the log acquired from application server；
Sending module, for the log to be imported to be sent to search server cluster, so that described search server cluster
The log to be imported is segmented, the corresponding index file of the log to be imported is generated, the index file is used for
When receiving the search key that the user terminal is sent, corresponding target journaling is obtained from the log to be imported.
7. transfer server cluster as claimed in claim 6, which is characterized in that the processing module is specifically used for:
The corresponding target index tab of the log to be imported is determined according to the attribute information of the log to be imported；
The log to be imported is obtained from the message-oriented middleware according to the target index tab.
8. a kind of log query system characterized by comprising
Message-oriented middleware, transfer server cluster, search server cluster；
The message-oriented middleware, for storing the log acquired from application server；
The transfer server cluster, for receiving the importing task of user terminal transmission, the importing task includes day to be imported
The attribute information of will；The day to be imported is obtained from the message-oriented middleware according to the attribute information of the log to be imported
Will；The log to be imported is sent to described search server cluster；
It is corresponding to generate the log to be imported for segmenting to the log to be imported for described search server cluster
Index file；The search key that the user terminal is sent is received, according to described search keyword and the index file from institute
It states and obtains corresponding target journaling in log to be imported, and the target journaling is sent to the user terminal.
9. a kind of computer equipment including memory, processor and stores the meter that can be run on a memory and on a processor
Calculation machine program, which is characterized in that the processor is realized described in Claims 1 to 5 any claim when executing described program
The step of method.
10. a kind of computer readable storage medium, which is characterized in that it is stored with the computer journey that can be executed by computer equipment
Sequence, when described program is run on a computing device, so that computer equipment perform claim requirement 1~5 is any described
The step of method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910461187.8A CN110175152A (en) | 2019-05-30 | 2019-05-30 | A kind of log inquiring method, transfer server cluster and log query system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910461187.8A CN110175152A (en) | 2019-05-30 | 2019-05-30 | A kind of log inquiring method, transfer server cluster and log query system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110175152A true CN110175152A (en) | 2019-08-27 |
Family
ID=67696665
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910461187.8A Pending CN110175152A (en) | 2019-05-30 | 2019-05-30 | A kind of log inquiring method, transfer server cluster and log query system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110175152A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111478889A (en) * | 2020-03-27 | 2020-07-31 | 新浪网技术(中国)有限公司 | Alarm method and device |
| CN111581060A (en) * | 2020-05-11 | 2020-08-25 | 金蝶软件（中国）有限公司 | Prometheus-based log alarm system and method and related equipment |
| CN112948328A (en) * | 2021-01-28 | 2021-06-11 | 长沙市到家悠享网络科技有限公司 | Retrieval method, device, equipment and medium of log data |
| CN113177023A (en) * | 2021-04-19 | 2021-07-27 | 杭州海康威视系统技术有限公司 | Log retrieval method and device and electronic equipment |
| CN115168314A (en) * | 2022-09-08 | 2022-10-11 | 北京亿赛通网络安全技术有限公司 | Log data processing method and system |
-
2019
- 2019-05-30 CN CN201910461187.8A patent/CN110175152A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111478889A (en) * | 2020-03-27 | 2020-07-31 | 新浪网技术(中国)有限公司 | Alarm method and device |
| CN111581060A (en) * | 2020-05-11 | 2020-08-25 | 金蝶软件（中国）有限公司 | Prometheus-based log alarm system and method and related equipment |
| CN111581060B (en) * | 2020-05-11 | 2024-03-12 | 金蝶软件（中国）有限公司 | Prometaus-based log alarm system, method and related equipment |
| CN112948328A (en) * | 2021-01-28 | 2021-06-11 | 长沙市到家悠享网络科技有限公司 | Retrieval method, device, equipment and medium of log data |
| CN113177023A (en) * | 2021-04-19 | 2021-07-27 | 杭州海康威视系统技术有限公司 | Log retrieval method and device and electronic equipment |
| CN115168314A (en) * | 2022-09-08 | 2022-10-11 | 北京亿赛通网络安全技术有限公司 | Log data processing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110175152A (en) | A kind of log inquiring method, transfer server cluster and log query system | |
| US20200293549A1 (en) | Blockchain-based data processing method and device | |
| CN102236581B (en) | Mapping reduction method and system thereof for data center | |
| CN104424229B (en) | A kind of calculation method and system that various dimensions are split | |
| CN109034993A (en) | Account checking method, equipment, system and computer readable storage medium | |
| CN109151824B (en) | Library data service expansion system and method based on 5G architecture | |
| CN107391502B (en) | Time interval data query method and device and index construction method and device | |
| CN102043863B (en) | Method for Web service clustering | |
| CN103368986A (en) | Information recommendation method and information recommendation device | |
| CN106528894B (en) | The method and device of label information is set | |
| CN103345484A (en) | Report form processing system based on dynamic domain and method | |
| US10540386B2 (en) | Method for processing and displaying real-time social data on map | |
| CN109145051A (en) | The data summarization method and device and electronic equipment of distributed data base | |
| CN104077415A (en) | Searching method and device | |
| CN104050193B (en) | Generate the method for message and realize the data handling system of this method | |
| CN104424325A (en) | Data inquiry method and device | |
| CN110297959A (en) | Date storage method, device, storage medium and Edge Server | |
| Zhang et al. | Quegel: A general-purpose system for querying big graphs | |
| CN103677488A (en) | Artificial intelligence user interface | |
| CN112925954A (en) | Method and apparatus for querying data in a graph database | |
| CN106452816B (en) | Information processing method, server and client | |
| CN102902593A (en) | Protocol distribution processing system based on cache mechanism | |
| CN109918277A (en) | Electronic device, the evaluation method of system log cluster analysis result and storage medium | |
| CN114070791B (en) | Speed limiting processing method and device for data traffic | |
| CN115098738A (en) | Service data extraction method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |