CN110990178A - Keyword collection monitoring method, device and equipment and computer storage medium - Google Patents
Keyword collection monitoring method, device and equipment and computer storage medium Download PDFInfo
- Publication number
- CN110990178A CN110990178A CN201911181363.9A CN201911181363A CN110990178A CN 110990178 A CN110990178 A CN 110990178A CN 201911181363 A CN201911181363 A CN 201911181363A CN 110990178 A CN110990178 A CN 110990178A
- Authority
- CN
- China
- Prior art keywords
- keyword
- target
- plug
- path configuration
- log file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0766—Error or fault reporting or storing
- G06F11/0775—Content or structure details of the error report, e.g. specific table structure, specific error fields
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- Finance (AREA)
- General Business, Economics & Management (AREA)
- Development Economics (AREA)
- Accounting & Taxation (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to the technical field of financial technology (Fintech) and discloses a keyword acquisition monitoring method, which comprises the following steps: acquiring path configuration of a program log file sent by a server, acquiring a target plug-in with a keyword acquisition function according to the path configuration of the program log file, and scanning an application log of an acquisition end through the target plug-in and the path configuration of the program log file; if a target keyword is scanned in the application log, determining the position information and the hit times of the target keyword in the application log; and storing the position information and the hit times to a temporary file of the acquisition end, and outputting the temporary file when receiving an acquisition reporting instruction sent by the server end. The invention also discloses a keyword acquisition monitoring device, equipment and a computer storage medium. The invention improves the speed of positioning the keyword information in the application log.
Description
Technical Field
The invention relates to the technical field of financial technology (Fintech), in particular to a keyword acquisition monitoring method, a keyword acquisition monitoring device, keyword acquisition monitoring equipment and a computer storage medium.
Background
With the development of computer technology, more and more technologies are applied in the financial field, and the traditional financial industry is gradually changing to financial technology (Fintech), but higher requirements are also put forward on the technologies due to the requirements of the financial industry on safety and real-time performance. For example, when a user writes a script at a client, the user needs to scan logs at regular time and report data to a monitoring service, that is, keyword data hit every minute is reported to a server, when the user wants to check keyword information, the user can only see the number of times of keyword hit, but cannot quickly obtain details of keywords, and can only obtain the details of keywords by checking an original log after receiving an exception. Therefore, how to quickly locate the keyword information in the application log becomes a technical problem to be solved urgently at present.
Disclosure of Invention
The invention mainly aims to provide a method, a device and equipment for collecting and monitoring keywords and a computer storage medium, aiming at improving the speed of positioning the keyword information in an application log.
In order to achieve the above object, the present invention provides a keyword collection monitoring method, which comprises the following steps:
acquiring path configuration of a program log file sent by a server, acquiring a target plug-in with a keyword acquisition function according to the path configuration of the program log file, and scanning an application log of an acquisition end through the target plug-in and the path configuration of the program log file;
if a target keyword is scanned in the application log, determining the position information and the hit times of the target keyword in the application log;
and storing the position information and the hit times to a temporary file of the acquisition end, and outputting the temporary file when receiving an acquisition reporting instruction sent by the server end.
Optionally, the step of obtaining a target plug-in with a keyword collection function according to the program log file path configuration includes:
acquiring a target keyword in the program log file path configuration, and detecting whether a matching plug-in matched with the target keyword exists in the server;
and if so, acquiring a matching plug-in the server, and taking the matching plug-in as a target plug-in with a keyword acquisition function.
Optionally, after the step of detecting whether a matching plug-in matching the target keyword exists in the server, the method includes:
and if the target keyword does not exist in the target plug-in center, acquiring version information of the acquisition end, acquiring the matched plug-in at the plug-in center corresponding to the acquisition end according to the version information and the target keyword, and taking the matched plug-in as the target plug-in with a keyword acquisition function.
Optionally, the step of scanning the application log of the acquisition end through the target plug-in and the program log file path configuration includes:
loading the program log file path configuration through the target plug-in to scan an application log at a collecting end, and determining whether a primary keyword corresponding to the program log file path configuration exists in the application log based on the scanning result;
and if so, determining a target keyword according to the program log file path configuration and the primary keyword.
Optionally, the step of determining a target keyword according to the program log file path configuration and the primary keyword includes:
detecting the primary keyword according to the program log file path configuration, and determining whether the primary keyword meets the configuration requirement of the program log file path configuration;
and if so, taking the primary keyword as a target keyword.
Optionally, after the step of determining whether the primary key satisfies the configuration requirement of the program log file path configuration, the method includes:
and if the primary keyword does not meet the configuration requirement of the program log file path configuration, excluding the primary keyword according to the configuration requirement.
Optionally, the step of scanning the application log at the acquisition end by loading the program log file path configuration through the target plug-in includes:
acquiring a preset time period in the target plug-in, and acquiring a collection mode of key words in the target plug-in;
and loading the program log file path configuration based on the preset time period and the acquisition mode to scan the application log of the acquisition end.
In addition, in order to achieve the above object, the present invention further provides a keyword collection monitoring apparatus, including:
the acquisition module is used for acquiring the path configuration of the program log file sent by the server, acquiring a target plug-in with a keyword acquisition function according to the path configuration of the program log file, and scanning the application log of the acquisition end through the target plug-in and the path configuration of the program log file;
the determining module is used for determining the position information and the hit times of the target keyword in the application log if the target keyword is scanned in the application log;
and the output module is used for storing the position information and the hit times to a temporary file of the acquisition end and outputting the temporary file when receiving an acquisition reporting instruction sent by the server end.
In addition, in order to achieve the above object, the present invention further provides a keyword collection monitoring apparatus, including: the system comprises a memory, a processor and a keyword collection monitoring program which is stored on the memory and can run on the processor, wherein the steps of the keyword collection monitoring method are realized when the keyword collection monitoring program is executed by the processor.
In addition, in order to achieve the above object, the present invention further provides a computer storage medium, where a keyword collection monitoring program is stored on the computer storage medium, and when being executed by a processor, the keyword collection monitoring program implements the steps of the keyword collection monitoring method as described above.
The method comprises the steps of obtaining a program log file path configuration sent by a server, obtaining a target plug-in with a keyword acquisition function according to the program log file path configuration, and scanning an application log of an acquisition end through the target plug-in and the program log file path configuration; if a target keyword is scanned in the application log, determining the position information and the hit times of the target keyword in the application log; and storing the position information and the hit times to a temporary file of the acquisition end, and outputting the temporary file when receiving an acquisition reporting instruction sent by the server end. The target plug-in is obtained according to the program log file path configuration sent by the server side, and the target keywords are scanned according to the target plug-in and the program log file path configuration, so that a user can modify concerned keywords at any time due to flexible configuration of the plug-in, and the position information and the hit times of the target keywords can be output according to the collection reporting instruction, so that the user can clearly know the positions of the target keywords in the application log without inquiring in the source file again, and the efficiency of positioning the keyword information is improved.
Drawings
FIG. 1 is a schematic diagram of an apparatus architecture of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a keyword collection monitoring method according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram of a device module of the keyword collection monitoring device according to the present invention;
FIG. 4 is a schematic flow chart of keyword collection performed by a server in the keyword collection monitoring method according to the present invention;
FIG. 5 is a schematic flow chart of acquiring a plug-in center plug-in the keyword collection monitoring method according to the present invention;
FIG. 6 is a schematic flow chart of keyword collection performed by a collection end in the keyword collection monitoring method according to the present invention;
fig. 7 is a schematic flow chart of the server viewing keywords in the keyword collection monitoring method of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present invention.
The keyword collection monitoring equipment in the embodiment of the invention can be a PC (personal computer) or server equipment, and a Java virtual machine runs on the keyword collection monitoring equipment.
As shown in fig. 1, the keyword collection monitoring apparatus may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration of the apparatus shown in fig. 1 is not intended to be limiting of the apparatus and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a keyword collection monitoring program.
In the device shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be configured to call the keyword collection monitoring program stored in the memory 1005 and perform operations in the keyword collection monitoring method described below.
Based on the hardware structure, the embodiment of the keyword collection monitoring method is provided.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the keyword collection monitoring method of the present invention, where the method includes:
step S10, acquiring program log file path configuration sent by a server, acquiring a target plug-in with a keyword acquisition function according to the program log file path configuration, and scanning an application log of an acquisition end through the target plug-in and the program log file path configuration;
in the embodiment, the collection is realized by designing a plug-in receiving mode, the keyword collection is realized by plug-in design, and the function iteration can be realized by issuing the plug-in without updating the frame main body, so that the whole process is rapid and automatic, and a large amount of release solution is not needed. The plug-in is to make the function of collecting the keywords into a plug-in and place the plug-in the plug-in center, and when the plug-in needs to be updated, only the plug-in needs to be placed in the plug-in center and then an update command is issued. The acquisition end can automatically download a new plug-in to the plug-in center and reload the acquisition task.
In this embodiment, the plug-in generation process is as follows: the basic method for acquiring the plug-in generation is to abstract a parent class of each plug-in and inherit a thread class, and the parent class comprises some basic methods, such as a thread starting method, a collection task stopping method, a collection time acquiring method and other general methods. And acquiring acquisition methods of the acquisition keywords, coding based on the acquisition methods to generate files (i.e., plugins), such as python (computer programming language) files, and storing the plugins in the server, wherein at least one acquisition method exists in each plugin. After the target plug-in is determined in each plug-in, the acquisition method in the target plug-in is started according to a preset time period, for example, the acquisition method in the target plug-in is started in a way of running once per minute. The plug-in dynamically loads the configuration file of the user, reads the application log in real time, and scans the application log every minute according to the configuration. And when the functional plug-in needs to be modified, only the plug-in with the collection method needs to be modified, and the parent class of the plug-in does not need to be modified. It should be noted that, in this embodiment, when the acquisition end scans the application log according to the target plug-in, the target plug-in may acquire the application log according to a method in its parent class.
When a user wants to quickly check keywords, the path configuration of the program log file which needs to be paid attention by the user can be carried out in the server according to the self requirement, for example, the ERROR (ERROR) needing to be paid attention to the application, the ERROR keywords are configured and issued by the server, and then the ERROR keywords are issued to the acquisition end in real time. For example, as shown in fig. 4, the server configures a program log file path configuration with keywords, issues the configuration to the acquisition end, acquires the configuration acquisition in real time, and reports the latest acquisition result, so that the user can acquire the keyword information in real time.
And acquiring the path configuration of a program log file sent by the server in the acquisition end, and acquiring a target plug-in with a keyword acquisition function in the server according to the path configuration of the program log file, but if the target plug-in with the keyword acquisition function does not exist in the server, pulling the target plug-in a plug-in center so as to scan the application log of the acquisition end through the target plug-in and the path configuration of the program log file. The method for acquiring the target plug-in the plug-in center by the acquisition end is shown in fig. 5, the server end issues the version number to the acquisition end, the acquisition end receives the version information, pulls the corresponding plug-in, namely the target plug-in, in the plug-in center according to the version information and the keywords in the program log file path configuration, reloads the target plug-in, and restarts the acquisition task. It should be noted that, in this embodiment, through the design of plug-in, the keyword collection plug-in can dynamically iterate, collect uninterruptedly, and simultaneously can roll back quickly. When rollback is needed, only the server side is needed to issue the previous plug-in version. The collection can be continued using the previous card by rolling back to the previous card. That is, when the user does not want to use the current plug-in for keyword collection, the plug-in that has been used in the history may be selected for keyword collection. The plug-in is to package the program into an independent component, and is flexible to use. The plug-in center is a central server for storing various plug-ins. The keywords are words or phrases that the user is interested in, such as ERROR in the program, Exception, WARNING, etc. In this embodiment, since different plug-ins have different collection methods and each plug-in at least includes one collection method, the target plug-in also includes a target collection method required by the user and is stored in the server in a Python file manner. And in essence the target plug-in is a piece of editable program to facilitate the user to modify the collection method in the target plug-in as desired.
Step S20, if the target keyword is scanned in the application log, determining the position information and the hit frequency of the target keyword in the application log;
when the application log of the acquisition end is scanned through the program log file path configuration and the target plug-in, namely the application log of the acquisition end is scanned through the target plug-in loading program log file path configuration, whether primary keywords (namely the keywords of the application log matched with the keywords in the program log file path configuration) exist in the application log is determined according to the scanning result, and if the primary keywords exist, the primary keywords are excluded and checked according to the exclusion function of the program log file path configuration to obtain the target keywords. For example, when a user needs to pay attention to the ERROR of an application, the server configures ERROR keywords and issues the ERROR keywords to the acquisition end in real time, and when an application log is scanned, the latest configured keywords are loaded, the log is scanned, and the application log takes effect in real time. And because the program log file path configuration has an exclusion function, if a user pays attention to ERROR but does not pay attention to IOERROR (input or output communication ERROR), the exclusion keyword can be configured, namely the IO is excluded. Therefore, information which is not focused by a user can be excluded, and information focused by the user can be focused.
When the target keyword is scanned in the application log, the position information and the hit times of the target keyword in the application log need to be determined, that is, the position information of how many times the target keyword appears in the application log and appears each time is determined. Since the conventional keyword alarm can only be performed or the number of times of occurrence of the keyword is reached, the user needs to manually check the original file when the user wants to acquire the position information of the keyword (namely, the context information of the keyword). Therefore, in addition to the number of keywords (i.e., the number of hits), the position information of each keyword in the application log is also acquired in the present embodiment.
And step S30, storing the position information and the hit times to a temporary file of the acquisition end, and outputting the temporary file when receiving an acquisition reporting instruction sent by the server end.
After the hit times of the keywords and the position information of each keyword are obtained, the position information and the hit times are required to be collected into a temporary file of the collection end, and the file address of the temporary file is reported to the server end in an accessible link mode. For example, as shown in fig. 6, the collector determines whether the keyword is hit according to the configuration scan file (i.e., the application log is configured and scanned according to the path of the program log file), if so, records the context of the keyword to the local temporary file, counts the number of hits of the keyword, and then reports the index (i.e., the number of times that the keyword appears in the application log) and the keyword temporary file to the server to complete the collection. But if the keyword is not hit, only reporting the index to the server to complete the collection.
A Remote Procedure Call (RPC) service is started in the server, and when a link click by a user is detected, that is, an acquisition report instruction input by the user is obtained, the content of accessing a local cache file (that is, a temporary file) is called, and the context of hitting keywords during acquisition is recorded in the cache file. This allows the context of the hit key to be quickly seen without searching into the source file. By means of caching the context of the keywords, the efficiency of keyword positioning is improved, and users can use the method more conveniently. The problem is located more quickly. Among these, the RPC service is a remote procedure call, a request for a service from a remote computer program over a network. For example, as shown in fig. 7, the RPC service is locally started by a log collection processing tool (agent), the server side clicks the link reported by collection, the link calls the RPC service to obtain a hit context, the context is displayed, and error information is quickly displayed to complete quick information positioning.
It should be noted that, currently, many users use the ELK to realize log collection and monitoring, the ELK needs to report all logs to the server, and then the users actively search for corresponding keywords. However, in this embodiment, the statistics of the keywords is performed once in a preset time period (e.g., every minute), the times of occurrence of the keywords in a single minute are counted, the keyword hit conditions are displayed very intuitively, and the keyword context is cached by the client side, so that bandwidth is not consumed, and the keyword context can be located quickly. Among them, ELK is an abbreviation of three open source software, namely Elasticsearch (open source distributed search engine), logstack (tool for collection, analysis, filtering logs of logs), Kibana (open source and free tool).
In the embodiment, by obtaining the path configuration of a program log file sent by a server, and obtaining a target plug-in with a keyword acquisition function according to the path configuration of the program log file, scanning an application log of an acquisition end through the target plug-in and the path configuration of the program log file; if a target keyword is scanned in the application log, determining the position information and the hit times of the target keyword in the application log; and storing the position information and the hit times to a temporary file of the acquisition end, and outputting the temporary file when receiving an acquisition reporting instruction sent by the server end. The target plug-in is obtained according to the program log file path configuration sent by the server side, and the target keywords are scanned according to the target plug-in and the program log file path configuration, so that a user can modify concerned keywords at any time due to flexible configuration of the plug-in, and the position information and the hit times of the target keywords can be output according to the collection reporting instruction, so that the user can clearly know the positions of the target keywords in the application log without inquiring in the source file again, and the efficiency of positioning the keyword information is improved.
Further, based on the first embodiment of the keyword collection monitoring method of the present invention, a second embodiment of the keyword collection monitoring method of the present invention is proposed. This embodiment is a step S10 of the first embodiment of the present invention, and a refinement of the step of obtaining a target plug-in with a keyword collection function according to the program log file path configuration includes:
step a, acquiring a target keyword in the program log file path configuration, and detecting whether a matching plug-in matched with the target keyword exists in the server;
after the acquisition end acquires the path configuration of the program log file sent by the server end, keywords in the path configuration of the program log file are extracted, whether a matching plug-in matched with the keywords exists in the server end or not is detected according to the keywords, and different operations are executed according to different detection results.
And b, if the keyword exists, acquiring a matching plug-in the server, and taking the matching plug-in as a target plug-in with a keyword acquisition function.
And when the matching plug-in which is matched with the target keyword exists in the server through judgment, acquiring the matching plug-in the server in the collector, and taking the matching plug-in as the target plug-in with the keyword acquisition function to acquire the keyword for the application log.
In this embodiment, whether a matching plug-in exists in the server is detected according to a target keyword in the path configuration of the program log file, and if the matching plug-in exists, the matching plug-in is acquired and is used as the target plug-in, so that the accuracy of acquiring the target plug-in is improved.
Further, after the step of detecting whether a matching plug-in matching the target keyword exists in the server, the method includes:
and c, if the target keyword does not exist in the target keyword collection system, acquiring the version information of the collection end, acquiring the matching plug-in at the plug-in center corresponding to the collection end according to the version information and the target keyword, and taking the matching plug-in as the target plug-in with the keyword collection function.
When the matching plug-in which is matched with the target keyword does not exist in the server after judgment, the version information of the acquisition end, such as the system model, the terminal type and the like, needs to be acquired, and the matching plug-in is acquired at the plug-in center corresponding to the acquisition end according to the version information and the target keyword, namely the matching plug-in meeting the requirements of the version information and the target keyword is pulled from the plug-in center, and the matching plug-in is used as the target plug-in with the keyword acquisition function to acquire the keyword for the application log.
In this embodiment, when it is determined that no matching plug-in exists in the server, the matching plug-in is obtained in the plug-in center according to the version information of the acquisition end and the target keyword, and the matching plug-in is used as the target plug-in, so that the accuracy of obtaining the target plug-in is ensured.
Further, based on any one of the first to second embodiments of the keyword collection monitoring method of the present invention, a third embodiment of the keyword collection monitoring method of the present invention is provided. This embodiment is a step S10 of the first embodiment of the present invention, and the refinement of the step of scanning the application log at the acquisition end through the target plug-in and the program log file path configuration includes:
step d, the application log of the acquisition end is scanned by loading the program log file path configuration through the target plug-in, and whether a primary keyword corresponding to the program log file path configuration exists in the application log is determined based on the scanning result;
in the acquisition end, the parent class of the target plug-in is abstracted, and the thread class is inherited. The parent class is common to all plug-ins and comprises a thread starting method, a collection task stopping method, a collection time obtaining method and other general methods. The acquisition method in the target plug-in unit is started in a mode of running once in a preset time period (such as every minute), the target plug-in unit loads program log file path configuration to read an application log of an acquisition end in real time, scans the application log according to the configuration every minute, and determines whether primary keywords corresponding to the program log file path configuration exist in the application log according to a scanning result. And performs different operations according to the determination result. The primary keyword is a keyword matched with the path configuration of the program log file in the application log.
And e, if the program log file path exists, determining a target keyword according to the program log file path configuration and the primary keyword.
When the primary keyword is found to exist in the application log through judgment, the primary keyword is further required to be screened according to an exclusion function in the program log file path configuration so as to determine the target keyword.
In this embodiment, the application log is scanned by loading the program log file path configuration by the target plug-in, and when the primary keyword is acquired, the target keyword is determined according to the program log file path configuration, so that the accuracy of acquiring the target keyword is ensured.
Specifically, the step of determining a target keyword according to the program log file path configuration and the primary keyword includes:
step f, detecting the primary keyword according to the program log file path configuration, and determining whether the primary keyword meets the configuration requirement of the program log file path configuration;
when the acquisition end acquires the primary keyword, the primary keyword is required to be detected according to the path configuration of the program log file so as to determine whether the primary keyword meets the configuration requirement in the path configuration of the program log file, and different operations are executed according to different determination results. The configuration requirement may be a requirement for setting a primary keyword to be more refined, for example, if there are two types of keywords, namely ERROR and IO ERROR, in the primary keyword, and the configuration requirement is that only the ERROR keyword is needed, then the keyword of the IO ERROR type in the primary keyword may be excluded, and only the ERROR keyword is retained.
And g, if so, taking the primary keyword as a target keyword.
And when the primary keyword is found to meet the configuration requirement of the program log file path configuration through judgment, the primary keyword can be used as a target keyword.
In this embodiment, the primary keyword is detected according to the program log file path configuration, and when the primary keyword meets the configuration requirement, the primary keyword is used as the target keyword, so that the accuracy of the obtained target keyword is ensured.
Specifically, after the step of determining whether the primary keyword meets the configuration requirement of the program log file path configuration, the method includes:
and h, if the primary keyword does not meet the configuration requirement of the program log file path configuration, excluding the primary keyword according to the configuration requirement.
When the primary keyword is found to not meet the configuration requirement of the program log file path configuration through judgment, the primary keyword can be subjected to exclusion processing, such as deletion, hiding and the like, according to a preset processing rule in the configuration requirement.
In this embodiment, when it is determined that the primary keyword does not meet the configuration requirement, the primary keyword is excluded according to the configuration requirement, so that accuracy of subsequently acquiring the target keyword is ensured.
Further, the step of scanning the application log of the acquisition end by loading the program log file path configuration through the target plug-in includes:
step k, acquiring a preset time period in the target plug-in, and acquiring an acquisition mode for acquiring keywords in the target plug-in;
the method comprises the steps of obtaining a preset time period (such as every minute) in a target plug-in a collecting end, obtaining a collecting mode of collecting keywords in the target plug-in, and collecting the keywords of an application log according to the preset time period and the collecting mode.
And m, loading the program log file path configuration based on the preset time period and the acquisition mode to scan the application log of the acquisition end.
After the preset time period and the acquisition mode are obtained, the application log of the acquisition end can be scanned by loading the program log file path configuration according to the preset time period and the acquisition mode, for example, the application log of the acquisition end is scanned by loading the program log file path configuration according to the acquisition mode every two minutes.
In the embodiment, the application log is scanned by acquiring the preset time period and the acquisition mode in the target plug-in and loading the program log file path configuration through the preset time period and the acquisition mode, so that the times of occurrence of the target keywords in each preset time period can be intuitively known.
The present invention also provides a keyword collection monitoring apparatus, referring to fig. 3, the keyword collection monitoring apparatus includes:
the acquisition module is used for acquiring the path configuration of the program log file sent by the server, acquiring a target plug-in with a keyword acquisition function according to the path configuration of the program log file, and scanning the application log of the acquisition end through the target plug-in and the path configuration of the program log file;
the determining module is used for determining the position information and the hit times of the target keyword in the application log if the target keyword is scanned in the application log;
and the output module is used for storing the position information and the hit times to a temporary file of the acquisition end and outputting the temporary file when receiving an acquisition reporting instruction sent by the server end.
Optionally, the obtaining module is further configured to:
acquiring a target keyword in the program log file path configuration, and detecting whether a matching plug-in matched with the target keyword exists in the server;
and if so, acquiring a matching plug-in the server, and taking the matching plug-in as a target plug-in with a keyword acquisition function.
Optionally, the obtaining module is further configured to:
and if the target keyword does not exist in the target plug-in center, acquiring version information of the acquisition end, acquiring the matched plug-in at the plug-in center corresponding to the acquisition end according to the version information and the target keyword, and taking the matched plug-in as the target plug-in with a keyword acquisition function.
Optionally, the obtaining module is further configured to:
loading the program log file path configuration through the target plug-in to scan an application log at a collecting end, and determining whether a primary keyword corresponding to the program log file path configuration exists in the application log based on the scanning result;
and if so, determining a target keyword according to the program log file path configuration and the primary keyword.
Optionally, the obtaining module is further configured to:
detecting the primary keyword according to the program log file path configuration, and determining whether the primary keyword meets the configuration requirement of the program log file path configuration;
and if so, taking the primary keyword as a target keyword.
Optionally, the obtaining module is further configured to:
and if the primary keyword does not meet the configuration requirement of the program log file path configuration, excluding the primary keyword according to the configuration requirement.
Optionally, the obtaining module is further configured to:
acquiring a preset time period in the target plug-in, and acquiring a collection mode of key words in the target plug-in;
and loading the program log file path configuration based on the preset time period and the acquisition mode to scan the application log of the acquisition end.
The method executed by each program module can refer to each embodiment of the keyword collection monitoring method of the present invention, and is not described herein again.
The invention also provides a computer storage medium.
The computer storage medium of the invention stores a keyword collection monitoring program, and the steps of the keyword collection monitoring method are realized when the keyword collection monitoring program is executed by a processor.
The method implemented when the keyword collection monitoring program running on the processor is executed may refer to each embodiment of the keyword collection monitoring method of the present invention, and details are not described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A keyword collection monitoring method is characterized by comprising the following steps:
acquiring path configuration of a program log file sent by a server, acquiring a target plug-in with a keyword acquisition function according to the path configuration of the program log file, and scanning an application log of an acquisition end through the target plug-in and the path configuration of the program log file;
if a target keyword is scanned in the application log, determining the position information and the hit times of the target keyword in the application log;
and storing the position information and the hit times to a temporary file of the acquisition end, and outputting the temporary file when receiving an acquisition reporting instruction sent by the server end.
2. The method for monitoring keyword collection according to claim 1, wherein the step of obtaining a target plug-in with a keyword collection function according to the program log file path configuration comprises:
acquiring a target keyword in the program log file path configuration, and detecting whether a matching plug-in matched with the target keyword exists in the server;
and if so, acquiring a matching plug-in the server, and taking the matching plug-in as a target plug-in with a keyword acquisition function.
3. The keyword collection monitoring method according to claim 2, wherein after the step of detecting whether a matching plug-in matching the target keyword exists in the server, the method comprises:
and if the target keyword does not exist in the target plug-in center, acquiring version information of the acquisition end, acquiring the matched plug-in at the plug-in center corresponding to the acquisition end according to the version information and the target keyword, and taking the matched plug-in as the target plug-in with a keyword acquisition function.
4. The keyword collection monitoring method according to any one of claims 1 to 3, wherein the step of scanning the application log of the collection end through the target plug-in and the program log file path configuration comprises:
loading the program log file path configuration through the target plug-in to scan an application log at a collecting end, and determining whether a primary keyword corresponding to the program log file path configuration exists in the application log based on the scanning result;
and if so, determining a target keyword according to the program log file path configuration and the primary keyword.
5. The keyword collection monitoring method according to claim 4, wherein the step of determining the target keyword according to the program log file path configuration and the primary keyword comprises:
detecting the primary keyword according to the program log file path configuration, and determining whether the primary keyword meets the configuration requirement of the program log file path configuration;
and if so, taking the primary keyword as a target keyword.
6. The keyword spotting monitoring method of claim 5 wherein said step of determining whether said primary keyword meets configuration requirements for said program log file path configuration is followed by the steps of:
and if the primary keyword does not meet the configuration requirement of the program log file path configuration, excluding the primary keyword according to the configuration requirement.
7. The keyword collection monitoring method according to claim 4, wherein the step of scanning the application log of the collection end by loading the program log file path configuration through the target plug-in includes:
acquiring a preset time period in the target plug-in, and acquiring a collection mode of key words in the target plug-in;
and loading the program log file path configuration based on the preset time period and the acquisition mode to scan the application log of the acquisition end.
8. A keyword collection monitoring apparatus, comprising:
the acquisition module is used for acquiring the path configuration of the program log file sent by the server, acquiring a target plug-in with a keyword acquisition function according to the path configuration of the program log file, and scanning the application log of the acquisition end through the target plug-in and the path configuration of the program log file;
the determining module is used for determining the position information and the hit times of the target keyword in the application log if the target keyword is scanned in the application log;
and the output module is used for storing the position information and the hit times to a temporary file of the acquisition end and outputting the temporary file when receiving an acquisition reporting instruction sent by the server end.
9. A keyword collection monitoring apparatus, characterized in that the keyword collection monitoring apparatus comprises: a memory, a processor and a keyword collection monitoring program stored on the memory and executable on the processor, the keyword collection monitoring program when executed by the processor implementing the steps of the keyword collection monitoring method according to any one of claims 1 to 7.
10. A computer storage medium having a keyword collection monitoring program stored thereon, wherein the keyword collection monitoring program when executed by a processor implements the steps of the keyword collection monitoring method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911181363.9A CN110990178A (en) | 2019-11-27 | 2019-11-27 | Keyword collection monitoring method, device and equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911181363.9A CN110990178A (en) | 2019-11-27 | 2019-11-27 | Keyword collection monitoring method, device and equipment and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110990178A true CN110990178A (en) | 2020-04-10 |
Family
ID=70087292
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911181363.9A Pending CN110990178A (en) | 2019-11-27 | 2019-11-27 | Keyword collection monitoring method, device and equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110990178A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112506755A (en) * | 2020-12-18 | 2021-03-16 | 北京顺达同行科技有限公司 | Log collection method and device, computer equipment and storage medium |
| CN112910994A (en) * | 2021-01-29 | 2021-06-04 | 重庆长安汽车股份有限公司 | Remote uploading and acquiring method for application logs |
| CN113176978A (en) * | 2021-04-30 | 2021-07-27 | 平安壹钱包电子商务有限公司 | Monitoring method, system and device based on log file and readable storage medium |
| CN113381907A (en) * | 2021-06-04 | 2021-09-10 | 挂号网（杭州）科技有限公司 | Log collection method and device, electronic equipment and storage medium |
-
2019
- 2019-11-27 CN CN201911181363.9A patent/CN110990178A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112506755A (en) * | 2020-12-18 | 2021-03-16 | 北京顺达同行科技有限公司 | Log collection method and device, computer equipment and storage medium |
| CN112506755B (en) * | 2020-12-18 | 2023-11-03 | 北京顺达同行科技有限公司 | Log acquisition method, device, computer equipment and storage medium |
| CN112910994A (en) * | 2021-01-29 | 2021-06-04 | 重庆长安汽车股份有限公司 | Remote uploading and acquiring method for application logs |
| CN113176978A (en) * | 2021-04-30 | 2021-07-27 | 平安壹钱包电子商务有限公司 | Monitoring method, system and device based on log file and readable storage medium |
| CN113381907A (en) * | 2021-06-04 | 2021-09-10 | 挂号网（杭州）科技有限公司 | Log collection method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110990178A (en) | Keyword collection monitoring method, device and equipment and computer storage medium | |
| US8126930B2 (en) | Micro-bucket testing for page optimization | |
| US9590880B2 (en) | Dynamic collection analysis and reporting of telemetry data | |
| US8365062B2 (en) | Auditing a website with page scanning and rendering techniques | |
| CN111897625B (en) | Resource event backtracking method, system and electronic equipment based on Kubernetes cluster | |
| US11637792B2 (en) | Systems and methods for a metadata driven integration of chatbot systems into back-end application services | |
| CN112650688B (en) | Automated regression testing method, associated device and computer program product | |
| CN104572327A (en) | Method, device and system for processing browser crash | |
| US7913233B2 (en) | Performance analyzer | |
| CN112394908A (en) | Method and device for automatically generating embedded point page, computer equipment and storage medium | |
| CN106649120A (en) | Data acquisition method, and data analysis method and system | |
| US20070005679A1 (en) | Server-client hybrid search systems, methods, and apparatuses | |
| CN112817817A (en) | Buried point information query method and device, computer equipment and storage medium | |
| Nayebi et al. | Anatomy of functionality deletion: an exploratory study on mobile apps | |
| CN111352612A (en) | Scheduling method and device for code deployment, electronic equipment and storage medium | |
| US20170139803A1 (en) | Obtaining and analyzing a reduced metric data set | |
| CN109522191A (en) | A kind of method and device of the attribute information of acquisition interbehavior instruction | |
| US20100138833A1 (en) | Resource coverage and analysis | |
| CN114661551A (en) | Method and device for monitoring software running state and computer readable storage medium | |
| CN109002550B (en) | Test method and device for reduction equipment | |
| US20140123126A1 (en) | Automatic topology extraction and plotting with correlation to real time analytic data | |
| CN113010208A (en) | Version information generation method, version information generation device, version information generation equipment and storage medium | |
| CN117271584A (en) | Data processing method and device, computer readable storage medium and electronic equipment | |
| US7051230B2 (en) | Method and system for allowing customization of remote data collection in the event of a system error | |
| CN114090514A (en) | Log retrieval method and device for distributed system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |