CN115687014A - Service monitoring method, system and related device - Google Patents

Abstract

The application discloses a service monitoring method, a system and a related device, which relate to the technical field of smart home/smart home, and the service monitoring system comprises: a client software development tool component, an output device, and a database. The client software development tool component is used for collecting and tracking log data of the application service, preprocessing the collected log data and sending the preprocessed data to corresponding output equipment; the output equipment is used for transmitting the preprocessed data to the database for storage, and the preprocessing comprises data filtering and/or data analysis; and the database is used for interacting with the visualization end to provide business query service. By reducing the access links, the access cost, the resource consumption and the data loss risk are reduced, the real-time performance is improved, the flexibility is improved, and the expansibility is enhanced.

Description

Service monitoring method, system and related device

Technical Field

The application belongs to the technical field of smart home/smart home, and particularly relates to a service monitoring method, a service monitoring system and a related device.

Background

The high-speed development of internet services brings increasing traffic pressure, service logic is becoming more and more complex, traditional single-machine application cannot meet the requirements, and more websites gradually adopt a distributed deployment architecture. Meanwhile, with the continuous maturity of basic development frameworks such as Spring Cloud/Dubbo and the like, more and more enterprises start to vertically split the website framework according to the service modules, and a distributed micro-service framework which is more suitable for team collaborative development and rapid iteration is formed.

The distributed micro-service architecture has advancement in development efficiency, but brings great challenges to service monitoring, operation and maintenance and diagnosis technologies. Taking service monitoring as an example, depending on the current distributed micro-service architecture, an application service (APP Server) accesses a data collection engine Agent (logstack Agent), then transmits data/logs to Kafka (or Redis), indirectly transmits messages or data in a queue to a data collection engine (logstack), and transmits the data to a distributed search and analysis engine (elastic search) for storage after filtering and analyzing by the data collection engine; finally, the logs and data are presented to the user by the analysis and visualization platform (Kibana). The service monitoring scheme at least has the problems of high access cost, poor real-time performance and the like.

Disclosure of Invention

In order to solve the above problems, that is, to solve the problems of a service monitoring scheme, at least, high access cost, poor real-time performance, and the like, the present application provides a service monitoring method, system, and related apparatus.

In a first aspect, the present application provides a service monitoring system, including: a client software development tool component, an output device and a database; the client software development tool component is used for collecting and tracking log data of the application service, preprocessing the collected log data and sending the preprocessed data to corresponding output equipment; the output equipment is used for transmitting the preprocessed data to the database for storage, and the preprocessing comprises data filtering and/or data analysis; and the database is used for interacting with the visualization end to provide business query service.

In one possible embodiment, the client software development tool component comprises: a log link tracker and a log link collector, wherein: a log link tracker for tracking a log link of an application service; and the log link collector is used for collecting the log data of the log link traced by the log link tracker, preprocessing the collected log data and sending the preprocessed data to the corresponding output equipment.

In one possible implementation, a log link collector includes a collection component and a survey component, wherein: the survey component is used for managing the collection component and determining a log file to be read; and the collection component is used for reading the file content in the log file to be read, preprocessing the file content and sending the read file content to the corresponding output equipment.

In one possible embodiment, the database employs the StarRocks architecture.

In a possible implementation manner, the service monitoring system further includes: and the visualization end is used for interacting with the database so as to visualize the data inquired by the business inquiry service, wherein the visualized data comprises data for reflecting the incidence relation between the business and the service call.

In a possible implementation manner, a visualization tool is integrated in a visualization end, the visualization tool displays queried data through a visualization interface, the visualization interface includes a first area and a second area, the first area is used for displaying the queried data, the second area includes controls corresponding to at least one data processing manner, the visualization end is used for performing data processing on the queried data according to the data processing manner corresponding to the target control when detecting that the queried data is dragged to the target control in the second area, and the data processing manner includes data sharing and chart making.

In a second aspect, the present application provides a service monitoring method, which may be applied to a client software development tool component in the service monitoring system provided in the first aspect, where the service monitoring method includes: collecting and tracking log data of an application service; preprocessing the collected log data, wherein the preprocessing comprises data filtering and/or data analysis; and sending the preprocessed data to corresponding output equipment, wherein the output equipment is used for transmitting the preprocessed data to a database for storage, and the database is used for providing service query service.

One possible implementation, the client software development tool component includes a log link tracker and a log link collector, collects and tracks log data of an application service, including: tracking a log link of an application service by a log link tracker; and collecting log data of the log link traced by the log link tracker through the log link collector, preprocessing the collected log data, and sending the preprocessed data to corresponding output equipment.

In a third aspect, the present application provides an electronic device, comprising: the client software development tool component in the business monitoring system of the first aspect is configured to execute the business monitoring method of the second aspect.

In a fourth aspect, the present application provides a computer-readable storage medium, in which computer program instructions are stored, and the computer program instructions are executed by a processor to implement the traffic monitoring method of the second aspect.

In a fifth aspect, the present application provides a computer program product comprising a computer program for implementing the traffic monitoring method of the second aspect when the computer program is executed by a processor.

The service monitoring method, the service monitoring system and the related device are characterized in that a client software development tool component of the service monitoring system is used for collecting and tracking log data of application services, preprocessing the collected log data and sending the preprocessed data to corresponding output equipment; the output equipment is used for transmitting the preprocessed data to the database for storage, and the preprocessing comprises data filtering and/or data analysis; and the database is used for interacting with the visualization end to provide business query service. The client software development tool component collects and tracks log data of the application service, preprocesses the collected log data, and sends the preprocessed data to the corresponding output equipment, so that access cost, resource consumption and data loss risk can be reduced by reducing access links, real-time performance is improved, and the method is more flexible and expansibility is enhanced.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive labor.

FIG. 1 is a schematic diagram of a currently widely used architecture of a traffic monitoring system;

FIG. 2 is a schematic diagram of an overall design framework of a business monitoring system provided in an embodiment of the present application;

fig. 3 is a schematic architecture diagram of a service monitoring system according to an embodiment of the present application;

FIG. 4 is a schematic sequence diagram of an example HTTP trace in the traffic monitoring system according to the embodiment of the present application;

fig. 5 is a schematic diagram of an operation process of a log link collector in a traffic monitoring system according to an embodiment of the present application;

fig. 6 is a schematic diagram of an architecture of a database used in a business monitoring system according to an embodiment of the present application;

FIG. 7 is a schematic view of a visualization interface of a business monitoring system provided in an embodiment of the present application;

fig. 8 is a flowchart of a service monitoring method according to an embodiment of the present application.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

For the huge challenges brought to the service monitoring, operation and maintenance and diagnosis technology by the distributed micro-service architecture, taking the process of the practice of the distributed micro-service architecture as an example, the main challenges are encountered:

the positioning problem is difficult: a website request in a distributed micro-service architecture usually needs to return a result after passing through a plurality of services/nodes, once the request has an error, the problem can be preliminarily positioned by repeatedly turning over logs on a plurality of machines, and the examination of simple problems also usually involves a plurality of groups;

the discovery of the bottleneck is difficult: is it difficult to find out the bottleneck quickly when the user feedbacks the website with a pause phenomenon, which is the network problem from the user terminal to the server? Or too high a load on the server side results in slow response? Or too much database pressure? Even if the link causing the jamming is positioned, the problem of the code level is difficult to quickly position;

the framework is difficult to comb: after the business logic becomes increasingly complex, it is difficult to comb from the code level which downstream services (databases, HTTP APIs, caches) a certain application depends on and which external calls depend on. The grooming of business logic, administration of architecture, and planning of capacity also becomes more difficult.

Taking the service monitoring as an example, fig. 1 is a schematic diagram of a generally adopted service monitoring system. As shown in fig. 1, the service monitoring system relies on a conventional distributed micro-service architecture at present, an application service (APPServer) accesses a data collection engine Agent (logstack Agent), then transmits data/logs to a database (Kafka or Redis) or a corresponding log topic in a database cluster, indirectly transmits messages or data in a log topic queue to a data collection engine (logstack), and the data collection engine is used as a database consumption group, and transmits the data to a distributed search and analysis engine (Elasticsearch) for storage after filtering and analyzing; the logs and data are finally presented to the user by the analysis and visualization platform (Kibana). However, the technology has the problems of high access cost, long link, more consumed resources, higher risk of data loss, poor flexibility, low expansibility and the like.

In addition, the above-mentioned technology is to obtain the data of the database through certain technologies such as data extraction, cleaning, conversion, loading, etc., then put into the data warehouse (i.e. distributed search and analysis engine), then carry on statistical processing to the data of the data warehouse, achieve the effect of monitoring the business data. The method is more common in the occasion with low timeliness, but can not be adopted in the occasion with high timeliness requirement.

In order to solve the above problems, the present application provides a service monitoring system, in which a collection end Logstash is replaced by a client software development tool component, the client software development tool component collects and tracks log data of an application service, pre-processes the collected log data, and stores the pre-processed data, so as to reduce access cost, resource consumption and data loss risk by reducing access links, improve real-time performance, and enhance flexibility and expansibility.

The Software Development tool component is a client Software Development Kit (SDK) component, wherein English corresponding to the Software Development tool is a Software Development Kit (SDK), and the SDK component can be referred to as the SDK component for short.

The technical solution of the present application will be described in detail below with reference to specific examples. It should be noted that the following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

Fig. 2 is a schematic diagram of an overall design framework of a service monitoring system according to an embodiment of the present application. As shown in fig. 2, the service monitoring system performs operations such as data acquisition, data storage, data processing, and data prediction. Specifically, the collected data includes log data of application services, wherein the application services may include, but are not limited to, applications (apps), h 5/applets, pc, trading systems, order systems, invoice systems, user systems, and the like; for data prediction, visualization or warning of various icon types, such as a line graph or a bar graph, may be performed based on the result of the data prediction.

In the service monitoring system, data storage is carried out on data obtained by executing data acquisition; then, data processing is carried out on the acquired data; and finally, performing data prediction on the data after data processing to obtain a data prediction result required by a user, and displaying the data prediction result through a visual interface.

In the above-mentioned service monitoring system, as an example, the specific manner of the alarm may be to pop up a user selection interface box through a visual interface, based on the user selection interface box, the user may select to re-read the log data of the application service and perform monitoring, or select to abandon monitoring, and the like.

Fig. 3 is a schematic structural diagram of a service monitoring system according to an embodiment of the present application. As shown in fig. 3, the traffic monitoring system 30 includes: a client software development tool component 31, an output device 32, and a database 33. The client software development tool component 31 is configured to collect and track log data of an application service, preprocess the collected log data, and send the preprocessed data to the corresponding output device 32; the output device 32 is used for transmitting the preprocessed data to the database 33 for storage, and the preprocessing comprises data filtering and/or data analysis; the database 33 is used to interact with a visualization terminal (not shown) to provide business query services.

By way of example, the client software development tool component 31 may be identified as log _ sdk. The service monitoring system 30 can collect and report log data in real time in a low-intrusion or zero-intrusion manner such as Java-agent.

Alternatively, the client software development tool component 31 may be installed in various types of clients for tracking and collecting log data of application services installed in the clients. For example, referring to fig. 3, the application services installed in the client include app1, app2, and app3, and the client software development tool component 31 may track and collect log data of the three application services of app1, app2, and app3, as permitted by the user.

For example, the database may be specifically a relational database such as an MPP database, and the output device 32 may be a Message Queue (MQ) cluster, but the embodiment of the present application is not limited thereto.

In the embodiment of the application, the client software development tool component collects and tracks log data of the application service, preprocesses the collected log data, and sends the preprocessed data to the corresponding output equipment, and the data in the database is not required to be stored in the database first and then put into a data warehouse after data extraction, cleaning, conversion, loading and other operations, so that the access cost, resource consumption and data loss risk are reduced by reducing the access link, the real-time performance is improved, and the method is more flexible and the expansibility is enhanced.

In the above embodiment, the client software development tool component 31 may further include: a log link tracker and a log link collector. The log link tracker is used for tracking a log link of the application service; and the log link collector is used for collecting the log data of the log link traced by the log link tracker, preprocessing the collected log data and sending the preprocessed data to the corresponding output equipment. And collecting and reporting the service data in real time by using a low-intrusion or zero-intrusion mode such as Java-agent and the like.

The application service can correspond to a plurality of micro services, the micro services and the micro services can be mutually called in a Remote Procedure Call (RPC) mode or a hypertext Transfer Protocol (HTTP) mode, and one request can relate to a plurality of micro services, and the micro services can depend on other micro services, so that the whole request path forms a mesh Call chain, and once a certain node in the whole Call chain is abnormal, the stability of the whole Call chain can be influenced. Therefore, it is necessary to collect and report system behavior and link information so as to quickly locate and solve problems when a fault occurs, and the working principle is as illustrated in fig. 4. FIG. 4 illustrates an exemplary sequence of HTTP trace, where a User initiates a call request via HTTP, a User Code (User Code) calls resource/foo, and the log link tracker receives the call request (invoke request) to call the resource and generates a thread number, such as X-B3-TraceId: aa and X-B3-Spanisd: 6B. In the calling process, calling information such as recording tags (recorded tags), added Trace headers (added Trace headers), recording time stamps (recorded timestamps), recording interval durations (recorded duration) and the like is recorded, the recording process is realized in a code zero-intrusion mode, and the recorded calling information is stored in a Trace Collector (Trace Collector). After the information is successfully called, a feedback is given to the user, such as 200OK. The resource calling process generates a time span, and the user code reports the time span after receiving the HTTP response.

Further, the log link collector can include a collection component and a survey component that can work in concert to send file changes to specified outputs. Specifically, the method comprises the following steps:

the survey component is used to manage the collection component and determine the log files to be read. Illustratively, the log link collector may be embodied as sg _ log _ collector, the log link tracker may be embodied as sg _ log _ trace, and the survey component is embodied as prospectors. The survey component finds the log files corresponding to the application service, e.g., finds all info log files under/apps/logs/, and starts a collection component for each log file. The survey component will examine each log file to determine whether the collection component to which the log file corresponds has been started, or needs to be started, or whether the log file can be ignored. If the collection component is closed, the survey component will only perform checks if the log file size changes. Alternatively, the survey component can only detect local log files.

The collection assembly is used for reading file contents in the log file to be read, preprocessing the file contents and sending the read file contents to corresponding output equipment. Illustratively, the collection component may be harvesters.

As described above, each log file corresponds to one collection component, each collection component reads the corresponding log file line by line, and the read log files can be sent to a handler (spaoller), log management, formatting, and the like, and then the file content is sent to a designated output, and the working process is as illustrated in fig. 5.

In addition, the collection component is also responsible for opening and closing the log file, which means that while the collection component is running, the file descriptor is in an open state, and if the log file is renamed or deleted in the collection, the log link collector will continue to read this log file, so the disk will not be released until the collection component is closed. By default, the collection component will maintain the file open until the close condition is satisfied. Illustratively, if the collection component is on, the log link collector will close the file handles that are no longer updated for a specified duration, which is timed from the time the collection component reads the last row; if the log file changes after the file handle is closed, a new collection component is started.

Alternatively, the time of closing the file handle is not dependent on the modification time of the file, and if the parameter related to the time of closing the file handle is not properly configured, a situation may occur in which log data is not collected in real time, and for example, the parameter is set to 10 seconds by default.

Optionally, the collection component uses the internal timestamp to record the time when the file was last collected, for example, the time when the file was last collected is default to 5 minutes, then after the collection component reads the last line of the file, the collection component starts to count down for 5 minutes, and if the file read within 5 minutes has no change, then the file handle is closed.

In some embodiments, the log link collector records the file status of the log file in the log file. The file state may characterize the offset at which the collection component collected the log file. If the output device is not connected, the log link collector records the last line before transmission and continues to transmit when the connection is available. The status of the survey component is recorded in memory while the log link collector is running. The log link collector is rebuilt at restart using the registered (registered) state for restoring to the state before restart. Each survey component will record a state for each log file found, and for each log file, the log link collector stores a unique identifier to detect whether the log file was previously collected.

It should be noted that the log link collector can ensure that events are passed to the output of the configuration at least once without data loss. Because the log link collector saves the delivery state of each event in the log file. Without the confirmation of the exporter, the log link collector will try to send until a response is received. If the log link collector is shut down during transmission, then all events are not acknowledged before shut down. Events that are not acknowledged before the log link collector is shut down may be resent after the log link collector is restarted. This ensures that the transmission is at least once, but may be repeated. At this time, the time to wait for the event response before turning off (default disabled) may be set by setting a power-down time limit (shutdown _ timeout) parameter.

Alternatively, the database may employ the StarRocks architecture. A schematic diagram of the StarRocks architecture is shown in fig. 6, the StarRocks architecture is simple and includes a front-end node (FE for short) and a back-end node (BE for short), and is convenient to deploy and maintain without depending on external components. In addition, both the front-end node and the back-end node can be expanded on line horizontally, and both metadata and data have copy mechanisms, so that the whole database is ensured to have no single point.

The front-end node comprises a Catalog management program (Catalog Manager) and a Query Optimizer (Query Optimizer), and is responsible for managing metadata, managing client connection, performing Query planning, query scheduling and other work. The front-end node has two roles depending on the configuration: follower (Follower) node and Observer (Observer) node. Each front-end node retains a complete copy of the metadata in memory so that each front-end node can provide indiscriminate services.

Optionally, the follower node selects a Leader node through a BDBJE protocol of Paxos-like; more than half of follower node instances in the selected main required cluster are alive; the leader node is used for writing the metadata; the non-leader node will automatically route the metadata write request to the leader node. Each time metadata is written, most follower nodes must be successful to confirm that the write was successful.

Optionally, the observer node does not participate in the election master operation, and only asynchronously synchronizes and plays back the log, mainly for expanding the query concurrency capability of the cluster.

The backend node comprises a Storage Engine (Storage Engine) and an Execution Engine (Execution Engine), and is responsible for data Storage, structured Query Language (SQL) Execution and other tasks.

In the aspect of data storage, the back-end nodes are peer-to-peer, and the front-end nodes distribute data to the corresponding back-end nodes according to a certain strategy. When data is imported, the data can be directly written into the back-end node without being transferred through the front-end node, and the back-end node is responsible for writing the imported data into a corresponding format and generating a related index.

Optionally, when the SQL calculation is executed, an SQL statement is first planned into a logical execution unit according to a specific semantic, and then is split into specific physical execution units according to the distribution of data, and the physical execution units are executed on the nodes of the data storage, so that transmission and copying of data can be avoided, and thus, the extreme query performance can be obtained.

Optionally, a MySQL protocol interface is exposed outside of the entire StarRocks architecture adopted by the database, and supports standard SQL syntax, so that a user can conveniently query and analyze data in the database adopting the StarRocks architecture through an existing MySQL client.

Considering that most of the service monitoring tools in the prior art measure the application health degree from the aspects of infrastructure, application systems, requests and the like, however, the measurement indexes lack service semantics and cannot be embodied intuitively. Based on this, the traffic monitoring system may further include: and the visualization terminal is used for interacting with the database so as to visualize the data inquired by the business inquiry service. The visualized data may include, but is not limited to, data for reflecting an association relationship between the service and the service call, for example, a mapping relationship between certain service information and a Uniform Resource Locator (URL) and an RPC interface, including information to be matched and a split dimension, to complete association between the service and the service call. The embodiment of the application monitors the whole link of the key transaction of the business by tracking and collecting the business information in the application service, has high instantaneity and strong flexibility, displays the indexes of the business level in real time, such as the response time, the times and the error rate of the business, solves the problem that the mapping association between the application program and the business performance cannot be realized, and achieves the purposes of visualization and friendliness.

Optionally, a visualization tool is integrated in the visualization end, and the visualization tool displays the queried data through a visualization interface. Optionally, the visualization interface includes a first area and a second area, the first area is used for displaying the queried data, the second area includes at least one control corresponding to each data processing mode, the visualization end is used for processing the queried data according to the data processing mode corresponding to the target control when the target control dragging the queried data to the second area is detected, and the data processing mode includes data sharing, chart making and the like.

Illustratively, FIG. 7 shows a visualization interface of a visualization tool. As shown in FIG. 7, the visualization interface includes several modules, a dashboard, a view, a data set, a data source, and management and security.

Each module in the visual interface is as follows:

instrument panel: the instrument panel can be realized based on Apache EChats, rich chart types are supported, and the instrument panel is quickly manufactured in a dragging mode; fast and safe sharing is supported; multi-end viewing is supported; uniform Resource Locator (URL) viewing is supported. The instrument panel can be added with a plurality of charts, each chart can be provided with a plurality of charts, and the chart types comprise column charts, line charts and the like. The generated chart can be made into a link for rapid and safe sharing, and the view of others is supported. And multi-end viewing, including a PC end, a mobile end or large-screen equipment and the like.

View: the method supports multiple image libraries, index calculation and filtering rules, supports multi-level drilling, and has multiple attributes and styles through visual matching.

Data set: the method supports a direct connection mode and a local mode, wherein the local mode can be realized based on Apache Doris/StarRocks, and supports multi-data set association, data set field calculation, timing synchronization and the like.

A data source: the system comprises a data warehouse/data lake, a distributed relational database, data files, an Application Program Interface (API) data source and the like, wherein the data types comprise a relational database and a non-relational database, the relational database comprises an Amazon Redshift, an open source type database (clickhouse), mySQL and the like, and the non-relational database comprises NoSQL and the like. The distributed relational database includes an On-Line transaction Processing (OLTP) database and an On-Line Analytical Processing (OLAP) database. Data files such as Excel files.

Management and security: the system comprises a click login component, a user and tenant component, a role and permission component, a log audit component, an embedded integrated component, a presentation layer State Transfer (REST) interface component and the like, and is a basic component of a page system. Optionally, after the user logs in the system, the log information may be checked according to the existing user authority.

The visualization tool provided by the embodiment of the application has at least the following advantages:

1. simple and easy to use: the data sets can be clicked and dragged through a mouse, and analysis of the corresponding data sets is completed. Wherein the data set can be dragged to the dashboard, thereby generating a corresponding chart type.

2. And (4) safe sharing: and various data sharing modes are supported, and data security is ensured.

3. Response in seconds: the visualization tool integrates Apache Doris/StarRocks and supports second-level query return delay under the condition of ultra-large data volume.

Optionally, the visualization end may specifically be a PC end, a mobile end, or a large screen device.

The service monitoring system provided by the embodiment of the application has a short link, and each node is more convenient to maintain, so that the defect of difficulty in problem troubleshooting can be overcome.

On the basis of the service monitoring system, the embodiment of the present application further provides a service monitoring method applied to the client software development tool component in the service monitoring system, and the service monitoring method is explained next.

Fig. 8 is a flowchart of a service monitoring method according to an embodiment of the present application. As shown in fig. 8, the service monitoring method includes:

s801: log data for the application service is collected and tracked.

S802: preprocessing the collected log data, including data filtering and/or data analysis.

S803: and sending the preprocessed data to corresponding output equipment, wherein the output equipment is used for transmitting the preprocessed data to a database for storage, and the database is used for providing service query service.

Optionally, the client software development tool component includes a log link tracker and a log link collector. Correspondingly, S801 may further include: tracking a log link of an application service by a log link tracker; and collecting log data of the log link traced by the log link tracker through the log link collector, preprocessing the collected log data, and sending the preprocessed data to corresponding output equipment.

The service monitoring method according to the embodiment of the present application has the similar implementation principle and technical effect as those of the above embodiments, and is not described herein again.

An embodiment of the present application further provides an electronic device, including: the client software development tool component in the business monitoring system is used for executing the business monitoring method.

The embodiment of the present application further provides a computer-readable storage medium, where a computer execution instruction is stored in the computer-readable storage medium, and when the processor executes the computer execution instruction, the service monitoring method is implemented.

Embodiments of the present application further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method for monitoring a service as above is implemented.

The service monitoring method is widely applied to full-House intelligent digital control application scenes such as intelligent homes (Smart Home), intelligent homes, intelligent Home equipment ecology, intelligent Home (Intelligent House) ecology and the like.

The network may include, but is not limited to, at least one of: wired networks, wireless networks.

The wired network may include, but is not limited to, at least one of: wide area networks, metropolitan area networks, local area networks, and the like.

The wireless network may include, but is not limited to, at least one of: wireless fidelity (WIFI for short), bluetooth, etc. The client (or called as "terminal device" or "user device" or "terminal" or the like) may include, but is not limited to, a Personal Computer (Personal Computer, abbreviated as PC), a mobile phone, a tablet PC, an intelligent air conditioner, an intelligent range hood, an intelligent refrigerator, an intelligent oven, an intelligent stove, an intelligent washing machine, an intelligent water heater, an intelligent washing device, an intelligent dish washer, an intelligent projection device, an intelligent television, an intelligent clothes hanger, an intelligent curtain, an intelligent audio/video, an intelligent socket, an intelligent sound box, an intelligent fresh air device, an intelligent kitchen and toilet device, an intelligent bathroom device, an intelligent floor sweeping robot, an intelligent window wiping robot, an intelligent floor mopping robot, an intelligent air purifying device, an intelligent steam box, an intelligent microwave oven, an intelligent kitchen treasure, an intelligent purifier, an intelligent water dispenser, an intelligent door lock, and the like.

The computer readable storage medium may be implemented by any type of volatile or nonvolatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically Erasable Programmable Read-Only Memory (EEPROM), erasable Programmable Read-Only Memory (EPROM), programmable Read-Only Memory (PROM), read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk or optical disk. Readable storage media can be any available media that can be accessed by a general purpose or special purpose computer.

The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims (10)

1. A traffic monitoring system, comprising: a client software development tool component, an output device and a database;

the client software development tool component is used for collecting and tracking the log data of the application service, preprocessing the collected log data and sending the preprocessed data to corresponding output equipment;

the output equipment is used for transmitting the preprocessed data to the database for storage, and the preprocessing comprises data filtering and/or data analysis;

and the database is used for interacting with the visualization end to provide service query service.

2. The traffic monitoring system of claim 1, wherein the client software development tool component comprises: a log link tracker and a log link collector, wherein:

the log link tracker is used for tracking a log link of an application service;

and the log link collector is used for collecting the log data of the log link traced by the log link tracker, preprocessing the collected log data and sending the preprocessed data to the corresponding output equipment.

3. The traffic monitoring system of claim 2, wherein the log link collector comprises a collection component and a survey component, wherein:

the survey component is used for managing the collection component and determining a log file to be read;

and the collection component is used for reading the file content in the log file to be read, preprocessing the file content and sending the read file content to corresponding output equipment.

4. The traffic monitoring system according to any of claims 1 to 3, characterized in that the database employs the StarRocks architecture.

5. The traffic monitoring system according to any of claims 1 to 3, further comprising: and the visualization end is used for interacting with the database so as to visualize the data queried by the business query service, wherein the visualized data comprises data for reflecting the incidence relation between the business and the service call.

6. The business monitoring system according to claim 5, wherein a visualization tool is integrated in the visualization end, the visualization tool displays queried data through a visualization interface, the visualization interface includes a first region and a second region, the first region is used for displaying the queried data, the second region includes at least one control corresponding to each data processing mode, the visualization end is used for processing the queried data according to the data processing mode corresponding to the target control when detecting that the queried data is dragged to the target control in the second region, and the data processing mode includes data sharing and chart making.

7. A business monitoring method applied to a client software development tool component in the business monitoring system according to any one of claims 1 to 6, the business monitoring method comprising:

collecting and tracking log data of an application service;

preprocessing the collected log data, wherein the preprocessing comprises data filtering and/or data analysis;

and sending the preprocessed data to corresponding output equipment, wherein the output equipment is used for transmitting the preprocessed data to the database for storage, and the database is used for providing service query service.

8. The traffic monitoring method of claim 7, wherein the client software development tool component comprises a log link tracker and a log link collector, and wherein collecting and tracking log data of an application service comprises:

tracking, by the log link tracker, a log link of an application service;

and collecting the log data of the log link traced by the log link tracker through the log link collector, preprocessing the collected log data, and sending the preprocessed data to corresponding output equipment.

9. An electronic device, comprising: the client software development tool component in the traffic monitoring system according to any of claims 1 to 6, for performing the traffic monitoring method according to claim 7 or 8.

10. A computer-readable storage medium having computer program instructions stored therein which, when executed, implement the traffic monitoring method of claim 7 or 8.

Images