CN116775440B - Log-level link tracking method and system based on airport production operation system - Google Patents
Log-level link tracking method and system based on airport production operation system Download PDFInfo
- Publication number
- CN116775440B CN116775440B CN202311048005.7A CN202311048005A CN116775440B CN 116775440 B CN116775440 B CN 116775440B CN 202311048005 A CN202311048005 A CN 202311048005A CN 116775440 B CN116775440 B CN 116775440B
- Authority
- CN
- China
- Prior art keywords
- service
- link
- time
- server
- log
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 109
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 26
- 230000008569 process Effects 0.000 claims abstract description 65
- 230000004044 response Effects 0.000 claims abstract description 26
- 238000005516 engineering process Methods 0.000 claims description 32
- 238000012545 processing Methods 0.000 claims description 18
- 230000000694 effects Effects 0.000 claims description 7
- 238000013480 data collection Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 5
- 230000003993 interaction Effects 0.000 claims description 3
- 230000001360 synchronised effect Effects 0.000 claims 1
- 238000003745 diagnosis Methods 0.000 abstract description 8
- 238000004590 computer program Methods 0.000 description 10
- 235000010627 Phaseolus vulgaris Nutrition 0.000 description 7
- 244000046052 Phaseolus vulgaris Species 0.000 description 7
- 238000012544 monitoring process Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 229920001872 Spider silk Polymers 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- NFEIBWMZVIVJLQ-UHFFFAOYSA-N mexiletine hydrochloride Chemical compound [Cl-].CC([NH3+])COC1=C(C)C=CC=C1C NFEIBWMZVIVJLQ-UHFFFAOYSA-N 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
Abstract
The application belongs to the technical field of link tracking in airport operation, and discloses a log-level link tracking method and system based on an airport production operation system. The method calls the dubbo service of each system of the airport, caches the data, accesses the database and records the front-end response through the zipkin; the method comprises the steps of combining a clock synchronization system, obtaining a time stamp of a calling process, and establishing link tracking; the method comprises the steps that the zipkin original record information is generalized and adjusted at a server; and realizing the complete record request full-flow link of the request end at the request end. The application aims to provide complete call link restoration for application developers and maintainers, can intuitively display the time consumption condition of each step of the call, can help the developers to quickly analyze and diagnose the performance bottleneck under the system architecture, and improves the fault diagnosis efficiency.
Description
Technical Field
The application belongs to the technical field of link tracking in airport operation, and particularly relates to a log-level link tracking method and system based on an airport production operation system.
Background
When an aviation field system encounters a problem to be checked, such as a system abnormality or a system performance problem, the conventional problem checking means usually finds out spider silk and horse feet from a log file recorded by the system. The scattered log files have large information quantity, the workload of searching the problems from the log is large, the log does not necessarily contain necessary information, and for a certain request of a user to call back-end services, each service execution condition is unlikely to be obtained from the log.
For the traditional monitoring alarm platform, the monitoring of platform resources only comprises CPU, memory, network bandwidth conditions and the like, and the monitoring of indexes (average response time, slow endpoint conditions and the like) of service micro-service application is not convenient.
The link tracking system can record information within the scope of the request, such as the execution process and time consumption of one remote method call, and is an important tool for checking system problems and system performance.
Through the above analysis, the problems and defects existing in the prior art are as follows: the calling link in the prior art cannot intuitively display the time consumption condition of each step of the calling, so that the fault diagnosis efficiency is lower.
Disclosure of Invention
In order to overcome the problems in the related art, the disclosed embodiments of the present application provide a log-level link tracking method and system based on an airport production operation system.
The technical scheme is as follows: the log-level link tracking method based on the airport production operation system is used for recording the time consumption condition of each step in service call and specifically comprises the following steps:
s101, recording the time consumption of invoking the dubbo service, the data cache, the database access and the front-end response of an airport through a zipkin technology;
s102, enabling clocks of each service in the call to be consistent by using a clock synchronization system, acquiring a time stamp of the call process, and establishing link tracking based on the time stamp;
the step of acquiring the time stamp of the calling process, and establishing link tracking based on the time stamp comprises the following steps:
the local interceptor automatically records the starting time of the call, and the link tracking time stamp is based on the time of the clock synchronization system; all components are accessed into an airport clock synchronization system, and clock synchronization is realized through an NTP network protocol, so that the time stamp of span of each component in the process is consistent;
the NTP network protocol includes: firstly, a clock synchronization server acquires accurate time information through GPS signals; the other servers periodically send time requests to the clock synchronization server; the NTP network protocol estimates the time delay of network transmission by using an algorithm of time delay and offset, adjusts a local clock according to the response of a server, and continuously carries out the process until the local clock is consistent with the clock of a time server;
s103, the server records time consumption at each stage in the process from the beginning to the end of the service through the zipkin technology, and generalizes and adjusts the original recorded information of the zipkin technology in combination with the actual service requirements of the airport;
the server records time consumption of each stage in the process from the start to the end of the service through a zipkin technology, and the method comprises the following steps:
after the request reaches the server, the server interceptor (3) automatically records the starting time of the server, and the timestamp keeps consistent with the timestamp of the request end by using the timestamp of the clock synchronization system, so that the accuracy of the timestamp is ensured;
the airport actual business requirements include: recording the time consumption of different service operations by using the zipkin to exceed a certain time at the interface;
for example, a manual refresh operation requires a relatively low operation time, which should be recorded over 1 s; other operations, such as automatically processing external messages, etc., are not perceived by the user, so more than 3s is re-recorded;
s104, recording and calling the whole process at the request end based on the zipkin technology, adding the content of a response head of the service end dubbo protocol for recording the information of the request link of the service end, collecting the information of the log of the service end by the service end, summarizing the log of the request end by using the traceid, and finally realizing the complete recording of the request whole process link of the request end.
In step S101, the dubbo service call to the airport by the zip technique includes:
automatically acquiring link tracking configuration information from a link configuration service, and recording a link after the interface needing to be tracked and the response time consumption exceed a certain threshold; the system completes specific link tracking record based on the complete configuration of the link configuration service center.
Further, the recording of time consumption of each stage in the process from the start to the end of the service by the server through the zipkin technology further comprises:
respectively adapting aiming at a data grid cache, an oracle database and an A-MQ message middleware, recording specific time stamps of calling before and after each component is called, and storing the specific time stamps in the tracking; the system records specific time consumption as a basic span unit according to each component, and integrates the whole process in the whole call response process in a tree mode.
Further, when the request is called, the request end and the server end interceptor can automatically record the time stamp of the clock synchronization system used by the calling time stamp to ensure the time consistency.
In step S104, the implementing the server to gather the server log information includes:
after the service call is finished, the requested party completes link data collection based on the zipkin technology and automatically generates all span information in the process, records the span information in a json format into a log, and ensures time consistency by using the time stamp of the clock synchronization system.
Further, the method for realizing the server to collect the log information of the server further comprises the following steps:
after the service call is finished, the requester collects and sorts the span information of the whole call service and the span information of the service of the requester together, and records the span information and the span information into the service log.
Further, the method for realizing the server to collect the log information of the server further comprises the following steps:
after receiving the message containing all link tracking data, the span processor comprehensively processes the local link data, the network link data and the server link data, uses the track and the log summary of the request terminal, and finally prints all span information called locally into the log file at the same time to realize the complete record of the request full-flow link of the request terminal.
Another object of the present application is to provide a log-level link tracking system based on an airport production operation system, implementing the log-level link tracking method based on the airport production operation system, the system comprising:
the system automatically acquires link tracking configuration information from the link configuration service when the service is started, and records the link after the interface needing to be tracked and the response time consumption exceed a certain threshold;
the local interceptor is used for automatically recording the starting time of the call after the call of the request end;
the server interceptor is used for automatically recording the starting time of the server after the request reaches the server, and the timestamp is consistent with the timestamp of the clock synchronization system used by the request;
the specific timestamp calling module is used for respectively adapting to the DataGrid cache, the oracle database and the A-MQ message middleware by combining with the airport system technical structure, realizing the specific timestamps of calling before and after each component is called and storing the specific timestamps in the tracking;
the link tracking module is mutually called between the micro services, and when a request is called, the request end and the server end interceptor can automatically record the time stamp of a clock synchronization system used by the calling time stamp to ensure the time consistency;
the information generation module is used for completing link data collection and automatically generating all span information in the process based on the zipkin technology after the requested party finishes service call based on the inter-call of the micro-service technology, recording the span information into a log in json format, and ensuring time consistency by using the time stamp of the clock synchronization system;
the collection and arrangement module is used for collecting and arranging span information of the whole calling service and span information of the requested party service together after service calling is finished by the solving party based on the inter-calling of the micro service technology, and recording the span information and the span information into the service log;
and the request full-flow link effect display module is used for comprehensively processing the local link data, the network link data and the server link data after the span processor receives the message containing all the link tracking data after the service request is finished, summarizing the logs of the request terminal by using the traceid and the request terminal, and finally printing all the span information of the local call into the log file at the same time to realize the complete record of the request full-flow link of the request terminal.
By combining all the technical schemes, the application has the advantages and positive effects that: the innovation of the application is the tracking of systems built in airports using older architectures, which are currently used less. The application aims to provide complete call link restoration for application developers and maintainers, can intuitively display the time consumption condition of each step of the call, can help the developers to quickly analyze and diagnose the performance bottleneck under the system architecture, and improves the fault diagnosis efficiency.
The application also has the following advantages: (1) link query and diagnosis: all link information from the client to the background, database, cache, etc. can be tracked and aggregated into a call chain. Once which link in the link fails, the link can be quickly positioned by calling. (2) performance analysis and optimization: the execution performance of each link can be analyzed and optimized according to the time consumption condition of each link. When a request with higher average response time or higher error rate is found, a single link with higher time consumption can be located, so that performance optimization is performed. (3) The link tracking of the present application can span multiple services, such as clients, background services, databases, caches, etc., and by analyzing time consuming conditions in the process, performance bottlenecks in each step can be analyzed. The system can be sampled as required, only records the interfaces of key requests and time consuming, and can be integrated with other monitoring systems so as to quickly respond and solve the problems. (4) The application improves the efficiency of fault diagnosis, and the original problem confirmation needs about one hour, and the application can shorten the efficiency of fault diagnosis to about 5 minutes. The application solves the problem that the frame does not realize link tracking all the time in the airport production operation system in the prior art.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure;
FIG. 1 is a flow chart of a log-level link tracking method based on an airport production operation system provided by an embodiment of the application;
FIG. 2 is a schematic diagram of a log-level link tracking method based on an airport production operation system according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a log-level link tracking system based on an airport production operating system provided by an embodiment of the present application;
in the figure: 1. a link configuration service module; 2. a local interceptor; 3. a server interceptor; 4. a specific timestamp calling module; 5. the link tracking module is mutually called among the micro services; 6. an information generation module; 7. collecting and arranging modules; 8. and requesting a full-flow link effect display module.
Detailed Description
In order that the above objects, features and advantages of the application will be readily understood, a more particular description of the application will be rendered by reference to the appended drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. The application may be embodied in many other forms than described herein and similarly modified by those skilled in the art without departing from the spirit or scope of the application, which is therefore not limited to the specific embodiments disclosed below.
In embodiment 1, the log-level link tracking method based on the airport production operation system provided by the embodiment of the application adopts technologies such as zip and dubbo, integrates characteristics of a zip automatic tracking link by utilizing a dubbo framework, and establishes a specific link tracking scheme based on airport business.
As shown in fig. 1, the method specifically comprises the following steps:
s101, recording the time consumption of invoking the dubbo service, the data cache, the database access and the front-end response of an airport through a zipkin technology;
s102, enabling clocks of each service in the call to be consistent by using a clock synchronization system, acquiring a time stamp of the call process, and establishing link tracking based on the time stamp;
s103, the server records time consumption at each stage in the process from the beginning to the end of the service through the zipkin technology, and generalizes and adjusts the original recorded information of the zipkin technology in combination with the actual service requirements of the airport;
s104, recording and calling the whole process at the request end based on the zipkin technology, adding the content of a response head of the service end dubbo protocol for recording the information of the request link of the service end, collecting the information of the log of the service end by the service end, summarizing the log of the request end by using the traceid, and finally realizing the complete recording of the request whole process link of the request end.
Illustratively, in step S101, for the dubbo micro service framework used in the airport, the call log record is completed through the whole call of the zipkin to the dubbo service call, the data cache, the database access, the front end response and the like of the system by combining the frame characteristics of the dubbo such as load balancing and interceptor, and finally the complete record of the call complete process is completed.
It can be understood that the airport uses the dubbo micro-service framework, load balancing is realized through the zookeeper, the zipkin has special plug-ins for the dubbo framework, cache, database and the like, and the processor Bean for response can be registered when the dubbo call, the cache call and the database call start and end, so that the information can be statistically reported;
after the front end of the airport triggers the dubbo service call, the background receives the request to process data, a cache or a database may be used in the process, and after the data processing is completed, a result or a state is returned to the front end;
the data processing is interface processing, and the interface processes according to parameters, returns information, writes a library and the like.
In step S102, under the condition that the clock synchronization system obtains the unique time accurately and without errors, the clock synchronization system is combined to achieve the time stamp accuracy of each calling process, and further achieve process tracking that the network, the link establishment and the like in the whole calling process cannot be recorded through the system.
It will be appreciated that each component will obtain a uniform timestamp through the clock synchronization system. When service call starts, each passing component comprising a front end, a background, a cache, a database and the like records starting and ending time stamps, so that tracking of the whole link is realized;
wherein, the tracking of the whole link comprises: the Zipkin has different plug-ins for different architectures, the plug-ins are similar to interceptors, the plug-ins can be registered in the system when the system is started, and when the front end, the background, the cache and the database process requests, all the plug-ins pass through the interceptors in the plug-ins, so that each time point of the requests can be recorded.
All components are accessed into an airport clock synchronization system, and clock synchronization is realized through an NTP network protocol, so that the time stamp of each component in the process is consistent.
The clock synchronization protocol used is NTP network protocol, firstly the clock synchronization server obtains accurate time information through GPS signal. Other servers periodically send time requests to the clock synchronization server, and NTP network protocols use an algorithm of delay and offset to estimate the time delay of the network transmission and adjust the local clock based on the server's response. This process continues until the local clock and the time server clock remain consistent.
In step S103, for the server, time consuming of each stage of the whole process from the start to the end of the service is recorded through the zip, and the information of the zip original record is generalized and adjusted in combination with the actual service requirement of the airport. Fields such as the track, sr (background acceptance request time), ss (background transmission response time) and the like of the zipkin record are adapted to be in the form of time-consuming records such as cacheDurationTotal, sqlDurationTotal of each process span which is easier to understand, so that more readable log information is provided. Meanwhile, in order to ensure the running efficiency of the system, the log is recorded by the slow request through technical configuration, the log record is not made by the normal request, and the more accurate automatic filtering is performed on the basis of ensuring the stability of the original running efficiency of the system, so that the request which is more likely to need attention is filtered out.
The airport actual business requirements include: recording the time consumption of different service operations by using the zipkin to exceed a certain time at the interface;
for example, a manual refresh operation requires a relatively low operation time, which should be recorded over 1 s; other operations, such as automatically processing external messages, etc., are not perceived by the user and therefore are re-recorded for more than 3 seconds.
It can be appreciated that, for the server, the zip has a plug-in for the dubbo, the zip processing Bean may be registered before the dubbo interceptor, and when the dubbo intercepts the request, the zip processing Bean may be entered first. Fields such as a track, sr (background receiving request time), ss (background sending response time) and the like of the zipkin record are adapted to be more easily understood in the form of time-consuming records such as cacheDurationTotal, sqlDurationTotal of each process span, and more easily readable log information is provided;
where the span of Zipkin records the span start time and end time of each plug-in, and does not identify what the plug-in is, e.g., the span time spent cached, renamed cacheDuration.
In step S104, for the request end, on the basis of completing the whole process record of completing the request end call based on the zip, the content of the request header of the dubbo protocol of the server end is enriched, the request link information of the server end is increased, the server end gathers log information of the server end, and the complete record request whole process link of the request end is finally realized by using traceid and the log summary of the request end itself.
Wherein, the dubbo framework used by airport production operation, the transmission protocol is a dubbo protocol, and the dubbo protocol can add some additional information in the protocol header. The link information of the back end in the application is added in the header of the dubbo protocol in the form of key value pairs and is transmitted to the front end.
It will be appreciated that in the Dubbo invoked protocol header, process information may be stored. The service side span interceptor is used for integrating the service side information and then putting the service side information into a dubbo-called protocol header, and the request side can read the span of the service side from the service side span interceptor so as to integrate the service side information.
The dubbo protocol is the default transport protocol used by the dubbo framework, the application layer protocol, the underlying transport layer, or the use of TCP. The dubbo protocol is similar to the http protocol, except that the dubbo protocol is higher performance and less information is transmitted.
At present, the link tracking based on the log is mostly only capable of recording the link log of the rear end by the rear end, and the front end only records the independent link data of the front end, so that the detection is inconvenient. The application processes the situation, can transmit the link data of the back end to the front end, print the whole link content in the front end.
Embodiment 2 is an innovation point of the present application, which is applicable to the Dubbo architecture of the airport production operation system, and uses the airport NTP clock synchronization system to realize the validity of the link time stamp, and other link tracking systems cannot realize the validity of the time stamp. The link tracking method can collect and integrate span information, and can calculate the execution times and total time of a certain sql of the database, network time consumption and the like.
As another implementation manner of the present application, as shown in fig. 2, the log-level link tracking method based on the airport production operation system provided in the embodiment of the present application includes:
s0: when the service is started, the system can automatically acquire link tracking configuration information from the link configuration service, wherein the link tracking configuration information comprises interfaces which need to be tracked, and link recording is performed after the response time consumption exceeds a certain threshold value. The system completes specific link tracking record based on the complete configuration of the link configuration service center.
For example, the configuration information of the link tracking is managed by the zookeeper, and after the system is started, the configuration information which is maintained in advance is read by reading address information in the configuration file and connecting the address information to the zookeeper.
After the system reads the configuration file, the configuration information is analyzed according to agreed rules, wherein the configuration information comprises interfaces or methods needing to be tracked, time consumption thresholds needing to be recorded for specific interfaces or methods, and the like. When recording the link trace data, it will record according to the configuration information.
The link configuration service provides a unified configuration technology, solves the problem that parameters are possibly inconsistent due to configuration of multiple servers under a micro-service architecture, and improves the expandability and stability of link tracking.
S1: after the request end initiates the call, the local interceptor 2 automatically records the starting time of the call. The link tracking time stamp uses the accurate time of the clock synchronization system, and is combined with the clock synchronization system universal to an airport, so that the time stamp data of all the request terminals are ensured to be consistent, and the technical problem that link tracking information cannot be corresponding due to local time inconsistency under the conditions of multiple services and multiple applications is solved.
Illustratively, when a request end initiates a call, the dubbo interceptor intercepts the call request and saves the current timestamp in memory prior to the request.
All components are accessed into an airport clock synchronization system, and clock synchronization is realized through an NTP network protocol, so that the time stamp of each component in the process is consistent.
S2: after the request arrives at the server, the server interceptor 3 automatically records the start time of the server, and the timestamp keeps consistent with the timestamp of the request end by using the clock synchronization system, so that the accuracy of the timestamp is ensured.
Illustratively, all components are accessed to an airport clock synchronization system, and clock synchronization is realized through an NTP network protocol, so that the time stamp of each component in the process is consistent.
S3: and by combining with the technical structure of an airport system, the method is respectively adapted to specific technical components such as a data grid cache, an oracle database, an A-MQ message middleware and the like, realizes that the specific time stamps of calling are recorded before and after each component is called, and is stored in the tracking. The system records specific time consumption according to each component as a basic span unit, improves readability, integrates the whole process in the whole calling response process in a tree mode, and solves the technical problem that faults cannot be rapidly analyzed when a katen is applied by specific time consumption record analysis of different components.
Illustratively, the DataGrid cache, oracle database, a-MQ message middleware, etc., also access the airport clock synchronization system, thereby ensuring that the timestamps are consistent. The specific plug-in of Zipkin can intercept the call of these components, so that the current time stamp is recorded when the component call starts and ends, respectively.
The root node is the locally invoked tracingId, and the branches include components, and other components may be invoked in each component, so that the link trace data is integrated in a tree manner and printed in json format.
S4: the present technology supports link tracking when inter-calling between micro services. When a call is requested, the request end and the server end interceptor 3 can automatically record the call time stamp, and the time stamp of the clock synchronization system is used for guaranteeing the time consistency.
Illustratively, the micro-service may be built through a dubbo or other architecture, also connected to the clock synchronization system of the airport, and records the call time stamp at the beginning and end of the service call.
When the service call starts and ends, the call time stamp is recorded, the micro-service end also uses a dubbo framework, the zip is provided with a plug-in for the dubbo, the zip processing Bean can be registered before a dubbo interceptor, and when the dubbo intercepts a request, the processing Bean of the zip is entered first.
S5: based on the inter-calling of the micro-service technology, a requested party can complete link data collection and automatically generate all span information in the process based on the zipkin technology after the service calling is finished, record the span information into a log in json format, and guarantee time consistency by using the time stamp of a clock synchronization system.
Illustratively, since the micro-service is detected at the beginning and the end of the call, the time stamp is recorded at the beginning and the end of the call, so that the call information of the micro-service can be obtained. The call information of the called party is put into the dubbo call protocol header and transmitted to the calling party. The caller can acquire span information of the entire call service and record it.
It can be appreciated that, for the server, the zip has a plug-in for the dubbo, the zip processing Bean may be registered before the dubbo interceptor, and when the dubbo intercepts the request, the zip processing Bean may be entered first.
On the basis of completing the whole process record of the completion request end based on the zipkin, the content of the dubbo protocol response header of the server end is enriched, the request link information of the server end is increased, the server end gathers the log information of the server end, the traceid is used for summarizing the log of the request end, and finally the whole process link of the request end complete record request is realized.
S6: after the back-end main service call is finished, the main service collects and sorts the span information of the main service and the span information of the micro service together and records the span information and the span information into a log of the service. The span information is put into a dubbo protocol head in the form of key value pairs and is transmitted to the front end;
s7: after the service request is finished, after receiving the message containing all the link tracking data, the span processor comprehensively processes the local link data, the network link data and the server link data, uses the traceid to summarize the logs of the request terminal, and finally prints all the span information called locally into the log file at the same time. The effect that the request end completely records the request full-flow link is achieved, and a user can see all the requests in each stage in the log of the client end.
The request for the service is initiated from the front end, and finally ends after being returned to the front end. The calling information in the whole process of the request is transmitted back to the requesting party through the dubbo calling protocol header, and finally all span is summarized at the client. The client performs summarization processing according to the span information;
the span information of the back end is put into the dubbo protocol header in the form of key value pairs. After the front end is taken out, the data are processed. Illustratively, the backend will return: and after the front end receives the front end start time stamp and the rear end time stamp, calculating the time of the two paths of the front end transmitted to the rear end and the rear end transmitted back to the front end according to the front end start time stamp and the rear end time stamp.
For example, the network time consuming from the front end to the server can be calculated by calculating the front end call start time stamp and the server call start time stamp, so that the network state can be reflected.
The above embodiment shows that the present application improves the efficiency of fault diagnosis, and the original problem confirmation requires about one hour, and the present application can shorten the efficiency of fault diagnosis to about 5 minutes.
Embodiment 3 as shown in fig. 3, the log-level link tracking system based on the airport production operation system provided in the embodiment of the present application includes:
the link configuration service module 1 is configured to automatically acquire link tracking configuration information from a link configuration service when the service is started, where the link configuration service includes which interfaces need to be tracked, and recording the link after the response time exceeds a certain threshold. The system completes specific link tracking record based on the complete configuration of the link configuration service center.
And the local interceptor 2 is used for automatically recording the starting time of the call by the local interceptor 2 after the call of the request end. The link tracking time stamp is based on the standard accurate time of the clock synchronization system, and the clock synchronization system universal to an airport is combined to ensure that the time stamp data of all the request terminals are consistent;
and the server interceptor 3 is used for automatically recording the starting time of the server after the request reaches the server, and the timestamp and the requesting end keep a database timestamp used together, so that the accuracy of the timestamp is ensured.
The specific timestamp calling module 4 is used for respectively adapting specific technical components such as a DataGrid cache, an oracle database, an A-MQ message middleware and the like by combining with an airport system technical structure, realizing that the specific timestamps of calling are recorded before and after each component is called, and storing the specific timestamps in the tracking.
The inter-micro service call link tracking module 5 is used for automatically recording call time stamps by both the request end and the server end interceptor 3 when the call is requested, and the time stamps of the clock synchronization system are used for ensuring the time consistency.
The information generating module 6 is configured to, based on the interaction of the micro-service technology, complete link data collection and automatically generate all span information in the process based on the zip technology after the service call is completed, record the span information in a json format into a log, and use a database timestamp to ensure time consistency.
And the collection and arrangement module 7 is used for collecting and arranging the span information of the whole calling service and the span information of the requested service together after the service call is finished by the requester based on the mutual call of the micro service technology, and recording the span information and the span information into the service log.
And the request full-flow link effect display module 8 is used for comprehensively processing the local link data, the network link data and the server link data after the span processor receives the message containing all the link tracking data after the service request is finished, summarizing the logs of the request terminal by using the traceid and the request terminal, and finally printing all the span information called locally into the log file at the same time. The effect that the request end completely records the request full-flow link is achieved, and a user can see all the requests in each stage in the log of the client end.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
The content of the information interaction and the execution process between the devices/units and the like is based on the same conception as the method embodiment of the present application, and specific functions and technical effects brought by the content can be referred to in the method embodiment section, and will not be described herein.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. For specific working processes of the units and modules in the system, reference may be made to corresponding processes in the foregoing method embodiments.
Based on the technical solutions described in the embodiments of the present application, the following application examples may be further proposed.
According to an embodiment of the present application, there is also provided a computer apparatus including: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, which when executed by the processor performs the steps of any of the various method embodiments described above.
Embodiments of the present application also provide a computer readable storage medium storing a computer program which, when executed by a processor, performs the steps of the respective method embodiments described above.
The embodiment of the application also provides an information data processing terminal, which is used for providing a user input interface to implement the steps in the method embodiments when being implemented on an electronic device, and the information data processing terminal is not limited to a mobile phone, a computer and a switch.
The embodiment of the application also provides a server, which is used for realizing the steps in the method embodiments when being executed on the electronic device and providing a user input interface.
Embodiments of the present application also provide a computer program product which, when run on an electronic device, causes the electronic device to perform the steps of the method embodiments described above.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing device/terminal apparatus, recording medium, computer Memory, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunications signals, and software distribution media. Such as a U-disk, removable hard disk, magnetic or optical disk, etc.
While the application has been described with respect to what is presently considered to be the most practical and preferred embodiments, it is to be understood that the application is not limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications, equivalents, and alternatives falling within the spirit and scope of the application.
Claims (8)
1. The log-level link tracking method based on the airport production operation system is characterized by being used for recording the time consumption condition of each step in service call and specifically comprising the following steps:
s101, recording the time consumption of invoking the dubbo service, the data cache, the database access and the front-end response of an airport through a zipkin technology;
s102, enabling clocks of each service in the call to be consistent by using a clock synchronization system, acquiring a time stamp of the call process, and establishing link tracking based on the time stamp;
the step of acquiring the time stamp of the calling process, and establishing link tracking based on the time stamp comprises the following steps:
the local interceptor (2) automatically records the starting time of the call, and the link tracking time stamp is based on the time of the clock synchronization system; all components are accessed into an airport clock synchronization system, and clock synchronization is realized through an NTP network protocol, so that the time stamp of span of each component in the process is consistent;
the NTP network protocol includes: firstly, a clock synchronization server acquires accurate time information through GPS signals; the other servers periodically send time requests to the clock synchronization server; the NTP network protocol estimates the time delay of network transmission by using an algorithm of time delay and offset, adjusts a local clock according to the response of a server, and continuously carries out the process until the local clock is consistent with the clock of a time server;
s103, the server records time consumption at each stage in the process from the beginning to the end of the service through the zipkin technology, and generalizes and adjusts the original recorded information of the zipkin technology in combination with the actual service requirements of the airport;
the server records time consumption of each stage in the process from the start to the end of the service through a zipkin technology, and the method comprises the following steps:
after the request reaches the server, the server interceptor (3) automatically records the starting time of the server, and the timestamp keeps consistent with the timestamp of the request end by using the timestamp of the clock synchronization system, so that the accuracy of the timestamp is ensured;
the airport actual business requirements include: recording the time consumption of different service operations by using the zipkin to exceed a certain time at the interface;
s104, recording and calling the whole process at the request end based on the zipkin technology, adding the content of a response head of the service end dubbo protocol for recording the information of the request link of the service end, collecting the information of the log of the service end by the service end, summarizing the log of the request end by using the traceid, and finally realizing the complete recording of the request whole process link of the request end.
2. The method for log-level link tracking based on airport production operating system of claim 1, wherein invoking the dubbo service of the airport via the zipkin technique in step S101 comprises:
automatically acquiring link tracking configuration information from a link configuration service, and recording a link after the interface needing to be tracked and the response time consumption exceed a certain threshold; the system completes specific link tracking record based on the complete configuration of the link configuration service center.
3. The method for tracking the log-level link based on the airport production operating system according to claim 1, wherein the server records time consumption of each stage in the process from the start to the end of the service through a zipkin technology, further comprising:
respectively adapting aiming at a data grid cache, an oracle database and an A-MQ message middleware, recording specific time stamps of calling before and after each component is called, and storing the specific time stamps in the tracking; the system records specific time consumption as a basic span unit according to each component, and integrates the whole process in the whole call response process in a tree mode.
4. A log-level link tracking method based on airport production operation system according to claim 3, characterized in that when the request is invoked, both the request end and the server end interceptor (3) automatically record the use of the invoking time stamp to ensure the time consistency by the time stamp synchronized by the clock.
5. The method for log-level link tracking based on airport production operating system according to claim 1, wherein in step S104, implementing the server-side gathering of server-side log information comprises:
after the service call is finished, the requested party completes link data collection based on the zipkin technology and automatically generates all span information in the process, records the span information in a json format into a log, and ensures time consistency by using the time stamp of the clock synchronization system.
6. The method for log-level link tracking based on airport production operating system of claim 5, wherein implementing the server to gather server log information further comprises:
after the service call is finished, the requester collects and sorts the span information of the whole call service and the span information of the service of the requester together, and records the span information and the span information into the service log.
7. The method for log-level link tracking based on airport production operating system of claim 6, wherein implementing the server to gather server log information further comprises:
after receiving the message containing all link tracking data, the span processor comprehensively processes the local link data, the network link data and the server link data, uses the track and the log summary of the request terminal, and finally prints all span information called locally into the log file at the same time to realize the complete record of the request full-flow link of the request terminal.
8. A log-level link tracking system based on an airport production operating system, characterized in that it implements the log-level link tracking method based on an airport production operating system according to any one of claims 1-7, the system comprising:
the link configuration service module (1) is used for automatically acquiring link tracking configuration information from a link configuration service when the service is started, and recording the link after the interface needing to be tracked and the response time consumption exceed a certain threshold;
the local interceptor (2) is used for automatically recording the starting time of the call after the call of the request end;
the server interceptor (3) is used for automatically recording the starting time of the server after the request reaches the server, and the timestamp is consistent with the timestamp of the clock synchronization system used by the request terminal;
the specific timestamp calling module (4) is used for respectively adapting to the DataGrid cache, the oracle database and the A-MQ message middleware by combining with the airport system technical structure, realizing the specific timestamps of calling before and after each component is called, and storing the specific timestamps in the tracking;
the link tracking module (5) is mutually called between the micro services, and when a request is called, the request end and the server end interceptor (3) can automatically record the calling time stamp to use the time stamp of the clock synchronization system to ensure the time consistency;
the information generation module (6) is used for completing link data collection and automatically generating all span information in the process based on the zipkin technology after the requested party finishes service call based on the interaction of the micro-service technology, recording the span information into a log in json format, and ensuring time consistency by using the time stamp of the clock synchronization system;
the collecting and sorting module (7) is used for collecting and sorting the span information of the whole calling service and the span information of the requested party service together after the service call is ended by the party based on the mutual call of the micro service technology, and recording the span information into the service log;
and the request full-flow link effect display module (8) is used for comprehensively processing the local link data, the network link data and the server link data after the span processor receives the message containing all the link tracking data after the service request is finished, summarizing the logs of the request terminal by using the traceid and the request terminal, and finally printing all the span information of the local call into the log file at the same time to realize the complete record of the request full-flow link of the request terminal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311048005.7A CN116775440B (en) | 2023-08-21 | 2023-08-21 | Log-level link tracking method and system based on airport production operation system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311048005.7A CN116775440B (en) | 2023-08-21 | 2023-08-21 | Log-level link tracking method and system based on airport production operation system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116775440A CN116775440A (en) | 2023-09-19 |
| CN116775440B true CN116775440B (en) | 2023-12-01 |
Family
ID=87993385
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311048005.7A Active CN116775440B (en) | 2023-08-21 | 2023-08-21 | Log-level link tracking method and system based on airport production operation system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116775440B (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107622000A (en) * | 2017-09-19 | 2018-01-23 | 北京京东尚科信息技术有限公司 | A kind of collection of application crash information and report method, device |
| CN109450633A (en) * | 2018-09-25 | 2019-03-08 | 平安科技（深圳）有限公司 | Information encrypts sending method and device, electronic equipment, storage medium |
| CN109522198A (en) * | 2018-11-26 | 2019-03-26 | 北京微播视界科技有限公司 | Processing method, device, electronic equipment and the readable storage medium storing program for executing of application program |
| CN109992465A (en) * | 2017-12-29 | 2019-07-09 | 中国电信股份有限公司 | Service tracks method, apparatus and computer readable storage medium |
| CN111078504A (en) * | 2019-12-25 | 2020-04-28 | 深圳前海环融联易信息科技服务有限公司 | Distributed call chain tracking method and device, computer equipment and storage medium |
| WO2020233015A1 (en) * | 2019-05-20 | 2020-11-26 | 平安普惠企业管理有限公司 | Link tracking method and apparatus |
| CN112612675A (en) * | 2020-12-25 | 2021-04-06 | 山东经伟晟睿数据技术有限公司 | Distributed big data log link tracking method and system under micro-service architecture |
| CN112968747A (en) * | 2021-03-04 | 2021-06-15 | 广州市百果园网络科技有限公司 | Time calibration method and device, computer equipment and storage medium |
| CN113946499A (en) * | 2021-10-22 | 2022-01-18 | 神州数码系统集成服务有限公司 | Micro-service link tracking and performance analysis method, system, equipment and application |
| WO2022027867A1 (en) * | 2020-08-04 | 2022-02-10 | 紫光云(南京)数字技术有限公司 | Mdc-based multithreaded link tracking method and apparatus, and computer-readable storage medium |
| CN114116401A (en) * | 2021-11-30 | 2022-03-01 | 平安养老保险股份有限公司 | Method, device, equipment and medium for tracking and monitoring all links of cluster message |
| CN114297028A (en) * | 2021-12-28 | 2022-04-08 | 厦门安胜网络科技有限公司 | Micro-service log link tracking method and system |
| CN115580607A (en) * | 2022-10-13 | 2023-01-06 | 国家体育总局体育彩票管理中心 | Lottery system link monitoring method, device and system |
| CN116244339A (en) * | 2023-02-23 | 2023-06-09 | 国网信息通信产业集团有限公司 | Method, device, terminal and medium for fusing operation service monitoring data of business center |
| CN116467189A (en) * | 2023-03-31 | 2023-07-21 | 青岛民航凯亚系统集成有限公司 | Method and system for interface call completion performance pressure measurement and full link data monitoring |
-
2023
- 2023-08-21 CN CN202311048005.7A patent/CN116775440B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107622000A (en) * | 2017-09-19 | 2018-01-23 | 北京京东尚科信息技术有限公司 | A kind of collection of application crash information and report method, device |
| CN109992465A (en) * | 2017-12-29 | 2019-07-09 | 中国电信股份有限公司 | Service tracks method, apparatus and computer readable storage medium |
| CN109450633A (en) * | 2018-09-25 | 2019-03-08 | 平安科技（深圳）有限公司 | Information encrypts sending method and device, electronic equipment, storage medium |
| CN109522198A (en) * | 2018-11-26 | 2019-03-26 | 北京微播视界科技有限公司 | Processing method, device, electronic equipment and the readable storage medium storing program for executing of application program |
| WO2020233015A1 (en) * | 2019-05-20 | 2020-11-26 | 平安普惠企业管理有限公司 | Link tracking method and apparatus |
| CN111078504A (en) * | 2019-12-25 | 2020-04-28 | 深圳前海环融联易信息科技服务有限公司 | Distributed call chain tracking method and device, computer equipment and storage medium |
| WO2022027867A1 (en) * | 2020-08-04 | 2022-02-10 | 紫光云(南京)数字技术有限公司 | Mdc-based multithreaded link tracking method and apparatus, and computer-readable storage medium |
| CN112612675A (en) * | 2020-12-25 | 2021-04-06 | 山东经伟晟睿数据技术有限公司 | Distributed big data log link tracking method and system under micro-service architecture |
| CN112968747A (en) * | 2021-03-04 | 2021-06-15 | 广州市百果园网络科技有限公司 | Time calibration method and device, computer equipment and storage medium |
| CN113946499A (en) * | 2021-10-22 | 2022-01-18 | 神州数码系统集成服务有限公司 | Micro-service link tracking and performance analysis method, system, equipment and application |
| CN114116401A (en) * | 2021-11-30 | 2022-03-01 | 平安养老保险股份有限公司 | Method, device, equipment and medium for tracking and monitoring all links of cluster message |
| CN114297028A (en) * | 2021-12-28 | 2022-04-08 | 厦门安胜网络科技有限公司 | Micro-service log link tracking method and system |
| CN115580607A (en) * | 2022-10-13 | 2023-01-06 | 国家体育总局体育彩票管理中心 | Lottery system link monitoring method, device and system |
| CN116244339A (en) * | 2023-02-23 | 2023-06-09 | 国网信息通信产业集团有限公司 | Method, device, terminal and medium for fusing operation service monitoring data of business center |
| CN116467189A (en) * | 2023-03-31 | 2023-07-21 | 青岛民航凯亚系统集成有限公司 | Method and system for interface call completion performance pressure measurement and full link data monitoring |
Non-Patent Citations (2)
| Title |
|---|
| 温小斌 ; 张达 ; 诸映晴 ; .轻量级分布式追踪系统的设计与实现.计算机时代.2020,(第09期),全文. * |
| 轻量级分布式追踪系统的设计与实现;温小斌;张达;诸映晴;;计算机时代(第09期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116775440A (en) | 2023-09-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7685270B1 (en) | Method and apparatus for measuring latency in web services | |
| US9634825B2 (en) | Apparatus and method for correcting time stamps of transactions performed by multiple servers | |
| US7203623B2 (en) | Distributed data gathering and aggregation agent | |
| US8868727B2 (en) | Methods and computer program products for storing generated network application performance data | |
| EP1386240B1 (en) | Synthetic transaction monitor | |
| CN111581054B (en) | Log embedded point service analysis alarm system and method based on ELK | |
| CN108471366A (en) | A kind of stereoscopic monitoring system of facing cloud native applications | |
| US9576010B2 (en) | Monitoring an application environment | |
| CN101997925A (en) | Server monitoring method with early warning function and system thereof | |
| US20020174421A1 (en) | Java application response time analyzer | |
| CN112307057A (en) | Data processing method and device, electronic equipment and computer storage medium | |
| CN112039701B (en) | Interface call monitoring method, device, equipment and storage medium | |
| US20200042424A1 (en) | Method, apparatus and system for processing log data | |
| CN111367760A (en) | Log collection method and device, computer equipment and storage medium | |
| CN115580607A (en) | Lottery system link monitoring method, device and system | |
| CN108549714B (en) | Data processing method and device | |
| CN116775440B (en) | Log-level link tracking method and system based on airport production operation system | |
| CN110633191B (en) | Method and system for monitoring service health of software system in real time | |
| CN107515864B (en) | Method and equipment for monitoring workflow | |
| Sah | A New Architecture for Managing Enterprise Log Data. | |
| CN114598622A (en) | Data monitoring method and device, storage medium and computer equipment | |
| CN115878721A (en) | Data synchronization method, device, terminal and computer readable storage medium | |
| CN117312104B (en) | Visual link tracking method and system based on airport production operation system | |
| CN113778777A (en) | Log playback method and device | |
| CN114928530B (en) | Link information acquisition method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |