CN116668802A - Message queue abnormity monitoring method and device, storage medium and electronic equipment - Google Patents
Message queue abnormity monitoring method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN116668802A CN116668802A CN202310632851.7A CN202310632851A CN116668802A CN 116668802 A CN116668802 A CN 116668802A CN 202310632851 A CN202310632851 A CN 202310632851A CN 116668802 A CN116668802 A CN 116668802A
- Authority
- CN
- China
- Prior art keywords
- message queue
- event
- determining
- monitored
- waiting time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000012544 monitoring process Methods 0.000 title claims abstract description 53
- 230000002159 abnormal effect Effects 0.000 claims abstract description 32
- 238000007619 statistical method Methods 0.000 claims abstract description 10
- 230000006870 function Effects 0.000 claims description 36
- 238000012545 processing Methods 0.000 claims description 15
- 238000004458 analytical method Methods 0.000 claims description 14
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000012806 monitoring device Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 230000005856 abnormality Effects 0.000 abstract description 15
- 230000000694 effects Effects 0.000 abstract description 6
- 230000000903 blocking effect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000012886 linear function Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005315 distribution function Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012067 mathematical method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
- H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/142—Network analysis or design using statistical or mathematical methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
- H04L43/045—Processing captured monitoring data, e.g. for logfile generation for graphical visualisation of monitoring data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Analysis (AREA)
- Algebra (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Pure & Applied Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Debugging And Monitoring (AREA)
- Computer And Data Communications (AREA)
Abstract
The disclosure provides a message queue abnormality monitoring method, a device, a storage medium and electronic equipment, and relates to the technical field of computers. The message queue abnormality monitoring method comprises the following steps: determining waiting time consumption of each event unit in a message queue to be monitored in the message queue to be monitored; carrying out statistical analysis on each waiting time to obtain a probability distribution result; determining a fitting distribution trend function of the probability distribution result; and determining the event units with the quantity difference value larger than a preset threshold value from the fitting distribution trend function in the probability distribution result as abnormal stack events. The message queue abnormity monitoring method solves the technical problem that abnormal events causing user operation blocking in the current live video APP cannot be determined in the prior art, and achieves the technical effect of accurately determining the abnormal events.
Description
Technical Field
The disclosure relates to the field of computer technology, and in particular, to a method and a device for monitoring abnormality of a message queue, a computer readable storage medium and electronic equipment.
Background
The android system is a processing system which is commonly used at present, a user often installs Application programs such as a video live broadcast APP (Application program) when using the android system, but the model and configuration of the android device used by the user are different, so that in order to improve the user experience of the video live broadcast APP, the user is generally monitored online, and the video live broadcast APP is optimized in a targeted manner based on online monitoring information.
The traditional online monitoring method mainly monitors ANR (Application Not Response, which refers to a dialog box displayed to a user by a system when an application program in an android system is not sensitive enough in response) data and Crash data of online users. The ANR problem is that the application does not respond within a period of time, resulting in a reduction of user experience, but the ANR time is not reached for part of the execution time, but the user operation experience is indeed affected by the delay time, and the industry currently lacks an effective analysis method for the problem.
Therefore, an abnormal event causing user operation jamming in the video live APP cannot be determined at present.
Disclosure of Invention
The disclosure provides a message queue anomaly monitoring method, a device, a computer readable storage medium and electronic equipment, so as to improve the technical effect of determining the data delay efficiency and effect in video live APP.
In a first aspect, an embodiment of the present disclosure provides a method for monitoring an abnormality of a message queue, including:
determining waiting time consumption of each event unit in the message queue to be monitored;
carrying out statistical analysis on each waiting time to obtain a probability distribution result; the probability distribution result at least comprises the event quantity of each waiting time-consuming corresponding event unit; determining a fitting distribution trend function of the probability distribution result;
and determining event units with the number difference value between the number of the events in the probability distribution result and the number of the events in the fitting distribution trend function being larger than a preset threshold value as abnormal stack events.
In a second aspect, an embodiment of the present disclosure provides a message queue anomaly monitoring device, including:
the first determining module is used for determining waiting time consumption of each event unit in the message queue to be monitored;
the analysis module is used for carrying out statistical analysis on each waiting time to obtain a probability distribution result; the probability distribution result at least comprises the event quantity of each waiting time-consuming corresponding event unit;
the second determining module is used for determining a fitting distribution trend function of the probability distribution result;
and the third determining module is used for determining event units with the number difference value between the number of the events in the probability distribution result and the number of the events in the fitting distribution trend function being larger than a preset threshold value as abnormal stack events.
In a third aspect, one embodiment of the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method as above.
In a fourth aspect, one embodiment of the present disclosure provides an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method as above via execution of the executable instructions.
The technical scheme of the present disclosure has the following beneficial effects:
according to the message queue anomaly monitoring method, the waiting time of each event unit in the message queue to be monitored is firstly determined, then the waiting time is statistically analyzed to obtain a probability distribution result, event units with the quantity difference value between the probability distribution result and the fitting distribution trend function being larger than the preset threshold value are determined to be the anomaly stack events, the anomaly stack events can be accurately determined through the quantity difference value between the waiting time of each event unit and the integral fitting distribution trend function, namely the reasons of data delay in the live video APP can be rapidly and accurately determined through the anomaly stack event analysis, the technical problem that the anomaly events causing user operation clamping in the live video APP cannot be determined at present is solved, the anomaly stack events can be accurately and efficiently determined, and the technical effect of improving the use smoothness of the live video APP is further achieved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely some embodiments of the present disclosure and that other drawings may be derived from these drawings without undue effort.
FIG. 1 illustrates a flow chart of a message queue anomaly monitoring method in the present exemplary embodiment;
fig. 2 shows a diagram of waiting time consumption of CC live broadcast in different models of mobile phones in the present exemplary embodiment;
FIG. 3 illustrates a flow chart of a message queue anomaly monitoring method in the present exemplary embodiment;
FIG. 4 is a flow chart illustrating a method of message queue anomaly monitoring in the present exemplary embodiment;
fig. 5 shows a distribution curve and a fitting distribution trend curve of waiting time consumption of CC live broadcast in different models of mobile phones in the present exemplary embodiment;
FIG. 6 is a graphical user interface diagram illustrating a method for monitoring anomalies in a message queue in accordance with one exemplary embodiment;
fig. 7 is a schematic diagram showing a message queue abnormality monitoring apparatus according to the present exemplary embodiment;
fig. 8 shows a schematic structural diagram of an electronic device in the present exemplary embodiment.
Detailed Description
Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present disclosure. However, those skilled in the art will recognize that the aspects of the present disclosure may be practiced with one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.
The flow diagrams depicted in the figures are exemplary only and not necessarily all steps are included. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
In the related art, the android system is a processing system which is relatively commonly used at present, and a user often installs an Application program such as a video live broadcast APP (Application, third party Application program) when using the android system, but the model and configuration of the android device used by the user are different, so as to improve the user experience of the video live broadcast APP, generally perform online monitoring on the user, and perform targeted optimization on the video live broadcast APP based on online monitoring information.
The traditional online monitoring method mainly monitors ANR (Application Not Response, which refers to a dialog box displayed to a user by a system when an application program in an android system is not sensitive enough in response) data and Crash data of online users. The ANR problem is that the application does not respond within a period of time, resulting in a reduction of user experience, but the ANR time is not reached for part of the execution time, but the user operation experience is indeed affected by the delay time, and the industry currently lacks an effective analysis method for the problem. Therefore, an abnormal event causing user operation jamming in the video live APP cannot be determined at present.
In view of the above problems, the embodiment of the disclosure provides a message queue anomaly monitoring method, which can accurately locate an anomaly stack event in a message queue, can perform more accurate analysis on data delay in video live broadcast APP based on the anomaly stack event, and provides better video live broadcast APP use experience for users. The following briefly describes an application environment of a message queue anomaly monitoring method provided by an embodiment of the present disclosure:
the message queue abnormality monitoring method provided by the embodiment of the present disclosure is used for updating a message processing mechanism such as a terminal device UI (User Interface Design, interface design) based on a message queue, where the terminal device may be a computer, a server, a notebook computer, or any wearable device, and is not exhaustive herein. The live video broadcast can be operated at the terminal equipment, and can also be operated at other electronic equipment outside the terminal equipment, the other electronic equipment is in communication connection with the terminal equipment, and the data in the live video broadcast APP operation process can be uploaded to the terminal equipment in real time.
Taking the terminal equipment as an execution main body, taking the current widely used CC live broadcast APP as an example, and taking the message queue abnormality monitoring method applied to the terminal equipment for monitoring abnormal stack events in the live broadcast process as an example for illustration. Referring to fig. 1, the message queue anomaly monitoring method provided in the embodiment of the present disclosure includes the following steps 101 to 104:
step 101, determining waiting time consumption of each event unit in the message queue to be monitored.
The message queue to be monitored refers to a message queue formed by data generated in the running process of the video live broadcast APP, and the message queue includes a plurality of event units arranged according to uploading time, wherein one event unit may be a set field, or may be a method unit (a basic unit of program code) set by a developer and capable of executing an independent function, which is not particularly limited herein, and may be arbitrarily set according to practical situations. The waiting time consumption is the time consumed between the event unit and the processing from uploading to the message queue to be monitored, and can be determined according to the data consumption node or the length updating node of the message queue to be monitored, or can be determined according to the unit identification of the event unit, and the purpose of determining the waiting time consumption only needs to be achieved.
And 102, carrying out statistical analysis on each waiting time to obtain a probability distribution result.
The probability distribution result at least comprises the event number of each waiting time consumption corresponding to the event unit. It should be explained that the number of events may be a specific number of events, or may be processed by other mathematical methods, for example, taking the logarithm or not taking the logarithm, and only the number of events corresponding to each waiting time need to be represented. Taking the CC live APP as an example, the waiting time of each event unit in the CC live broadcast accords with the power rate distribution, and of course, other manufacturer APPs may exist in different android APPs, and at the software level in each manufacturer APP, the waiting time caused by some special time-consuming event units has a certain deviation from other reference time, that is, the probability distribution result is abnormal. The probability distribution result may be in the form of a numerical value, a table, a graph, or the like, and is not limited in any way.
And step 103, determining a fitting distribution trend function of the probability distribution result.
The fitted distribution trend function is used for representing the time-consuming distribution trend or distribution rule of each event unit, and the representation form of the fitted distribution trend function can be any form such as a mathematical expression or a chart, and the like, and is not limited in any way.
And 104, determining event units with the number difference value between the number of the events in the probability distribution result and the number of the events in the fitting distribution trend function being larger than a preset threshold value as abnormal stack events.
Taking the fitting distribution trend function as a power function, taking the logarithm of the waiting time consumption and the number of event units, taking the example that the waiting time consumption and the event units are in a double-logarithm coordinate system, distributing the waiting time consumption values around the linear function in the probability distribution result, namely, the difference between the ordinate of the two functions, namely, the difference between the number logarithm of the event is the difference of the number, which is larger than a preset threshold, namely, the whole trend that the waiting time consumption does not accord with the probability distribution result, determining the event units corresponding to the waiting time consumption as abnormal stack events, and determining the event units, which are smaller than or equal to the preset threshold in number of the event units corresponding to the fitting distribution trend function, as normal stack events.
According to the message queue anomaly monitoring method provided by the embodiment of the disclosure, the waiting time of each event unit in the message queue to be monitored is firstly determined, then statistical analysis is carried out on each waiting time to obtain a probability distribution result, event units with the number difference value of the event number in the probability distribution result and the fitting distribution trend function being larger than the preset threshold value are determined to be abnormal stack events, the abnormal stack events can be accurately determined through the number difference value between the waiting time of each event unit and the integral fitting distribution trend function, namely, the reason of data delay in the video live APP can be rapidly and accurately determined through the analysis of the abnormal stack events, so that the technical problem that the abnormal events causing the user operation to be stuck in the video live APP cannot be determined is solved, the abnormal stack events can be accurately and efficiently determined, and the technical effect of improving the use smoothness of the video live APP is further achieved.
In an alternative embodiment of the present disclosure, the waiting time and the number of events may be respectively logarithmic, a double-logarithmic coordinate system is adopted, a distribution trend function is fitted in the double-logarithmic coordinate system to be a straight line, the intuitiveness of determining the abnormal stack event is better, no staff is required to have abundant professional basis or mathematical statistics basis, and the application is wider.
In an alternative embodiment of the present disclosure, the video live APP takes the CC live APP as an example, and provides a fitting distribution trend function determining manner in the CC live APP:
firstly, setting the degree of complexity of executing an event unit as x, wherein the probability of executing the event unit in unit time is as follows:
in the formula (1), L represents the length of a message queue to be monitored, gamma is an adjustable parameter, and i represents an ith event unit in the message queue to be monitored;
when γ=0, the probability that the event unit is executed in a unit timeThat is to say the probability of execution at this time is equal to the random selection.
When γ= infinity, the probability of the event unit being executed in unit time, n (x) =1, indicates that the current event unit must be executed;
when 0 < gamma < +++, if the event cell task is smaller (i.e. x is larger), the probability of being performed over a unit time is greater, this is consistent with the actual situation, so it can be considered that gamma represents the congestion degree of the message queue to be monitored, and the larger gamma is, the smoother the video live APP is.
The probability that an event element of complexity x is executed after waiting for time t is:
f(x,t)＝[1-Π(x)] t-1 Π(x)(2)
the average waiting time tau of the event unit x in the message queue to be monitored is:
will be equal toSubstituting into equation (3) yields an approximate expression of average latency:
assuming that the degree of heaviness x of the enqueuing messages of the message queue to be monitored obeys a certain random distribution ρ (x), there is ρ (x) dx=p (τ) dτ, the probability distribution of the waiting time can be calculated:
using the obtained approximation relationshipSubstitution of x in equation (4) yields:
the final latency probability distribution is thus obtained as:
wherein ρ (τ) -1/γ ) This part can be considered to be approximately constant (where only a sufficiently large τ is considered, i.e., τ > 1000 ms), where the probability distribution can be seen to be a power law distribution, i.e.,
if lnP (τ) is taken as the variable y and lnτ is taken as the variable x, this is a straight line, which is equivalent to lnP (τ) and in a double logarithmic coordinate systemlnτ is a linear function with a slope of. It can even be said that under the message queue based handset mechanism, the latency distribution of all event elements is a power law distribution. The part that deviates from the power law distribution is the outlier, i.e. belonging to an outlier stack event, and the power law distribution form is controlled by only one parameter τ.
Furthermore, it should be further explained that if the message queue is regarded as a queuing system, the mobile phone program is in a critical state that the enqueuing speed of a message (i.e. event unit) is equal to the dequeuing speed, because the mobile phone screen is refreshed at intervals to draw the page and perform various operations of the user. As long as the user is not stuck, there is always a message in the message queue in a queued state. When the enqueue speed is less than the dequeue speed, then the messages in the message queue are always consumed and the waiting time consumption profile P (τ) of the task is exponentially decreased.
Referring to fig. 2, fig. 2 is a diagram showing a waiting time-consuming distribution of CC live broadcast in different models of mobile phones, and as can be seen from fig. 2, the presence of a power law distribution in an external network user is illustrated as long as the data presents a straight line in the log-log graph. In addition, there are currently some congestion situations in the external network subscribers during individual time periods. For example, the part of the mobile phone of model 1 waits for a bulge which is obviously deviated from a theoretical straight line, and the data of other time periods generally conform to the straight line except for individual abnormal points.
It should be noted that the abnormal data is raised upward from the straight line and not downward, because a task is blocked in the queue and cannot be executed in advance by the queue, so when analyzing the data points of the deviation distribution, only a stack with high frequency in the problem interval needs to be found. In addition, the existence of a certain jitter at the end of the straight line is a normal phenomenon because the number of reports is insufficient with the increase of the waiting time τ. The number of tail reports as in FIG. 2 is only 10 2 To 10 3 Magnitude. In addition, the points of significant deviation are all potential abnormal stacksStack distribution.
In an alternative embodiment of the present disclosure, referring to fig. 3, before determining that each event unit in the message queue to be monitored is waiting for time in the message queue to be monitored in step 101, the message queue anomaly monitoring method further includes the following steps 301-303:
step 301, source data in an object to be monitored is acquired.
The object to be monitored refers to a video live broadcast APP to be monitored, such as a CC live broadcast program and the like; correspondingly, the source data refers to data reported by the CC live program in real time or data reported by the katon stack, for example, including transmission rate, smoothness, network speed and the like of the data stream, which are not exhaustive herein.
And 302, performing stack conversion on the source data according to a pre-configured data type model to obtain event units of different types.
Wherein each event unit contains stack data of the same category. The data class model is used for stack translation of source data. For example, a stack field in source data reported to kibana (an open source analysis and visualization platform) in CC live broadcast is a mixed stack name, which is used in computer science to define a linear table that performs an insert or delete operation only at the end of the table, and it is difficult to trace back the katon stack without translation.
The categories may be divided according to the type, prefix, time interval, etc. of the data, and the embodiments of the present disclosure are not particularly limited, and may be specifically configured according to actual situations, and only need to achieve the purpose of classification.
And 303, adding each event unit to a pre-configured message queue to be monitored.
The message queue to be monitored can be one path, and event units of different categories are sequentially added into the message queue to be monitored according to the sequence of stack conversion; the message queue to be monitored can also be multiple paths, and one type of event unit is one path, so that the message queue to be monitored is convenient to store and call.
According to the message queue abnormality monitoring method, the obtained source data in the object to be monitored are subjected to stack conversion according to the pre-configured data type model, so that each event unit is added into the pre-configured message queue to be monitored after each event unit of different types is obtained, the event units in the message queue to be monitored at the follow-up degree are convenient to read and process, the efficiency is higher, and the efficiency of monitoring the message queue abnormality in the embodiment of the disclosure can be improved.
In an alternative embodiment of the present disclosure, the preconfigured data class model is a class model constructed based on data prefix characters.
The preconfigured data class model is a class model constructed based on data prefix characters, that is, the stack call chains are grouped and aggregated by using a prefix matching method in the embodiment of the disclosure, so that grouping is convenient, stacks with similar call chains can be classified into the same group, the data readability after processing is greatly improved, a developer can rapidly position the call chains of the stacks, and the efficiency of monitoring message queue abnormality in the embodiment of the disclosure is further improved.
In an optional embodiment of the disclosure, before adding each event unit to the pre-configured message queue to be monitored in step 303, the message queue anomaly monitoring method further includes the following steps:
adding unit identification to each event unit in the message queue to be monitored.
For example, taking the CC live video APP as an example, matrix (a collection of development tools when application software is built by a SDK (Software Development Kit, a software development kit, generally a specific software package, a software framework, a hardware platform, an operating system, etc.) for katon monitoring, which is a vacation open source), the CC live program An Zhuoduan currently introduces slow method monitoring (evol method) in Matrix, for recording the time consumed for message execution in a message queue, is implemented based on ASM (a key in c++, for embedding assembly language in c++ source code) instrumentation, the principle of which is to scan and instrumentation all class files with ASM tools, i.e., the added unit identifier in the disclosed embodiment, through a task transformclasswiscathtdutsk during proxy compilation, taking a global class file as input. For example, an appmethodboat.i method is inserted at the beginning of each event element, and an appmethodboat.o method is inserted as element identification at the end of the event element.
Correspondingly, the step 101 of determining waiting time consumption of each event unit in the message queue to be monitored includes the following steps:
and determining waiting time consumption of each event unit in the message queue to be monitored based on the unit identification.
Taking the event identification in the above steps as an example, the time stamps of the AppMethodBast.i method and the AppMethodBast.o method are recorded, so that the execution time of the event unit can be known, and the bottom layer implementation of Matrix is also to register a callback for starting the execution of Message and a callback for ending the execution by monitoring the Looper of the Handler, so that the recorded method time is the main thread time, namely the waiting time of the event unit.
Of course, in an alternative embodiment, the time may be recorded by setting a waiting time consumption threshold, for example, only recording event units that wait for more than 1000 milliseconds, so as to reduce the sample size of subsequent analysis, thereby improving the efficiency of monitoring the message queue for anomalies.
In an alternative embodiment of the present disclosure, the step 102 performs a statistical analysis on each waiting time to obtain a probability distribution result, and includes the following steps:
constructing a time-consuming distribution curve of each waiting time; the time consumption distribution curve at least comprises the number of events waiting for the time consumption corresponding to the event unit.
Correspondingly, referring to fig. 4, in the step 104, determining an event unit with a number difference between the number of events in the probability distribution result and the number of events in the fitting distribution trend function being greater than a preset threshold as an abnormal stack event includes the following steps 401-402:
step 401, determining a fitting trend line of a time-consuming distribution curve;
step 402, determining event units in the time-consuming distribution curve, the number difference between the event units and the fitted trend line of which is greater than a preset threshold value, as abnormal stack events.
For example, referring to fig. 5, the waiting time of an event unit in an exemplary live video APP is shown in fig. 5, the abscissa is the waiting time log τ, log is shown in ms, for example, the waiting time of an event unit is 1000ms, and is shown in ln (1000) on the abscissa, that is, a position around 6.9, and the ordinate is the occurrence number (waiting time reports logN (τ)) of the event unit, which is represented by log of occurrence number, for example, the event unit has occurred 1000 times in 1000ms, and is shown in fig. 5 as (6.9) coordinates, where fig. 5 includes a time consumption distribution curve Sample Point and a Fitting trend line Fitting line. Under normal conditions, the slope of the time-consuming distribution curve is basically the same as the slope of the fitted trend line, that is, the slope difference of the time-consuming distribution curve and the fitted trend line is in the range of a preset threshold value, once the slope of the time-consuming distribution curve changes, the vertical coordinate difference value of the two curves can be increased, namely, an event unit with the quantity difference value larger than the preset threshold value is determined as an abnormal stack event. For example, two distinct bumps appear in fig. 5, with log of waiting time being around 8 and 8.7, respectively, corresponding to log of occurrence of 3 and 5. The former causes a user-loaded stuck while the latter causes a portion of ANR that bulges meaning that the X5 kernel event unit occurs more frequently than normal in this time interval.
According to the method and the device for monitoring the message queue abnormality, the fitting trend line of the time-consuming distribution curve is firstly determined, then the event units, in the time-consuming distribution curve, with the number difference value between the fitting trend line and the time-consuming distribution curve being larger than the preset threshold value are determined to be abnormal stack events, the abnormal stack events are determined through the number difference value between the fitting trend line and the time-consuming distribution curve, time-consuming waiting analysis is not needed for each event unit, computing resources are greatly saved, and therefore the efficiency of monitoring the message queue abnormality is improved.
In an alternative embodiment of the present disclosure, the step 102 performs a statistical analysis on each waiting time to obtain a probability distribution result, and includes the following steps:
the probability of each event unit being executed within a unit time is determined.
I.e. the waiting time of an event unit is characterized by the probability that the event unit is executed per unit time. The probability of being executed may be determined using the above formula (1), and will not be described in detail herein.
Correspondingly, step 103, determining a fitting distribution trend function of the probability distribution result, including the following steps:
a fitted distribution trend function of the probability of being performed is determined.
Substituting the obtained executed probability into the power law distribution function in the formulas (2) - (8) so as to obtain the fitting distribution trend function aiming at the executed probability.
The embodiment of the disclosure firstly determines the executed probability of each event unit in unit time, and then determines the fitting distribution trend function of the executed probability, so that the abnormal stack event is determined through the power law distribution of the fitting distribution trend function.
In an optional embodiment of the disclosure, the method for monitoring abnormality of a message queue further includes:
and displaying the monitoring content of each event unit in the message queue to be monitored on the graphical user interface.
For example, referring to FIG. 6, a web front-end page is implemented by Vue (an animation and rendering tool) separate from the data processing logic of the back-end. The graphical user interface implemented through the front end is used to expose the background persisted data. Wherein, the monitoring content at least comprises: at least one of the data content and the data processing process is not exhaustive herein.
In an alternative embodiment of the present disclosure, the interface of the back end may be requested through a custom query screening condition, and after the back end returns the result packet, the data is presented on the graphical user interface of the front end. The page contains filtering conditions such as intranet and extranet, time and version number, and the number of the leftmost columns in fig. 6 is the number of packets under the condition of changing query, and the second column of stack columns is the packet name under the prefix matching call chain, with "-" as the interval of the stack. The lower red is a current event unit captured by matrix, the scene is an activity system component where the current program is located, and the right handler, state, remarks and actions (bridge between user request and service logic), each Action serving as a service agent of a client, are used for assisting the developer in recording. Of course, the form and content illustrated in fig. 6 and the embodiments of the present disclosure are merely examples, and are not intended to limit the form and content of the graphical user interface display.
To implement the message queue anomaly monitoring method described above, in one embodiment of the present disclosure, a message queue anomaly monitoring device 700 is provided. Fig. 7 shows a schematic architecture diagram of a message queue anomaly monitoring device 700, comprising: a first determination module 710, an analysis module 720, a second determination module 730, and a third determination module 740, wherein:
the first determining module 710 is configured to determine waiting time consumption of each event unit in the message queue to be monitored;
the analysis module 720 is configured to perform statistical analysis on each waiting time to obtain a probability distribution result; the probability distribution result at least comprises the event quantity of each waiting time-consuming corresponding event unit;
the second determining module 730 is configured to determine a fitting distribution trend function of the probability distribution result;
the third determining module 740 is configured to determine, as an abnormal stack event, an event unit in which a difference between the number of events in the probability distribution result and the number of events in the fitting distribution trend function is greater than a preset threshold.
In an optional embodiment of the disclosure, the first module is further configured to obtain source data in an object to be monitored; performing stack conversion on source data according to a pre-configured data type model to obtain event units of different types; wherein each event unit contains stack data of the same category; and adding each event unit into a pre-configured message queue to be monitored.
In an alternative embodiment of the present disclosure, the preconfigured data class model is a class model built based on data prefix characters.
In an optional embodiment of the disclosure, the first module is further configured to add a unit identifier to each event unit in the message queue to be monitored; and determining waiting time consumption of each event unit in the message queue to be monitored based on the unit identification.
In an alternative embodiment of the present disclosure, the analysis module 720 is specifically configured to construct each waiting time-consuming profile; the time consumption distribution curve at least comprises the number of the events waiting for the time consumption corresponding to the event units; determining a fitted trend line of the time-consuming distribution curve; and determining event units with the number difference value between the time-consuming distribution curve and the fitted trend line being larger than a preset threshold value as abnormal stack events.
In an alternative embodiment of the present disclosure, the analysis module 720 is specifically configured to determine the probability of each event unit being executed in a unit time; a fitted distribution trend function of the probability of being performed is determined.
In an alternative embodiment of the present disclosure, the third determining module 740 is further configured to display, on the graphical user interface, the monitored content of each event unit in the message queue to be monitored; wherein, the monitoring content at least comprises: at least one of data content and data processing procedure.
Exemplary embodiments of the present disclosure also provide a computer readable storage medium, which may be implemented in the form of a program product comprising program code for causing an electronic device to carry out the steps according to the various exemplary embodiments of the disclosure as described in the above section of the "exemplary method" when the program product is run on the electronic device. In one embodiment, the program product may be implemented as a portable compact disc read only memory (CD-ROM) and includes program code and may be run on an electronic device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider). In the embodiments of the present disclosure, any of the steps in the above message queue abnormality monitoring method may be implemented when the program code stored in the computer-readable storage medium is executed.
Referring to fig. 8, the exemplary embodiment of the present disclosure further provides an electronic device 800, which may be a background server of the information platform. The electronic device 800 is described below with reference to fig. 8. It should be understood that the electronic device 800 shown in fig. 8 is merely an example and should not be construed to limit the functionality and scope of use of embodiments of the present disclosure in any way.
As shown in fig. 8, the electronic device 800 is embodied in the form of a general purpose computing device. Components of electronic device 800 may include, but are not limited to: at least one processing unit 810, at least one memory unit 820, a bus 830 that connects the different system components, including the memory unit 820 and the processing unit 810.
Wherein the storage unit stores program code that is executable by the processing unit 810 such that the processing unit 810 performs steps according to various exemplary embodiments of the present invention described in the above section of the "exemplary method" of the present specification. For example, the processing unit 810 may perform the method steps shown in fig. 1, etc.
Storage 820 may include volatile storage such as Random Access Memory (RAM) 821 and/or cache memory 822, and may further include read-only memory (ROM) 823.
The storage unit 820 may also include a program/utility 824 having a set (at least one) of program modules 825, such program modules 825 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 830 may include a data bus, an address bus, and a control bus.
The electronic device 800 may also communicate with one or more external devices 2000 (e.g., keyboard, pointing device, bluetooth device, etc.) via an input/output (I/O) interface 840. Electronic device 800 may also communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet through network adapter 850. As shown, network adapter 850 communicates with other modules of electronic device 800 via bus 830. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 800, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
In the embodiment of the disclosure, any step in the message queue abnormality monitoring method may be implemented when the program code stored in the electronic device is executed.
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit in accordance with exemplary embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
Those skilled in the art will appreciate that the various aspects of the present disclosure may be implemented as a system, method, or program product. Accordingly, various aspects of the disclosure may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A method for monitoring anomalies in a message queue, comprising:
determining waiting time consumption of each event unit in a message queue to be monitored in the message queue to be monitored;
carrying out statistical analysis on each waiting time to obtain a probability distribution result; wherein the probability distribution result at least comprises event numbers corresponding to the event units in each waiting time;
determining a fitting distribution trend function of the probability distribution result;
and determining the event units with the number difference value larger than a preset threshold value between the number difference value of the event units in the probability distribution result and the number of the events in the fitting distribution trend function as abnormal stack events.
2. The message queue exception monitoring method of claim 1, wherein prior to said determining that each event unit in a message queue to be monitored is waiting for time in the message queue to be monitored, the method further comprises:
acquiring source data in an object to be monitored;
performing stack conversion on the source data according to a pre-configured data type model to obtain event units of different types; wherein each event unit contains stack data of the same category;
and adding each event unit to the pre-configured message queue to be monitored.
3. The message queue exception monitoring method of claim 2, wherein the preconfigured data class model is a class model constructed based on data prefix characters.
4. The message queue anomaly monitoring method of claim 2, wherein prior to said adding each of the event units to the pre-configured message queue to be monitored, the method further comprises:
adding a unit identifier to each event unit in the message queue to be monitored;
correspondingly, the determining the waiting time consumption of each event unit in the message queue to be monitored includes:
and determining the waiting time consumption of each event unit in the message queue to be monitored based on the unit identification.
5. The method for monitoring anomalies in a message queue according to claim 1, wherein the statistically analyzing each of the waiting time periods to obtain a probability distribution result includes:
constructing each waiting time-consuming distribution curve; wherein the time-consuming profile comprises at least the number of events of the waiting time-consuming corresponding to the event cell;
correspondingly, the determining the event unit with the number difference value between the probability distribution result and the number of the events in the fitting distribution trend function being greater than a preset threshold value as an abnormal stack event comprises:
determining a fitted trend line of the time-consuming profile;
and determining the event units, which are larger than a preset threshold value, in the time-consuming distribution curve and have the quantity difference value with the fitted trend line as the abnormal stack events.
6. The method for monitoring anomalies in a message queue according to claim 1, wherein the statistically analyzing each of the waiting time periods to obtain a probability distribution result includes:
determining the execution probability of each event unit in unit time;
correspondingly, the determining the fitting distribution trend function of the probability distribution result includes:
the fitted distribution trend function of the executed probability is determined.
7. The message queue exception monitoring method of claim 1, further comprising:
displaying monitoring content of each event unit in the message queue to be monitored on a graphical user interface; wherein, the monitoring content at least comprises: at least one of data content and data processing procedure.
8. A message queue anomaly monitoring device, the device comprising:
the first determining module is used for determining waiting time consumption of each event unit in the message queue to be monitored;
the analysis module is used for carrying out statistical analysis on each waiting time to obtain a probability distribution result; wherein the probability distribution result at least comprises event numbers corresponding to the event units in each waiting time;
the second determining module is used for determining a fitting distribution trend function of the probability distribution result;
and a third determining module, configured to determine, as an abnormal stack event, the event unit in the probability distribution result, where the number difference between the number of events in the probability distribution result and the number of events in the fitting distribution trend function is greater than a preset threshold.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method of any of claims 1 to 7.
10. An electronic device, comprising:
a processor; and
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the method of any one of claims 1 to 7 via execution of the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310632851.7A CN116668802A (en) | 2023-05-30 | 2023-05-30 | Message queue abnormity monitoring method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310632851.7A CN116668802A (en) | 2023-05-30 | 2023-05-30 | Message queue abnormity monitoring method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116668802A true CN116668802A (en) | 2023-08-29 |
Family
ID=87727280
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310632851.7A Pending CN116668802A (en) | 2023-05-30 | 2023-05-30 | Message queue abnormity monitoring method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116668802A (en) |
-
2023
- 2023-05-30 CN CN202310632851.7A patent/CN116668802A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106844181B (en) | Method, system and mobile terminal for recording user behavior | |
| US20130346917A1 (en) | Client application analytics | |
| JP2022048088A (en) | Method and apparatus for testing code, electronic device, storage medium and computer program | |
| WO2021051589A1 (en) | Data storage method and apparatus, electronic device, and storage medium | |
| US20220138074A1 (en) | Method, electronic device and computer program product for processing data | |
| CN111177113A (en) | Data migration method and device, computer equipment and storage medium | |
| CN111026400A (en) | Method and device for analyzing service data stream | |
| US10140377B2 (en) | Data processing, data collection | |
| CN110188308B (en) | Client automatic dotting reporting method, storage medium, equipment and system | |
| CN110059064B (en) | Log file processing method and device and computer readable storage medium | |
| CN110602207A (en) | Method, device, server and storage medium for predicting push information based on off-network | |
| CN108768823B (en) | Method and equipment for processing message and displaying message | |
| CN111352644A (en) | Applet updating method, device, server and storage medium | |
| CN116668802A (en) | Message queue abnormity monitoring method and device, storage medium and electronic equipment | |
| CN111741046B (en) | Data reporting method, data acquisition method, device, equipment and medium | |
| CN111680240B (en) | Page display method and device and electronic equipment | |
| US11182042B2 (en) | Application input and feedback system | |
| CN113407102A (en) | Virtual key display method, device, system and storage medium | |
| CN111767111A (en) | Page data processing method and device, electronic equipment and storage medium | |
| CN113760317A (en) | Page display method, device, equipment and storage medium | |
| CN113778977A (en) | Data processing method and data processing device | |
| CN114064376A (en) | Page monitoring method and device, electronic equipment and medium | |
| CN110928940A (en) | Data writing method and device based on kafka cluster, electronic equipment and storage medium | |
| CN112287643A (en) | Message monitoring method, device, equipment and computer readable storage medium | |
| CN111625524B (en) | Data processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |