CN117041005A - Alarm method, alarm device, electronic equipment and storage medium - Google Patents

Alarm method, alarm device, electronic equipment and storage medium Download PDF

Info

Publication number
CN117041005A
CN117041005A CN202310946974.8A CN202310946974A CN117041005A CN 117041005 A CN117041005 A CN 117041005A CN 202310946974 A CN202310946974 A CN 202310946974A CN 117041005 A CN117041005 A CN 117041005A
Authority
CN
China
Prior art keywords
early warning
root cause
determining
time period
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310946974.8A
Other languages
Chinese (zh)
Inventor
郭维
朱丽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202310946974.8A priority Critical patent/CN117041005A/en
Publication of CN117041005A publication Critical patent/CN117041005A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • H04L41/0631Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Alarm Systems (AREA)

Abstract

The disclosure provides an alarm method, an alarm device, electronic equipment and a storage medium, relates to the technical field of artificial intelligence, and particularly relates to the field of big data processing and network security. The specific implementation scheme is as follows: determining a history early warning characteristic based on history early warning information of a first early warning for a first object in a second time period when the current early warning frequency of the first early warning for the first object in the first time period is greater than or equal to a set threshold, wherein the first time period and the second time period have the same time characteristic; determining a target early warning root cause based on the historical early warning characteristics and current early warning characteristics of the first early warning for the first object in the first time period; and determining whether to alarm for the first object based on the target early warning root cause. By adopting the technical scheme disclosed by the invention, the alarm accuracy can be improved.

Description

Alarm method, alarm device, electronic equipment and storage medium
Technical Field
The disclosure relates to the technical field of artificial intelligence, in particular to the field of big data processing and network security, and can be applied to a scene of a smart city. The disclosure relates in particular to an alarm method, an alarm device, electronic equipment and a storage medium.
Background
With the wide application of internet technology in various fields of society, the stability of online service has an increasingly important influence on business development. The online inspection can help the business to find service abnormality in time, and is an important means for guaranteeing service stability.
Currently, there are many open source software such as Prometheus, zabbix on the market, and many businesses will employ such a routing inspection scheme. However, with the complex development of services, this solution cannot meet the requirements of the services on comprehensive inspection, fine granularity and intellectualization.
Disclosure of Invention
The disclosure provides an alarm method, an alarm device, electronic equipment and a storage medium.
According to an aspect of the present disclosure, there is provided an alarm method including:
determining a history early warning characteristic based on history early warning information of a first early warning for a first object in a second time period when the current early warning frequency of the first early warning for the first object in the first time period is greater than or equal to a set threshold, wherein the first time period and the second time period have the same time characteristic;
determining a target early warning root cause based on the historical early warning characteristics and current early warning characteristics of the first early warning for the first object in the first time period; the method comprises the steps of,
And determining whether to alarm for the first object based on the target early warning root cause.
According to another aspect of the present disclosure, there is provided an alarm device including:
a history feature determining module, configured to determine a history feature based on history early warning information for performing a first early warning in a second time period for a first object, where the first time period and the second time period have the same time feature, when a current early warning number of times of performing the first early warning in the first time period for the first object is greater than or equal to a set threshold;
the target root cause determining module is used for determining a target early warning root cause based on the history early warning characteristics and the current early warning characteristics of the first early warning of the first object in the first time period; the method comprises the steps of,
and the alarm module is used for determining whether to alarm for the first object based on the target early warning root cause.
According to another aspect of the present disclosure, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform any one of the alert methods of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform any of the alert methods according to the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements any of the alarm methods according to the embodiments of the present disclosure.
According to the technology disclosed by the invention, when the current early warning frequency of the first early warning of the first object in the first time period is larger than or equal to the set threshold value, the historical early warning characteristic of the first early warning of the first object in the second time period with the same time characteristic as the first time period and the current early warning characteristic of the first early warning of the first object in the first time period are utilized, so that the target early warning root cause can be accurately obtained, and the alarm accuracy can be improved when the root cause is utilized to determine whether to alarm or not, and the alarm caused by accidental instability of the first object is reduced.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a flow chart of an alert method of an embodiment of the present disclosure;
FIG. 2 is an architecture diagram of an intelligent alert scheme of an embodiment of the present disclosure;
FIG. 3 is a flow chart of an alert method of another embodiment of the present disclosure;
FIG. 4 is a flow chart of an alarm record analysis method of another embodiment of the present disclosure;
FIG. 5 is a block diagram of an alert device according to another embodiment of the present disclosure;
FIG. 6 is a block diagram of an alert device according to another embodiment of the present disclosure;
FIG. 7 is a block diagram of an electronic device for implementing an alert method of an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. The term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, e.g., including at least one of A, B, C, may mean including any one or more elements selected from the group consisting of A, B and C. The terms "first" and "second" herein mean a plurality of similar technical terms and distinguishes them, and does not limit the meaning of the order, or only two, for example, a first feature and a second feature, which means that there are two types/classes of features, the first feature may be one or more, and the second feature may be one or more.
In addition, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order not to obscure the present disclosure.
FIG. 1 is a flow chart of an alert method of an embodiment of the present disclosure. The method can be applied to an electronic device. The electronic device is, for example, a terminal, a server, or other processing device, where the terminal may be a desktop computer, a mobile device, a PDA (Personal Digital Assistant ), a handheld device, a computing device, an in-vehicle device, a wearable device, or other User Equipment (UE). In some implementations, the electronic device may implement the alert method of embodiments of the present disclosure by way of a processor invoking computer readable instructions stored in a memory. As shown in fig. 1, the alarm method includes:
s110, determining a history early warning characteristic based on history early warning information of the first object for the first early warning in a second time period when the current early warning frequency of the first object for the first time period is greater than or equal to a set threshold, wherein the first time period and the second time period have the same time characteristic;
S120, determining a target early warning root cause based on the historical early warning characteristics and current early warning characteristics of the first object for the first early warning in a first time period; the method comprises the steps of,
s130, determining whether to alarm for the first object based on the target early warning root cause.
The above scheme may be applied to a scene of detecting or inspecting the first object, when detecting that the first object has an abnormality, performing corresponding early warning on the first object, and executing the warning method of steps S110 to S130 when the first object meets the corresponding early warning condition.
The first object may also be referred to as a detection object or a patrol object, for example. The first object may be one or more services or service indicators of an online service, such as interface availability, consistency check of interface data, validity of page elements, response time of a service, etc.
For example, the set threshold may be different for different first objects. Or, setting the set threshold based on the history early warning information of the first object.
For example, the set threshold may be different for different time periods of the same first object.
Illustratively, in the case where the current number of early warning is less than the set threshold, no warning is made.
For example, the first time period may be a period of time from a first time to a current time, wherein the first time is located before the current time.
Illustratively, the second time period is located before the first time period.
Illustratively, the first time period and the second time period have the same time characteristics, e.g., a certain time period of the day has the same time characteristics as the same time period of the previous day. For example, 7 on the same day: 00-8:00 and 7 of the previous day: 00-8:00 have the same temporal characteristics.
For example, the first warning may refer to a warning of a certain error code. For example, at 8 for the first object: 00-9: and (00) carrying out early warning of the error code 01 and triggering early warning analysis of the steps S110-S130 at the moment if the early warning times reach a set threshold value so as to determine whether to alarm for the first object.
In an exemplary embodiment, when the early warning analysis is triggered, the historical early warning information of the first object for performing the early warning of the error code in a second time period having the same time characteristic as the first time period is searched for in the historical early warning information of the control object by using the error code corresponding to the first early warning and the time characteristic of the first time period.
Illustratively, the early warning information may include an early warning object, an early warning time, an error code, an early warning number of times, an early warning root cause, and the like.
The current warning times may be the accumulated warning times in the first time period or the maximum continuous warning times in the first time period. For example, the first object is detected according to the set frequency, if the detection result does not meet the set condition, early warning is performed, and if the detection result meets the set condition, early warning is not performed. Therefore, when the detection is not performed for a certain time, the continuous early warning times can be reset to zero again.
For example, the second time period may be multiple, and the corresponding history early-warning features are respectively determined by using the history early-warning information corresponding to each second time period. And obtaining a plurality of history early warning features.
Based on the historical early warning features and the current early warning features of the first object for the first early warning in the first time period, determining the target early warning root cause may include: and determining target historical early-warning features in the plurality of historical early-warning features based on the similarity degree of each historical early-warning feature and the current early-warning feature, and accurately determining target early-warning root factors based on early-warning root factors corresponding to the target historical early-warning features.
Illustratively, determining the target early warning root cause based on the historical early warning characteristics and the current early warning characteristics for the first object to perform the first early warning within the first time period may include: and inputting the historical early warning characteristics and the current early warning characteristics into a root cause prediction model to obtain a target early warning root cause output by the root cause prediction model. The root cause prediction model is a model obtained by training based on one or more training samples, wherein the training samples comprise early warning features and early warning root causes corresponding to the early warning features.
Illustratively, determining the target early warning root cause based on the historical early warning characteristics and the current early warning characteristics for the first object to perform the first early warning within the first time period may include: and determining a first early warning root cause based on the historical early warning characteristics, determining a second early warning root cause based on the current early warning characteristics, and determining a target early warning root cause by utilizing the first early warning root cause and the second early warning root cause.
In an exemplary case, in which the target early warning root cause is a preset root cause, it is determined to alert the first object. The alarm mode may include various modes, such as notifying corresponding staff by mail, short message, telephone, etc. And under the condition that the target early warning root cause is not the preset root cause, the early warning information of the first object can be continuously analyzed. If the early warning condition is serious, an alarm can be given, and the safety of the first object is improved.
Illustratively, the preset root cause may be an instance failure or an online problem, etc. The non-preset root cause may be an unstable condition of an online occasion. If the on-line occasional unstable conditions are lighter, no alarm is given, and if the on-line occasional unstable conditions are heavier, an alarm is still needed to ensure the quality of the on-line service.
According to the technology disclosed by the invention, when the current early warning frequency of the first early warning of the first object in the first time period is larger than or equal to the set threshold value, the target early warning root cause can be accurately obtained by utilizing the historical early warning characteristic of the first object in the second time period with the same time characteristic as the first time period and the current early warning characteristic of the first early warning of the first object in the first time period. Therefore, the alarm accuracy can be improved when the root cause is used for determining whether the alarm is given, and the alarm caused by accidental instability of the first object is reduced.
In an exemplary embodiment, determining a target early warning root cause based on historical early warning features and current early warning features for a first object to perform a first early warning in a first time period includes: determining a first early warning root cause based on the historical early warning features; determining a second early warning root cause based on current early warning characteristics of the first early warning for the first object in a first time period; and determining a target early warning root cause based on the first early warning root cause and the second early warning root cause.
Illustratively, the historical early warning features are input into a root cause prediction model, and a first early warning root cause output by the root cause prediction model is obtained. Or, according to the history early warning features, searching at least one adjacent first early warning feature in the early warning feature space, and determining the first early warning root cause based on the early warning root causes corresponding to the first early warning features respectively.
For example, if there are a plurality of history prediction features, each corresponding to a second time period, the early warning root cause corresponding to each history prediction feature may be determined separately, and the early warning root cause with the highest probability may be selected from the early warning root causes as the first early warning root cause.
The current early warning feature is input into the root cause prediction model, and a second early warning root cause output by the root cause prediction model is obtained. Or, according to the current early warning characteristics, searching at least one adjacent second early warning characteristic in the early warning characteristic space, and determining a second early warning root cause based on the early warning root causes corresponding to the second early warning characteristics respectively.
Illustratively, the first early warning root cause and the second early warning root cause are taken as target early warning root causes. Or, one of the first early warning root cause and the second early warning root cause is taken as a preset root cause, and the target early warning root cause is taken as the target early warning root cause.
According to the embodiment, the first early warning root cause and the second early warning root cause are respectively determined by utilizing the history early warning characteristic and the current early warning characteristic, and the target early warning root cause is determined based on the two root causes. Therefore, missing root causes can be avoided, and as long as any one of the two root causes is a preset root cause, early warning can be carried out aiming at the target early warning root cause so as to ensure the service quality of the first object.
Under the condition that the target early warning root cause is not the preset root cause, the current early warning severity can be further analyzed to determine whether to warn the first object.
In an exemplary embodiment, determining whether to alert for the first object based on the target pre-warning root cause includes: under the condition that the target early warning root cause is not the preset root cause, determining the severity of early warning based on the target early warning root cause and the current early warning times; and under the condition that the early warning severity meets a first preset condition, warning is carried out aiming at the first object.
In practical applications, the preset root cause may be an instance failure or an online problem, etc. The non-preset root cause may be an unstable condition of an online occasion. At this time, the target early warning root cause and the current early warning times can be utilized to determine the severity of the early warning. Therefore, if the pre-warning severity is light for the on-line occasional unstable condition, no warning is performed, and if the pre-warning severity is heavy for the on-line occasional unstable condition, the warning is still required to be performed so as to ensure the quality of the on-line service provided by the first object.
The target early warning root cause may be either one of the first early warning root cause and the second early warning root cause, or both of them.
For example, the target early warning root cause and the current early warning times can be respectively scored, and the severity of the early warning can be determined according to the scores of the target early warning root cause and the current early warning times. Or, weighting and summing the scores to obtain the early warning severity. Or multiplying the scores to obtain the early warning severity.
For example, in the case that the severity of the early warning does not meet the first preset condition, no warning is performed for the first object, so that warning caused by accidental instability of the first object can be reduced.
According to the embodiment, for the non-preset root cause, the severity of the current early warning is evaluated by using the root cause and the early warning times, and if the severity is heavy, the first object still needs to be warned. Therefore, when the root cause is the on-line occasional unstable condition, if the early warning severity is lighter, no warning is performed, and the warning caused by accidental instability of the first object can be reduced. If the pre-warning severity is heavy, an alarm is still required, which can guarantee the quality of the online service provided by the first object.
In an exemplary embodiment, determining the pre-warning severity based on the target pre-warning root cause and the current number of pre-warnings comprises: determining a first numerical value based on the importance degree of the target early warning root cause; determining a second value based on the current early warning times; the pre-warning severity is determined based on a product of the first value and the second value.
For example, the importance level of each early warning root cause may be preset. The mapping relation between the importance degree and the numerical value of each early warning root cause can also be preset. For example, a first value corresponding to the importance degree of the target early warning root cause is determined from the mapping relation between the importance degree and the values of the early warning root causes. Or setting a first function, and calculating the importance degree of the target early warning root cause by using the first function to obtain a first numerical value. The first function characterizes a functional relation between the importance degree and the numerical value of the early warning root cause.
For example, the number of early warning times may be classified by rank. For example, the number of early warning times is smaller, the grade is lower, the number of early warning times is larger, and the grade is higher. The lower the level, the lower the second value, the higher the level, and the higher the second value. Or setting a second function, and calculating the early warning times by using the second function to obtain a second numerical value. The second function characterizes a functional relation between the early warning times and the numerical value.
According to the embodiment, the importance degree of the target early warning root cause and the corresponding numerical value of the current early warning times are multiplied, so that the early warning severity degree of the first early warning of the first object in the first time period can be accurately scored. Therefore, when whether the alarm is given or not is determined according to the early warning severity, the alarm accuracy is improved, and the quality of online service is further improved.
For the early warning severity, the priority of the first object can be used for modifying the early warning severity, so that the importance of the first object can be represented by the early warning severity.
In an exemplary embodiment, further comprising: determining a third value based on the priority of the first object; and correcting the early warning severity degree based on the third numerical value.
For example, the priority of each first object may be set in advance. The mapping relationship between the priority and the numerical value of each first object may also be preset. For example, from the mapping relationship between the priority of each first object and the numerical value, a third numerical value corresponding to the priority of the first object is determined. Or setting a third function, and calculating the priority of the first object by using the third function to obtain a third numerical value. Wherein the third function characterizes a functional relationship between the priority of the first object and the numerical value.
Illustratively, correcting the pre-warning severity based on the third value may include: and multiplying the third value by the value representing the early warning severity to obtain the corrected early warning severity.
According to the embodiment, the early warning severity is corrected by using the value corresponding to the priority of the first object, so that the importance of the first object can be reflected by the early warning severity, and the accuracy of the early warning severity is improved. Therefore, when whether the alarm is given or not is determined according to the early warning severity, the alarm accuracy can be further improved, and the quality of online service is further improved.
Under the condition that the target early warning root cause is not the preset root cause, besides analyzing the early warning severity, the current early warning stability can be analyzed to determine whether to warn the first object. Or, under the condition that the severity of the early warning does not meet the preset condition, analyzing the current early warning stability to determine whether to warn the first object
In an exemplary embodiment, determining whether to alert for the first object based on the target pre-warning root cause includes: under the condition that the target early warning root cause is not a preset root cause or the early warning severity degree does not meet a first preset condition, determining first early warning stability based on the current early warning times and the historical early warning times of first early warning for a first object in a third time period, wherein the third time period and the first time period have the same time characteristic; and under the condition that the first early warning stability meets the second preset condition, warning is carried out aiming at the first object.
For example, the history early-warning times may be an average value or a median of the history early-warning times of performing the first early-warning for the first object in the plurality of third time periods.
Illustratively, the first warning stability is determined using a difference between the current warning times and the historical warning times. Or determining the first early warning stability by using the ratio of the difference value between the current early warning times and the historical early warning times to the current early warning times. The first early warning stability characterizes the deviation condition of the current early warning times and the historical early warning times of the first object. If the deviation is larger, the early warning stability is poorer. If the deviation is smaller, the early warning stability is better.
For example, if the early warning stability is poor, the first object is alerted, and if the early warning stability is good, the first object is not alerted.
According to the embodiment, under the condition that the target early warning root cause is not the preset root cause, the current early warning times and the historical early warning times under the same early warning condition are further utilized to carry out comparison analysis, so that the first early warning stability is obtained, deviation between the early warning times and the historical early warning times can be evaluated, and if the deviation is too large, an alarm is given to ensure the quality of service provided by the first object.
Therefore, when the root cause is the on-line occasional unstable condition, if the first early warning stability is good, no warning is performed, and the warning caused by accidental instability of the first object can be reduced. If the first pre-warning stability is poor, an alarm is still required, which can guarantee the quality of the online service provided by the first object.
In an exemplary embodiment, the first object is an online service, and the method provided by the foregoing embodiment further includes: determining a current early warning frequency based on the current early warning times and the current service times of the online service in the first time period; determining a historical early warning frequency based on the historical early warning times and the historical service times of the online service in a third time period; determining a second early warning stability based on the current early warning frequency and the historical early warning frequency; and correcting the first early warning stability based on the second early warning stability.
In this example, the on-line service may be detected in a patrol manner, and the frequency of providing the on-line service is different due to the different first objects, and each time the service corresponds to one detection. Therefore, the early warning stability is evaluated according to the deviation between the early warning times and the historical early warning times, and the accuracy is not high. Therefore, the deviation between the early warning frequency and the historical early warning frequency is utilized to correct the early warning stability, so that the corrected early warning stability is more accurate and more reasonable.
For example, the above-described history service number may be an average value or a median of the history service number of the online service in the plurality of third time periods, or the like.
Illustratively, the ratio between the current number of early warning and the current number of service is taken as the current early warning frequency. And taking the ratio of the historical early warning times to the historical service times as the historical early warning frequency.
Illustratively, the second warning stability is determined based on a difference between the current warning frequency and the historical warning frequency. Or determining that the second early warning is stable based on the ratio of the difference between the current early warning frequency and the historical early warning frequency to the current early warning frequency.
Illustratively, an average value between the second pre-warning stability and the first pre-warning stability is utilized as the corrected first pre-warning stability. Or, using the value obtained by the weighted summation of the second early warning stability and the first early warning stability as the corrected first early warning stability.
According to the embodiment, the first early warning stability is corrected by using the second early warning stability representing the deviation between the early warning frequency and the historical early warning frequency, so that the corrected first early warning stability is more accurate and more reasonable. Therefore, when the first corrected early warning stability is used for determining whether to carry out warning, the warning accuracy is further improved.
FIG. 2 is an architecture diagram of an intelligent alert scheme according to an embodiment of the present disclosure. Fig. 3 is a flow chart of an alert method of another embodiment of the present disclosure. FIG. 4 is a flow chart of an alarm record analysis method according to another embodiment of the present disclosure.
The following will describe an online service intelligent alarm scheme based on log streams with reference to fig. 2 to fig. 4, which specifically includes the following steps:
1. and (5) business inspection.
As shown in fig. 2, each product combines with a service form to make a patrol item (i.e., a first object), and sets a detection coverage of the first object, for example, including interface http availability, interface data consistency check, extension room detection, service upstream and downstream scenes, page element validity, service response time, and the like. To reduce the impact of inspection on the on-line traffic, the detection may be performed on the first object at regular time using environmental mirroring, dynamic containers, pipelining, etc. Wherein, the dynamic container ensures the independence of each round of automatic execution, and reduces false alarms caused by online environment or tools. Meanwhile, aiming at the current early-warning scene, the method can quickly retry based on container expansion, so that waiting for a patrol period is reduced, and feedback problems are quickly found.
2. And (5) collecting logs.
As shown in fig. 2, a log receiving service is built. And after the dynamic container performs automatic inspection, sending a local report to the server. The server receives, analyzes, formats the log and the like. After the logs are collected, the log information is classified and processed according to an abnormality judgment rule to obtain abnormal information; clustering the abnormal information based on basic early warning rules, and then performing preliminary screening to obtain early warning callback information.
3. Intelligent determination
As shown in fig. 2, for the early warning callback information passing through the preliminary screening, a personalized early warning threshold is configured, and a hierarchical inspection strategy is adopted for on-line services of different levels. And the historical alarm data is combined to analyze the early warning callback information, so that the alarm caused by unstable online service is reduced, the real meaningful alarm information is provided, and the online problem is effectively recalled.
3.1 personalized threshold setting
As shown in fig. 2, the personalized alarm rule configuration is performed on the inspection item according to the stability condition and the early warning severity of the online service. For example, the cumulative number of failures and the continuous failure number of the early warning are set as thresholds for the traffic direction, the type of inspection items, the error code, the 4 dimensions of the time period, and the like. If the current early warning times do not exceed the threshold value, the warning is not performed due to unstable partial service. And if the current early warning times exceed the threshold value, carrying out the next step of judgment. For example, if the service a has an unstable return condition in the early morning for a long time in a certain direction, the configured alarm threshold is higher in the corresponding time period, and the service a has higher tolerance to online service abnormality.
3.2 early warning analysis
As shown in fig. 3, the following describes a specific process of analyzing the early warning callback information to determine whether to alarm:
3.2.1 determining whether to alarm by using root cause and early warning severity
Specifically, in a historical scene of the same error code early warning in the same time period of the first object and the current early warning scene, determining the probability of the corresponding alarm root cause, and selecting the root cause with the highest probability as the root cause corresponding to the current early warning scene. If the root cause is an instance failure or an online problem, an alert notification is generated. If the root cause is not an instance failure or an online problem, such as an unstable condition where the root cause may be an online service incident, the severity of the pre-warning is further evaluated to determine whether to alert. The calculation formula of the early warning severity is as follows:
SC=RC×FC×PC
where SC is the root importance score. The root cause severity of the early warning problem is utilized to regulate the importance of the root cause, and the more serious the problem is, the higher the importance score of the corresponding root cause is. FC is the score of the current number of early warning. The early warning times are divided according to different grades, and the score is higher as the early warning times are more. Importance is a score of the priority of the first object, the higher its score. SC is the comprehensive score obtained by the 3 dimensions of the importance degree of the comprehensive root cause, the early warning times and the first object priority, and can reflect the serious condition of the current early warning.
And 3.2.2, determining whether to alarm by utilizing the early warning stability.
Specifically, if the severity of the early warning does not meet the set condition, comparing the current early warning times of the first object with the historical early warning times in the same past time period, if the deviation of the current early warning times and the historical early warning times is too large, the first object is abnormal, and an alarm notification is generated. Because the service frequencies of the different first objects are inconsistent, comprehensive analysis can be performed from two dimensions of the early warning times and the early warning frequencies so as to determine whether to alarm. If the difference between the current situation and the historical situation is larger, the early warning stability is poorer. If the deviation between the current situation and the historical situation is not large, the better the early warning stability is. The calculation formula of the early warning stability is as follows:
the CF characterizes the current early warning times of the first object in the current first time period. ACF characterizes the average number of pre-warnings of the first subject over the same period of time as the first period of time for the first 7 days. RF represents the current pre-warning frequency of the first object during the current first time period. ARF represents the average warning frequency of the first subject over the same period of time as the first period of time for the first 7 days. TC represents an average value obtained by normalizing the early warning frequency deviation.
And 3.2.3, determining a root cause by utilizing the current early warning characteristic so as to determine whether to alarm or not.
Specifically, based on the current early warning characteristics, a corresponding root cause is determined, and if the root cause is an instance fault or an on-line problem, an alarm notification is generated. For example, the similarity between the current early warning feature and the feature in the early warning feature space is calculated, one or more features which are adjacent to the current early warning feature and similar to the current early warning feature are selected from the early warning feature space by using the similarity, and the root causes corresponding to the features are used for determining the target root cause. If the target root cause is the specified root cause, an alert notification is generated. For another example, extracting features such as service direction, sub-direction, error code, failure times, continuous failure times, time dimension and the like in the history early warning information to obtain a feature sample; and aiming at the characteristic samples, the corresponding alarming root causes are used as classification labels of the samples. And training the model by using the marked characteristic sample to obtain a root cause prediction model. And processing the current early warning characteristic by using the root cause prediction model to obtain a target root cause. If the target root cause is the specified root cause, an alert notification is generated.
4. And (5) alarming.
And 4.1, intelligent analysis and recording. When the alarm is determined to be carried out on the first object, carrying out error code extraction, root cause positioning and other records on the early warning information of the first object.
As shown in fig. 4, the primary decision is made as to which layer of the full link the alarm belongs to, by the log of the full link of the service and the error code mapping rule. And extracting a request address of the early warning scene, analyzing whether a machine room or an example convergence feature exists according to the request address, and judging a fault level to assist machine positioning. And acquiring service calling party information, bottom layer service information and time consumption of each layer of service, thereby recording the online service performance. And summarizing the analysis results and synchronizing the analysis results into the alarm notification so as to facilitate the service party to follow up the alarm problem.
And 4.2, notifying an alarm. And synchronously sending and judging the alarm notification to staff of the corresponding service party by means of mail, short messages, social tools and the like. Meanwhile, the staff can acquire detailed information of the alarm through a link in the alarm notification.
5. And (5) data analysis.
As shown in fig. 2, the data of the inspection alarm is dropped, and month analysis and aggregation are performed in multiple dimensions such as root cause, error code, alarm time, business direction of the first object, and the like. And (3) gathering log data, and displaying the information such as the pre-warning times of the inspection pre-warning and the duty ratio of the service times, the duty ratio change trend, the ring ratio graph, the error code distribution (problem clustering) and the like in real time.
The embodiment can be widely applied to the inspection construction of online service. For example, for a service in a public network environment, for example, for network access through http call, patrol is performed, and if abnormality exists, preliminary early warning is performed. Specifically, based on automatic construction and carding to perfect inspection items, environments such as python, docker, mysql and dependent packages are installed, an automatic inspection container environment is built, log monitoring service is started, intelligent judgment, alarm notification and data disc-falling modules are called, and therefore floor construction of the inspection scheme is completed.
According to the embodiment, the inspection item is inspected by the dynamic container on the inspection construction of the on-line service, so that the environment isolation of automatic inspection can be ensured, and the influence of machine resources and external information on the inspection item is reduced. In the abnormal judgment stage, a solution based on personalized threshold value, history analysis and similarity analysis is provided, intelligent division is carried out on abnormal conditions on the line, and alarming caused by unstable jitter on the line is effectively reduced. On the basis of alarm notification, the embodiment expands an intelligent analysis and alarm notification module, can strengthen the positioning and analysis capability of abnormal problems, and can be used for butting various notification modes, thereby improving the problem investigation efficiency.
FIG. 5 is a block diagram of an alert device according to an embodiment of the present disclosure.
As shown in fig. 5, the alarm device may include:
a history feature determining module 510, configured to determine a history feature based on history early warning information for performing a first early warning for a first object in a second time period when a current early warning number of times of performing the first early warning for the first object in the first time period is greater than or equal to a set threshold, where the first time period and the second time period have the same time feature;
a target root cause determining module 520, configured to determine a target root cause of early warning based on the historical early warning feature and a current early warning feature of the first object for the first early warning in the first time period; the method comprises the steps of,
and an alarm module 530, configured to determine whether to alarm for the first object based on the target early warning root cause.
FIG. 6 is a block diagram of an alert device according to another embodiment of the present disclosure.
As shown in fig. 6, the history feature determination module 610, the target root determination module 620, and the alarm module 630 in fig. 6 are identical in structure and function to the corresponding history feature determination module 510, the target root determination module 520, and the alarm module 530 in fig. 5, and are not described herein.
In an exemplary embodiment, the target root cause determination module 620 includes:
a first root cause determining unit 621 configured to determine a first early warning root cause based on the history early warning feature;
a second root cause determining unit 622, configured to determine a second root cause of the early warning based on a current early warning feature of the first early warning for the first object in the first period of time;
a third root cause determining unit 623, configured to determine the target early warning root cause based on the first early warning root cause and the second early warning root cause.
In an exemplary embodiment, the alarm module 630 includes:
the early warning severity determining unit 631 is configured to determine an early warning severity based on the target early warning root cause and the current early warning times when the target early warning root cause is not a preset root cause;
the first alarm unit 632 is configured to alarm the first object if the early warning severity meets a first preset condition.
In an exemplary embodiment, the early warning severity determining unit 631 is specifically configured to:
determining a first numerical value based on the importance degree of the target early warning root cause;
Determining a second value based on the current early warning times;
and determining the early warning severity degree based on the product of the first value and the second value.
In an exemplary embodiment, the system further includes an early warning severity correction unit 633, specifically configured to:
determining a third value based on the priority of the first object;
and correcting the early warning severity degree based on the third numerical value.
In an exemplary embodiment, the alarm module 630 includes:
a first stability determining unit 634, configured to determine, when the target early warning root cause is not a preset root cause, a first early warning stability based on the current early warning frequency and a historical early warning frequency of the first early warning for the first object in a third time period, where the third time period has the same time characteristic as the first time period;
and a second alarm unit 635 configured to alarm the first object when the first pre-alarm stability meets a second preset condition.
In an exemplary embodiment, the first object is an online service, and the alert module 630 further includes:
A first frequency determining unit 636 configured to determine a current early warning frequency based on the current early warning times and the current service times of the online service in the first time period;
a second frequency determining unit 637 configured to determine a historical early warning frequency based on the historical early warning times and the historical service times of the online service in the third period;
a second stability determining unit 638, configured to determine a second early warning stability based on the current early warning frequency and the historical early warning frequency;
and a stability correction unit 639 configured to correct the first warning stability based on the second warning stability.
In an exemplary embodiment, the alarm module 630 includes:
and a third alarm unit 6310, configured to alarm the first object when the target early warning root cause is a preset root cause.
For descriptions of specific functions and examples of each module and sub-module of the apparatus in the embodiments of the present disclosure, reference may be made to the related descriptions of corresponding steps in the foregoing method embodiments, which are not repeated herein.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
Fig. 7 illustrates a schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile apparatuses, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing apparatuses. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 7, the apparatus 700 includes a computing unit 701 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 may also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
Various components in device 700 are connected to I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, etc.; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, an optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The computing unit 701 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the respective methods and processes described above, for example, an alarm method. For example, in some embodiments, an alert method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 700 via the ROM 702 and/or the communication unit 707. When a computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of one of the alert methods described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform an alerting method by any other suitable means (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions, improvements, etc. that are within the principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims (19)

1. An alert method comprising:
determining a history early warning characteristic based on history early warning information of a first early warning for a first object in a second time period when the current early warning frequency of the first early warning for the first object in the first time period is greater than or equal to a set threshold, wherein the first time period and the second time period have the same time characteristic;
determining a target early warning root cause based on the historical early warning characteristics and current early warning characteristics of the first early warning for the first object in the first time period; the method comprises the steps of,
And determining whether to alarm for the first object based on the target early warning root cause.
2. The method of claim 1, wherein the determining a target pre-warning root cause based on the historical pre-warning feature and a current pre-warning feature of the first pre-warning for the first object over the first time period comprises:
determining a first early warning root cause based on the historical early warning features;
determining a second early warning root cause based on current early warning characteristics of the first early warning for the first object in the first time period;
and determining the target early warning root cause based on the first early warning root cause and the second early warning root cause.
3. The method of claim 1 or 2, wherein the determining whether to alert for the first object based on the target pre-warning root cause comprises:
under the condition that the target early warning root cause is not a preset root cause, determining early warning severity based on the target early warning root cause and the current early warning times;
and under the condition that the early warning severity meets a first preset condition, warning is carried out on the first object.
4. The method of claim 3, wherein the determining the pre-warning severity based on the target pre-warning root cause and the current number of pre-warnings comprises:
Determining a first numerical value based on the importance degree of the target early warning root cause;
determining a second value based on the current early warning times;
and determining the early warning severity degree based on the product of the first value and the second value.
5. The method of claim 3 or 4, further comprising:
determining a third value based on the priority of the first object;
and correcting the early warning severity degree based on the third numerical value.
6. The method of claim 1 or 2, wherein the determining whether to alert for the first object based on the target pre-warning root cause comprises:
determining a first early warning stability based on the current early warning times and the historical early warning times of the first early warning for the first object in a third time period under the condition that the target early warning root cause is not a preset root cause, wherein the third time period and the first time period have the same time characteristic;
and under the condition that the first early warning stability meets a second preset condition, warning is carried out on the first object.
7. The method of claim 6, wherein the first object is an online service, the method further comprising:
Determining a current early warning frequency based on the current early warning times and the current service times of the online service in the first time period;
determining a historical early warning frequency based on the historical early warning times and the historical service times of the online service in the third time period;
determining a second early warning stability based on the current early warning frequency and the historical early warning frequency;
and correcting the first early warning stability based on the second early warning stability.
8. The method of any of claims 1-7, wherein the determining whether to alert for the first object based on the target pre-warning root cause comprises:
and under the condition that the target early warning root cause is a preset root cause, alarming is carried out aiming at the first object.
9. An alert device comprising:
a history feature determining module, configured to determine a history feature based on history early warning information for performing a first early warning in a second time period for a first object, where the first time period and the second time period have the same time feature, when a current early warning number of times of performing the first early warning in the first time period for the first object is greater than or equal to a set threshold;
The target root cause determining module is used for determining a target early warning root cause based on the history early warning characteristics and the current early warning characteristics of the first early warning of the first object in the first time period; the method comprises the steps of,
and the alarm module is used for determining whether to alarm for the first object based on the target early warning root cause.
10. The apparatus of claim 9, wherein the target root cause determination module comprises:
the first root cause determining unit is used for determining a first early warning root cause based on the history early warning characteristics;
a second root cause determining unit, configured to determine a second root cause of the first early warning based on a current early warning feature of the first early warning for the first object in the first period of time;
and the third root cause determining unit is used for determining the target early warning root cause based on the first early warning root cause and the second early warning root cause.
11. The apparatus of claim 9 or 10, wherein the alert module comprises:
the early warning severity determining unit is used for determining the early warning severity based on the target early warning root cause and the current early warning times under the condition that the target early warning root cause is not a preset root cause;
And the first alarming unit is used for alarming the first object under the condition that the early warning severity meets a first preset condition.
12. The apparatus of claim 11, wherein the early warning severity determination unit is specifically configured to:
determining a first numerical value based on the importance degree of the target early warning root cause;
determining a second value based on the current early warning times;
and determining the early warning severity degree based on the product of the first value and the second value.
13. The apparatus according to claim 11 or 12, further comprising an early warning severity correction unit, in particular for:
determining a third value based on the priority of the first object;
and correcting the early warning severity degree based on the third numerical value.
14. The apparatus of claim 9 or 10, wherein the alert module comprises:
a first stability determining unit, configured to determine a first early warning stability based on the current early warning times and a historical early warning time of the first early warning for the first object in a third time period, where the third time period has the same time characteristic as the first time period, when the target early warning root cause is not a preset root cause;
And the second alarming unit is used for alarming the first object under the condition that the first early warning stability meets a second preset condition.
15. The apparatus of claim 14, wherein the first object is an online service, the apparatus further comprising:
the first frequency determining unit is used for determining the current early warning frequency based on the current early warning times and the current service times of the online service in the first time period;
the second frequency determining unit is used for determining a historical early warning frequency based on the historical early warning times and the historical service times of the online service in the third time period;
the second stability determining unit is used for determining second early warning stability based on the current early warning frequency and the historical early warning frequency;
and the stability correction unit is used for correcting the first early warning stability based on the second early warning stability.
16. The apparatus of any of claims 9-15, wherein the alert module comprises:
and the third alarming unit is used for alarming the first object under the condition that the target early warning root cause is a preset root cause.
17. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.
18. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-8.
19. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-8.
CN202310946974.8A 2023-07-28 2023-07-28 Alarm method, alarm device, electronic equipment and storage medium Pending CN117041005A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310946974.8A CN117041005A (en) 2023-07-28 2023-07-28 Alarm method, alarm device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310946974.8A CN117041005A (en) 2023-07-28 2023-07-28 Alarm method, alarm device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN117041005A true CN117041005A (en) 2023-11-10

Family

ID=88625494

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310946974.8A Pending CN117041005A (en) 2023-07-28 2023-07-28 Alarm method, alarm device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117041005A (en)

Similar Documents

Publication Publication Date Title
CN110333995A (en) The method and device that operation of industrial installation is monitored
CN115033463B (en) System exception type determining method, device, equipment and storage medium
CN114580916A (en) Enterprise risk assessment method and device, electronic equipment and storage medium
CN116010220A (en) Alarm diagnosis method, device, equipment and storage medium
CN115328733A (en) Alarm method and device applied to business system, electronic equipment and storage medium
CN115333783A (en) API call abnormity detection method, device, equipment and storage medium
CN114697247B (en) Fault detection method, device, equipment and storage medium of streaming media system
CN116668264A (en) Root cause analysis method, device, equipment and storage medium for alarm clustering
CN116755974A (en) Cloud computing platform operation and maintenance method and device, electronic equipment and storage medium
CN117041005A (en) Alarm method, alarm device, electronic equipment and storage medium
CN116226644A (en) Method and device for determining equipment fault type, electronic equipment and storage medium
CN116471174A (en) Log data monitoring system, method, device and storage medium
CN114861909A (en) Model quality monitoring method and device, electronic equipment and storage medium
CN116820826B (en) Root cause positioning method, device, equipment and storage medium based on call chain
CN116627695B (en) Alarm event root cause recommendation method, device, equipment and storage medium
CN113138903B (en) Method and apparatus for tracking performance of a storage system
CN117194813A (en) Method and device for judging shortest distance between target point and region
CN115640202A (en) Performance detection method and device of service program and storage medium
CN117608904A (en) Fault positioning method and device, electronic equipment and storage medium
CN118170555A (en) Distributed service abnormality positioning method, device, electronic equipment and storage medium
CN117149486A (en) Alarm and root cause positioning method, model training method, device, equipment and medium
CN117667895A (en) Data processing method, system, equipment and medium based on data braiding
CN117493060A (en) Database component anomaly detection method, device, equipment and medium
CN117934152A (en) Risk assessment method, device, equipment and storage medium after system change
CN115689574A (en) Transaction risk early warning method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination