CN114401494B - Short message issuing abnormality detection method, device, computer equipment and storage medium - Google Patents
Short message issuing abnormality detection method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114401494B CN114401494B CN202210043421.7A CN202210043421A CN114401494B CN 114401494 B CN114401494 B CN 114401494B CN 202210043421 A CN202210043421 A CN 202210043421A CN 114401494 B CN114401494 B CN 114401494B
- Authority
- CN
- China
- Prior art keywords
- data
- short message
- task
- preset
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/12—Messaging; Mailboxes; Announcements
- H04W4/14—Short messaging services, e.g. short message services [SMS] or unstructured supplementary service data [USSD]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention relates to the field of operation process optimization, and discloses a short message issuing abnormality detection method, a short message issuing abnormality detection device, computer equipment and a storage medium, wherein the method comprises the following steps: receiving feedback data of a short message pushing task, wherein the feedback data comprises short message sending success data of a plurality of operators; processing the feedback data through a multi-element linear prediction model to obtain expected success data; the multi-element linear prediction model comprises constant items and dimension factors, wherein one dimension parameter corresponds to one dimension factor; acquiring a preset normal interval matched with the short message pushing task, and judging whether expected successful data are out of the preset normal interval; and if the expected successful data is outside the predicted normal interval, sending out an abnormal reminding. The invention can rapidly detect whether the fluctuation of the feedback data is in the predicted normal interval, and further judge whether the current short message pushing task is abnormal or not in time.
Description
Technical Field
The present invention relates to the field of operation process optimization, and in particular, to a method and apparatus for detecting an abnormality in sending a short message, a computer device, and a storage medium.
Background
Short message service is widely applied and plays a very important role in the fields of OTP (One Time Password, one-time password) verification, enterprise message notification and the like. The short message service has the advantages of low cost, high touch efficiency, better conversion rate and the like.
However, based on the mobile phone number of the operator as a carrier, the short message service is easily affected by factors such as regions, operators and the like, so that the success rate of sending the short message fluctuates. Such fluctuations have a serious negative impact on some time-efficient service events, severely affecting the conversion rate of the service events. Moreover, when a fluctuation event occurs, the problem is not easy to find in time and eliminated.
Disclosure of Invention
Based on this, it is necessary to provide a method, a device, a computer device and a storage medium for detecting the abnormality of the short message delivery, so as to detect whether the short message delivery is abnormal or not in time.
A short message issuing abnormality detection method includes:
receiving feedback data of a short message pushing task, wherein the feedback data comprises short message sending success data of a plurality of operators;
processing the feedback data through a multi-element linear prediction model to obtain expected success data; the multi-element linear prediction model comprises constant items and dimension factors, wherein one dimension parameter corresponds to one dimension factor;
Acquiring a preset normal interval matched with the short message pushing task, and judging whether expected successful data are out of the preset normal interval;
and if the expected successful data is outside the predicted normal interval, sending out an abnormal reminding.
A short message issuing abnormality detection device includes:
the feedback data receiving module is used for receiving feedback data of the short message pushing task, wherein the feedback data comprises short message sending success data of a plurality of operators;
the prediction expectation module is used for processing the feedback data through a multi-element linear prediction model to acquire expected success data; the multi-element linear prediction model comprises constant items and dimension factors, wherein one dimension parameter corresponds to one dimension factor;
the matching normal interval module is used for acquiring a preset normal interval matched with the short message pushing task and judging whether expected successful data are outside the preset normal interval;
and the abnormality reminding module is used for sending out an abnormality reminder if the expected successful data is out of the predicted normal interval.
A computer device comprises a memory, a processor and computer readable instructions stored in the memory and capable of running on the processor, wherein the processor realizes the short message issuing abnormality detection method when executing the computer readable instructions.
One or more readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform a method of short message issue anomaly detection as described above.
According to the short message issuing abnormality detection method, the short message issuing abnormality detection device, the computer equipment and the storage medium, the feedback data of the short message pushing task is received, and the feedback data comprise the successful data of the short message issuing of a plurality of operators so as to monitor the issuing condition of the short message in real time. Processing the feedback data through a multi-element linear prediction model to obtain expected success data; the multi-element linear prediction model comprises constant items and dimension factors, wherein one dimension parameter corresponds to one dimension factor, so that real-time variable feedback data are converted into expected success data with better predictability, and the recognition accuracy of abnormal conditions is improved. And acquiring a preset normal interval matched with the short message pushing task, and judging whether the expected successful data is out of the preset normal interval or not so as to evaluate whether the expected successful data is abnormal or not through the preset normal interval. If the expected successful data is outside the predicted normal interval, an abnormal prompt is sent out, so that a worker can obtain the corresponding abnormal prompt in time. The invention can rapidly detect whether the fluctuation of the feedback data is in the predicted normal interval, and further judge whether the current short message pushing task is abnormal or not in time.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an application environment of a method for detecting an abnormality in a SMS transmission in an embodiment of the present invention;
FIG. 2 is a flow chart of a method for detecting an abnormality in SMS delivery according to an embodiment of the present invention;
FIG. 3 is a plurality of prediction curves according to an embodiment of the present invention;
FIG. 4 is a schematic view of a point distribution of a first image according to an embodiment of the present invention;
FIG. 5 is a schematic view of a point distribution of a second image according to an embodiment of the present invention;
FIG. 6 is a schematic view of a dot distribution of a third image according to an embodiment of the present invention;
FIG. 7 is a schematic view of a point distribution of an outlier detection image according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a short message sending abnormality detection apparatus according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a computer device in accordance with an embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The short message sending abnormality detection method provided by the embodiment can be applied to an application environment as shown in fig. 1, wherein a client communicates with a server. Clients include, but are not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server may be implemented by a stand-alone server or a server cluster formed by a plurality of servers.
In an embodiment, as shown in fig. 2, a method for detecting an abnormality in sending a short message is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps S60-S90.
S60, receiving feedback data of the short message pushing task, wherein the feedback data comprises short message sending success data of a plurality of operators.
The short message pushing task is understood to mean a task of sending a short message containing specified contents to a target crowd. Corresponding specified content can be written based on specific service requirements, and then corresponding short message pushing tasks are generated. The business requirements may be marketing requirements, campaign requirements, and the like. For example, a company may need to push certain activity information to a million customers on a certain day.
And submitting the short message pushing task to an operator. Then, during task execution, feedback data of the short message push task is received from different operators in real time. Here, the feedback data includes short message delivery success data of several operators. The short message delivery success data may include the number of successful delivery of the short message per minute. If the short message pushing task relates to a plurality of areas, the short message successful issuing data can comprise area identifiers for distinguishing the successful issuing quantity of different areas.
S70, processing the feedback data through a multi-element linear prediction model to obtain expected success data; the multi-element linear prediction model comprises constant items and dimension factors, wherein one dimension parameter corresponds to one dimension factor.
Understandably, the multiple linear prediction model may be a multiple linear prediction model obtained based on historical data training, which has a prediction function and can predict the expected number of short messages to be sent. The multiple linear prediction model includes constant terms and dimension factors. In one example, the multiple linear prediction model may be expressed as:
Y＝β 0 +β 1 X 1 +β 2 X 2 +β 3 X 3 +…+β n X n
wherein Y is a predicted value;
β 0 is a constant term;
X 1 represents the 1 st dimension parameter, and so on, X n Representing the nth dimension parameter, wherein n is the total number of the dimension parameters;
β 1 ～β n respectively with X 1 ～X n Corresponding dimension factors.
And extracting values of each dimension parameter from the feedback data, and substituting the values into a multi-element linear prediction model to obtain a predicted value. The expected success data includes predicted values for various points in time. For example, the number of successful pushing operations for the first minute is extracted from the feedback data, parameter values of each dimension parameter are obtained, and then substitutedA multi-element linear prediction model for obtaining a predicted value Y of the first minute 1 The method comprises the steps of carrying out a first treatment on the surface of the Extracting the successful pushing quantity of the second minute from the feedback data to obtain parameter values of each dimension parameter, and substituting the parameter values into a multi-element linear prediction model to obtain a predicted value Y of the second minute 2 The method comprises the steps of carrying out a first treatment on the surface of the And so on.
In the example of fig. 3, expected success data may be represented using a predictor curve 02.
S80, acquiring a preset normal interval matched with the short message pushing task, and judging whether the expected successful data is outside the preset normal interval.
It will be appreciated that there is generally a certain difference between the push groups of each short message push task, so it is necessary to find a plurality of history push tasks that are most similar (even identical) to the push group of the current short message push task, and no abnormal event occurs in these history push tasks. And processing feedback data of the historical pushing tasks through a multi-element linear prediction model, so that a plurality of historical expected success data can be obtained, and corresponding preset normal intervals can be drawn according to the historical expected success data. In the example of fig. 3, a highest success curve 01, a lowest success curve 04, and a fitting success curve 03 are included. The preset normal interval may refer to an interval between the highest success curve 01 and the fitting success curve 03.
And S90, if the expected successful data is outside the predicted normal interval, an abnormal prompt is sent out.
Understandably, if the currently calculated expected successful data is within the predicted normal interval, it is indicated that the current short message transmission is normal. If the currently calculated expected successful data is outside the predicted normal interval, the current short message transmission is indicated to have a possible problem, and therefore, a corresponding abnormal prompt needs to be sent. Here, the exception alert includes, but is not limited to, a pop-up alert, a mail alert, and a short message alert.
In one example, in a marketing campaign of a certain day, at a certain time point, the amount of work done by a marketing message corresponding to a certain operator becomes low, as shown in table 1 below.
Table 1 feedback data and expected success data in an example
| Time | X 1 | X 2 | X 3 | X 4 | Y |
| 09:01:00 | 755.5 | 613.5 | 423.5 | 2213 | 2051.93015 |
| 09:02:00 | 754.5 | 615.5 | 424.5 | 2204 | 2047.40373 |
| 09:03:00 | 553.0 | 617.0 | 425.0 | 2195 | 1871.60428 |
| 09:04:00 | 557.5 | 613.0 | 426.5 | 2195 | 1872.32626 |
Wherein X is 1 、X 2 、X 3 X are respectively a first operator, a second operator and a third operator 4 And (3) representing the region identifier, wherein Y is a predicted value. The first operator sends out a work amount 755.5 in a short message of 09:01.
As can be seen from Table 1, at times 09:03 and 09:04, X 1 The column, i.e. the successful message amount of the first operator, decreases, resulting in a decrease of the final predicted value, i.e. the Y column value. The method has the advantages that the method has an excessively large difference from the expected success amount in a normal sample, and can reflect the problem of the link of the first operator in the region at present and needs to be adjusted after being compared with the normal value of the same region.
In step S60-S90, feedback data of the short message pushing task is received, where the feedback data includes successful data of the short message delivery of several operators, so as to monitor the delivery status of the short message in real time. Processing the feedback data through a multi-element linear prediction model to obtain expected success data; the multi-element linear prediction model comprises constant items and dimension factors, wherein one dimension parameter corresponds to one dimension factor, so that real-time variable feedback data are converted into expected success data with better predictability, and the recognition accuracy of abnormal conditions is improved. And acquiring a preset normal interval matched with the short message pushing task, and judging whether the expected successful data is out of the preset normal interval or not so as to evaluate whether the expected successful data is abnormal or not through the preset normal interval. If the expected successful data is outside the predicted normal interval, an abnormal prompt is sent out, so that a worker can obtain the corresponding abnormal prompt in time. The embodiment can rapidly detect whether the fluctuation of the feedback data is in the predicted normal interval or not, and further timely judge whether the current short message pushing task is abnormal or not.
Optionally, before step S70, that is, before the feedback data is processed by the multiple linear prediction model, the method further includes:
s10, acquiring short message sending historical data of a specified time span;
s20, processing the short message sending history data into a multi-dimensional array set according to a preset data extraction rule, wherein the multi-dimensional array set comprises a plurality of multi-dimensional arrays, one multi-dimensional array corresponds to one unit time interval, the appointed time span is divided into a plurality of unit time intervals, and the multi-dimensional arrays comprise the successful number of short messages with a plurality of dimensions in the unit time intervals and the expected number;
s30, constructing a multiple linear regression model according to the multi-dimensional array set, and solving model parameters of the multiple linear regression model, wherein the model parameters comprise constant items, dimension factors and residual items;
s40, checking and correcting the multiple linear regression model by using a preset significance checking method to generate a multiple linear correction model;
and S50, verifying the multi-element linear correction model by using a preset model diagnosis method, and determining the verified multi-element linear correction model as the multi-element linear prediction model.
It is understood that the specified time span may be set according to actual needs, for example, may be 1 to 24 hours. The short message issuing historical data comprises target issuing data and actual issuing success data of a historical task.
The preset data extraction rule can be set according to actual needs, for example, the successful number of short message delivery of a certain operator in a certain area in a unit time interval can be counted. In an example, the preset data extraction rule may extract a plurality of multidimensional arrays from the short message sending historical data to form a multidimensional data set. The unit time interval can be set according to actual needs, for example, one minute. The dimensions of the multidimensional array can be set according to actual conditions. For example, the corresponding dimension parameters can be set according to different operators and different regions.
In one example, the multi-dimensional array set is shown in Table 2.
Table 2 multidimensional data collection in an example
Wherein each behavior is a multidimensional array.
Multiple linear regression models may be constructed, such as:
Y＝β 0 +β 1 X 1 +β 2 X 2 +β 3 X 3 +…+β n X n +ε
wherein beta is 0 Is a constant term, beta 1 、β 2 、β 3 、β n As dimension factor, X 1 、X 2 、X 3 、X n Y is a predicted value (dependent variable), epsilon is a residual term, and the residual term is the sum of the influence of all uncertain factors, and the value of the residual term is not observable. It can be assumed that ε obeys a normal distribution: n (0, sigma) 2 ) The value of which needs to be determined by specific residual analysis means. In an example, a multiple linear regression model is constructed with four dimension factors of region, first operator, second operator, and third operator, respectively, and then the model can be expressed as:
Y＝β 0 +β 1 X 1 +β 2 X 2 +β 3 X 3 +β 4 X 4 +ε
wherein Y is a predicted value;
β 0 is a constant term;
X 1 、X 2 、X 3 a first operator, a second operator and a third operator respectively;
X 4 representing a region identifier;
epsilon is the residual term.
Substituting the multidimensional array set into a multiple linear regression model, and solving model parameters through calculation. Model parameters include constant terms, individual dimension factors, and residual terms (subject to normal distribution, within a data range).
After the model parameters of the multiple linear regression model are solved, a preset significance calibration method can be used for calibrating and correcting the multiple linear regression model. The preset significance checking method can be selected according to actual needs, and can be a T-test method, an F-test method and an R-square test method. One or more saliency verification methods may be used. And obtaining a multi-element linear correction model through checksum correction of a preset significance verification method. The accuracy of the model parameters of the multi-element linear correction model is higher.
And then, verifying the multi-element linear correction model by using a preset model diagnosis method, analyzing residual terms and abnormal points of the multi-element linear correction model, and ensuring the accuracy of the multi-element linear correction model. If the verification is passed, the multiple linear correction model can be determined as a multiple linear prediction model. And if the model parameters of the multi-element linear model are not verified, adjusting the model parameters of the multi-element linear model or presetting data extraction rules. For example, when the preset data extraction rule is modified, the unit time interval may be modified, and the original one minute is changed to two minutes or thirty seconds.
Optionally, the preset significance checking method comprises a T-test method, an F-test method and an R-square test method;
step S40, namely, performing checksum correction on the multiple linear regression model by using a preset significance checking method, to generate a multiple linear correction model, including:
s401, checking the multiple linear regression model according to the T test method to generate a first check result;
s402, checking the multiple linear regression model according to the F test method to generate a second check result;
s403, checking the multiple linear regression model according to the R square check method to generate a third check result;
S404, if the first check result, the second check result and the third check result are all significant results, correcting the multiple linear regression model according to the first check result, the second check result and the third check result to obtain the multiple linear correction model.
Understandably, the preset saliency verification method includes a T-test method, an F-test method, and an R-square test method. And (3) checking the multiple linear regression model by using each checking method to obtain a corresponding checking result. The verification results include two types, one indicating that the multiple linear regression model has significance, i.e., is a significant result, and the other indicating that the multiple linear regression model has no significance, i.e., is a non-significant result.
Here, the T-test (student T-test) can test the independent variable X in the multiple linear regression model i (i=1 to n) significance for Y, typically determined by P-value less than 0.01 or less, indicating the argument X i The correlation with Y is remarkable.
The F test is used to test all the arguments { X ] i Linear significance of Y as a whole, also determined by P-value less than 0.01 or less, indicating the overall argument { X } i The correlation between the two components is remarkable.
R 2 The (R square) test method is used for judging the fitting degree of the regression equation, R 2 The values of (1) are (0, 1), and the closer to 1, the better the fitting degree is.
In an example, the verification results of the three preset significance verification methods are as follows:
first verification result of T-verification method: all independent variables are very significant;
second check result of F check method: the verification result is obvious, p-value:<2.2e -16 ；
R 2 third verification result of the verification method: the correlation was very strong at 0.972.
Before verification, the multiple linear regression model is:
Y＝212.8780+0.8542*X 1 +0.6672*X 2 -0.6674*X 3 +0.4821*X 4
after correction, the obtained multi-element linear correction model is as follows:
Y＝212.87996+0.85423*X 1 +0.66724*X 2 -0.66741*X 3 +0.48214*X 4
and after the verification is finished, the multi-element linear correction model is more accurate in relation coefficient compared with the multi-element linear regression model.
Optionally, the preset model diagnosis method comprises a residual analysis method and an abnormal point detection method;
step S50, namely, verifying the multiple linear correction model by using a preset model diagnosis method, determining the verified multiple linear correction model as the multiple linear prediction model, including:
s501, verifying the multi-element linear correction model according to the residual error analysis method to generate a first verification result;
s502, verifying the multi-element linear correction model according to the abnormal point detection method to generate a second verification result;
S503, if the first verification result and the second verification result both meet a preset verification requirement, judging that the multi-element linear correction model passes verification.
It is understood that in order to make the multivariate linear correction model as accurate as possible, the model can also be checked for correctness using residual analysis and outlier detection. In the residual analysis and outlier detection test, patterns for model diagnosis can be generated by means of a drawing tool for visual analysis to determine whether the multiple linear correction model passes verification. The preset verification requirements can be set according to actual needs. In some examples, the preset verification requirements include: the first verification result is pass verification, and the second verification result is pass verification.
Optionally, step S501, that is, the verifying the multiple linear correction model according to the residual analysis method, generates a first verification result, includes:
s5011, obtaining residual errors and fitting values of the multi-element linear correction model, filling the residual errors and the fitting values into a first image, analyzing first point distribution forms of the residual errors and the fitting values in the first image, and generating first analysis data;
S5012, obtaining a standardized residual error and a theoretical quantile of the multi-element linear correction model, filling the standardized residual error and the theoretical quantile into a second image, analyzing a second point distribution form of the standardized residual error and the theoretical quantile in the second image, and generating second analysis data;
s5013, obtaining a standardized residual square root of the multi-element linear correction model, filling the standardized residual square root and the fitting value into a third image, analyzing a third point distribution form of the standardized residual square root and the fitting value in the third image, and generating third analysis data;
s5014, generating the first verification result according to the first analysis data, the second analysis data, and the third analysis data.
It is understood that the residual error and the fitting value of the multi-element linear correction model can be obtained, the residual error and the fitting value are filled into the first image, and the first point distribution form of the residual error and the fitting value in the first image is analyzed to generate first analysis data. In one example, as shown in fig. 4, fig. 4 is a first image of a comparison of residual errors (residual) and fitting values (filtered values or shorthand). By identifying (here, manually observable, then manually input, also automatically identifiable by a graphical analysis tool) the residual and the first point distribution morphology of the fitting values in the first image, the following first analysis data can be obtained: the data points between the residual error and the fitting value are uniformly distributed on two sides of y=0, random distribution is presented, and a trend line is a stable curve and has no obvious shape characteristics.
And (3) acquiring the standardized residual error and the theoretical quantile of the multi-element linear correction model, filling the standardized residual error and the theoretical quantile into a second image, analyzing the second point distribution form of the standardized residual error and the theoretical quantile in the second image, and generating second analysis data. In one example, as shown in fig. 5, fig. 5 is a second image (i.e., normalQ-Q, standard qq plot) of normalized residuals (normalized residuals) compared to theoretical quantiles (theoretical quantiles). By identifying the second point distribution morphology of the normalized residual and the theoretical quantile in the second image, the following second analysis data may be obtained: the data points in the second image are arranged in a diagonal line, approach to a straight line, and are directly penetrated by the diagonal line, so that the data points intuitively accord with normal distribution.
And obtaining the normalized residual square root and the fitting value of the multi-element linear correction model, filling the normalized residual square root and the fitting value into a third image, analyzing the third point distribution form of the normalized residual square root and the fitting value in the third image, and generating third analysis data. In one example, as shown in fig. 6, fig. 6 is a third image (Scale-Location) of a normalized square root of residual (squarestoradifideprovided) compared with a fitting value (Fittedvalues). By identifying the third point distribution morphology of the normalized residual square root and the fitting value in the third image, the following third analysis data can be obtained: the data points in the third image are uniformly distributed on both sides of y=0, and random distribution is presented, and the curve in the third image is stable and has no obvious shape characteristics.
If the first analysis data, the second analysis data and the third analysis data are all abnormal, the first verification result is that the multiple linear correction model passes verification.
In other examples, normalized residuals and leverage values of the multiple linear correction model may be obtained, the normalized residuals and leverage values are filled into the outlier detection image, and a distribution form of the normalized residuals and leverage values in the outlier detection image is analyzed to generate outlier detection analysis data. In an example, as shown in fig. 7, fig. 7 is an outlier detection image in which normalized residuals (normalized residuals) and lever values (lever) are compared. By identifying the distribution pattern of the normalized residual and the leverage value in the outlier detection image, the following analysis data can be obtained: if no contour line appears, no abnormal point which has special influence on regression result is indicated in the data.
Optionally, in step S80, the acquiring a preset normal interval matched with the short message pushing task includes:
s801, extracting push keywords of the short message push task;
s802, corresponding normal task data are searched in a task database according to the pushing keywords, wherein the normal task data comprise task data of a plurality of historical normal pushing tasks;
S803, calculating the upper and lower issuing limits of each time point when the short message pushing task is executed according to the normal task data, and forming the preset normal interval.
Understandably, the push keywords include one or more keywords. The business characteristics or the regional attributes of the short message pushing task can be selected as keywords, and the keywords can be set according to the crowd characteristics corresponding to the pushing crowd. Extraction rules for extracting push keywords can be set according to actual needs.
The task database records task data for the history short message push task, where the task data includes feedback data and expected success data (calculated by the same multivariate linear prediction model). The historical normal pushing task refers to a pushing task which is confirmed to have no abnormal condition after the post verification. The normal task data includes a plurality of task data of historical normal push tasks.
The task data of each historical normal push task includes predicted values for a plurality of points in time. At the same time point (can be fixed time, such as 9:00, or can be relative time, such as taking the time fed back by an operator for the first time as a starting point), the predicted values of different historical normal pushing tasks are different, wherein the largest predicted value is set as an upper issuing limit, and the smallest predicted value is set as the upper issuing limit. In some cases, if the predicted value at the same point in time follows a normal distribution, the lower issue limit may be set to μ+2σ, and the lower issue limit may be set to μ -2σ. Here, μ is the average value of the predicted values, and σ is the variance. The preset normal interval comprises an upper issuing limit and a lower issuing limit of each time point.
Optionally, in step S80, the determining whether the expected successful data is outside a preset normal interval includes:
s804, judging whether each predicted value in the expected successful data is larger than the corresponding issuing upper limit or smaller than the issuing lower limit;
s805, when the predicted value in the expected successful data is greater than the corresponding upper issuing limit, calculating a first difference value between the predicted value and the upper issuing limit, and adding one to the exceeding accumulated value; or when the predicted value is smaller than the corresponding lower issuing limit, calculating a second difference value between the lower issuing limit and the predicted value, and adding one to the exceeding accumulated value; the initial value of the exceeding accumulated value is zero;
and S306, when the exceeding accumulated value is larger than a preset accumulated threshold, or the first difference value is larger than a first comparison threshold, or the second difference value is larger than a second comparison threshold, judging that the expected successful data is out of a preset normal interval.
Understandably, a certain predicted value in the expected successful data can be compared with the lower limit or the upper limit of the same time point, if the predicted value is larger than the corresponding lower limit, a first difference between the predicted value and the lower limit is calculated, and meanwhile, the exceeding accumulated value is increased by one; if the value is smaller than the corresponding lower issuing limit, a second difference value between the lower issuing limit and the predicted value is calculated, and meanwhile, the exceeding accumulated value is increased by one. Here, the initial value of the superscalar integrated value is zero.
And when the exceeding accumulated value is larger than a preset accumulated threshold, or the first difference value is larger than a first comparison threshold, or the second difference value is larger than a second comparison threshold, judging that the expected successful data is out of a preset normal interval. When the exceeding accumulated value is smaller than or equal to a preset accumulated threshold, the first difference value is smaller than or equal to a first comparison threshold, and the second difference value is smaller than or equal to a second comparison threshold, and the expected successful data is judged to be in a preset normal interval. The overstandard cumulative value can be set according to actual needs. The first comparison threshold and the second comparison threshold can be set according to actual needs, and can be fixed values or percentage values (e.g. calculated by taking an average value as a base).
Here, the over-standard accumulated value and the two comparison thresholds are set at the same time, so that whether the short message is issued abnormally or not can be better detected, and the occurrence of false alarm is reduced.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
In an embodiment, a short message sending abnormality detection device is provided, where the short message sending abnormality detection device corresponds to the short message sending abnormality detection method in the above embodiment one by one. As shown in fig. 8, the short message issuing abnormality detection apparatus includes a feedback data receiving module 60, a prediction expectation module 70, a matching normal interval module 80, and an abnormality alert module 90. The functional modules are described in detail as follows:
The feedback data receiving module 60 is configured to receive feedback data of a short message pushing task, where the feedback data includes short message sending success data of several operators;
the prediction expectation module 70 is configured to process the feedback data through a multiple linear prediction model to obtain expected success data; the multi-element linear prediction model comprises constant items and dimension factors, wherein one dimension parameter corresponds to one dimension factor;
the matching normal interval module 80 is configured to obtain a preset normal interval matched with the short message pushing task, and determine whether the expected successful data is outside the preset normal interval;
the anomaly reminding module 90 is configured to send out an anomaly reminder if the expected successful data is outside the predicted normal interval.
Optionally, the short message sending abnormality detection device further includes:
the history data acquisition module is used for acquiring the short message sending history data of the designated time span;
the multi-dimensional array module is used for processing the short message sending history data into a multi-dimensional array set according to a preset data extraction rule, wherein the multi-dimensional array set comprises a plurality of multi-dimensional arrays, one multi-dimensional array corresponds to one unit time interval, the appointed time span is divided into a plurality of unit time intervals, and the multi-dimensional arrays comprise the successful number of short message ordering of a plurality of dimensions in the unit time intervals and the expected number;
The regression model building module is used for building a multiple linear regression model according to the multi-dimensional array set and solving model parameters of the multiple linear regression model, wherein the model parameters comprise constant items, dimension factors and residual items;
the generating and correcting model module is used for checking and correcting the multiple linear regression model by using a preset significance checking method to generate a multiple linear correcting model;
and the prediction model generation module is used for verifying the multi-element linear correction model by using a preset model diagnosis method, and determining the verified multi-element linear correction model as the multi-element linear prediction model.
Optionally, the preset significance checking method comprises a T-test method, an F-test method and an R-square test method;
the generating a correction model module includes:
the first verification unit is used for verifying the multiple linear regression model according to the T test method to generate a first verification result;
the second verification unit is used for verifying the multiple linear regression model according to the F test method to generate a second verification result;
the third verification unit is used for verifying the multiple linear regression model according to the R square test method to generate a third verification result;
And the correction model obtaining unit is used for correcting the multiple linear regression model according to the first check result, the second check result and the third check result if the first check result, the second check result and the third check result are all obvious, so as to obtain the multiple linear correction model.
Optionally, the preset model diagnosis method comprises a residual analysis method and an abnormal point detection method;
the generating a correction model module includes:
the residual analysis unit is used for verifying the multi-element linear correction model according to the residual analysis method to generate a first verification result;
the abnormal point detection unit is used for verifying the multi-element linear correction model according to the abnormal point detection method to generate a second verification result;
and the passing verification unit is used for judging that the multi-element linear correction model passes verification if the first verification result and the second verification result both meet the preset verification requirement.
Optionally, the residual analysis unit includes:
the first image analysis unit is used for acquiring residual errors and fitting values of the multi-element linear correction model, filling the residual errors and the fitting values into a first image, analyzing first point distribution forms of the residual errors and the fitting values in the first image, and generating first analysis data;
The second image analysis unit is used for acquiring a standardized residual error and a theoretical quantile of the multi-element linear correction model, filling the standardized residual error and the theoretical quantile into a second image, analyzing a second point distribution form of the standardized residual error and the theoretical quantile in the second image, and generating second analysis data;
a third image analysis unit, configured to obtain a normalized residual square root of the multiple linear correction model, fill the normalized residual square root and the fitting value into a third image, and analyze a third point distribution form of the normalized residual square root and the fitting value in the third image, so as to generate third analysis data;
and a first verification result generation unit for generating the first verification result according to the first analysis data, the second analysis data and the third analysis data.
Optionally, the matching normal interval module 80 includes:
the keyword extraction unit is used for extracting the pushing keywords of the short message pushing task;
the matched task data unit is used for searching corresponding normal task data in a task database according to the pushing keywords, wherein the normal task data comprise task data of a plurality of historical normal pushing tasks;
And forming a normal interval unit, which is used for calculating the upper and lower issuing limits of each time point when the short message pushing task is executed according to the normal task data to form the preset normal interval.
Optionally, the matching normal interval module 80 includes:
the predicted value judging unit is used for judging whether each predicted value in the expected successful data is larger than the corresponding issuing upper limit or smaller than the issuing lower limit;
a difference calculating unit, configured to calculate a first difference between the predicted value and the upper issuing limit when the predicted value in the expected successful data is greater than the corresponding upper issuing limit, and meanwhile, add one to the exceeding accumulated value; or when the predicted value is smaller than the corresponding lower issuing limit, calculating a second difference value between the lower issuing limit and the predicted value, and adding one to the exceeding accumulated value; the initial value of the exceeding accumulated value is zero;
and the abnormality judging unit is used for judging that the expected successful data is out of a preset normal interval when the exceeding accumulated value is larger than a preset accumulated threshold value, or the first difference value is larger than a first comparison threshold value, or the second difference value is larger than a second comparison threshold value.
The specific limitation of the short message delivery abnormality detection device can be referred to the limitation of the short message delivery abnormality detection method hereinabove, and will not be described herein. All or part of each module in the short message issuing abnormality detection device can be realized by software, hardware and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 9. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a readable storage medium, an internal memory. The readable storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the execution of an operating system and computer-readable instructions in a readable storage medium. The database of the computer equipment is used for storing data related to the short message issuing abnormality detection method. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer readable instructions, when executed by a processor, implement a method for detecting a short message delivery exception. The readable storage medium provided by the present embodiment includes a nonvolatile readable storage medium and a volatile readable storage medium.
In one embodiment, a computer device is provided that includes a memory, a processor, and computer readable instructions stored on the memory and executable on the processor, when executing the computer readable instructions, performing the steps of:
receiving feedback data of a short message pushing task, wherein the feedback data comprises short message sending success data of a plurality of operators;
processing the feedback data through a multi-element linear prediction model to obtain expected success data; the multi-element linear prediction model comprises constant items and dimension factors, wherein one dimension parameter corresponds to one dimension factor;
acquiring a preset normal interval matched with the short message pushing task, and judging whether expected successful data are out of the preset normal interval;
and if the expected successful data is outside the predicted normal interval, sending out an abnormal reminding.
In one embodiment, one or more computer-readable storage media are provided having computer-readable instructions stored thereon, the readable storage media provided by the present embodiment including non-volatile readable storage media and volatile readable storage media. The readable storage medium has stored thereon computer readable instructions which when executed by one or more processors perform the steps of:
Receiving feedback data of a short message pushing task, wherein the feedback data comprises short message sending success data of a plurality of operators;
processing the feedback data through a multi-element linear prediction model to obtain expected success data; the multi-element linear prediction model comprises constant items and dimension factors, wherein one dimension parameter corresponds to one dimension factor;
acquiring a preset normal interval matched with the short message pushing task, and judging whether expected successful data are out of the preset normal interval;
and if the expected successful data is outside the predicted normal interval, sending out an abnormal reminding.
Those skilled in the art will appreciate that implementing all or part of the above described embodiment methods may be accomplished by instructing the associated hardware by computer readable instructions stored on a non-volatile readable storage medium or a volatile readable storage medium, which when executed may comprise the above described embodiment methods. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (9)
1. The short message issuing abnormality detection method is characterized by comprising the following steps:
receiving feedback data of a short message pushing task, wherein the feedback data comprises short message sending success data of a plurality of operators;
Processing the feedback data through a multi-element linear prediction model to obtain expected success data; the multi-element linear prediction model comprises constant items and dimension factors, wherein one dimension parameter corresponds to one dimension factor;
acquiring a preset normal interval matched with the short message pushing task, and judging whether expected successful data are out of the preset normal interval;
if the expected successful data is outside the predicted normal interval, an abnormal reminder is sent out;
the obtaining the preset normal interval matched with the short message pushing task comprises the following steps:
extracting the push keywords of the short message push task;
searching corresponding normal task data in a task database according to the push keywords, wherein the normal task data comprise task data of a plurality of historical normal push tasks;
and calculating the upper and lower issuing limits of each time point when the short message pushing task is executed according to the normal task data to form the preset normal interval.
2. The method for detecting abnormal delivery of short message according to claim 1, wherein before the feedback data is processed by the multiple linear prediction model to obtain the expected successful data, the method further comprises:
Acquiring short message sending historical data of a designated time span;
processing the short message sending historical data into a multi-dimensional array set according to a preset data extraction rule, wherein the multi-dimensional array set comprises a plurality of multi-dimensional arrays, one multi-dimensional array corresponds to one unit time interval, the appointed time span is divided into a plurality of unit time intervals, and the multi-dimensional arrays comprise the successful number of short message ordering of a plurality of dimensions in the unit time interval and the expected number;
constructing a multiple linear regression model according to the multi-dimensional array set, and solving model parameters of the multiple linear regression model, wherein the model parameters comprise constant items, dimension factors and residual items;
performing checksum correction on the multiple linear regression model by using a preset significance verification method to generate a multiple linear correction model;
and verifying the multi-element linear correction model by using a preset model diagnosis method, and determining the verified multi-element linear correction model as the multi-element linear prediction model.
3. The short message delivery abnormality detection method according to claim 2, wherein the preset significance checking method includes a T-test method, an F-test method, and an R-square test method;
The method for verifying and correcting the multiple linear regression model by using a preset significance verification method to generate a multiple linear correction model comprises the following steps:
checking the multiple linear regression model according to the T test method to generate a first check result;
checking the multiple linear regression model according to the F check method to generate a second check result;
checking the multiple linear regression model according to the R square checking method to generate a third checking result;
and if the first check result, the second check result and the third check result are all obvious results, correcting the multiple linear regression model according to the first check result, the second check result and the third check result to obtain the multiple linear correction model.
4. The short message distribution abnormality detection method according to claim 2, wherein the preset model diagnosis method includes a residual analysis method and an abnormal point detection method;
the method for diagnosing the multiple linear correction model by using the preset model verifies the multiple linear correction model, determines the verified multiple linear correction model as the multiple linear prediction model, and comprises the following steps:
Verifying the multi-element linear correction model according to the residual error analysis method to generate a first verification result;
verifying the multi-element linear correction model according to the abnormal point detection method to generate a second verification result;
and if the first verification result and the second verification result both accord with the preset verification requirement, judging that the multi-element linear correction model passes verification.
5. The method for detecting abnormal transmission of short message according to claim 4, wherein said verifying the multivariate linear correction model according to the residual analysis method, generating a first verification result, comprises:
acquiring residual errors and fitting values of the multi-element linear correction model, filling the residual errors and the fitting values into a first image, analyzing first point distribution forms of the residual errors and the fitting values in the first image, and generating first analysis data;
obtaining a standardized residual error and a theoretical quantile of the multi-element linear correction model, filling the standardized residual error and the theoretical quantile into a second image, analyzing a second point distribution form of the standardized residual error and the theoretical quantile in the second image, and generating second analysis data;
Obtaining a normalized residual square root of the multi-element linear correction model, filling the normalized residual square root and the fitting value into a third image, analyzing a third point distribution form of the normalized residual square root and the fitting value in the third image, and generating third analysis data;
and generating the first verification result according to the first analysis data, the second analysis data and the third analysis data.
6. The method for detecting abnormal sending of short message according to claim 1, wherein the determining whether the expected successful data is outside a preset normal interval comprises:
judging whether each predicted value in the expected successful data is larger than the corresponding issuing upper limit or smaller than the issuing lower limit;
when the predicted value in the expected successful data is larger than the corresponding upper issuing limit, calculating a first difference value between the predicted value and the upper issuing limit, and adding one to the exceeding accumulated value; or when the predicted value is smaller than the corresponding lower issuing limit, calculating a second difference value between the lower issuing limit and the predicted value, and adding one to the exceeding accumulated value; the initial value of the exceeding accumulated value is zero;
And when the exceeding accumulated value is larger than a preset accumulated threshold, or the first difference value is larger than a first comparison threshold, or the second difference value is larger than a second comparison threshold, judging that the expected successful data is out of a preset normal interval.
7. The short message issuing abnormality detection device is characterized by comprising:
the feedback data receiving module is used for receiving feedback data of the short message pushing task, wherein the feedback data comprises short message sending success data of a plurality of operators;
the prediction expectation module is used for processing the feedback data through a multi-element linear prediction model to acquire expected success data; the multi-element linear prediction model comprises constant items and dimension factors, wherein one dimension parameter corresponds to one dimension factor;
the matching normal interval module is used for acquiring a preset normal interval matched with the short message pushing task and judging whether expected successful data are outside the preset normal interval;
the abnormal reminding module is used for sending out abnormal reminding if the expected successful data is out of the predicted normal interval;
the matching normal interval module comprises:
the keyword extraction unit is used for extracting the pushing keywords of the short message pushing task;
The matched task data unit is used for searching corresponding normal task data in a task database according to the pushing keywords, wherein the normal task data comprise task data of a plurality of historical normal pushing tasks;
and forming a normal interval unit, which is used for calculating the upper and lower issuing limits of each time point when the short message pushing task is executed according to the normal task data to form the preset normal interval.
8. A computer device comprising a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, wherein the processor, when executing the computer readable instructions, implements the short message delivery anomaly detection method of any one of claims 1 to 6.
9. One or more readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the short message delivery anomaly detection method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210043421.7A CN114401494B (en) | 2022-01-14 | 2022-01-14 | Short message issuing abnormality detection method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210043421.7A CN114401494B (en) | 2022-01-14 | 2022-01-14 | Short message issuing abnormality detection method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114401494A CN114401494A (en) | 2022-04-26 |
| CN114401494B true CN114401494B (en) | 2023-05-26 |
Family
ID=81230770
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210043421.7A Active CN114401494B (en) | 2022-01-14 | 2022-01-14 | Short message issuing abnormality detection method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114401494B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008113290A1 (en) * | 2007-03-16 | 2008-09-25 | Huawei Technologies Co., Ltd. | Method and device for pushing information |
| CN105323763A (en) * | 2014-06-27 | 2016-02-10 | 中国移动通信集团湖南有限公司 | Method and apparatus for identifying spam messages |
| CN110888788A (en) * | 2019-10-16 | 2020-03-17 | 平安科技（深圳）有限公司 | Anomaly detection method and device, computer equipment and storage medium |
| CN111460294A (en) * | 2020-03-31 | 2020-07-28 | 汉海信息技术(上海)有限公司 | Message pushing method and device, computer equipment and storage medium |
| CN112231181A (en) * | 2020-12-08 | 2021-01-15 | 平安科技（深圳）有限公司 | Data abnormal update detection method and device, computer equipment and storage medium |
-
2022
- 2022-01-14 CN CN202210043421.7A patent/CN114401494B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008113290A1 (en) * | 2007-03-16 | 2008-09-25 | Huawei Technologies Co., Ltd. | Method and device for pushing information |
| CN105323763A (en) * | 2014-06-27 | 2016-02-10 | 中国移动通信集团湖南有限公司 | Method and apparatus for identifying spam messages |
| CN110888788A (en) * | 2019-10-16 | 2020-03-17 | 平安科技（深圳）有限公司 | Anomaly detection method and device, computer equipment and storage medium |
| CN111460294A (en) * | 2020-03-31 | 2020-07-28 | 汉海信息技术(上海)有限公司 | Message pushing method and device, computer equipment and storage medium |
| CN112231181A (en) * | 2020-12-08 | 2021-01-15 | 平安科技（深圳）有限公司 | Data abnormal update detection method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114401494A (en) | 2022-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10805151B2 (en) | Method, apparatus, and storage medium for diagnosing failure based on a service monitoring indicator of a server by clustering servers with similar degrees of abnormal fluctuation | |
| CN110493190B (en) | Data information processing method and device, computer equipment and storage medium | |
| CN111259791B (en) | Nuclear power station material acceptance method and device, computer equipment and storage medium | |
| CN111475370A (en) | Operation and maintenance monitoring method, device and equipment based on data center and storage medium | |
| CN110489314B (en) | Model anomaly detection method and device, computer equipment and storage medium | |
| CN111314173B (en) | Monitoring information abnormity positioning method and device, computer equipment and storage medium | |
| CN111709794A (en) | Abnormal order processing method and device, computer equipment and storage medium | |
| CN111193609A (en) | Application abnormity feedback method and device and application abnormity monitoring system | |
| CN110532461B (en) | Information platform pushing method and device, computer equipment and storage medium | |
| CN110888911A (en) | Sample data processing method and device, computer equipment and storage medium | |
| CN111767192A (en) | Service data detection method, device, equipment and medium based on artificial intelligence | |
| WO2019019429A1 (en) | Anomaly detection method, device and apparatus for virtual machine, and storage medium | |
| CN110928859A (en) | Model monitoring method and device, computer equipment and storage medium | |
| CN114401494B (en) | Short message issuing abnormality detection method, device, computer equipment and storage medium | |
| CN116720084A (en) | Data identification method, device, electronic equipment and computer readable storage medium | |
| CN109542947B (en) | Data statistical method, device, computer equipment and storage medium | |
| CN111147441A (en) | Method and device for automatically detecting fraud behaviors of online ticket purchasing and readable storage medium | |
| CN110795308A (en) | Server inspection method, device, equipment and storage medium | |
| CN115237796A (en) | Link abnormity identification method and device, storage medium and electronic equipment | |
| CN116340934A (en) | Terminal abnormal behavior detection method, device, equipment and storage medium | |
| CN115147236A (en) | Processing method, processing device and electronic equipment | |
| CN114896955A (en) | Data report processing method and device, computer equipment and storage medium | |
| CN109829681B (en) | Software development monitoring method, device, computer equipment and storage medium | |
| CN113888027A (en) | Qualification supervision method and device, computer equipment and storage medium | |
| CN112631929A (en) | Test case generation method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |