CN114793205A - Abnormal link detection method, device, equipment and storage medium - Google Patents
Abnormal link detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114793205A CN114793205A CN202210441702.8A CN202210441702A CN114793205A CN 114793205 A CN114793205 A CN 114793205A CN 202210441702 A CN202210441702 A CN 202210441702A CN 114793205 A CN114793205 A CN 114793205A
- Authority
- CN
- China
- Prior art keywords
- abnormal
- link
- detection model
- sample
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 281
- 238000001514 detection method Methods 0.000 title claims abstract description 175
- 238000012549 training Methods 0.000 claims abstract description 74
- 238000000034 method Methods 0.000 claims abstract description 38
- 238000004364 calculation method Methods 0.000 claims abstract description 20
- 238000004891 communication Methods 0.000 claims description 20
- 230000005856 abnormality Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000002547 anomalous effect Effects 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 101100465000 Mus musculus Prag1 gene Proteins 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 108700041286 delta Proteins 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0811—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking connectivity
Abstract
The application discloses a method, a device, equipment and a storage medium for detecting an abnormal link, wherein the method comprises the following steps: detecting a real-time link sample based on a preset first detection model and a preset second detection model to respectively obtain a first abnormal value and a second abnormal value, wherein the first detection model is determined based on historical data training, and the second detection model is determined based on-line sample training before a current conference; performing weighted calculation on the first abnormal value and the second abnormal value to obtain a target abnormal value; and if the target abnormal value is larger than the abnormal threshold value, determining that the data link is abnormal, and sending an abnormal notification. The method comprises the steps of training a relatively universal offline isolated forest model by adopting historical data, training an individualized online isolated forest model corresponding to a user by adopting flow data of the user in a conference, and further obtaining a double isolated forest model.
Description
Technical Field
The present application relates to the field of video communication technologies, and in particular, to a method, an apparatus, a device, and a storage medium for detecting an abnormal link.
Background
The current abnormal link detection of audio and video communication (such as a video conference) mainly comprises a mode of marking a training supervision model on an abnormal sample and detecting and judging an abnormal threshold value of a single index, wherein the mode of training the supervision model requires manual marking of the sample, a large amount of manpower is consumed, the abnormal sample is not completely covered, so that the model is not accurate, the single abnormal threshold value judgment can only detect a simple scene, and the detection is not accurate due to the difficulty in adapting to a complex scene, so that the current abnormal link detection of the audio and video communication has the problem of low accuracy.
Disclosure of Invention
The present application mainly aims to provide a method, an apparatus, a device and a storage medium for detecting an abnormal link, and aims to solve the technical problem in the prior art that the detection of an abnormal link in audio and video communication has low accuracy.
In order to achieve the above object, the present application provides an abnormal link detection method, where the abnormal link detection method includes:
detecting a real-time link sample based on a preset first detection model and a preset second detection model to respectively obtain a first abnormal value and a second abnormal value, wherein the first detection model is determined based on historical data training, and the second detection model is determined based on-line sample training before a current conference;
performing weighted calculation on the first abnormal value and the second abnormal value to obtain a target abnormal value;
and if the target abnormal value is larger than the abnormal threshold value, determining that the data link is abnormal, and sending an abnormal notification.
Optionally, the detecting a real-time link sample based on a preset first detection model and a preset second detection model to obtain a first outlier and a second outlier, respectively, where the first detection model is determined based on historical data training, and before the step of determining based on online sample training before the current conference, the detecting a real-time link sample based on a preset first detection model and a preset second detection model includes:
acquiring a historical sample set and an initial detection model;
training the initial detection model based on the historical sample set, and stopping training when a training result meets an end condition to obtain a first model parameter;
and inputting the first model parameter into the initial detection model to obtain a first detection model.
Optionally, before the step of detecting the real-time link sample based on the preset first detection model and the preset second detection model to obtain the first abnormal value and the second abnormal value, the method includes:
obtaining online samples in a preset time period before a current conference to form an online sample set;
inputting the online sample set into the initial detection model for training, and stopping training when a training result meets an end condition to obtain a second model parameter;
and inputting the second model parameter into the initial detection model to obtain a second detection model.
Optionally, the step of performing weighted calculation on the first outlier and the second outlier to obtain a target outlier includes:
acquiring preset weighting parameters;
and performing weighted calculation on the first abnormal value and the second abnormal value based on a weighted parameter to obtain a target abnormal value.
Optionally, if the target abnormal value is greater than the abnormal threshold, determining that the data link is abnormal, and sending an abnormal notification, including;
acquiring an abnormal threshold;
comparing the target outlier to the outlier threshold;
and if the target abnormal value is larger than the abnormal threshold value, determining that the real-time link sample is an abnormal sample and the data link is abnormal, and sending an abnormal notice.
Optionally, after the step of determining that the real-time link sample is an abnormal sample and the data link has an abnormality if the target abnormal value is greater than the abnormal threshold, and sending an abnormality notification, the method includes:
acquiring link characteristic information in a communication link;
and outputting an exception processing result corresponding to the exception state based on the exception state of the link characteristic information.
Optionally, the first detection model is an offline isolated forest model and the second detection model is an online isolated forest model.
The present application also provides an abnormal link detection apparatus, including:
the detection module is used for detecting a real-time link sample based on a preset first detection model and a preset second detection model to respectively obtain a first abnormal value and a second abnormal value, the first detection model is determined based on historical data training, and the second detection model is determined based on-line sample training before a current conference;
the calculation module is used for performing weighted calculation on the first abnormal value and the second abnormal value to obtain a target abnormal value;
and the judging module is used for determining that the data link is abnormal and sending an abnormal notification if the target abnormal value is greater than the abnormal threshold.
The present application further provides an abnormal link detection apparatus, including: a memory, a processor, and a program stored on the memory for implementing the abnormal link detection method,
the memory is used for storing a program for realizing the abnormal link detection method;
the processor is configured to execute a program for implementing the abnormal link detection method, so as to implement the steps of the abnormal link detection method.
The present application provides a storage medium, and the storage medium stores one or more programs, and the one or more programs are further executable by one or more processors for implementing the steps of the abnormal link detection method described in any one of the above.
Compared with the low detection accuracy of the audio and video communication abnormal link in the prior art, the method, the device, the equipment and the storage medium for detecting the abnormal link provided by the application detect the real-time link sample based on the preset first detection model and the preset second detection model to respectively obtain a first abnormal value and a second abnormal value, wherein the first detection model is determined based on historical data training, and the second detection model is determined based on-line sample training before the current conference; performing weighted calculation on the first abnormal value and the second abnormal value to obtain a target abnormal value; and if the target abnormal value is larger than the abnormal threshold value, determining that the data link is abnormal, and sending an abnormal notification. And training a relatively generalized offline isolated forest model by adopting historical data, and training an individualized online isolated forest model corresponding to a user by adopting flow data of the user in a conference so as to obtain a double isolated forest model. And in the process that the user participates in the conference, the trained double isolated forest models are used for participating in the abnormal recognition together, so that the accuracy of the abnormal recognition is ensured.
Drawings
Fig. 1 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a first embodiment of an abnormal link detection method according to the present application;
fig. 3 is a schematic view of a video conference architecture according to a first embodiment of the abnormal link detection method of the present application;
fig. 4 is a functional block diagram of the abnormal link detection apparatus according to the present application.
The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
As shown in fig. 1, fig. 1 is a schematic terminal structure diagram of a hardware operating environment according to an embodiment of the present application.
The terminal in the embodiment of the application may be a PC, or may be a mobile terminal device having a display function, such as a smart phone, a tablet computer, an e-book reader, an MP3(Moving Picture Experts Group Audio Layer III, motion Picture Experts compress standard Audio Layer 3) player, an MP4(Moving Picture Experts Group Audio Layer IV, motion Picture Experts compress standard Audio Layer 3) player, a portable computer, or the like.
As shown in fig. 1, the terminal may include: a processor 1001, e.g. a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Optionally, the terminal may further include a camera, a Radio Frequency (RF) circuit, a sensor, an audio circuit, a WiFi module, and the like. Such as light sensors, motion sensors, and other sensors, among others. Specifically, the light sensor may include an ambient light sensor that adjusts the brightness of the display according to the brightness of ambient light, and a proximity sensor that turns off the display and/or the backlight when the mobile terminal moves to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally, three axes), detect the magnitude and direction of gravity when the mobile terminal is stationary, and can be used for applications (such as horizontal and vertical screen switching, related games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer and tapping) and the like for recognizing the attitude of the mobile terminal; of course, the mobile terminal may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which are not described herein again.
Those skilled in the art will appreciate that the terminal structure shown in fig. 1 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a network operation control application program.
In the terminal shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be used to invoke a network operation control application stored in the memory 1005.
Referring to fig. 2, an embodiment of the present application provides an abnormal link detection method, where the abnormal link detection method includes:
step S100, detecting a real-time link sample based on a preset first detection model and a preset second detection model to respectively obtain a first abnormal value and a second abnormal value, wherein the first detection model is determined based on historical data training, and the second detection model is determined based on-line sample training before a current conference;
step S200, performing weighted calculation on the first abnormal value and the second abnormal value to obtain a target abnormal value;
step S300, if the target abnormal value is larger than the abnormal threshold, determining that the data link is abnormal, and sending an abnormal notification.
The method comprises the following specific steps:
step S100, detecting a real-time link sample based on a preset first detection model and a preset second detection model to respectively obtain a first abnormal value and a second abnormal value, wherein the first detection model is determined based on historical data training, and the second detection model is determined based on-line sample training before a current conference;
in this embodiment, it should be noted that the abnormal link detecting method belongs to an abnormal link detecting system, and the abnormal link detecting system belongs to an abnormal link detecting device.
In this embodiment, the specific application scenarios may be:
the current abnormal link detection of audio and video communication (such as a video conference) mainly comprises a mode of marking a training monitoring model for an abnormal sample and a mode of detecting and judging an abnormal threshold value of a single index, wherein the mode of training the monitoring model requires a large amount of manpower for manually marking the sample, the abnormal sample is not covered completely to cause model inaccuracy, the single abnormal threshold value judgment can only detect a simple scene, and the detection is not accurate due to the difficulty in adapting to a complex scene, so that the current abnormal link detection of the audio and video communication has the problem of low accuracy.
In the application, for an abnormal link detection system, a real-time link sample is detected based on a preset first detection model and a preset second detection model, and a first abnormal value and a second abnormal value are respectively obtained, wherein the first detection model is determined based on historical data training, and the second detection model is determined based on online sample training before a current conference; performing weighted calculation on the first abnormal value and the second abnormal value to obtain a target abnormal value; if the target abnormal value is larger than the abnormal threshold value, determining that the data link is abnormal, and sending an abnormal notification, namely, identifying abnormal flow in the real-time link from different angles through the two prediction models to obtain a first abnormal value and a second abnormal value, realizing the detection of the abnormal sample in the real-time link sample through the weighting of the two abnormal values, and improving the accuracy of the detection of the abnormal link.
Namely, in the application, the abnormal flow is identified from different angles through the dual models, and the problem of low accuracy in the detection of the abnormal audio and video communication link is solved.
In this embodiment, it should be noted that the first detection model and the second detection model are models for detecting and identifying abnormal traffic in the communication link, the first detection model is for detecting sample data with universal distribution, the second detection model is for detecting individual distribution of a single conference sample or strong correlation of a single sending user receiving end sample, it is to be appreciated that the first detection model can be an offline orphan forest model, the second detection model can be an online orphan forest model, wherein, the isolated Forest model is composed of N trees and is used for excavating abnormal (Anomaly) data, or outlier mining, it can be understood that data that does not conform to the rule of other data is found in a large pile of data, and is generally used for analysis such as attack detection and traffic anomaly in network security. The abnormal flow can be detected and identified from different angles through the two isolated forest models. In the following, the first detection model may be an offline isolated forest model, the second detection model may be an online isolated forest model, and the embodiments of the other detection models are substantially the same and are not described herein again.
Referring to fig. 3, in a video conference architecture diagram based on an SFU (selective forwarding unit), a user a pushes audio and video streams to an SFU server directly linked to the user a, and the server forwards the audio and video streams of the user a to the SFU server linked to the user in the same conference through SFU server cascade connection, and then pushes other users. Obtaining indexes such as bit number, bit rate, packet data, packet loss number, nack request number, pli request number, fir request number, frame rate, resolution ratio and the like of a transmitting or receiving data stream in a period T (generally 10s, 20s or 1min) as index characteristics of link quality, wherein the indexes are
s (i) link characteristic information (collected by a client end of a sending end) representing a sending flow of a user i;
fs (i) indicates that SFU service directly linked by user i receives stream link characteristic information (collected by SFU server);
fr (i, j) indicates that SFU service directly linked by the user j receives link characteristic information (collected by an SFU server) of a user i sending stream;
r (i, j) represents that the user j receives the link characteristic information of the stream sent by the user i (the client of the receiving end carries out embedded point acquisition);
as the meaning of the feature vector is known, s (i), fs (i), fr (i, j) and r (i, j) have significant correlation, so the set { fr (i,j) ,j∈M},{r (i,j) Correlation between samples within j ∈ M }The strong, representing the characteristic distribution of the receiving end of the user i sending stream;
set { s } (i) ,i∈M}，{fs (i) ,i∈M}，{fr (i,j) ,j∈M i∈M}，{r (i,j) J belongs to M i and belongs to a certain correlation between samples in M, which represents the characteristic distribution of the sample data in the same conference; where M represents a set of users in a conference.
Further, the detecting a real-time link sample based on a preset first detection model and a preset second detection model to obtain a first abnormal value and a second abnormal value, respectively, where the first detection model is determined based on historical data training, and the second detection model is determined based on online sample training before a current conference, and before the step of determining, the method includes:
step A1, acquiring a historical sample set and an initial detection model;
step A2, training the initial detection model based on the historical sample set, and stopping training when the training result meets the end condition to obtain a first model parameter;
step A3, inputting the first model parameter into the initial detection model to obtain a first detection model.
In the embodiment, a history sample set is formed based on offline data in history, and an isolated forest MODEL MODEL is created through the history sample set, wherein the isolated forest MODEL is a tree-shaped integrated MODEL formed by more than one isolated tree, and all nodes of each isolated tree have 2 child nodes or no child nodes; given a set of n samples X { X1, X2, …, xn }, grouped according to the frame rate fps, resolution w and h of the samples, forming a set of samples G, and for each set of samples, recursively selecting a set of samples X by randomly selecting a feature q of the data set and randomly selecting a split value p of the feature, thereby creating an isolated tree.
And performing model training on the initial detection model through the historical sample set, and stopping training when a training result meets an ending condition. Wherein the end condition may be that the depth of the orphan tree reaches a defined maximum; the nodes of the isolated tree have only one sample after a certain recursion; or after a certain recursionThe node data of the isolated tree all have the same value, and when the training result meets one of the ending conditions, the process of recursively establishing the isolated book is stopped. Therefore, each group of historical sample sets is sampled for t times, and an isolated number iTree is established for a part of data in each sampling (fps,w,h) And (5) sampling t times to establish t classes of isolated trees, and finally forming an isolated forest by G x t isolated trees.
After model training is finished, obtaining first model parameters such as sample data set characteristics q1 and characteristic split values p1, and establishing an isolated tree by recursion of a historical sample set X through the sample data set characteristics q1 and the characteristic split values p1, wherein the obtained offline isolated forest model is as follows:
wherein fps is the frame rate of the sample, w × h is the resolution, and μ 1 and μ 2 are the weighting coefficients of the above formula.
In addition, μ 1 and μ 2 are empirical values, and are obtained by historical training, and when it is detected that the target abnormal values obtained based on μ 1 and μ 2 are low in accuracy, the optimal adjustment is performed on μ 1 and μ 2. It can be understood that whether the difference is beyond the error range is judged by the difference between the target abnormal value and the actual abnormal value, and if so, the optimal adjustment on μ 1 and μ 2 is needed.
Therefore, when the real-time link sample is detected through the first detection model, the specific process of obtaining the first abnormal value is as follows:
path length h of leaf node x in isolated tree (fps,w,h) (x) Given a data set comprising n samples for which orphan trees are established, the number of edges that run from the root node of the orphan tree through to the leaf node where x is located, the average path length c (n) of an orphan tree is:
where h (i) is a harmonic number, this value may be h (i) ═ ln (i) + 0.577215664.
When the sample size is fixed to n, the average path length c (n) of different isolated trees is the same, and is used to normalize the path length h (x) of the sample x, and the abnormal score calculation formula of the sample x is as follows:
based on the above formula (1) and formula (2), a first abnormal value score is obtained offline . It is understood that the solitary forest is composed of n solitary trees, and for a certain sample x, the leaf node x of each solitary tree has a path length h (x), F (h (x)) is a weighted average of the path lengths h (x) of the x samples in different solitary trees in the solitary forest, and c (n) is an average path length of the solitary trees including the sample. Therefore, the larger the value of the abnormality score s (x, n) is, the more abnormal, that is, the larger the first abnormal value is.
Further, before the step of detecting the real-time link sample based on the preset first detection model and the preset second detection model to obtain the first abnormal value and the second abnormal value, the method includes:
step B1, acquiring online samples in a preset time period before the current conference to form an online sample set;
step B2, inputting the online sample set into the initial detection model for training, and stopping training when a training result meets an end condition to obtain a second model parameter;
and step B3, inputting the second model parameters into the initial detection model to obtain a second detection model.
In this embodiment, online samples in a period of time before a current conference of a communication link are obtained to form an online sample set, and the initial detection model is trained by using the online sample set to obtain a second detection model. In the training process, the training is stopped when the training result meets the end condition, and the end condition is the same as the end condition in the training of the first detection model, namely the end condition is the training end condition of the isolated forest model. After training is finished, second model parameters such as sample data set characteristics q2 and characteristic split values p2 are obtained, and a recursion history sample set X is obtained through the sample data set characteristics q1 and the characteristic split values p1, so that an isolated tree is established, and an online isolated forest model is obtained.
Therefore, the specific process of obtaining the second abnormal value by detecting the real-time link sample through the second detection model is as follows:
inputting the real-time link sample into the online isolated forest model to obtain F (h (x)) in the online isolated forest model, and calculating a second abnormal value through the following sample abnormal score formula:
score line ＝δ 1 *s offline (x,n)+δ 2 *s line (x,n)……(3)
wherein, delta 1 and delta 2 are weighting coefficients of the above formula, delta 1 +δ 2 ＝1,δ 1 <δ 2 ，s offline (x, n) is an off-line isolated forest model anomaly scoring function, s line (x, n) is an online isolated forest model anomaly scoring function, and scoreline is a second anomaly value.
It should be noted that the value of the abnormal value score is between (0-1), and the samples with larger abnormal values are more abnormal.
Step S200, performing weighted calculation on the first abnormal value and the second abnormal value to obtain a target abnormal value;
in this embodiment, the first abnormal value and the second abnormal value are weighted to obtain the target abnormal value, and the weighting formula is as follows:
where score (x) is the target outlier, and φ 1, φ 2 are the weighting coefficients of the above equation.
Step S300, if the target abnormal value is larger than the abnormal threshold, determining that the data link is abnormal, and sending an abnormal notification.
In this embodiment, an anomaly threshold set by a manager according to the detection requirement is obtained, and when the target anomaly value is greater than the anomaly threshold, it can be determined that the data link is anomalous, and an anomaly notification is sent, where the anomaly notification may be sent by sound, light, signal, or the like. If the target abnormal value is smaller than or equal to the abnormal threshold value, the current link is normal, and the link is continuously monitored.
Compared with the low detection accuracy of the audio and video communication abnormal link in the prior art, the method, the device, the equipment and the storage medium for detecting the abnormal link provided by the application detect the real-time link sample based on the preset first detection model and the preset second detection model to respectively obtain a first abnormal value and a second abnormal value, wherein the first detection model is determined based on historical data training, and the second detection model is determined based on-line sample training before the current conference; performing weighted calculation on the first abnormal value and the second abnormal value to obtain a target abnormal value; and if the target abnormal value is larger than the abnormal threshold value, determining that the data link is abnormal, and sending an abnormal notification. And training a relatively universal offline isolated forest model by using historical data, and training an individualized online isolated forest model corresponding to a user by using flow data of the user in the conference to further obtain a double isolated forest model. In the process that a user participates in a conference, the trained double isolated forest models are used for participating in abnormal recognition together, and therefore accuracy of the abnormal recognition is guaranteed.
The present application provides a second embodiment based on the first embodiment of the above abnormal link detection method, where the abnormal link detection method includes:
the second detection model comprises a first detection submodel based on a conference and a second detection submodel based on a user, the online sample set comprises a sending sample and a receiving sample, the initial detection model is trained based on the sending sample to obtain a first detection submodel, the initial detection model is trained based on the receiving sample to obtain a second detection submodel, abnormal values based on individual distribution of a single conference sample and abnormal values based on strong correlation detection of the sending user receiving end sample are respectively obtained through the first detection submodel and the second detection submodel, and the two abnormal values and the first abnormal value are subjected to weighted calculation to obtain a target abnormal value, and the specific process is as follows:
for transmitted samples s (i) ，fs (i) Obtaining a sending sample of a period of time before a current conference, and training an online isolated forest model room Then, the sample anomaly score is:
score room ＝∝ 1 *s offline (x,n)+∝ 2 *s room (x,n)
wherein ℃ - 1 +∝ 2 ＝1,∝ 1 <∝ 2 ，s offline (x, n) is an off-line isolated forest model anomaly scoring function, s room (x, n) an anomaly scoring function based on a room online isolated forest model;
for received samples r (i,j) ,fr (i,j) Obtaining a receiving sample of a period of time before a current conference, and training an on-line solitary Senry model room Obtaining a receiving sample of a user i sending stream, and training an online isolated forest model user Then, the sample anomaly score is:
score user ＝γ 1 *s offline (x,n)+γ 2 *s room (x,n)+γ 3 *s user (x,n)
wherein gamma is 1 +γ 2 +γ 3 ＝1,γ 1 <γ 2 <γ 3 ，s offline (x, n) is an off-line isolated forest model anomaly scoring function, s room (x, n) an anomaly scoring function based on a room online isolated forest model; s is user (x, n) is based on a send user online isolated forest model anomaly scoring function.
The above equation (4) for calculating the target abnormal value becomes:
score(x)＝β 1 score offline (x,n 1 )+β 2 score room (x,n 2 )+β 3 score user (x,n 3 )
and comparing the target abnormal value score (x) with an abnormal threshold, if the target abnormal value is greater than the abnormal threshold, judging that the data link is abnormal, and sending an abnormal notification.
In the embodiment, the relevance among sample data is fully considered, the sample data universality distribution is detected through an offline isolated forest model, the individual distribution of a single conference sample and the strong relevance of a single sending user receiving end sample are detected through an online model, and the offline model and the online model are used for weighting to perform anomaly detection, so that the detection accuracy is improved.
The present application further provides a third embodiment based on the first embodiment or the second embodiment of the above abnormal link detection method, where the abnormal link detection method includes:
if the target abnormal value is greater than the abnormal threshold value, determining that the real-time link sample is an abnormal sample, and the data link is abnormal, and sending an abnormal notification, wherein the method comprises the following steps:
step S400, link characteristic information in a communication link is obtained;
step S500, outputting an exception handling result corresponding to the exception state based on the exception state of the link characteristic information.
In this embodiment, the sending link s is acquired while receiving the notification of the detection abnormality () 、fs () Fr (i, j), based on the abnormal state of the link characteristic information, an abnormal processing result corresponding to the abnormal state is output.
As an example, if the transmission link s (i) ，fs (i) Detecting the abnormal state and continuously abnormal state in N continuous periods, which indicates that the SFU Server connected by the user i is not optimal, and informing a scheduling system to switch the SFU Server connected by the user i across machine rooms;
as an example, if the transmission link s (i) ，fs (i) If fr (i, j) plug flow link is abnormal, indicating that cascade connection between SFU servers is abnormal, and alarming to inform operation and maintenance personnel to perform problem troubleshooting;
as an example, if the fr (i, j) push flow link is normal, the user pull flow r (i, j) link is abnormal, and the link continues to be abnormal within N periods, it is indicated that the SFU Server connected to the receiving end user j is not optimal, and the scheduling system is notified to perform cross-machine switching on the SFU Server connected to the user j;
in this embodiment, when sending the exception notification, the specific data transmission problem in the abnormal data link is sent out, and a corresponding exception handling method or result is given, so as to improve the efficiency of detecting the abnormal link.
In a specific scenario, the abnormal link detection process is as follows:
a. training isolated forest models using offline historical data offline ；
b. Let M denote the set of users, samples, in a conference
Representing the time t, and receiving the audio and video characteristic information sent by the user i by the user j; for the sample
The real-time detection steps are as follows:
step b1, acquiring a sample set sent and received by the non-user i1 in the first y periods
Training a conference-based isolated forest modelrom;
step b2. obtains the sample set received by the user i1 sending non-user j1 in the previous y periods
Training model based on user isolated forest user ；
c. Computing samples
Abnormal value of (a):
samples are respectively passed through model offline 、model room 、model user Calculating outWeighting the abnormal value to obtain a target abnormal value:
score(x)＝γ 1 score offline (x,n 1 )+γ 1 score room (x,n 2 )+γ 1 score user (x,n 3 )
if score is greater than the anomaly threshold ε, then the sample is an anomalous sample.
The present application further provides an abnormal link detection device, including: a memory, a processor, and a program stored on the memory for implementing the abnormal link detection method,
the memory is used for storing a program for realizing the abnormal link detection method;
the processor is configured to execute a program for implementing the abnormal link detection method, so as to implement the steps of the abnormal link detection method.
The specific implementation of the abnormal link detection device of the present application is substantially the same as that of each embodiment of the abnormal link detection method described above, and is not described herein again.
The present application further provides an abnormal link detection apparatus, referring to fig. 4, the abnormal link detection apparatus includes:
the detection module 10 is configured to detect a real-time link sample based on a preset first detection model and a preset second detection model, and obtain a first abnormal value and a second abnormal value respectively, where the first detection model is determined based on historical data training, and the second detection model is determined based on online sample training before a current conference;
a calculation module 20, configured to perform weighted calculation on the first abnormal value and the second abnormal value to obtain a target abnormal value;
and the judging module 30 is configured to determine that the data link is abnormal and send an abnormal notification if the target abnormal value is greater than the abnormal threshold.
The specific implementation of the abnormal link detection apparatus of the present application is substantially the same as that of each of the above-mentioned embodiments of the abnormal link detection method, and is not described herein again.
The present application provides a storage medium, and the storage medium stores one or more programs, and the one or more programs are further executable by one or more processors for implementing the steps of the abnormal link detection method described in any one of the above.
The specific implementation of the storage medium of the present application is substantially the same as the embodiments of the above abnormal link detection method, and is not described herein again.
The present application provides a storage medium, and the storage medium stores one or more programs, and the one or more programs are further executable by one or more processors for implementing the steps of the abnormal link detection method described in any one of the above.
The specific implementation of the storage medium of the present application is substantially the same as that of each embodiment of the above-mentioned abnormal link detection method, and is not described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application or portions thereof contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above and includes several instructions for enabling a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method described in the embodiments of the present application.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are included in the scope of the present application.
Claims (10)
1. An abnormal link detection method, characterized in that the abnormal link detection method comprises:
detecting a real-time link sample based on a preset first detection model and a preset second detection model to respectively obtain a first abnormal value and a second abnormal value, wherein the first detection model is determined based on historical data training, and the second detection model is determined based on-line sample training before a current conference;
performing weighted calculation on the first abnormal value and the second abnormal value to obtain a target abnormal value;
and if the target abnormal value is larger than the abnormal threshold value, determining that the data link is abnormal, and sending an abnormal notification.
2. The method for detecting the abnormal link according to claim 1, wherein the detecting the real-time link sample based on a preset first detection model and a preset second detection model to obtain a first abnormal value and a second abnormal value respectively, the first detection model is determined based on historical data training, and the second detection model is determined based on online sample training before the current conference before the step of determining, includes:
acquiring a historical sample set and an initial detection model;
training the initial detection model based on the historical sample set, and stopping training when a training result meets an end condition to obtain a first model parameter;
and inputting the first model parameter into the initial detection model to obtain a first detection model.
3. The method for detecting an abnormal link according to claim 1, wherein the step of detecting the real-time link sample based on the preset first detection model and the preset second detection model to obtain the first abnormal value and the second abnormal value is preceded by the steps of:
acquiring online samples in a preset time period before a current conference to form an online sample set;
inputting the online sample set into the initial detection model for training, and stopping training when a training result meets an end condition to obtain a second model parameter;
and inputting the second model parameter into the initial detection model to obtain a second detection model.
4. The method of claim 1, wherein the step of performing a weighted calculation on the first outlier and the second outlier to obtain a target outlier comprises:
acquiring a preset weighting parameter;
and performing weighted calculation on the first abnormal value and the second abnormal value based on a weighted parameter to obtain a target abnormal value.
5. The method for detecting an abnormal link according to claim 1, wherein if the target abnormal value is greater than the abnormal threshold, the step of determining that the data link is abnormal and sending an abnormal notification includes;
acquiring an abnormal threshold;
comparing the target outlier to the outlier threshold;
and if the target abnormal value is larger than the abnormal threshold value, determining that the real-time link sample is an abnormal sample and the data link is abnormal, and sending an abnormal notice.
6. The method for detecting an abnormal link according to claim 1, wherein after the step of determining that the real-time link sample is an abnormal sample and the data link has an abnormality if the target abnormal value is greater than the abnormal threshold, and sending an abnormality notification, the method comprises:
acquiring link characteristic information in a communication link;
and outputting an exception processing result corresponding to the exception state based on the exception state of the link characteristic information.
7. The abnormal link detection method of claim 1, wherein the first detection model is an offline isolated forest model and the second detection model is an online isolated forest model.
8. An abnormal link detecting apparatus, comprising:
the detection module is used for detecting a real-time link sample based on a preset first detection model and a preset second detection model to respectively obtain a first abnormal value and a second abnormal value, the first detection model is determined based on historical data training, and the second detection model is determined based on-line sample training before a current conference;
the calculation module is used for performing weighted calculation on the first abnormal value and the second abnormal value to obtain a target abnormal value;
and the judging module is used for determining that the data link is abnormal and sending an abnormal notice if the target abnormal value is greater than the abnormal threshold.
9. An abnormal link detecting apparatus, characterized in that the abnormal link detecting apparatus comprises: a memory, a processor, and an abnormal link detection program stored on the memory and executable on the processor, the abnormal link detection program configured to implement the steps of the abnormal link detection method of any one of claims 1 to 7.
10. A storage medium having stored thereon an abnormal link detection program configured to implement the steps of the abnormal link detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210441702.8A CN114793205A (en) | 2022-04-25 | 2022-04-25 | Abnormal link detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210441702.8A CN114793205A (en) | 2022-04-25 | 2022-04-25 | Abnormal link detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114793205A true CN114793205A (en) | 2022-07-26 |
Family
ID=82462645
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210441702.8A Pending CN114793205A (en) | 2022-04-25 | 2022-04-25 | Abnormal link detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114793205A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180248905A1 (en) * | 2017-02-24 | 2018-08-30 | Ciena Corporation | Systems and methods to detect abnormal behavior in networks |
| CN108777873A (en) * | 2018-06-04 | 2018-11-09 | 江南大学 | The wireless sensor network abnormal deviation data examination method of forest is isolated based on weighted blend |
| US20190036795A1 (en) * | 2017-07-27 | 2019-01-31 | Verizon Patent And Licensing Inc. | Method and system for proactive anomaly detection in devices and networks |
| CN110807488A (en) * | 2019-11-01 | 2020-02-18 | 北京芯盾时代科技有限公司 | Anomaly detection method and device based on user peer-to-peer group |
| CN113032774A (en) * | 2019-12-25 | 2021-06-25 | 中移动信息技术有限公司 | Training method, device and equipment of anomaly detection model and computer storage medium |
| CN113656452A (en) * | 2021-07-28 | 2021-11-16 | 北京宝兰德软件股份有限公司 | Method and device for detecting abnormal index of call chain, electronic equipment and storage medium |
| CN113762350A (en) * | 2021-08-12 | 2021-12-07 | 南京星云数字技术有限公司 | Abnormal data detection method and device, computer equipment and storage medium |
| US20220086179A1 (en) * | 2020-09-12 | 2022-03-17 | Microsoft Technology Licensing, Llc | Service access data enrichment for cybersecurity |
| WO2022057321A1 (en) * | 2020-09-17 | 2022-03-24 | 华为技术有限公司 | Method and apparatus for detecting anomalous link, and storage medium |
-
2022
- 2022-04-25 CN CN202210441702.8A patent/CN114793205A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180248905A1 (en) * | 2017-02-24 | 2018-08-30 | Ciena Corporation | Systems and methods to detect abnormal behavior in networks |
| US20190036795A1 (en) * | 2017-07-27 | 2019-01-31 | Verizon Patent And Licensing Inc. | Method and system for proactive anomaly detection in devices and networks |
| CN108777873A (en) * | 2018-06-04 | 2018-11-09 | 江南大学 | The wireless sensor network abnormal deviation data examination method of forest is isolated based on weighted blend |
| CN110807488A (en) * | 2019-11-01 | 2020-02-18 | 北京芯盾时代科技有限公司 | Anomaly detection method and device based on user peer-to-peer group |
| CN113032774A (en) * | 2019-12-25 | 2021-06-25 | 中移动信息技术有限公司 | Training method, device and equipment of anomaly detection model and computer storage medium |
| US20220086179A1 (en) * | 2020-09-12 | 2022-03-17 | Microsoft Technology Licensing, Llc | Service access data enrichment for cybersecurity |
| WO2022057321A1 (en) * | 2020-09-17 | 2022-03-24 | 华为技术有限公司 | Method and apparatus for detecting anomalous link, and storage medium |
| CN113656452A (en) * | 2021-07-28 | 2021-11-16 | 北京宝兰德软件股份有限公司 | Method and device for detecting abnormal index of call chain, electronic equipment and storage medium |
| CN113762350A (en) * | 2021-08-12 | 2021-12-07 | 南京星云数字技术有限公司 | Abnormal data detection method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108234171B (en) | Data processing method, system and device | |
| CN111461548B (en) | Intelligent quality monitoring system and method based on block chain and readable storage medium | |
| WO2020185233A1 (en) | Responding to machine learning requests from multiple clients | |
| CN107688950B (en) | After-sale consultation method for household appliances, electronic equipment and computer readable storage medium | |
| WO2012142797A1 (en) | Video monitoring system and method | |
| US8559728B2 (en) | Image processing apparatus and image processing method for evaluating a plurality of image recognition processing units | |
| US20120115447A1 (en) | System and method for providing safety content service | |
| WO2022247519A1 (en) | Object model device management method and apparatus, device, system, and storage medium | |
| CN112463422A (en) | Internet of things fault operation and maintenance method and device, computer equipment and storage medium | |
| CN113595784A (en) | Network flow detection method, device, equipment, storage medium and program product | |
| CN112135119A (en) | Method and system for automatically monitoring and alarming network condition in real-time audio and video communication | |
| CN110008926B (en) | Method and device for identifying age | |
| CN113596001B (en) | DDoS attack detection method, device, equipment and computer readable storage medium | |
| CN114793205A (en) | Abnormal link detection method, device, equipment and storage medium | |
| CN115766401B (en) | Industrial alarm information analysis method and device, electronic equipment and computer medium | |
| CN114499983B (en) | Tor flow detection method and device, terminal equipment and storage medium | |
| CN116744357A (en) | Base station fault prediction method, device, equipment and medium | |
| CN109754319B (en) | Credit score determination system, method, terminal and server | |
| CN110650051A (en) | Self-service terminal monitoring method and system and computer readable storage medium | |
| KR20190075558A (en) | Terminal apparatus and controlling method of the terminal apparatus | |
| CN114627556A (en) | Motion detection method, motion detection device, electronic apparatus, and storage medium | |
| Hang et al. | Design and implementation of intelligent fire notification service using IP camera in smart home | |
| CN112333045A (en) | Intelligent flow baseline learning method, equipment and computer readable storage medium | |
| CN112883816A (en) | Information pushing method and device | |
| CN111818548A (en) | Data processing method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |