CN112256519A - Data flow abnormity monitoring method and device, electronic equipment and storage medium - Google Patents
Data flow abnormity monitoring method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112256519A CN112256519A CN202010965607.9A CN202010965607A CN112256519A CN 112256519 A CN112256519 A CN 112256519A CN 202010965607 A CN202010965607 A CN 202010965607A CN 112256519 A CN112256519 A CN 112256519A
- Authority
- CN
- China
- Prior art keywords
- abnormal
- connection information
- tree
- data flow
- time interval
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000012544 monitoring process Methods 0.000 title claims abstract description 39
- 230000002159 abnormal effect Effects 0.000 claims abstract description 187
- 238000012216 screening Methods 0.000 claims abstract description 41
- 230000005856 abnormality Effects 0.000 claims description 11
- 239000013598 vector Substances 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 13
- 238000010276 construction Methods 0.000 description 3
- 238000000513 principal component analysis Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009223 counseling Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000002407 reforming Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to the technical field of data flow abnormity monitoring, in particular to a data flow abnormity monitoring method and device, electronic equipment and a storage medium. The monitoring method comprises the following steps: generating region connection information from the data stream information; screening abnormal region connection information according to the region connection information; generating an abnormal connection tree according to the abnormal region connection information; searching an abnormal subtree with the occurrence frequency higher than a preset abnormal threshold value in the abnormal connection tree, locking a data flow abnormal area by the abnormal connection tree and the abnormal subtree, and further obtaining an accidental traffic abnormal situation and a continuous traffic abnormal situation which occur in the area within a period of time, wherein the former reflects the influence of some emergency events on the traffic situation of the city or the area, and the latter reflects the problems existing in the current traffic planning of the city or the area.
Description
Technical Field
The invention relates to the technical field of data flow abnormity monitoring, in particular to a data flow abnormity monitoring method and device, electronic equipment and a storage medium.
Background
In a city, the development level of the road reflects the development level of the city to a certain extent, and in recent years, more and more cities start to perform road improvement and road construction. However, some cities fall into a strange circle of road planning and reforming, and a great amount of manpower and material resources are put into the urban road construction, but the road traffic problem cannot be fundamentally solved. The road construction consumes long time, occupies large scale, has huge investment cost, and if the final income is very little, the cost of loss is very huge.
The abnormal trajectory pattern of the moving objects on the roads usually reflects abnormal traffic flow on the road network, which may be caused by non-periodic events (abnormal events), such as celebration, parade, counseling, traffic control, and traffic jam. At present, many efforts exist in the industry for monitoring abnormal values, for example, Principal Component Analysis (PCA) is widely used for monitoring abnormal values in networks, and since PCA does not consider the difference of different object volumes and is highly dependent on the setting of data parameters, some large abnormal values may cause the calculated result to be far from the expected result.
In practice, the inventors found that the above prior art has the following disadvantages:
the monitoring method provided in the prior art can not substantially improve the urban road planning problem.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide a method, an apparatus, an electronic device and a storage medium for monitoring data stream anomalies, wherein the adopted technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a method for monitoring data flow anomalies, where the method includes the following steps:
generating region connection information from the data stream information;
screening abnormal region connection information according to the region connection information;
generating an abnormal connection tree according to the abnormal region connection information;
and searching an abnormal subtree with the occurrence frequency higher than a preset abnormal threshold value in the abnormal connection tree, and locking the abnormal area of the data flow by the abnormal connection tree and the abnormal subtree.
Further, the method for screening abnormal regional connection information according to the regional connection information comprises the following steps:
respectively calculating Euclidean distances between the region connection information in the current time interval and the feature vectors of the region connection information in corresponding time intervals in a plurality of adjacent continuous periods to obtain the minimum distance in all the Euclidean distances;
and taking the area connection information corresponding to the minimum distance larger than a preset minimum distance threshold value in each time interval as abnormal area connection information.
Further, the method for screening abnormal regional connection information according to the regional connection information comprises the following steps:
for each time interval, establishing a space rectangular coordinate system taking # Obj, Pcoto and Pctd as axes;
calculating mahalanobis distances between the region connection information points in the coordinate system;
screening extreme values in the coordinate system, and taking area connection information corresponding to the extreme values as abnormal area connection information;
where # Obj represents the traffic volume passing through the Link in the current sub-interval, Pcto represents the proportion of the traffic volume passing through the Link to the traffic volume traveling out from the start point of the Link in the current sub-interval, and Pctd represents the proportion of the traffic volume passing through the Link to the traffic volume traveling into the end point of the Link in the current sub-interval.
Further, the method for generating the abnormal connection tree according to the abnormal region connection information comprises the following steps:
traversing the abnormal region connection information according to a time sequence, wherein each time the abnormal region connection information in a current time interval is traversed, the abnormal region connection information in the next time interval is correspondingly traversed;
and when the abnormal region connection information in the current time interval and the abnormal region connection information in the next time interval meet the condition of the abnormal connection tree, establishing the abnormal connection tree in a recursive mode.
In a second aspect, an embodiment of the present invention provides a device for monitoring data flow anomalies, where the device for monitoring data flow anomalies includes:
the data generation module is used for generating the area connection information from the data stream information;
the abnormal screening module is used for screening abnormal region connection information according to the region connection information;
the abnormal connection tree generation module is used for generating an abnormal connection tree according to the abnormal region connection information;
and the abnormal locking module is used for searching an abnormal sub-tree with the occurrence frequency higher than a preset abnormal threshold value in the abnormal connection tree, and locking the abnormal area of the data flow by the abnormal connection tree and the abnormal sub-tree.
Further, the exception screening module comprises a temporal screening module, the temporal screening module comprising:
a minimum distance obtaining module, configured to calculate euclidean distances between the region connection information in a current time interval and feature vectors of the region connection information in corresponding time intervals in consecutive multiple adjacent cycles, respectively, and obtain a minimum distance among all the euclidean distances; and
and the time anomaly screening module is used for taking the area connection information corresponding to the minimum distance greater than a preset minimum distance threshold value in each time interval as the abnormal area connection information.
Further, the exception screening module includes a spatial screening module, which includes:
the coordinate system establishing module is used for establishing a space rectangular coordinate system taking # Obj, Pcoto and Pctd as axes for each time interval; where # Obj represents the traffic volume passing through the Link in the current sub-interval, Pcto represents the proportion of the traffic volume passing through the Link to the traffic volume traveling out from the start point of the Link in the current sub-interval, and Pctd represents the proportion of the traffic volume passing through the Link to the traffic volume traveling into the end point of the Link in the current sub-interval.
The distance acquisition module is used for calculating the Mahalanobis distance between the region connection information points in the coordinate system; and
the spatial anomaly screening module is used for screening extreme values in the coordinate system and taking the area connection information corresponding to the extreme values as abnormal area connection information;
further, the abnormal connection tree generating module includes:
the traversal tree-building module is used for traversing the abnormal region connection information according to a time sequence, and correspondingly traversing the abnormal region connection information in the next time interval when the abnormal region connection information in the current time interval is traversed; and when the abnormal region connection information in the current time interval and the abnormal region connection information in the next time interval meet the condition of the abnormal connection tree, establishing the abnormal connection tree in a recursive mode.
In a third aspect, an embodiment of the present invention provides an electronic device, which includes a memory and a processor, where:
the memory is used for storing one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement any one of the methods for monitoring data flow anomalies provided in the above method.
In a fourth aspect, an embodiment of the present invention provides a storage medium, where the storage medium stores a computer-readable program, and when the program is executed, the method for monitoring data flow abnormality is implemented as any one of the methods provided in the foregoing method.
The invention has the following beneficial effects:
the embodiment of the invention provides a method for monitoring data flow abnormity, which comprises the steps of generating area connection information by data flow information; screening abnormal region connection information according to the region connection information; generating an abnormal connection tree according to the abnormal region connection information; searching an abnormal subtree with the occurrence frequency higher than a preset abnormal threshold value in the abnormal connection tree, locking a data flow abnormal area by the abnormal connection tree and the abnormal subtree, and further obtaining an accidental traffic abnormal situation and a continuous traffic abnormal situation which occur in the area within a period of time, wherein the former reflects the influence of some emergency events on the traffic situation of the city or the area, and the latter reflects the problems existing in the current traffic planning of the city or the area. The embodiment of the invention discloses the repeated interaction among the space-time abnormal values, finds the potential influence in the design of the conventional switching network, and further can more accurately monitor the abnormality of the data stream.
Drawings
Fig. 1 is a flowchart of a method for monitoring data flow anomalies according to an embodiment of the present invention;
FIG. 2 is a process diagram of generating region connection information from data stream information between different regions according to an embodiment of the present invention;
FIG. 2a is a schematic diagram of geographic information about an area to be monitored according to an embodiment of the present invention;
FIG. 2b is a diagram illustrating a plurality of regions according to an embodiment of the present invention;
FIG. 2c is a schematic diagram of a data flow for generating a traffic track between different zones according to an embodiment of the present invention;
FIG. 2d is a schematic diagram of traffic track data flow between different zones according to an embodiment of the present invention;
fig. 3 is a block diagram of a device for monitoring data flow anomalies according to another embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention adopted to achieve the predetermined objects, the following detailed description of a method, an apparatus, an electronic device and a storage medium for monitoring data stream anomalies according to the present invention with reference to the accompanying drawings and preferred embodiments, and the detailed implementation, structure, features and effects thereof are described below. In the following description, different "one embodiment" or "another embodiment" refers to not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The embodiment of the invention takes traffic data flow as an example, and specifically describes specific schemes of a method, a device, electronic equipment and a storage medium for monitoring data flow abnormity provided by the invention with reference to the accompanying drawings. Referring to fig. 1, a flow chart of a method for monitoring data flow anomaly according to the present invention is shown, wherein the method comprises the following steps:
in step S001, the area connection information is generated from the data stream information.
According to the forming rule of the traffic data stream, the time is divided into a plurality of time intervals, and each time interval is composed of a plurality of sub-intervals. In each sub-interval, the traffic data stream is dynamically formed in each area. For the setting of the time interval, in this embodiment, one hour may be set as one time interval, and then there are 24 time intervals each day, and then there are corresponding time intervals between two adjacent days; taking the 24 time intervals as a period, there is a corresponding time interval between two adjacent periods. Similarly, a time interval can be set every day, so that a corresponding time interval exists between adjacent weeks, and abnormal values can be conveniently detected; that is, with one time interval per day and one cycle per week, there is a corresponding time interval between adjacent cycles. The invention monitors the traffic data stream in each time interval to obtain an abnormal traffic data stream. The embodiment of the present invention refers to the area connection information as Link, and is not stated below.
Firstly, acquiring geographic information of an area to be monitored, and dividing the area into a plurality of areas according to roads; corresponding data stream information is collected and collated, including vehicle trajectories and corresponding times. In time order, for a vehicle track in one data stream information: p1 → P2 → P3 … … → Pn, and if two adjacent nodes, P (k +1), belong to two different areas, area Link information Link of P (k) → P (k +1) is generated.
Definition 1: in the first placejAt a time interval ofiThe feature vectors for each Link are:
f
i，j
=<#obj,Pct
o
,Pct
d
>
definition 2: for the jth time interval containing q sub-intervalsiThe feature vectors for each Link are:
F
i，j
=<f
i,j-q+1
,f
i,j-q+2
,…… , f
i，j
>
after obtaining the region connection information, three attribute values of all links in the current sub-interval are calculated, wherein the three attribute values are respectively as follows: # Obj, Pcoto, Pcdd. Where, # Obj represents the traffic volume passing through the Link in the current subinterval, Pcto represents the proportion of the traffic volume passing through the Link to the traffic volume traveling out from the start point of the Link in the current subinterval, and Pctd represents the proportion of the traffic volume passing through the Link to the traffic volume traveling into the end point of the Link in the current subinterval.
Fig. 2 is a process diagram of generating regional connection information from data stream information between different regions, where fig. 2a is a schematic diagram of geographic information about a region to be monitored according to an embodiment of the present invention, which shows geographic information of a local region; FIG. 2b is a diagram of dividing a plurality of regions according to an embodiment of the present invention, which shows a graph after dividing the regions; FIG. 2c is a schematic diagram of a data flow for generating a traffic track between different zones according to an embodiment of the present invention, showing traffic data information for the red-lined area in FIG. 2 b; FIG. 2d is a schematic diagram of traffic track data flow between different areas according to an embodiment of the present invention. Link information of a → b is generated between two neighboring area nodes a, b in FIG. 2 d. The Link has a starting point and an end point.
Since the traffic volume passing through the Link is 2 at the current time interval, the traffic volume exiting from the start point a of the Link is (2+3), and the traffic volume entering the end point b of the Link is (2+ 6). Thus, # Obj =2, Pcto =2/(2+3), Pctd =2/(2+ 6).
Therefore, the feature vector for Link of a → b at this time interval is obtained as:
f i，j =<#obj,Pct o ,Pct d > =<#obj=2,Pct o =2/(2+3)=0.4,Pct d =2/(2+6) =0.25>=<2,0.4, 0.25>
in step S002, abnormal area connection information is screened out based on the area connection information.
And (3) screening abnormal region connection information in time and space according to the region connection information obtained in the step (S001), wherein the screening step specifically comprises the following steps:
in terms of time, in this embodiment, assuming that an hour is a time interval, two adjacent days have corresponding time intervals, that is, an hour is a time interval, a 24 hour is a period, and adjacent periods have corresponding time intervals. For a certain Link in the current time interval, respectively calculating the Euclidean distances of the Link and the characteristic vectors of the Link in the corresponding time intervals in the adjacent continuous multiple periods to obtain the minimum distance in all the Euclidean distances; and taking Link corresponding to the minimum distance larger than the preset minimum distance threshold in each time interval as abnormal area connection information. The preset minimum distance threshold is set according to needs or actual conditions.
Preferably, after the minimum distance for each Link is calculated, the influence of different regions is eliminated by subtracting a minimum value and then dividing by a maximum value.
Step S003, the abnormal connection tree is generated according to the abnormal area connection information.
The conditions that the abnormal junction tree should satisfy are: and taking the abnormal region connection information as a node, wherein the time interval of the father node is earlier than the time interval of the child node, and the termination region of the father node is equal to the start region of the child node.
The generation process of the abnormal connection tree comprises the following steps: traversing each abnormal region connection information obtained in the step S002 according to the time sequence, traversing the abnormal region connection information in the next time interval every time when one abnormal region connection information is traversed, judging whether the abnormal region connection information can be a child node, and if so, recursively building a tree.
Step S004, searching an abnormal sub-tree with the occurrence frequency higher than a preset abnormal threshold value in the abnormal connection tree, and locking the abnormal area of the data flow by the abnormal connection tree and the abnormal sub-tree.
When the occurrence frequency of a certain abnormal subtree is greater than a preset abnormal threshold value, the abnormal subtree is placed into an abnormal subtree set, and the screened abnormal connection tree and the abnormal subtree set are displayed in a regional map, so that the traffic abnormal condition is visually displayed. The preset abnormal threshold is set according to needs or actual conditions.
In summary, the embodiment of the present invention provides a method for monitoring data stream anomaly, where the method includes generating area connection information from data stream information; screening abnormal region connection information according to the region connection information; generating an abnormal connection tree according to the abnormal region connection information; searching an abnormal subtree with the occurrence frequency higher than a preset abnormal threshold value in the abnormal connection tree, locking a data flow abnormal area by the abnormal connection tree and the abnormal subtree, and further obtaining an accidental traffic abnormal situation of the area reflected by the abnormal connection tree in a period of time and a continuous traffic abnormal situation reflected by the abnormal subtree, wherein the former reflects the influence of some emergencies on the traffic situation of the city or the area, and the latter reflects the problems existing in the current traffic planning of the city or the area. The embodiment of the invention can reveal the repeated interaction among the space-time abnormal values and discover the potential influence in the design of the conventional switching network, and can provide a data driving method for road traffic analysis and more accurately monitor the abnormality of the data flow. In addition, the embodiment of the invention can be applied to monitoring other various data streams, such as network data or climate data.
Referring to fig. 3, it shows a monitoring apparatus for data flow anomaly according to another embodiment of the present invention, the monitoring apparatus includes a data generating module 301, an anomaly filtering module 302, an anomaly connection tree generating module 303, and an anomaly locking module 304, specifically:
a data generating module 301, configured to generate the region connection information from the data stream information. Firstly, acquiring geographic information of an area to be monitored, and dividing the area into a plurality of areas according to roads; corresponding data stream information is collected and collated, including vehicle trajectories and corresponding times. Then three attribute values are calculated for all links in a certain sub-interval: # Obj, Pcoto, Pcdd.
An exception screening module 302, configured to screen out exception region connection information according to the region connection information. The abnormity screening module comprises a time screening module and a space screening module.
Specifically, the time filtering module 3021 is configured to calculate, at the ith Link in the jth time interval, euclidean distances of feature vectors of corresponding links in time intervals corresponding to consecutive days adjacent to the ith Link, respectively, and obtain a minimum distance among all euclidean distances; and taking Link corresponding to the minimum distance larger than the preset minimum distance threshold in each time interval as abnormal area connection information.
Specifically, the spatial screening module 3022 is configured to establish a spatial rectangular coordinate system with # Obj, Pcto, and Pctd as axes for each time interval; each Link in the coordinate system corresponds to a point, and the Mahalanobis distance is calculated; and screening extreme values in the coordinate system, and taking Link corresponding to the extreme values as abnormal region connection information.
An abnormal connection tree generating module 303, configured to generate an abnormal connection tree according to the abnormal region connection information. The abnormal connection tree generating module 303 includes a traversal tree building module 3031, configured to traverse each abnormal region connection information according to a time sequence, traverse the abnormal region connection information in a next time interval every time when one abnormal region connection information is traversed, and determine whether the abnormal region connection information may be a child node, and if so, recursively build a tree.
And an exception locking module 304, configured to find an exception sub-tree in the exception connection tree, where the occurrence frequency of the exception sub-tree is higher than a preset exception threshold, and lock an exception area of the data flow by the exception connection tree and the exception sub-tree. When the occurrence frequency of a certain abnormal subtree is greater than a preset abnormal threshold value, the abnormal subtree is placed into an abnormal subtree set, and the screened abnormal connection tree and the abnormal subtree set are displayed in a regional map, so that the traffic abnormal condition is visually displayed.
In summary, an embodiment of the present invention provides a device for monitoring data flow anomalies, where the device includes a data generation module, an anomaly screening module, an anomaly connection tree generation module, and an anomaly locking module, where the data generation module obtains region connection information, the anomaly screening module obtains anomaly region connection information by screening in time and space according to the region connection information, the anomaly connection tree generation module generates an anomaly connection tree according to the anomaly region connection information, and finally, the anomaly locking module obtains an anomaly subtree whose occurrence frequency is greater than a preset threshold according to the anomaly connection tree, and places the obtained anomaly subtree in an anomaly subtree set, so as to display a corresponding anomaly connection tree and an anomaly subtree set. And then obtaining the accidental traffic abnormal situation and the continuous traffic abnormal situation which occur in the region in a period of time, wherein the former reflects the influence of some emergency events on the traffic situation of the city or the region, and the latter reflects the problems existing in the current traffic planning of the city or the region.
Referring to fig. 4, which shows a schematic structural diagram of an electronic device including a memory 401 and a processor 402 based on the same inventive concept, it will be understood by those skilled in the art that the structure of the terminal device shown in fig. 4 does not constitute a limitation of the terminal device, and may include more or less components than those shown, or combine some components, or arrange different components. Wherein:
the memory 401 is used to store instructions required by the processor 402 to perform tasks.
The processor 402 is used for executing the instructions stored in the memory 401, and generating the area connection information from the data flow information; screening abnormal region connection information according to the region connection information; generating an abnormal connection tree according to the abnormal region connection information; and searching an abnormal subtree with the occurrence frequency higher than a preset abnormal threshold value in the abnormal connection tree, and locking the abnormal area of the data flow by the abnormal connection tree and the abnormal subtree.
In other embodiments, the electronic device further comprises a communication interface 403 connected to the memory 401 and the processor 402 by a bus or other means for enabling the main body to communicate with other devices or communication networks.
Preferably, the memory 401 is used for storing one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement any one of the data flow anomaly monitoring methods provided in the above embodiments.
The embodiment of the invention also provides a storage medium, which can store a computer-readable program, and when the program is executed, the method for monitoring data flow abnormality provided by any one of the above embodiments is executed.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for monitoring data flow abnormity is characterized by comprising the following steps:
generating region connection information from the data stream information;
screening abnormal region connection information according to the region connection information;
generating an abnormal connection tree according to the abnormal region connection information;
and searching an abnormal subtree with the occurrence frequency higher than a preset abnormal threshold value in the abnormal connection tree, and locking the abnormal area of the data flow by the abnormal connection tree and the abnormal subtree.
2. The method for monitoring data flow abnormality according to claim 1, wherein the method for screening abnormal regional connection information according to the regional connection information includes the following steps:
respectively calculating Euclidean distances between the region connection information in the current time interval and the feature vectors of the region connection information in corresponding time intervals in a plurality of adjacent continuous periods to obtain the minimum distance in all the Euclidean distances;
and taking the area connection information corresponding to the minimum distance larger than a preset minimum distance threshold value in each time interval as abnormal area connection information.
3. The method for monitoring data flow abnormality according to claim 1 or 2, wherein the method for screening abnormal regional connection information according to the regional connection information includes the following steps:
for each time interval, establishing a space rectangular coordinate system taking # Obj, Pcoto and Pctd as axes;
calculating mahalanobis distances between the region connection information points in the coordinate system;
screening extreme values in the coordinate system, and taking area connection information corresponding to the extreme values as abnormal area connection information;
where # Obj represents the traffic volume passing through Link in the current sub-interval, Pcto represents the proportion of the traffic volume passing through the Link to the traffic volume coming out from the start of the Link in the current sub-interval, and Pctd represents the proportion of the traffic volume passing through the Link to the traffic volume coming into the end of the Link in the current sub-interval.
4. The method for monitoring data flow abnormality according to claim 1, wherein the method for generating an abnormal connection tree according to the abnormal region connection information includes the following steps:
traversing the abnormal region connection information according to a time sequence, wherein each time the abnormal region connection information in a current time interval is traversed, the abnormal region connection information in the next time interval is correspondingly traversed;
and when the abnormal region connection information in the current time interval and the abnormal region connection information in the next time interval meet the condition of the abnormal connection tree, establishing the abnormal connection tree in a recursive mode.
5. A device for monitoring data flow anomalies, the device comprising:
the data generation module is used for generating the area connection information from the data stream information;
the abnormal screening module is used for screening abnormal region connection information according to the region connection information;
the abnormal connection tree generation module is used for generating an abnormal connection tree according to the abnormal region connection information;
and the abnormal locking module is used for searching an abnormal sub-tree with the occurrence frequency higher than a preset abnormal threshold value in the abnormal connection tree, and locking the abnormal area of the data flow by the abnormal connection tree and the abnormal sub-tree.
6. The apparatus for monitoring data flow abnormality according to claim 5, wherein said abnormality screening module includes a time screening module, said time screening module includes:
a minimum distance obtaining module, configured to calculate euclidean distances between the region connection information in a current time interval and feature vectors of the region connection information in corresponding time intervals in consecutive multiple adjacent cycles, respectively, and obtain a minimum distance among all the euclidean distances; and
and the time anomaly screening module is used for taking the area connection information corresponding to the minimum distance greater than a preset minimum distance threshold value in each time interval as the abnormal area connection information.
7. The apparatus for monitoring data flow anomaly according to claim 5 or 6, wherein said anomaly filtering module comprises a spatial filtering module, said spatial filtering module comprises:
the coordinate system establishing module is used for establishing a space rectangular coordinate system taking # Obj, Pcoto and Pctd as axes for each time interval; wherein # Obj represents the traffic volume passing through Link in the current sub-interval, Pcto represents the proportion of the traffic volume passing through the Link to the traffic volume driving out from the start point of the Link in the current sub-interval, and Pctd represents the proportion of the traffic volume passing through the Link to the traffic volume driving into the end point of the Link in the current sub-interval;
the distance acquisition module is used for calculating the Mahalanobis distance between the region connection information points in the coordinate system; and
and the spatial anomaly screening module is used for screening extreme values in the coordinate system and taking the area connection information corresponding to the extreme values as abnormal area connection information.
8. The apparatus for monitoring data flow abnormality according to claim 5, wherein said abnormal connection tree generation module includes:
the traversal tree-building module is used for traversing the abnormal region connection information according to a time sequence, and correspondingly traversing the abnormal region connection information in the next time interval when the abnormal region connection information in the current time interval is traversed; and when the abnormal region connection information in the current time interval and the abnormal region connection information in the next time interval meet the condition of the abnormal connection tree, establishing the abnormal connection tree in a recursive mode.
9. An electronic device comprising a memory and a processor, wherein:
the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement a method for monitoring data flow anomalies as claimed in any one of claims 1 to 4.
10. A storage medium, characterized in that the storage medium stores a computer-readable program, which when executed implements the method for monitoring data flow abnormality according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010965607.9A CN112256519A (en) | 2020-09-15 | 2020-09-15 | Data flow abnormity monitoring method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010965607.9A CN112256519A (en) | 2020-09-15 | 2020-09-15 | Data flow abnormity monitoring method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112256519A true CN112256519A (en) | 2021-01-22 |
Family
ID=74232833
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010965607.9A Pending CN112256519A (en) | 2020-09-15 | 2020-09-15 | Data flow abnormity monitoring method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112256519A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573116A (en) * | 2015-02-05 | 2015-04-29 | 哈尔滨工业大学 | Taxi GPS data mining based traffic abnormality recognition method |
| WO2019090753A1 (en) * | 2017-11-13 | 2019-05-16 | Beijing Didi Infinity Technology And Development Co., Ltd. | Systems and methods for monitoring traffic congestion |
-
2020
- 2020-09-15 CN CN202010965607.9A patent/CN112256519A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573116A (en) * | 2015-02-05 | 2015-04-29 | 哈尔滨工业大学 | Taxi GPS data mining based traffic abnormality recognition method |
| WO2019090753A1 (en) * | 2017-11-13 | 2019-05-16 | Beijing Didi Infinity Technology And Development Co., Ltd. | Systems and methods for monitoring traffic congestion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8606727B2 (en) | Method and system for traffic prediction based on space-time relation | |
| Bennacer et al. | Self-diagnosis technique for virtual private networks combining Bayesian networks and case-based reasoning | |
| Gupta et al. | Optimal output feedback control using two remote sensors over erasure channels | |
| CN111552246B (en) | Equipment production line scheduling method applied to smart park and cloud computing server | |
| Aljohani et al. | An algorithm for accessing traffic database using wireless technologies | |
| CN113904921B (en) | Dynamic network topology graph generation method, system, processing equipment and storage medium based on log and graph | |
| CN111815946A (en) | Method and device for determining abnormal road section, storage medium and electronic equipment | |
| Lim et al. | Stochastic motion planning with path constraints and application to optimal agent, resource, and route planning | |
| Uke et al. | UML based modeling for data aggregation in secured wireless sensor network | |
| CN114757389A (en) | Federal learning-based urban traffic flow space-time prediction method | |
| CN112364290B (en) | Method and system for constructing visual calculation model based on stream-oriented calculation | |
| CN112256519A (en) | Data flow abnormity monitoring method and device, electronic equipment and storage medium | |
| CN111010311B (en) | Intelligent network fault diagnosis method based on knowledge graph | |
| CN114417633B (en) | Network shooting range scene construction method and system based on parallel simulation six-tuple | |
| CN116258253A (en) | Vehicle OD prediction method based on Bayesian neural network | |
| CN113012412B (en) | Intelligent data acquisition method and system based on dynamic acquisition statistical analysis of instrument and video data | |
| CN111654413B (en) | Method, equipment and storage medium for selecting effective measurement points of network flow | |
| CN114493206A (en) | BIM-based global airport management system and method | |
| CN114063126A (en) | Electronic fence system and method for establishing electronic fence accurate track model | |
| CN114911654A (en) | Fault classification method, device and system | |
| CN112564933A (en) | Smart city free network system based on Internet of vehicles and construction method | |
| CN111210088A (en) | Traffic state index prediction method based on space-time factors | |
| Zhuo et al. | Network situation assessment based on RST | |
| CN113742909B (en) | Method for constructing unmanned vehicle group self-cooperative model in closed scene | |
| CN114051273B (en) | Large-scale network dynamic self-adaptive path planning method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |