CN110278465A - Video CDN distributed HTTP full link data collection and analysis method and system - Google Patents
Video CDN distributed HTTP full link data collection and analysis method and system Download PDFInfo
- Publication number
- CN110278465A CN110278465A CN201910541292.2A CN201910541292A CN110278465A CN 110278465 A CN110278465 A CN 110278465A CN 201910541292 A CN201910541292 A CN 201910541292A CN 110278465 A CN110278465 A CN 110278465A
- Authority
- CN
- China
- Prior art keywords
- snapshot
- log
- track
- server
- stratum
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 32
- 238000013480 data collection Methods 0.000 title abstract 3
- 238000000034 method Methods 0.000 claims abstract description 17
- 238000005070 sampling Methods 0.000 claims description 19
- 238000006243 chemical reaction Methods 0.000 claims description 13
- PMHURSZHKKJGBM-UHFFFAOYSA-N isoxaben Chemical compound O1N=C(C(C)(CC)CC)C=C1NC(=O)C1=C(OC)C=CC=C1OC PMHURSZHKKJGBM-UHFFFAOYSA-N 0.000 claims description 12
- 230000002159 abnormal effect Effects 0.000 claims description 7
- 230000010354 integration Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 210000001367 artery Anatomy 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
- H04N21/64322—IP
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
- H04N21/6473—Monitoring network processes errors
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a video CDN distributed HTTP full link data collection and analysis method and a system, wherein the video CDN distributed HTTP full link data collection and analysis method comprises the following steps: step A, respectively generating snapshots of logs in a first-layer server, a second-layer server, a third-layer server and a quality log server; step B, collecting the generated snapshot; step C, generating a track for all snapshots corresponding to the same user request, and storing the track in a track database; and D, embedding the snapshots belonging to the same track in a time range to form a snapshot hierarchical structure. The method can reproduce the complete path in the whole request life cycle of the user and the time consumption of all links in the path, can analyze the time consumption bottleneck of the path, defines the fault reason and is greatly helpful for identifying the hotlink; the online code does not need to be modified, the application layer is not sensed, and the system is not invasive to the original system; has certain expansibility and is easy to analyze; the service overhead is small, and the whole resource consumption is small.
Description
Technical field
The invention belongs to video CD N technical field, in particular to a kind of full link data of video CD N distribution HTTP is collected
With analysis method and system.
Background technique
Currently, the general framework of video CD N (is adjusted by a stratum server (video content index), two stratum servers
Degree), three layers of service device (video offers) and client (PCWEB/APP/PC client/ set-top box etc.) form, operating process
Be: client is generated from stratum server browsing click behavior incidentally requests the request string of ID and obtains video ID, then to ask
It asks ID and video ID to go two layers of dispatch server of access to obtain the video link specifically containing three layers of service device address, then leads to
Crossing specific video link goes three layers of service device to request video, receives client terminal playing video after video, sees attached drawing 1.
In playing process, the brief log of client will be submitted to quality log server by client.In this process
In, the log of one or two three layers of service devices and each self-forming of client oneself and being stored in respectively is locally stored, when analysis respectively
The machine log is analyzed, can not be effectively presented the life cycle that one is requested completely.Some robber's chain websites pass through two layers of tune of caching
Degree server returns the result directly to chain user offer three layers of service device video link is stolen, to cause economic loss.Thus, it is complete
Site preparation, which tracks user's request activity, can be not only used for debugging the failure of entire request process, be dedicated to release rate and matter
Throughput bottleneck, while effectively can be screened and be hit to chain behavior is stolen.However, there is no full traces one in the prior art
A user requests movable technical solution.
Summary of the invention
It is an object of the present invention in view of the above shortcomings of the prior art, provide a kind of full chain of video CD N distribution HTTP
Circuit-switched data collect with analysis method and system, can collect and the video CD N that connects is scattered in the logs of the whole network various regions, thus by one
The Life cycle of a request is reproduced, for post analysis processing.
In order to solve the above technical problems, the technical scheme adopted by the invention is that:
A kind of full link data of video CD N distribution HTTP is collected and analysis method, its main feature is that the following steps are included:
Step A gives birth to the log in a stratum server, two stratum servers, three layers of service device, quality log server respectively
At snapshot；
The snapshot generated in step B, collection step A；
All snapshots of the same user's request of correspondence are generated a track, track are stored in track data by step C
In library；
Step D will belong to impinging upon in time range fastly for track and be embedded in, forms the hierarchical structure of snapshot.
As a preferred method, in the step A, using the dispatch id of two stratum servers generation as track ID, wherein
Track ID is for all snapshots for belonging to same user's request of integration；
Log in two stratum servers generates corresponding snapshot by major key of the track ID；
Log in three layers of service device generates corresponding snapshot by major key of the track ID；
Log in quality log server generates corresponding snapshot by major key of the track ID；
The method that log in one stratum server generates corresponding snapshot includes: corresponding fast according to log in two stratum servers
According to obtaining corresponding request ID, match the log in a corresponding stratum server according to request ID and generate corresponding snapshot；Its
In, the corresponding snapshot of a stratum server is using the corresponding track ID of corresponding two stratum server as major key；
In the step C, the snapshot of corresponding same user's request is differentiated according to the track id information that snapshot includes.
As a preferred method, in the step A, in a stratum server, two stratum servers, three layers of service device, quality
In log server, snapshot is used and generated to log using sampling algorithm, wherein sampling algorithm includes:
Consistent hashing calculating is done to track ID with global sampling rate parameter, is 0 by consistent hashing calculated result
Log is converted to be stored in accordance with the form provided fastly；
SNAPSHOT INFO is contained in the snapshot of generation, wherein SNAPSHOT INFO includes the snapshot that the level according to locating for snapshot generates
Name, snapshot ID, IP, module level, specified field are stored in the KV field of snapshot in the form of key-value pair；Wherein, two layers of clothes
Request ID when the KV field of the corresponding snapshot of business device includes request scheduling.
Snap-shot action is collected in step B as a preferred method, to issue in system idle time.
Further, further include step E, abnormal conditions are screened according to trace information.
Based on the same inventive concept, the present invention also provides a kind of full link datas of video CD N distribution HTTP to collect
With analysis system, its main feature is that including:
Log conversion and reporting unit: for taking a stratum server, two stratum servers, three layers of service device, quality log
Log in business device generates snapshot respectively, and the snapshot of generation is reported to snapshot and compiles server；
Snapshot compiles server: the snapshot reported for receiving log conversion and reporting unit, and will be corresponding same
All snapshots of a user's request generate a track；
Track database: the track of server generation is compiled for saving snapshot；
Trajectory analysis unit: it is embedded in for impinging upon in time range fastly for track will to be belonged to, forms snapshot
Hierarchical structure.
As a preferred method, in the log conversion and reporting unit:
Log in two stratum servers generates corresponding snapshot by major key of the track ID；
Log in three layers of service device generates corresponding snapshot by major key of the track ID；
Log in quality log server generates corresponding snapshot by major key of the track ID；
According to the corresponding snapshot of log in two stratum servers, corresponding request ID is obtained, it is corresponding according to request ID matching
Log in one stratum server simultaneously generates corresponding snapshot；Wherein, the corresponding snapshot of a stratum server is serviced with corresponding two layers
The corresponding track ID of device is as major key；
The snapshot is compiled in server, and the track id information for including according to snapshot differentiates that the corresponding same user asks
The snapshot asked.
As a preferred method, in the log conversion and reporting unit, in a stratum server, two stratum servers, three
In stratum server, quality log server, snapshot is used and generated to log using sampling algorithm, wherein sampling algorithm
Include:
Consistent hashing calculating is done to track ID with global sampling rate parameter, is 0 by consistent hashing calculated result
Log is converted to be stored in accordance with the form provided fastly；
SNAPSHOT INFO is contained in the snapshot of generation, wherein SNAPSHOT INFO includes the snapshot that the level according to locating for snapshot generates
Name, snapshot ID, IP, module level, specified field are stored in the KV field of snapshot in the form of key-value pair；Wherein, two layers of clothes
Request ID when the KV field of the corresponding snapshot of business device includes request scheduling.
As a preferred method, in the log conversion and reporting unit: reporting snap-shot action in system idle time
It issues.
Further, the trajectory analysis unit is also used to screen abnormal conditions according to trace information.
Compared with prior art, the invention has the following advantages:
First, energy reappearing user entirely requests all links in fullpath and path in life cycle time-consuming, newly
All Depending modules of rendering requests and critical path, energy resolution path time-consuming bottleneck, clear failure reason, and have to chain identification is stolen
Huge help.
Second, it is based on existing video CD N framework, is analyzed using the existing log for being scattered in local, it is not necessary to modify lines
Upper code, to application layer unaware, to original system without invasive.
Third with unified format collection and according to each log snapshot information of request time sequential series, and is ultimately formed and is asked
Calling hierarchical structure is sought, there is certain scalability and is easy to analyze.
4th, log snapshot is generated using sampling algorithm, reduces costs of services, whole resource consumption is small.
Detailed description of the invention
Fig. 1 is video CD N normal request flow chart.
Fig. 2 is principle of the invention figure.
Specific embodiment
The full link data of video CD N distribution HTTP of the present invention collect with analysis method the following steps are included:
Step A, a stratum server, two stratum servers, three layers of service device, quality log server dispose corresponding day respectively
Will conversion and upload program, read original log, by a stratum server, two stratum servers, three layers of service device, quality log service
Log in device generates snapshot respectively；Wherein, a stratum server is video index server, and two stratum servers are dispatch service
Device, three layers of service device are video content server.
The snapshot generated in step B, collection step A；
All snapshots of the same user's request of correspondence are generated a track, track are stored in track data by step C
In library；
Step D will belong to impinging upon in time range fastly for track and be embedded in, forms the hierarchical structure of snapshot.
Step E screens abnormal conditions according to trace information.According to setting conditional filtering abnormal data, for example, if
Only two or three layers record are likely to be robber's chain, if two layers of scheduling three layers of access of corresponding flood tide are also likely to be to steal chain etc.；May be used also
By extracting all kinds of trajectory analysis speed bottle-necks, critical path etc..In addition, also providing a degree of visualization function.
In the step A, using the dispatch id that two stratum servers generate as track ID (TID), wherein track ID is for uniting
Close all snapshots for belonging to same user's request.TID and track ID, for all snapshots for belonging to the same request of integration,
The effect of license plate number is functioned similarly to, must be theoretically recorded in one or two three layers of service devices and client.But because can not
Code on line is modified, the data field of existing log can only be utmostly utilized.And on line environment none implement entirely
The ID of process.Because two layers of dispatch server are the key positions for being connected user and three layers of service device, and two layers of dispatch server sheet
Body is also that can generate a globally unique dispatch id to verify for three layers, so TID is directly served as using overall scheduling ID, and
Program Generating is not needed.One layer of log can not directly be filed using TID, the subsequent explanation of the acquisition of one layer of TID.
Log in two stratum servers generates corresponding snapshot by major key of the track ID；
Log in three layers of service device generates corresponding snapshot by major key of the track ID；
Log in quality log server generates corresponding snapshot by major key of the track ID；
The method that log in one stratum server generates corresponding snapshot includes: corresponding fast according to log in two stratum servers
According to obtaining corresponding request ID, match the log in a corresponding stratum server according to request ID and generate corresponding snapshot；Its
In, the corresponding snapshot of a stratum server is using the corresponding track ID of corresponding two stratum server as major key.
When snapshot is received with arranging, receives and be stored in local data base according to TID after snapshot, created if without TID
New track record.According to snapshot ID, module level field, which is arranged successively, is placed in track database (being shown in Table 1), if discovery two
Layer snapshot, according to its institute with key assignments centering request ID from a stratum server acquisition corresponding requests ID snapshot.Because of a stratum server
It is less, it is possible to be obtained by this inquiry mode.
Table 1: track database saves format
In the step C, the snapshot of corresponding same user's request is differentiated according to the track id information that snapshot includes.
It is sharp in a stratum server, two stratum servers, three layers of service device, quality log server in the step A
Snapshot is used and generated to log with sampling algorithm, wherein sampling algorithm includes:
Consistent hashing calculating is done to track ID with global sampling rate parameter, calculated result is divided according to sample rate
Match, the log that consistent hashing calculated result is 0 is included in sampling, is converted to and is stored in accordance with the form provided fastly.For example, sample rate is
When 1/16, consistent hashing calculating is carried out to TID, result is to select in 0 expression, its log is converted to and (is shown in Table in accordance with the form provided fastly
2) it is stored.
Watch 2: fastly in accordance with the form provided
SNAPSHOT INFO is contained in the snapshot of generation, wherein SNAPSHOT INFO includes the snapshot that the level according to locating for snapshot generates
Name, snapshot ID, IP, module level, specified field are stored in the KV field of snapshot in the form of key-value pair；Wherein, it needs to infuse
Meaning be the KV field of the corresponding snapshot of two stratum servers must include request ID when reporting request scheduling, and request ID is one
The key of stratum server snapshot filing.
Snap-shot action is collected in step B to issue in system idle time (i.e. host light hours).
As shown in Fig. 2, the full link data of video CD N distribution HTTP of the present invention is collected with analysis system and includes:
Log conversion and reporting unit: a stratum server, two stratum servers, three layers of service device and quality log clothes are installed on
Business device, for converting local log, by the day in a stratum server, two stratum servers, three layers of service device, quality log server
Will generates snapshot respectively, is that " snapshot " of unified format is locally stored after being screened, in system idle time by snapshot
It is uploaded to snapshot and compiles server.
Snapshot preserves the primary request that globally unique track ID (TID) is used to identity user.It is deployed in one layer of one layer
Slightly different for service, but function is essentially identical:
Log in two stratum servers generates corresponding snapshot by major key of the track ID；
Log in three layers of service device generates corresponding snapshot by major key of the track ID；
Log in quality log server generates corresponding snapshot by major key of the track ID；
According to the corresponding snapshot of log in two stratum servers, corresponding request ID is obtained, it is corresponding according to request ID matching
Log in one stratum server simultaneously generates corresponding snapshot；Wherein, the corresponding snapshot of a stratum server is serviced with corresponding two layers
The corresponding track ID of device is as major key.
In a stratum server, two stratum servers, three layers of service device, quality log server, sampling algorithm pair is utilized
Log is used and is generated snapshot, and wherein sampling algorithm includes:
Consistent hashing calculating is done to track ID with global sampling rate parameter, is 0 by consistent hashing calculated result
Log is converted to be stored in accordance with the form provided fastly；
SNAPSHOT INFO is contained in the snapshot of generation, wherein SNAPSHOT INFO includes the snapshot that the level according to locating for snapshot generates
Name, snapshot ID, IP, module level, specified field are stored in the KV field of snapshot in the form of key-value pair；Wherein, two layers of clothes
Request ID when the KV field of the corresponding snapshot of business device includes request scheduling.
Snapshot compiles server: the snapshot reported for receiving log conversion and reporting unit, and will be corresponding same
All snapshots of a user's request generate a track.In the present embodiment, is differentiated according to the track id information that snapshot includes and corresponded to
The snapshot of same user's request is major key to reporting snapshot that track database is written using unified TID, and each is known as track,
Storage is to provide subsequent analysis.
Track database: the track of server generation is compiled for saving snapshot.
Trajectory analysis unit: it is embedded in for impinging upon in time range fastly for track will to be belonged to, forms snapshot
Hierarchical structure.Because snapshot is generated in different servers, for time standard there may be difference, trajectory analysis unit can will be same
Belong to impinging upon within the scope of certain time fastly for track to be embedded in, be formed hierarchical structure (being shown in Table 3).
Table 3: track combined analysis
The trajectory analysis unit is also used to screen abnormal conditions according to trace information, and to missing or exception
Link is prompted.
To sum up, regard entire video CD N as traffic route, present invention visualization be described as follows:
Each main traffic artery has camera, and general camera, which is used only to look into, breaks rules and regulations or do flow monitoring.
And increase a set of Trajectory System now, specific license plate number can be identified from the picture that camera is shot, and pass through license plate number
The timing node that code appears in each road camera video recording is known that the route of the vehicle, travel speed, if
Deck etc..Here camera is exactly the log that original each stratum server bears respective function, and newly-increased Trajectory System is exactly this
The realization of inventive technique scheme.Fake license plate vehicle is to steal chain.
The embodiment of the present invention is described with above attached drawing, but the invention is not limited to above-mentioned specific
Embodiment, the above mentioned embodiment is only schematical, rather than limitation, those skilled in the art
Under the inspiration of the present invention, without breaking away from the scope protected by the purposes and claims of the present invention, it can also make very much
Form, within these are all belonged to the scope of protection of the present invention.
Claims (10)
1. a kind of full link data of video CD N distribution HTTP is collected and analysis method, which comprises the following steps:
Step A generates the log in a stratum server, two stratum servers, three layers of service device, quality log server fastly respectively
According to；
The snapshot generated in step B, collection step A；
All snapshots of the same user's request of correspondence are generated a track, track are stored in track database by step C
In；
Step D will belong to impinging upon in time range fastly for track and be embedded in, forms the hierarchical structure of snapshot.
2. the full link data of video CD N distribution HTTP as described in claim 1 is collected and analysis method, which is characterized in that
In the step A, the dispatch id generated using two stratum servers is used as track ID, and wherein track ID is used for that integration to be all belongs to
The snapshot of same user's request；
Log in two stratum servers generates corresponding snapshot by major key of the track ID；
Log in three layers of service device generates corresponding snapshot by major key of the track ID；
Log in quality log server generates corresponding snapshot by major key of the track ID；
The method that log in one stratum server generates corresponding snapshot includes: the corresponding snapshot of log in two stratum servers of foundation,
Corresponding request ID is obtained, the log in a corresponding stratum server is matched according to request ID and generates corresponding snapshot；Wherein,
The corresponding snapshot of one stratum server is using the corresponding track ID of corresponding two stratum server as major key；
In the step C, the snapshot of corresponding same user's request is differentiated according to the track id information that snapshot includes.
3. the full link data of video CD N distribution HTTP as claimed in claim 2 is collected and analysis method, which is characterized in that
In the step A, in a stratum server, two stratum servers, three layers of service device, quality log server, using adopting
Sample algorithm is used to log and is generated snapshot, and wherein sampling algorithm includes:
Consistent hashing calculating is done to track ID with global sampling rate parameter, the log for being 0 by consistent hashing calculated result
It is converted to and is stored in accordance with the form provided fastly；
SNAPSHOT INFO is contained in the snapshot of generation, wherein SNAPSHOT INFO include the level according to locating for snapshot generate snapshot name,
Snapshot ID, IP, module level, specified field are stored in the KV field of snapshot in the form of key-value pair；Wherein, two stratum server
Request ID when the KV field of corresponding snapshot includes request scheduling.
4. the full link data of video CD N distribution HTTP as described in claim 1 is collected and analysis method, which is characterized in that
Snap-shot action is collected in step B to issue in system idle time.
5. the full link data of video CD N distribution HTTP as described in claim 1 is collected and analysis method, which is characterized in that
Further include:
Step E screens abnormal conditions according to trace information.
6. a kind of full link data of video CD N distribution HTTP is collected and analysis system characterized by comprising
Log conversion and reporting unit: it is used for a stratum server, two stratum servers, three layers of service device, quality log server
In log generate snapshot respectively, and the snapshot of generation is reported into snapshot and compiles server；
Snapshot compiles server: the snapshot reported for receiving log conversion and reporting unit, and will the corresponding same use
All snapshots of family request generate a track；
Track database: the track of server generation is compiled for saving snapshot；
Trajectory analysis unit: it is embedded in for impinging upon in time range fastly for track will to be belonged to, forms the level of snapshot
Structure.
7. the full link data of video CD N distribution HTTP as claimed in claim 6 is collected and analysis system, which is characterized in that
In the log conversion and reporting unit:
Log in two stratum servers generates corresponding snapshot by major key of the track ID；
Log in three layers of service device generates corresponding snapshot by major key of the track ID；
Log in quality log server generates corresponding snapshot by major key of the track ID；
According to the corresponding snapshot of log in two stratum servers, corresponding request ID is obtained, it is one layer corresponding according to request ID matching
Log in server simultaneously generates corresponding snapshot；Wherein, the corresponding snapshot of a stratum server is with corresponding two stratum server pair
The track ID answered is as major key；
The snapshot is compiled in server, and the track id information for including according to snapshot differentiates that the corresponding same user requests
Snapshot.
8. the full link data of video CD N distribution HTTP as claimed in claim 7 is collected and analysis system, which is characterized in that
In the log conversion and reporting unit, in a stratum server, two stratum servers, three layers of service device, quality log server,
Snapshot is used and generated to log using sampling algorithm, wherein sampling algorithm includes:
Consistent hashing calculating is done to track ID with global sampling rate parameter, the log for being 0 by consistent hashing calculated result
It is converted to and is stored in accordance with the form provided fastly；
SNAPSHOT INFO is contained in the snapshot of generation, wherein SNAPSHOT INFO include the level according to locating for snapshot generate snapshot name,
Snapshot ID, IP, module level, specified field are stored in the KV field of snapshot in the form of key-value pair；Wherein, two stratum server
Request ID when the KV field of corresponding snapshot includes request scheduling.
9. the full link data of video CD N distribution HTTP as claimed in claim 6 is collected and analysis system, which is characterized in that
In the log conversion and reporting unit: snap-shot action being reported to issue in system idle time.
10. the full link data of video CD N distribution HTTP as claimed in claim 6 is collected and analysis system, which is characterized in that
The trajectory analysis unit is also used to screen abnormal conditions according to trace information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910541292.2A CN110278465B (en) | 2019-06-21 | 2019-06-21 | Video CDN distributed HTTP full link data collection and analysis method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910541292.2A CN110278465B (en) | 2019-06-21 | 2019-06-21 | Video CDN distributed HTTP full link data collection and analysis method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110278465A true CN110278465A (en) | 2019-09-24 |
| CN110278465B CN110278465B (en) | 2021-06-18 |
Family
ID=67962229
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910541292.2A Active CN110278465B (en) | 2019-06-21 | 2019-06-21 | Video CDN distributed HTTP full link data collection and analysis method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110278465B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102637206A (en) * | 2012-03-21 | 2012-08-15 | 浪潮集团山东通用软件有限公司 | Method for inquiring mass data |
| CN103051707A (en) * | 2012-12-20 | 2013-04-17 | 浪潮集团有限公司 | Dynamic user behavior-based cloud forensics method and dynamic user behavior-based cloud forensics system |
| US20130227074A1 (en) * | 2012-02-23 | 2013-08-29 | Mobitv, Inc. | Efficient delineation and distribution of media segments |
| CN105224433A (en) * | 2014-06-23 | 2016-01-06 | 阿里巴巴集团控股有限公司 | A kind of internal memory monitoring method and server |
| WO2016055893A1 (en) * | 2014-10-09 | 2016-04-14 | Telefonaktiebolaget L M Ericsson (Publ) | Method, traffic monitor (tm), request router (rr) and system for monitoring a content delivery network (cdn) |
| CN107707414A (en) * | 2017-11-22 | 2018-02-16 | 北京搜狐新媒体信息技术有限公司 | The monitoring system and method for CDN |
| US20180150364A1 (en) * | 2012-12-19 | 2018-05-31 | Amazon Technologies, Inc. | Cached data replication for cache recovery |
| CN108322533A (en) * | 2018-01-31 | 2018-07-24 | 广州鼎甲计算机科技有限公司 | Configuration and synchronization method between distributed type assemblies node based on operation log |
| CN109165363A (en) * | 2018-08-27 | 2019-01-08 | 成都深思科技有限公司 | A kind of configuration method of network data snapshot |
-
2019
- 2019-06-21 CN CN201910541292.2A patent/CN110278465B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130227074A1 (en) * | 2012-02-23 | 2013-08-29 | Mobitv, Inc. | Efficient delineation and distribution of media segments |
| CN102637206A (en) * | 2012-03-21 | 2012-08-15 | 浪潮集团山东通用软件有限公司 | Method for inquiring mass data |
| US20180150364A1 (en) * | 2012-12-19 | 2018-05-31 | Amazon Technologies, Inc. | Cached data replication for cache recovery |
| CN103051707A (en) * | 2012-12-20 | 2013-04-17 | 浪潮集团有限公司 | Dynamic user behavior-based cloud forensics method and dynamic user behavior-based cloud forensics system |
| CN105224433A (en) * | 2014-06-23 | 2016-01-06 | 阿里巴巴集团控股有限公司 | A kind of internal memory monitoring method and server |
| WO2016055893A1 (en) * | 2014-10-09 | 2016-04-14 | Telefonaktiebolaget L M Ericsson (Publ) | Method, traffic monitor (tm), request router (rr) and system for monitoring a content delivery network (cdn) |
| CN107707414A (en) * | 2017-11-22 | 2018-02-16 | 北京搜狐新媒体信息技术有限公司 | The monitoring system and method for CDN |
| CN108322533A (en) * | 2018-01-31 | 2018-07-24 | 广州鼎甲计算机科技有限公司 | Configuration and synchronization method between distributed type assemblies node based on operation log |
| CN109165363A (en) * | 2018-08-27 | 2019-01-08 | 成都深思科技有限公司 | A kind of configuration method of network data snapshot |
Non-Patent Citations (1)
| Title |
|---|
| 刘以鼎等: "济南电台制播网三级等保实施研究", 《有线电视技术》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110278465B (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107273267A (en) | Log analysis method based on elastic components | |
| CN102929903B (en) | Rapid video retrieval method based on layered structuralized description of video information | |
| CN105095052B (en) | Fault detection method under SOA environment and device | |
| CN106371986A (en) | Log treatment operation and maintenance monitoring system | |
| CN107229556A (en) | Log Analysis System based on elastic components | |
| CN107222472A (en) | A kind of user behavior method for detecting abnormality under Hadoop clusters | |
| CN101330406B (en) | System and method for monitoring WAP imperfect picture | |
| CN111259073A (en) | Intelligent business system running state studying and judging system based on logs, flow and business access | |
| CN111241050B (en) | Linkage analysis system and method for big data platform | |
| CN103475535A (en) | Log management system of cloud computing server | |
| CN110262949A (en) | Smart machine log processing system and method | |
| CN109905276A (en) | A kind of cloud service quality monitoring method and system | |
| US20180003513A1 (en) | Incident response analytic maps | |
| CN106021052A (en) | GJB289A bus testing system and method | |
| CN105141478A (en) | Method for monitoring state of sas card hard disk of linux server | |
| CN117235169A (en) | Wisdom fortune dimension data storage platform | |
| CN110505444A (en) | Safety defense monitoring system based on big data | |
| CN115309871A (en) | Industrial big data processing method and system based on artificial intelligence algorithm | |
| CN110278465A (en) | Video CDN distributed HTTP full link data collection and analysis method and system | |
| CN113222563A (en) | Audit data evidence storing and tracing system based on block chain | |
| CN117176441A (en) | System and method for detecting security log event of network equipment | |
| Sun et al. | LogPal: A generic anomaly detection scheme of heterogeneous logs for network systems | |
| CN115130994A (en) | Digital city management system | |
| CN113360313A (en) | Behavior analysis method based on massive system logs | |
| Sun et al. | Design of a Media Resource Management System for Colleges Based on Cloud Service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |