CN113824954A - OTT video quality monitoring method, device, equipment and storage medium - Google Patents

OTT video quality monitoring method, device, equipment and storage medium Download PDF

Info

Publication number
CN113824954A
CN113824954A CN202111392683.6A CN202111392683A CN113824954A CN 113824954 A CN113824954 A CN 113824954A CN 202111392683 A CN202111392683 A CN 202111392683A CN 113824954 A CN113824954 A CN 113824954A
Authority
CN
China
Prior art keywords
data
video quality
time
real
ott video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111392683.6A
Other languages
Chinese (zh)
Other versions
CN113824954B (en
Inventor
王子立
陆天钦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen SDMC Technology Co Ltd
Original Assignee
Shenzhen SDMC Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen SDMC Technology Co Ltd filed Critical Shenzhen SDMC Technology Co Ltd
Priority to CN202111392683.6A priority Critical patent/CN113824954B/en
Publication of CN113824954A publication Critical patent/CN113824954A/en
Application granted granted Critical
Publication of CN113824954B publication Critical patent/CN113824954B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00Diagnosis, testing or measuring for television systems or their details
    • H04N17/004Diagnosis, testing or measuring for television systems or their details for digital television systems

Abstract

The invention discloses a method, a device, equipment and a storage medium for monitoring the quality of an OTT video, wherein the method comprises the following steps: s1: acquiring original data by using a soft probe on an internet equipment terminal; s2: reporting the original data to a data platform; s3: calculating the original data through the data platform, and storing the calculation result into the data platform; s4: and monitoring the video quality based on the calculation result. The invention integrally processes the quality problem when a large amount of videos are played by using the data lake flow batch, solves the problems of insufficient computing power and long delivery time delay of a large data platform caused by the report of a large amount of data in a multi-user process, and realizes the query and early warning of real-time reports. And a real-time computing task state is introduced during real-time processing, so that data blockage and accumulation can be processed, and early warning abnormity or report time abnormity is avoided. And the same data processing logic is adopted when the data is processed in a flow batch mode, so that the development labor cost and the maintenance cost are reduced.

Description

OTT video quality monitoring method, device, equipment and storage medium
Technical Field
The invention relates to the technical field of internet videos, in particular to a method, a device, equipment and a storage medium for monitoring the OTT video quality.
Background
In recent years, the internet-based 0TT video service has been rapidly developed. Problems of picture starting delay, picture pause, poor picture definition and the like often occur in the playing process of the 0TT video, and user experience is influenced. Therefore, it is common practice to monitor the 0TT video quality in order to improve the user experience. In the prior art, a large data platform is mainly used for solving the problem of data volume and the problem of efficiency in data utilization, but the following problems still exist: 1. in the video detection process, data are reported regularly in a heartbeat mode and the abnormal data of the whole link are reported in a triggering mode, and under the condition of a large amount of equipment, the large data computing power is insufficient and the delivery time delay of a large data platform is long. 2. In terms of time sequence, if a user watches the event data without a network, the event data cannot be immediately returned, and only when the state is recovered, the click event is returned, so that in the process, the acquired data is delayed greatly, lost greatly and inaccurate, and early warning or report time is abnormal. 3. Real-time monitoring and data report presentation generally adopt a Lambda architecture, and for the same requirement, two teams are required to develop simultaneously, so that logic may be different, and finally result table inconsistency is caused, and the labor cost consumption is high.
Disclosure of Invention
Therefore, the technical problem to be solved by the present invention is to overcome the above-mentioned defects in the prior art, so as to provide a method, an apparatus, a device and a storage medium for monitoring OTT video quality.
The invention provides an OTT video quality monitoring method, which comprises the following steps:
s1: acquiring original data by using a soft probe on an internet equipment terminal;
s2: reporting the original data to a data platform;
s3: calculating the original data through the data platform, and storing the calculation result into the data platform;
s4: and monitoring the video quality based on the calculation result.
Furthermore, the data platform adopts a data lake architecture integrating stream and batch, and a data processing logic is used for representing the same service requirement in a computing layer.
Further, step S1 includes:
s11: installing a soft probe on an internet equipment terminal, wherein the internet equipment terminal comprises a mobile phone, a tablet personal computer, a television and a set top box;
s12: performing active test on the OTT video quality, wherein the active test comprises ping/Tracert/Http/video/bandwidth test indexes to obtain corresponding active test data;
s13: automatically identifying OTT video stream and generating OTT video quality index through Netfilter architecture and PF _ PACKET mode to obtain passive test data corresponding to QoS/QoE abnormal index;
s14: the active test data and the passive test data constitute original data.
Further, step S2 includes: the original data are accessed into the data lake in a kafka mode, and are stored into the data lake after passing through a unified ETL logic.
Further, step S3 includes:
s31: storing original data in the data lake, and then performing flow processing real-time calculation and batch processing off-line calculation of Flink, wherein SQL execution logic is adopted for flow processing and batch processing;
s32: and storing the flow processing real-time calculation result into a drive in the data lake, and storing the batch processing off-line calculation result into a Hive in the data lake.
Further, in step S31, the stream processing real-time calculation includes:
s311: performing aggregation calculation on the original data according to a time window;
s312: introducing a real-time computing task state based on the change rate of the time window index, and judging whether the time window index is stable;
s313: if the real-time computing task state is normal, the time window index is stable, and the service can be provided to the outside; and if the calculation is blocked and accumulated or the abnormality is already existed in the restarting process, continuously waiting for iterative processing until the real-time calculation task state is normal.
Further, step S4 includes:
s41: using Impala to make query, and synchronously covering off-line calculation results with OLAP to calculate results in real time;
s42: monitoring the data of the time window based on the real-time calculation result to realize the monitoring of the streaming media frame rate and the code rate second level; generating statistical reports of different dimensions based on the off-line calculation result; and (3) combining QoS abnormal index diagnosis comprising blocking, black screen and second opening, finding out the abnormal reason of the OTT video in time, and giving an early warning to the operation and maintenance to realize fault recovery and scheduling.
The invention also provides an OTT video quality monitoring device, which comprises:
the data acquisition module is used for acquiring original data by using a soft probe on an Internet equipment terminal;
the data reporting module is used for reporting the original data to the data platform;
the data calculation module is used for calculating the original data through the data platform and storing the calculation result into the data platform;
and the quality monitoring module is used for monitoring the video quality based on the calculation result.
The invention also provides a terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method as described above when executing the computer program.
The invention also provides a computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method as described above.
According to the OTT video quality monitoring method, device, equipment and storage medium provided by the invention, the quality problem of a large amount of video playing is integrally processed by using the data lake stream batch, so that the problems of insufficient calculation capacity and long delivery time delay of a large data platform caused by reporting of a large amount of data by multiple users are solved, and real-time report inquiry and early warning are realized. And a real-time computing task state is introduced during real-time processing, so that data blockage and accumulation can be processed, and early warning abnormity or report time abnormity is avoided. And the same data processing logic is adopted when the data is processed in a flow batch mode, so that the development labor cost and the maintenance cost are reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic flow chart of an OTT video quality monitoring method according to the present invention;
fig. 2 is a schematic structural diagram of an OTT video quality monitoring apparatus provided in the present invention;
fig. 3 is a schematic structural diagram of a terminal device provided in the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In addition, the technical features involved in the different embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Example 1
As shown in fig. 1, this embodiment provides an OTT video quality monitoring method, which includes the following steps:
s1: the method comprises the following steps of using a soft probe to collect original data on an internet equipment terminal, and specifically comprising the following steps:
s11: installing a soft probe on an internet equipment terminal, wherein the internet equipment terminal comprises a mobile phone, a tablet personal computer, a television and a set top box;
s12: performing active test on the OTT video quality, wherein the active test comprises ping/Tracert/Http/video/bandwidth test indexes to obtain corresponding active test data;
s13: automatically identifying OTT video stream and generating OTT video quality index through Netfilter architecture and PF _ PACKET mode to obtain passive test data corresponding to QoS/QoE abnormal index;
s14: the active test data and the passive test data constitute original data.
S2: reporting original data to a data platform, wherein the data platform adopts a data lake architecture integrating flow and batch, and a data processing logic is used for representing the same service requirement on a computing layer; the method specifically comprises the following steps: the original data are accessed into the data lake in a kafka mode, and are stored into the data lake after passing through a unified ETL logic.
S3: the method comprises the following steps of calculating original data through a data platform, and storing a calculation result into the data platform, and specifically comprises the following steps:
s31: storing original data in the data lake, and then performing flow processing real-time calculation and batch processing off-line calculation of Flink, wherein SQL execution logic is adopted for flow processing and batch processing; the stream processing real-time computation comprises: s311: performing aggregation calculation on the original data according to a time window; s312: introducing a real-time computing task state based on the change rate of the time window index, and judging whether the time window index is stable; s313: if the real-time computing task state is normal, the time window index is stable, and the service can be provided to the outside; if the calculation is stuck and piled up or the abnormality is already in the restarting process, continuously waiting for iterative processing until the real-time calculation task state is normal;
s32: and storing the flow processing real-time calculation result into a drive in the data lake, and storing the batch processing off-line calculation result into a Hive in the data lake.
S4: monitoring the video quality based on the calculation result, which specifically comprises the following steps:
s41: using Impala to make query, and synchronously covering off-line calculation results with OLAP to calculate results in real time;
s42: monitoring the data of the time window based on the real-time calculation result to realize the monitoring of the streaming media frame rate and the code rate second level; generating statistical reports of different dimensions based on the off-line calculation result, such as statistics of video quality conditions in the past time of different areas, daily high-frequency time periods in which abnormality occurs in the past 7 days, and the like; and (3) combining QoS abnormal index diagnosis comprising blocking, black screen and second opening, finding out the abnormal reason of the OTT video in time, and giving an early warning to the operation and maintenance to realize fault recovery and scheduling.
In this embodiment, the data lake itself can support both streaming and batch reading and writing, and the data lake itself can be consumed in real time, so it can perform both real-time calculation and off-line calculation, and then write the data back to the data lake in a unified manner. When the query is made, the offline calculation result and the real-time calculation result are used for unified integration, and the offline calculation result covers the real-time calculation result so as to save the storage space.
The invention solves the problems of insufficient computing power and long delivery time delay of a large data platform caused by the report of a large amount of data in multiple users by integrally processing the quality problem of a large amount of video playing in batches of the data lake streams, and realizes the query and early warning of real-time reports. And a real-time computing task state is introduced during real-time processing, so that data blockage and accumulation can be processed, and early warning abnormity or report time abnormity is avoided. And the same data processing logic is adopted when the data is processed in a flow batch mode, so that the development labor cost and the maintenance cost are reduced.
Example 2
As shown in fig. 2, this embodiment provides an OTT video quality monitoring apparatus, including:
the data acquisition module is used for acquiring original data by using a soft probe on an Internet equipment terminal;
the data reporting module is used for reporting the original data to the data platform;
the data calculation module is used for calculating the original data through the data platform and storing the calculation result into the data platform;
and the quality monitoring module is used for monitoring the video quality based on the calculation result.
It should be understood that the apparatus corresponds to the OTT video quality monitoring method in embodiment 1, and can perform the steps related to the above method embodiments, and the specific functions of the apparatus can be referred to the description above, and in order to avoid repetition, the detailed description is appropriately omitted here. The device includes at least one software function that can be stored in memory in the form of software or firmware (firmware) or solidified in the Operating System (OS) of the device.
Example 3
As shown in fig. 3, this embodiment provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the OTT video quality monitoring method in embodiment 1.
The present embodiment also provides a computer-readable storage medium, where a computer program is stored, and when being executed by a processor, the computer program implements the OTT video quality monitoring method in embodiment 1.
The storage medium may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic Memory, a flash Memory, a magnetic disk, or an optical disk.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules of the embodiments in the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.

Claims (10)

1. An OTT video quality monitoring method is characterized by comprising the following steps:
s1: acquiring original data by using a soft probe on an internet equipment terminal;
s2: reporting the original data to a data platform;
s3: calculating the original data through the data platform, and storing the calculation result into the data platform;
s4: and monitoring the video quality based on the calculation result.
2. The OTT video quality monitoring method according to claim 1, wherein the data platform employs a stream-batch-integrated data lake architecture, which uses a data processing logic at a computing layer to represent the same service requirement.
3. The OTT video quality monitoring method according to claim 2, wherein the step S1 includes:
s11: installing a soft probe on an internet equipment terminal, wherein the internet equipment terminal comprises a mobile phone, a tablet personal computer, a television and a set top box;
s12: performing active test on the OTT video quality, wherein the active test comprises ping/Tracert/Http/video/bandwidth test indexes to obtain corresponding active test data;
s13: automatically identifying OTT video stream and generating OTT video quality index through Netfilter architecture and PF _ PACKET mode to obtain passive test data corresponding to QoS/QoE abnormal index;
s14: the active test data and the passive test data constitute original data.
4. The OTT video quality monitoring method according to claim 3, wherein the step S2 includes: the original data are accessed into the data lake in a kafka mode, and are stored into the data lake after passing through a unified ETL logic.
5. The OTT video quality monitoring method according to claim 4, wherein the step S3 comprises:
s31: storing original data in the data lake, and then performing flow processing real-time calculation and batch processing off-line calculation of Flink, wherein SQL execution logic is adopted for flow processing and batch processing;
s32: and storing the flow processing real-time calculation result into a drive in the data lake, and storing the batch processing off-line calculation result into a Hive in the data lake.
6. The OTT video quality monitoring method according to claim 5, wherein in step S31, the real-time calculation of stream processing includes:
s311: performing aggregation calculation on the original data according to a time window;
s312: introducing a real-time computing task state based on the change rate of the time window index, and judging whether the time window index is stable;
s313: if the real-time computing task state is normal, the time window index is stable, and the service can be provided to the outside; and if the calculation is blocked and accumulated or the abnormality is already existed in the restarting process, continuously waiting for iterative processing until the real-time calculation task state is normal.
7. The OTT video quality monitoring method according to claim 6, wherein the step S4 comprises:
s41: using Impala to make query, and synchronously covering off-line calculation results with OLAP to calculate results in real time;
s42: monitoring the data of the time window based on the real-time calculation result to realize the monitoring of the streaming media frame rate and the code rate second level; generating statistical reports of different dimensions based on the off-line calculation result; and (3) combining QoS abnormal index diagnosis comprising blocking, black screen and second opening, finding out the abnormal reason of the OTT video in time, and giving an early warning to the operation and maintenance to realize fault recovery and scheduling.
8. An OTT video quality monitoring apparatus, comprising:
the data acquisition module is used for acquiring original data by using a soft probe on an Internet equipment terminal;
the data reporting module is used for reporting the original data to the data platform;
the data calculation module is used for calculating the original data through the data platform and storing the calculation result into the data platform;
and the quality monitoring module is used for monitoring the video quality based on the calculation result.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
CN202111392683.6A 2021-11-23 2021-11-23 OTT video quality monitoring method, device, equipment and storage medium Active CN113824954B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111392683.6A CN113824954B (en) 2021-11-23 2021-11-23 OTT video quality monitoring method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111392683.6A CN113824954B (en) 2021-11-23 2021-11-23 OTT video quality monitoring method, device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113824954A true CN113824954A (en) 2021-12-21
CN113824954B CN113824954B (en) 2022-02-08

Family

ID=78919707

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111392683.6A Active CN113824954B (en) 2021-11-23 2021-11-23 OTT video quality monitoring method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113824954B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106549794A (en) * 2015-09-23 2017-03-29 中国移动通信集团公司 A kind of mass monitoring system of OTT business, apparatus and method
CN107465526A (en) * 2016-06-03 2017-12-12 德科仕通信(上海)有限公司 Internet video CDN server mass monitoring system and method
CN109684352A (en) * 2018-12-29 2019-04-26 江苏满运软件科技有限公司 Data analysis system, method, storage medium and electronic equipment
CN111815449A (en) * 2020-07-13 2020-10-23 上证所信息网络有限公司 Flow calculation-based anomaly detection method and system for multi-host market quotation system
CN113568938A (en) * 2021-08-04 2021-10-29 北京百度网讯科技有限公司 Data stream processing method and device, electronic equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106549794A (en) * 2015-09-23 2017-03-29 中国移动通信集团公司 A kind of mass monitoring system of OTT business, apparatus and method
CN107465526A (en) * 2016-06-03 2017-12-12 德科仕通信(上海)有限公司 Internet video CDN server mass monitoring system and method
CN109684352A (en) * 2018-12-29 2019-04-26 江苏满运软件科技有限公司 Data analysis system, method, storage medium and electronic equipment
CN111815449A (en) * 2020-07-13 2020-10-23 上证所信息网络有限公司 Flow calculation-based anomaly detection method and system for multi-host market quotation system
CN113568938A (en) * 2021-08-04 2021-10-29 北京百度网讯科技有限公司 Data stream processing method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN113824954B (en) 2022-02-08

Similar Documents

Publication Publication Date Title
JP4942939B2 (en) Method and system for troubleshooting misconfiguration of a computer system based on the configuration of another computer system
EP3849142B1 (en) Automatic root cause diagnosis in networks
US9842017B1 (en) Collection and aggregation of device health metrics
US7936260B2 (en) Identifying redundant alarms by determining coefficients of correlation between alarm categories
US8914504B2 (en) End user performance monitoring for mobile applications
US9584395B1 (en) Adaptive metric collection, storage, and alert thresholds
CN112422344A (en) Log abnormity warning method and device, storage medium and electronic device
US20220050765A1 (en) Method for processing logs in a computer system for events identified as abnormal and revealing solutions, electronic device, and cloud server
CN110362455B (en) Data processing method and data processing device
Song et al. Q-score: Proactive service quality assessment in a large IPTV system
US11144376B2 (en) Veto-based model for measuring product health
CN101197621A (en) Method and system for remote diagnosing and locating failure of network management system
CN110149653A (en) A kind of cloud fault of mobile phone monitoring method and system
CN101399883A (en) Exception monitoring management method and device
US8166162B2 (en) Adaptive customer-facing interface reset mechanisms
US20210357281A1 (en) Using User Equipment Data Clusters and Spatial Temporal Graphs of Abnormalities for Root Cause Analysis
CN113824954B (en) OTT video quality monitoring method, device, equipment and storage medium
CN108039956A (en) Using monitoring method, system and computer-readable recording medium
CN109840178A (en) A kind of method, mobile terminal and the device of monitoring journal information
CN109547271B (en) Network state real-time monitoring alarm system based on big data
CN110543410A (en) Method for processing cluster index, method and device for inquiring cluster index
US20110173165A1 (en) Management of performance data
CN114500178B (en) Self-operation intelligent Internet of things gateway
CN116781568A (en) Data monitoring alarm method, device, equipment and storage medium
WO2016076841A1 (en) Systems and methods of measuring quality of video surveillance infrastructure

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant