CN115134617B - Live broadcast quality inspection system, live broadcast quality inspection method, web server and storage medium - Google Patents
Live broadcast quality inspection system, live broadcast quality inspection method, web server and storage medium Download PDFInfo
- Publication number
- CN115134617B CN115134617B CN202210682572.7A CN202210682572A CN115134617B CN 115134617 B CN115134617 B CN 115134617B CN 202210682572 A CN202210682572 A CN 202210682572A CN 115134617 B CN115134617 B CN 115134617B
- Authority
- CN
- China
- Prior art keywords
- monitor
- quality inspection
- live
- live broadcast
- data stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 222
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000003860 storage Methods 0.000 title claims abstract description 14
- 230000002159 abnormal effect Effects 0.000 claims abstract description 94
- 238000004891 communication Methods 0.000 claims abstract description 70
- 238000012544 monitoring process Methods 0.000 claims abstract description 39
- 238000009826 distribution Methods 0.000 claims description 64
- 239000002131 composite material Substances 0.000 claims description 33
- 238000007711 solidification Methods 0.000 claims description 15
- 230000008023 solidification Effects 0.000 claims description 15
- 230000000007 visual effect Effects 0.000 claims description 13
- 230000003068 static effect Effects 0.000 claims description 11
- 238000004064 recycling Methods 0.000 claims description 5
- 238000011084 recovery Methods 0.000 claims description 3
- 238000011161 development Methods 0.000 description 14
- 239000008186 active pharmaceutical agent Substances 0.000 description 13
- 238000001514 detection method Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 11
- 238000012545 processing Methods 0.000 description 11
- 230000005856 abnormality Effects 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 9
- 230000003111 delayed effect Effects 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 238000005538 encapsulation Methods 0.000 description 4
- 150000001875 compounds Chemical class 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 241000256836 Apis Species 0.000 description 2
- 238000012550 audit Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000003973 irrigation Methods 0.000 description 2
- 230000002262 irrigation Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 102100023413 GRB2-related adapter protein Human genes 0.000 description 1
- 101000829735 Homo sapiens GRB2-related adapter protein Proteins 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000008713 feedback mechanism Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
- H04L51/046—Interoperability with other network applications or services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Databases & Information Systems (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The invention discloses a live broadcast quality inspection system, a live broadcast quality inspection method, a Web server and a storage medium. The system comprises: the system comprises a quality inspection and checking module, an instant communication module and a data display module; the quality inspection auditing module is respectively connected with the instant communication module and the data display module, the instant communication module is connected with external instant communication software, and the data display module is connected with the Web front end; the quality inspection auditing module is used for acquiring and monitoring the live broadcast data stream; the instant communication module is used for acquiring abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and outputting the abnormal data; the data display module is used for acquiring the abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and displaying the abnormal data. The invention aims to provide a real-time and flexible live broadcast quality inspection system, which realizes high-efficiency multi-dimensional quality inspection and abnormal real-time notification.
Description
Technical Field
The invention relates to the technical field of audio and video processing, in particular to a live broadcast quality inspection system, a live broadcast quality inspection method, a Web server and a storage medium.
Background
Currently, in the field of audio and video processing, a large number of audio and video implementation libraries such as huffyuv, webRTC (Web Real-Time Communication, web instant messaging), live555 and the like are emerging. As an outstanding development technology in the field, FFmpeg (Fast Forward Mpeg, fast-forward Mpeg) is a preferred choice for audio and video developers by virtue of active community atmosphere, sound protocol support, powerful functional coverage and flexible development entrance. By means of rich command instructions and directed acyclic graph model design provided by FFmpeg, a developer can disassemble service requirements into DAGs (DIRECTEDACYCLIC GRAPH) by using basic operations such as encapsulation, encoding and decoding, filtering and the like as nodes; the audio and video data are input by the DAG entrance, and the node with the output degree of 0 outputs the processing result, so that the live broadcast quality inspection is realized. The working environment of the mode only needs to install FFmpeg, and a developer only needs to convert the requirement into a shell instruction. However, the method has the defect of insufficient flexibility, a developer can only adopt some basic components provided by FFmpeg, and high-order custom logic is not easy to access; when the service complexity is slightly high, the requirement is complicated to disassemble, and the instruction is inconvenient to splice.
Disclosure of Invention
The embodiment of the invention mainly aims to provide a live broadcast quality inspection system, a live broadcast quality inspection method, a Web server and a storage medium, and aims to provide a set of real-time and flexible live broadcast quality inspection system for realizing efficient multi-dimensional quality inspection and abnormal real-time notification.
In order to achieve the above objective, an embodiment of the present invention provides a live broadcast quality inspection system, where the live broadcast quality inspection system is developed based on JavaCV, FFmpeg, openCV, and the live broadcast quality inspection system includes:
The system comprises a quality inspection and checking module, an instant communication module and a data display module; the quality inspection auditing module is respectively connected with the instant communication module and the data display module, the instant communication module is connected with external instant communication software, and the data display module is connected with the Web front end;
the quality inspection auditing module is used for acquiring and monitoring the live broadcast data stream;
the instant communication module is used for acquiring abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and outputting the abnormal data;
the data display module is used for acquiring the abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and displaying the abnormal data.
Optionally, the quality inspection auditing module includes a monitor abstraction layer, an asynchronous frame distribution queue, a visual frame monitor abstraction layer, a composite monitor abstraction layer, a sound frame monitor abstraction layer, an image distribution queue, a composite frame distribution queue, an audio distribution queue, a resource recycling layer, a black screen monitor, a fuzzy monitor, a static field monitor, a brightness monitor, a two-dimensional code monitor, an image auditing monitor, a flower screen monitor, a solidification monitor, a silence monitor, and a voice auditing monitor;
the monitor abstract layer is connected with the asynchronous frame distribution queue, the asynchronous frame distribution queue is respectively connected with the visual frame monitor abstract layer, the composite monitor abstract layer and the audio frame monitor abstract layer, the visual frame abstract layer is connected with the image distribution queue, the image distribution queue is respectively connected with the black screen monitor, the fuzzy monitor, the static field monitor, the brightness monitor, the two-dimensional code monitor and the image auditing monitor, the composite monitor abstract layer is connected with the composite frame distribution queue, the composite frame distributing queue is connected with the screen monitor and the solidification monitor, the sound frame monitor is connected with the audio distributing queue, the audio distributing queue is connected with the mute monitor and the voice checking monitor, the black screen monitor, the fuzzy monitor, the mute monitor, the brightness monitor and the two-dimensional code monitor are connected with the image checking monitor, the screen monitor, the solidification monitor, the mute monitor and the voice checking monitor respectively and connected with the resource recycling layer.
Optionally, the data presentation module is further configured to:
acquiring the live broadcast data stream and displaying a live broadcast state corresponding to the live broadcast data stream;
Displaying live comments contained in the live data stream;
and summarizing the abnormal data contained in the live data stream to obtain an analysis report, and displaying the analysis report.
The embodiment of the invention provides a live broadcast quality inspection method, which comprises the following steps:
acquiring a data stream through a quality inspection and checking module and monitoring the data stream in real time;
If the live broadcast data stream is monitored to contain abnormal data through the quality inspection auditing module, outputting the abnormal data through the instant communication module, and displaying the abnormal data through the data display module.
Optionally, the quality inspection auditing module includes a monitor abstraction layer and an asynchronous frame distribution queue, and the step of acquiring and monitoring the live data stream includes:
Acquiring the live data stream through the monitor abstraction layer, and writing the live data stream into the asynchronous frame distribution queue after unpacking;
The live broadcast data stream is distributed to a corresponding monitor after being decoded through the asynchronous frame distribution queue;
And monitoring the live data stream through a monitor corresponding to the live data stream.
Optionally, the step of acquiring the data stream through the quality inspection auditing module and monitoring in real time includes:
The abnormal data is converted into communication information through the instant communication module;
And outputting the communication information to external instant communication software through the instant communication module.
Optionally, the step of acquiring the data stream through the quality inspection auditing module and monitoring in real time includes:
if the quality inspection auditing module monitors that the live broadcast corresponding to the live broadcast data stream is not timely started, the quality inspection auditing module judges whether the live broadcast corresponding to the live broadcast data stream is open broadcast or not;
If the quality inspection auditing module monitors that the live broadcast corresponding to the live broadcast data stream is not open broadcast, determining that the live broadcast corresponding to the live broadcast data stream is slow broadcast, and outputting the information of the live broadcast corresponding to the live broadcast data stream through the instant messaging module and the data display module.
Optionally, the step of acquiring the data stream through the quality inspection auditing module and monitoring in real time includes:
judging whether the live data stream is continuous or not through the quality inspection auditing module;
if the live data stream is continuous, continuously monitoring the live data stream through the quality inspection auditing module;
and if the quality inspection auditing module detects that the live broadcast data flow is stopped, stopping monitoring the live broadcast data flow through the quality inspection auditing module.
In addition, to achieve the above object, the present invention also provides a Web server including: the system comprises a memory, a processor and a live broadcast quality inspection system which is stored in the memory and can run on the processor, wherein the live broadcast quality inspection program is executed by the processor to realize the steps of the live broadcast quality inspection system.
In addition, in order to achieve the above object, the present invention further provides a computer readable storage medium, on which a program for live quality inspection is stored, the program for live quality inspection realizing the steps of the live quality inspection system as described above when being executed by a processor.
The embodiment of the invention provides a live broadcast quality inspection system, a live broadcast quality inspection method, a Web server and a storage medium, wherein the live broadcast quality inspection system comprises: the system comprises a quality inspection and checking module, an instant communication module and a data display module; the quality inspection auditing module is respectively connected with the instant communication module and the data display module, the instant communication module is connected with external instant communication software, and the data display module is connected with the Web front end; the quality inspection auditing module is used for acquiring and monitoring the live broadcast data stream; the instant communication module is used for acquiring abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and outputting the abnormal data; the data display module is used for acquiring the abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and displaying the abnormal data. According to the live broadcast quality inspection system, the quality inspection auditing module monitors live broadcast data streams in real time, when the quality inspection auditing module monitors that abnormal data exist in the live broadcast data streams, the abnormal data are output through the instant communication module and the data display module, so that a user can know that the abnormal data exist in the live broadcast data streams in multiple ways, and real-time communication of the live broadcast quality inspection process is realized. Because the quality inspection auditing module, the instant communication module and the data display module have high expansibility, the quality inspection auditing module, the instant communication module and the data display module can be expanded according to requirements and connected with various devices, the problem of insufficient flexibility of live broadcast quality inspection is solved, and the quality inspection efficiency is improved.
Drawings
FIG. 1 is a schematic diagram of a functional module of a Web server to which a live broadcast quality inspection device of the present invention belongs;
FIG. 2 is a schematic diagram of functional modules of the live quality inspection system of the present invention;
FIG. 3 is a block diagram of a live quality inspection system according to the present invention;
FIG. 4 is a schematic diagram of an interface display involved in the live quality inspection system of the present invention;
fig. 5 is a schematic diagram of a live comment analysis PC end related to the live broadcast quality inspection system of the present invention;
FIG. 6 is a schematic diagram of a live comment analysis mobile terminal related to a live quality inspection system of the present invention;
FIG. 7 is a schematic diagram of a detection lifecycle related to the live quality inspection system of the present invention;
fig. 8 is a data flow diagram of a quality inspection auditing module related to the live quality inspection system of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The main solution of the embodiment of the invention is to disclose a live broadcast quality inspection system, which comprises: the system comprises a quality inspection and checking module, an instant communication module and a data display module; the quality inspection auditing module is respectively connected with the instant communication module and the data display module, the instant communication module is connected with external instant communication software, and the data display module is connected with the Web front end; the quality inspection auditing module is used for acquiring and monitoring the live broadcast data stream; the instant communication module is used for acquiring abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and outputting the abnormal data; the data display module is used for acquiring the abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and displaying the abnormal data.
Technical terms related to the embodiment of the invention:
robustness: robust is a transliteration of Robust, i.e., a Robust and strong meaning. It also represents the ability of the system to survive abnormal and dangerous situations. For example, in the case of input errors, disk failures, network overloads, or intentional attacks, computer software can be crashed or crashed, or the robustness of the software. By "robustness" is also meant the characteristic of the control system to maintain certain other properties under perturbation of certain (structural, size) parameters. The stability robustness and the performance robustness can be classified according to different definitions of performance. A fixed controller designed to target the robustness of a closed loop system is called a robust controller.
JavaCV: javaCV is a packaging library composed of a plurality of open source computer vision libraries based on JavaCPP calling mode (one-layer packaging of JNI), and encapsulates common libraries and utility classes in the computer vision field including FFmpeg, openCV, tensorflow, caffe, tesseract, libdc1394, openKinect, videoInput, ARToolKitPlus, etc.
Huffyuv: huffyuv is a lossless video compression coding method, invented by Ben Rudiak-Gould, for replacing uncompressed YCbCr as acquisition format.
WebRTC: (Web Real-Time Communication, web instant messaging), an API that supports a Web browser for Real-time voice conversations or video conversations.
Live555: is a cross-platform C++ open source project providing a solution for streaming media, which implements support for standard streaming media transport protocols such as RTP/RTCP, RTSP, SIP and the like. live555 enables streaming, receiving, processing, etc. support of audio and video data in a variety of audio and video coding formats, including MPEG, h.263+, DV, JPEG video, and a variety of audio encodings. Meanwhile, due to good design, live555 is very easy to expand support for other formats. live555 has been used for the realization of streaming media playing functions of a multi-player, such as VLC (video lan), mplyer.
FFmpeg: (Fast Forward Mpeg ) FFmpeg is a set of open source computer programs that can be used to record, convert digital audio, video, and convert it into streams. LGPL or GPL licenses are employed. It provides a complete solution for recording, converting and streaming audio and video. It contains a very advanced audio/video codec library libavcodec, and many codes in libavcodec have been developed from scratch in order to guarantee high portability and codec quality. FFmpeg was developed under the Linux platform, but it could just as well be compiled to run in other operating system environments, including Windows, mac OS X, etc.
DAG: (DIRECTEDACYCLIC GRAPH ), which is a graph structure in graph theory, consists of a limited number of vertexes and 'directed edges', and cannot return to the starting vertex after any vertex passes through a plurality of directed edges.
Shell: in computer science, shell is commonly called shell (to distinguish from core), which refers to software (command interpreter, command parser) that "provides an operation interface for user". It is similar to command. Com and later cmd. Exe under DOS. It receives the user command and then invokes the corresponding application.
API: (Application Programming Interface, application program interface), the main purpose of the API is to provide the application and developer with the ability to access a set of routines without having to access source code or understand the details of the internal operating mechanisms. Software that provides the functionality defined by an API is referred to as an implementation of this API. An API is an interface and therefore an abstraction.
ASR: (Automatic Speech Recognition, automatic speech recognition technology), a technology that converts human speech into text.
IM: (INSTANT MESSAGING, real-time communication) is a real-time communication system that allows two or more people to communicate text messages, files, voice and video in real time using a network.
The detection effect of live broadcast quality inspection depends on the technical model selection in the audio and video field. In the field of audio and video processing, audio and video protocols proposed by various manufacturers are various, a large number of audio and video implementation libraries such as huffyuv, webRTC, live and 555 are emerging, and various schemes are updated continuously along with development and popularization of audio and video hardware. While the technology is outstanding as the development technology in the field, FFmpeg is a first choice for audio and video developers by virtue of active community atmosphere, sound protocol support, powerful functional coverage and flexible development entrance. In the field of live broadcast quality inspection, FFmpeg becomes a medium-flow column for live broadcast quality inspection in the industry by virtue of strong audio and video processing capability. Currently, solutions for live broadcast quality inspection mainly include the following three types:
(1) WebRTC-based tool development
WebRTC integrates full-line functions such as audio and video acquisition, encoding and decoding, transmission, display and the like, is incorporated into the H5 standard, and is favored by vast developers. However, the technology has obvious defects of unstable transmission quality, incomplete protocol compatibility, and the like, and needs to be further perfected and developed.
(2) FFmpeg instruction set based development
With the rich command line instructions provided by FFmpeg and the directed acyclic graph model (DIRECTED ACYCLIC GRAP, DAG) design, developers can disassemble business requirements into DAGs with the basic operations of encapsulation, encoding, decoding, filtering, etc. as nodes. The audio and video data are input by the DAG entry, and the processing result is output by the node with the output degree of 0. The working environment of the mode only needs to install FFmpeg; the development is simple and convenient, and the developer only needs to convert the requirement into a shell instruction. But this approach also has significant drawbacks: firstly, the flexibility is insufficient, a developer can only adopt some basic components provided by FFmpeg, and high-order custom logic is not easy to access; when the service complexity is slightly high, the disassembly is complicated, and the instruction is inconvenient to splice; the audio and video data stream flows in the DAG relatively to the black box, is not easy to control, and is inconvenient to interact with an external system.
(3) FFMPEG API-based development
To more flexibly process audio-video data, some developers have chosen to develop through FFmpeg native APIs. The technical scheme gives the developer great development freedom, but the original API is too scattered, so that the technical details which the developer needs to pay attention to are more, the development cost is high, and the project period is long. Other advanced audio and video developers provide high-order API WRAPPER in an open source class library manner through further encapsulation on the basis of the original FFmpeg. The development threshold of a developer is reduced, and the realization of the service value is accelerated. A typical scheme is Python Wrapper integrated with computer vision ecology, and the scheme brings a larger imagination space for an audio and video application scene. However, due to the high performance requirements of live quality inspection, some performance metrics are sacrificed by the Python Wrapper. Meanwhile, most of enterprise services are currently constructed on Java services, and the scheme of Java Wrapper is more in line with the development direction of enterprises on environmental equipment and service operation and maintenance.
The invention provides a solution, and aims to provide a real-time and flexible live broadcast quality inspection system for realizing efficient multi-dimensional quality inspection and abnormal real-time notification.
Specifically, referring to fig. 1, fig. 1 is a schematic diagram of a Web service deployment of the live quality inspection system of the present invention. The live broadcast quality inspection method is deployed on a Web server in a software package form, and provides a front-end page. The user can access for use through the intelligent mobile terminal and the fixed terminal equipment with data processing capability.
In this embodiment, the live quality inspection system is composed of a presentation layer 110, a logic layer 120, a data layer 130, and a network layer 140.
The display layer 110 is mainly an interface, a page, etc. for interaction with an external system and a user; the logic layer 120 encapsulates a core program of live quality inspection; the data layer 130 provides persistence, caching, etc. of data generated during system operation; the network layer 140 provides support for HTTP, RTMP protocols.
Wherein the core logic code in the logic layer 120, when executed, performs the steps of:
acquiring a data stream through a quality inspection and checking module and monitoring the data stream in real time;
If the live broadcast data stream is monitored to contain abnormal data through the quality inspection auditing module, outputting the abnormal data through the instant communication module, and displaying the abnormal data through the data display module.
Further, the core logic code in the logic layer 120, when executed, further performs the steps of:
The abnormal data is converted into communication information through the instant communication module;
And outputting the communication information to external instant communication software through the instant communication module.
Further, the core logic code in the logic layer 120, when executed, further performs the steps of:
Acquiring the live data stream through the monitor abstraction layer, and writing the live data stream into the asynchronous frame distribution queue after unpacking;
The live broadcast data stream is distributed to a corresponding monitor after being decoded through the asynchronous frame distribution queue;
And monitoring the live data stream through a monitor corresponding to the live data stream.
Further, the core logic code in the logic layer 120, when executed, further performs the steps of:
if the quality inspection auditing module monitors that the live broadcast corresponding to the live broadcast data stream is not timely started, the quality inspection auditing module judges whether the live broadcast corresponding to the live broadcast data stream is open broadcast or not;
If the quality inspection auditing module monitors that the live broadcast corresponding to the live broadcast data stream is not open broadcast, determining that the live broadcast corresponding to the live broadcast data stream is slow broadcast, and outputting the information of the live broadcast corresponding to the live broadcast data stream through the instant messaging module and the data display module.
Further, the core logic code in the logic layer 120, when executed, further performs the steps of:
judging whether the live data stream is continuous or not through the quality inspection auditing module;
if the live data stream is continuous, continuously monitoring the live data stream through the quality inspection auditing module;
and if the quality inspection auditing module detects that the live broadcast data flow is stopped, stopping monitoring the live broadcast data flow through the quality inspection auditing module.
According to the embodiment, through the scheme, the quality inspection auditing module is used for acquiring the data stream and monitoring the data stream in real time; if the live broadcast data stream is monitored to contain abnormal data through the quality inspection auditing module, outputting the abnormal data through the instant communication module, and displaying the abnormal data through the data display module. According to the live broadcast quality inspection system, the quality inspection auditing module monitors live broadcast data streams in real time, when the quality inspection auditing module monitors that abnormal data exist in the live broadcast data streams, the abnormal data are output through the instant communication module and the data display module, so that a user can know that the abnormal data exist in the live broadcast data streams in multiple ways, and real-time communication of the live broadcast quality inspection process is realized. Because the quality inspection auditing module, the instant communication module and the data display module have high expansibility, the quality inspection auditing module, the instant communication module and the data display module can be expanded according to requirements and connected with various devices, the problem of insufficient flexibility of live broadcast quality inspection is solved, and the quality inspection efficiency is improved.
Referring to fig. 2, fig. 2 is a schematic diagram of functional modules of the live quality inspection system according to the present invention. The live broadcast quality inspection system comprises:
the quality inspection auditing module 10, the instant communication module 20 and the data display module 30; the quality inspection auditing module 10 is respectively connected with the instant communication module 20 and the data display module 30, the instant communication module 20 is connected with external instant communication software, and the data display module 30 is connected with the Web front end;
the quality inspection auditing module 10 is used for acquiring and monitoring the live data stream;
The instant communication module 20 is configured to obtain abnormal data included in the live broadcast data stream monitored by the quality inspection auditing module, and output the abnormal data;
the data display module 30 is configured to obtain abnormal data included in the live data stream monitored by the quality inspection auditing module, and display the abnormal data.
The quality inspection checking module 10 comprises a monitor abstract layer, an asynchronous frame distribution queue, a visual frame monitor abstract layer, a composite monitor abstract layer, a sound frame monitor abstract layer, an image distribution queue, a composite frame distribution queue, an audio distribution queue, a resource recovery layer, a black screen monitor, a fuzzy monitor, a static field monitor, a brightness monitor, a two-dimensional code monitor, an image checking monitor, a flower screen monitor, a solidification monitor, a silence monitor and a voice checking monitor;
the monitor abstract layer is connected with the asynchronous frame distribution queue, the asynchronous frame distribution queue is respectively connected with the visual frame monitor abstract layer, the composite monitor abstract layer and the audio frame monitor abstract layer, the visual frame abstract layer is connected with the image distribution queue, the image distribution queue is respectively connected with the black screen monitor, the fuzzy monitor, the static field monitor, the brightness monitor, the two-dimensional code monitor and the image auditing monitor, the composite monitor abstract layer is connected with the composite frame distribution queue, the composite frame distributing queue is connected with the screen monitor and the solidification monitor, the sound frame monitor is connected with the audio distributing queue, the audio distributing queue is connected with the mute monitor and the voice checking monitor, the black screen monitor, the fuzzy monitor, the mute monitor, the brightness monitor and the two-dimensional code monitor are connected with the image checking monitor, the screen monitor, the solidification monitor, the mute monitor and the voice checking monitor respectively and connected with the resource recycling layer.
The data presentation module 30 is further configured to:
acquiring the live broadcast data stream and displaying a live broadcast state corresponding to the live broadcast data stream;
Displaying live comments contained in the live data stream;
and summarizing the abnormal data contained in the live data stream to obtain an analysis report, and displaying the analysis report.
Based on the Web server architecture, but not limited to the above architecture, a first embodiment of the live quality inspection method of the present invention is presented. The live broadcast quality inspection method comprises the following steps:
And step S101, acquiring a data stream through a quality inspection auditing module and monitoring in real time.
Step S102, if the quality inspection auditing module monitors that the live broadcast data stream contains abnormal data, the abnormal data is output through the instant messaging module, and the abnormal data is displayed through the data display module.
The execution main body of the method of the embodiment can be an intelligent mobile terminal with data processing capability and a fixed Web server, and the method of the embodiment is deployed on the Web server in a software package form and provides a front-end page. Based on JavaCV as a technical scheme base stone, the embodiment can realize real-time quality inspection of live multi-mode content by using FFmpeg, openCV and other bottom layer libraries to carry out high-order custom of service logic. Compared with the mode of C and Python writers, the scheme of the embodiment has better affinity for enterprise server development, and the capability of Java ecological service can be fully utilized to expand the audio and video scene boundary. As shown in fig. 3, fig. 3 is a structural diagram related to a live broadcast quality inspection system, which includes a quality inspection auditing module, an instant messaging module, and a data display module; the quality inspection auditing module is used for acquiring and auditing the live broadcast data stream, the instant communication module is used for outputting auditing information acquired by the quality inspection auditing module to operators and a host in the form of information, and the data display module is used for displaying live broadcast contained in the live broadcast data stream, abnormal conditions confirmed by the quality inspection auditing module and the like so as to display a front-end picture. Because the live broadcast quality inspection system has high expansibility, if new algorithms such as green screen, picture shaking and the like are developed in the later stage, the live broadcast quality inspection system can be conveniently integrated into the framework, and other codes (such as various notification and statistical codes do not need to be changed) are hardly changed.
In order to solve the problem of insufficient flexibility of live broadcast quality inspection, after a live broadcast task is created by a host, a quality inspection auditing module allocates monitoring resources for the live broadcast task and creates a monitor, and the quality inspection auditing module integrates high-order algorithms such as real-time ASR, image auditing, text analysis and the like; the quality inspection auditing module can analyze live audio signals according to the real-time ASR, analyze live video images according to the image auditing algorithm, and analyze live comment characters according to the text analysis algorithm. After the anchor is opened, the live broadcast data stream of the anchor is acquired and monitored through a quality inspection auditing module so as to analyze and audit the live broadcast data stream and ensure the smooth proceeding of the live broadcast.
If the live broadcast data stream is monitored to contain abnormal data through the quality inspection auditing module, if the live broadcast corresponding to the live broadcast data stream is silent, pornographic, solidified, two-dimensional code, black field, violence and the like, the abnormal data is output through the instant communication module, and the abnormal data is displayed through the data display module, so that a host and operators know the problem existing in the current live broadcast, the fault is solved in time, and the normal broadcasting of the live broadcast is ensured.
Wherein, step S101 includes:
and A1, displaying a live broadcast state corresponding to the live broadcast data stream through the data display module.
And step A2, displaying the live comment contained in the live data stream through the data display module.
And step A3, summarizing abnormal data contained in the live data stream through the data display module to obtain an analysis report, and displaying the analysis report.
When the anchor is opened, firstly, the live state of the live data stream is confirmed through a quality inspection and auditing module, for example: live broadcast is broadcast in a delayed mode, live broadcast is broadcast in an open mode, live broadcast is broadcast on time, and the like, and then a live broadcast state corresponding to the live broadcast data stream is displayed through the data display module. And extracting live comment and live comment of the live broadcast data stream according to a text analysis algorithm by a quality inspection auditing module, and displaying the live comment and live comment contained in the live broadcast data stream by a data display module. And obtaining abnormal data according to the auditing live broadcast data stream through the quality inspection auditing module, displaying the abnormal data contained in the live broadcast data stream through the data display module, and/or summarizing the abnormal data contained in the live broadcast data stream through the data display module to obtain an analysis report, and displaying the analysis report.
Specifically, as shown in fig. 4, in the page provided by the data display module, the quality inspector or operator interface can observe on the upper side which live broadcasts currently have problems. The live broadcast task which is currently added and the state of the live broadcast task are conveniently checked from the live broadcast list on the right side. After clicking the mouse to select, the middle of the page displays the live broadcast detailed information, the live broadcast content and the abnormal scene which is finished. On both sides are respectively the abnormal scene in progress and the current comments of the audience. Besides basic real-time abnormal scene data display, the data display module also provides time period abnormal data summarization. The abnormal condition (such as the data of each live broadcasting mechanism, the frequency proportion of scene occurrence, index trend and the like) data of all live broadcasting in a period of time are measured, and are collected, rechecked, summarized and used for data support for improvement decisions by quality inspection personnel.
Wherein, step S101 further comprises:
and B1, if the quality inspection auditing module monitors that the live broadcast corresponding to the live broadcast data stream is not timely started, judging whether the live broadcast corresponding to the live broadcast data stream is open or not by the quality inspection auditing module.
And B2, if the quality inspection auditing module monitors that the live broadcast corresponding to the live broadcast data stream is not open broadcast, determining that the live broadcast corresponding to the live broadcast data stream is slow broadcast, and outputting the information of the live broadcast corresponding to the live broadcast data stream through the instant messaging module and the data display module.
In this embodiment, first, whether live broadcasting corresponding to a live data stream is on-time started is determined according to a start time of the live data stream, where a preset start time of the live broadcasting is already set when a live broadcasting task is created, if the start time of the live data stream meets the preset start time, on-time started of the live broadcasting corresponding to the live data stream is determined, and information on-time started of the live broadcasting corresponding to the live data stream is output through an instant messaging module and a data display module, so that operators can know the condition of the live broadcasting. If the start time of the live broadcast data stream does not meet the theoretical start time, determining that the live broadcast corresponding to the live broadcast data stream is not on time.
Further, whether the live broadcast corresponding to the live broadcast data stream is a delayed broadcast is judged, if the live broadcast data stream does not start within the preset time, the live broadcast corresponding to the live broadcast data stream is determined to be the delayed broadcast, and delayed broadcast information is output through the instant messaging module and the data display module so that operators can know the live broadcast condition and pay attention to the corresponding live broadcast host broadcast. If the live broadcasting is not started within the preset delay tolerance time after the live broadcasting is delayed, determining that the live broadcasting corresponding to the live broadcasting data stream is open broadcasting, and outputting open broadcasting information through the instant communication module and the data display module so that operators can know the live broadcasting condition and control the corresponding live broadcasting host; the preset delay tolerance time can be set according to actual conditions.
Wherein, step S101 further comprises:
and C1, judging whether the live data stream is continuous or not through the quality inspection auditing module.
And step C2, if the live data stream is continuous, continuously monitoring the live data stream through the quality inspection auditing module.
And C3, if the quality inspection auditing module detects that the live broadcast data stream is stopped, stopping monitoring the live broadcast data stream by the quality inspection auditing module.
In this embodiment, after the quality inspection auditing module monitors the live data stream, it is determined whether the live data stream is continuous, if the live data stream is discontinuous, that is, the live data stream is interrupted, the interruption abnormality is detected, and the information of the live data stream interruption is output through the instant communication module and the data display module, so that an operator can adjust the live broadcast according to the interruption information. If the live broadcast resumes the push stream after the adjustment, the data display module moves the current abnormality into the historical abnormality, which indicates the current abnormality. If the live data stream is continuous, the live data stream is continuously monitored. If the live data stream is monitored to stop, namely the push stream is stopped, the live data stream is stopped to be monitored, the monitoring is finished, and the live broadcast corresponding to the live data stream is finished.
Wherein, step S102 includes:
and D1, converting the abnormal data into communication information through the instant communication module.
And D2, outputting the communication information to the instant communication software through the instant communication module.
In this embodiment, the instant communication module adopts an adapter mode design, when the quality inspection task is started, a preset abnormal scene observer is injected into the quality inspection auditing module, and when the threshold of the number of frames configured by the user is reached, abnormal data is converted into a message of the IM tool and sent out, as shown in fig. 5 and 6, so as to help quality inspection personnel grasp the live broadcast quality condition at any time and any place; the abnormal scene observer is a preset user with permission to receive the live broadcast condition, and the user configuration frame number threshold value can be set according to the actual condition, which is not particularly limited in this embodiment.
Wherein, step S102 further comprises:
And E1, if the live broadcast data stream is monitored to contain abnormal data through the quality inspection auditing module, judging the duration time of the abnormal data through the quality inspection auditing module.
And E2, outputting the abnormal data through the instant communication module and displaying the abnormal data through the data display module if the duration meets a preset threshold.
If the quality inspection auditing module monitors that the live data stream contains abnormal data, such as a black field, judging the duration time (the frame number of the black field) of the abnormal data through the quality inspection auditing module; if the duration time meets the preset threshold value, outputting abnormal data through the instant communication module, and displaying the abnormal data through the data display module so that operators can know the live broadcast condition to control the corresponding live broadcast anchor; the preset threshold value can be set according to actual conditions.
For various anomalies, the monitor corresponding to the live broadcast data stream not only reports the monitored anomaly data and the anomaly type of the anomaly data, but also includes the occurrence time, duration time, instant live broadcast screenshot and the like of the anomaly. In order to increase the flexibility and robustness of abnormal scene detection, the judging index and duration threshold of each scene can be flexibly configured. Meanwhile, the monitor corresponding to the live data stream maintains a complete life cycle of the abnormality from the beginning to the end, and the life cycle is shown in fig. 7 by taking the setting that the black field lasts for 4 frames as an example. Wherein 0 represents normal; 1 represents that black field abnormality is detected; 2 indicates that the black field abnormality is detected, but the threshold of the number of frames (or duration) configured by the user is not reached, wherein the threshold of the number of frames (or duration) configured by the user can be set according to the requirement of the user so as to enhance the robustness of scene detection; 3 indicates that the user-configured frame number threshold is just reached; 4 indicates that the number of user configuration frames has been exceeded; and 5 indicates that the scene has ended and has been restored to normal. The observer component is informed of relevant information of all abnormal data such as time screenshot and the like, so that the relevant information of the abnormal data is output to operators and/or anchor through the observer component; before the live broadcast quality inspection task is initialized, a user-defined observer component is injected, so that all abnormal processes can be conveniently tracked and processed. It can be understood that when the duration of the abnormal picture/audio playing meets the threshold of the number of frames (or duration) configured by the user, the current live broadcast segment is represented to be abnormal; by carrying out self definition on the observer component, the observer component can be externally connected with the component, so that the expansibility of the system function can be realized, and the affinity of the live broadcast quality inspection system can be improved.
The invention discloses a live broadcast quality inspection system, a live broadcast quality inspection method, a Web server and a storage medium. The method comprises the following steps: acquiring a data stream through a quality inspection and checking module and monitoring the data stream in real time; if the live broadcast data stream is monitored to contain abnormal data through the quality inspection auditing module, outputting the abnormal data through the instant communication module, and displaying the abnormal data through the data display module. The invention adopts JavaCV technology, on the basis of keeping programming flexibility, provides more convenient and easy-to-use high-order API, is based on the strong cloud ecology of Java, and is convenient for expansion and fusion with other systems. The invention aims to carry out full-flow and multi-mode real-time accompany monitoring on live broadcast activities. The invention takes the live broadcast stream pulling address as input to simulate the playing process of the client. By means of the technical schemes of real-time frame extraction, voice service, image algorithm, deep learning and the like, whether quality problems exist in live broadcast or not is analyzed, an abnormal real-time feedback mechanism is provided by combining an instant communication tool, real-time communication in a live broadcast quality inspection process is achieved, and quality inspection efficiency is improved.
Based on the first embodiment of the live broadcast quality inspection method, a second embodiment of the live broadcast quality inspection system is provided. In the present embodiment, step S101: the step of obtaining the data stream and monitoring in real time through the quality inspection auditing module comprises the following steps:
and step S1011, acquiring the live data stream through the monitor abstraction layer, and writing the live data stream into the asynchronous frame distribution queue after decapsulation.
Step S1012, distributing the live data stream to the corresponding monitor after decoding through the asynchronous frame distribution queue.
Step S1013, monitoring the live data stream through a monitor corresponding to the live data stream.
Referring to fig. 8, the quality inspection auditing module includes a monitor abstraction layer, an asynchronous frame distribution queue, a visual frame monitor abstraction layer, a composite monitor abstraction layer, a sound frame monitor abstraction layer, an image distribution queue, a composite frame distribution queue, a sound frame distribution queue, and a resource reclamation layer. Due to the limitation of the graph, the monitor implemented by the system comprises a black screen monitor, a fuzzy monitor, a static field monitor, a brightness monitor, a two-dimensional code monitor, an image checking monitor, a flower screen monitor, a solidification monitor, a mute monitor and a voice checking monitor.
In this embodiment, the live broadcast quality inspection system performs decapsulation and decoding of frames of the live broadcast data stream through the monitor abstraction layer to obtain decoding information, and writes audio frame information and video frame information included in the live broadcast data stream into the asynchronous frame distribution queue, so as to distribute each mode data stream to the respective corresponding monitor through the asynchronous frame distribution queue.
The video frame information in the live broadcast data stream is distributed to a video frame monitor abstract layer through an asynchronous frame distribution queue, the video frame monitor abstract layer consumes the live broadcast data stream, is further decoded into an image, and distributes video frame pixel information to each video frame monitor through an image distribution queue, wherein the video frame monitors comprise a black screen monitor, a fuzzy monitor, a static field monitor, a brightness monitor, a two-dimensional code monitor and an image auditing monitor; the image dispatch queue is an asynchronous frame dispatch queue. Different computer vision and image auditing algorithms are built in each video frame monitor to judge whether the live broadcast data stream detects abnormality, the detection result is captured by an observer component, and the corresponding detection result is reported to quality inspectors and/or operators and/or anchor. For example, the video frame monitor uses computer vision and image auditing algorithm to judge whether static field, blurring, brightness, black screen and content violation occur in the live broadcast data stream, and reports the detection result to quality inspectors and/or operators and/or anchor.
It should be noted that, in order to improve the system performance of the live broadcast quality inspection system, all the distributions only copy references, but do not copy the memory data, and use the concurrency component to manage the potential concurrent access scene of the image data, so as to greatly improve the memory utilization rate.
The method comprises the steps that sound frame information in a live broadcast data stream is distributed to a sound frame monitor abstract layer through an asynchronous frame distribution queue, the sound frame monitor abstract layer consumes the live broadcast data stream, is further decoded into audio, and distributes sound frame sampling information to each sound frame monitor through the sound frame distribution queue, wherein the sound frame monitors comprise a mute monitor and a voice audit monitor; the audio frame distribution queue is an asynchronous frame distribution queue. Each sound frame monitor is internally provided with a computer hearing, real-time ASR and a voice auditing algorithm for judging whether the live broadcast data stream detects abnormality, the detection result is captured by an observer component, and the corresponding detection result is reported to a quality inspector and/or an operator and/or a host. For example, the audio frame monitor uses a real-time ASR algorithm to determine whether the live data stream is mute and the anchor floor is illegal, and sends the detection result to quality inspectors and/or operators and/or anchor.
It should be noted that, in order to improve the system performance of the live broadcast quality inspection system, all the distributions only copy the references, but do not copy the memory data, and use the concurrency component to manage the potential audio data concurrency access scene, so as to greatly improve the memory utilization rate.
Aiming at the situation that audio and video are required to be comprehensively analyzed and the scenes under some decoding information are processed by a composite monitor abstract layer, the composite monitor abstract layer consumes live data streams, video frame pixel information, video frame decoding information (code rate, coding mode, frame rate, key frames and the like), audio frame sampling information and audio frame decoding information (sampling rate, channel number and the like) of the live data streams acquired by the composite monitor abstract layer are further decoded and distributed to all composite monitors through a composite frame distribution queue, wherein the composite monitors comprise a flower screen monitor and a solidification monitor; the composite frame distribution queue is an asynchronous frame distribution queue. The compound monitor judges whether the live data stream detects the abnormality or not through algorithms such as computer vision, computer hearing, natural language processing, audio and video encoding and decoding, image auditing, voice auditing and the like, obtains a detection result, and reports the corresponding detection result to quality inspectors and/or operators and/or anchor. For example, the compound monitor judges whether the live data stream has solidification or not through computer vision and computer hearing algorithm; whether the live broadcast data stream has the screen-splash condition is judged by monitoring whether code rate dip, goP elongation and the like exist in the live broadcast, and the detection result is reported to quality inspection personnel and/or operators and/or anchor. As long as the monitors which can not simply judge by only the video frame and the audio frame are required to be distributed to the compound monitors for access processing.
It should be noted that, in order to improve the system performance of the live broadcast quality inspection system, all the distributions only copy references, but do not copy the memory data, and use the concurrency component to manage the potential data concurrency access scene, so as to greatly improve the memory utilization rate, and allow the system to bear more quality inspection tasks in unit time.
Further, in this embodiment, the quality inspection auditing module screens out ineffective irrigation comments in the live data stream by means of policies such as a natural language processing algorithm and a business rule, performs statistical analysis on quality-related feedback, and automatically classifies the audience popup obtained by screening into five categories of "violation, irrigation, happiness, complaint and business opportunity" as classification results; and finally, giving a label to the user corresponding to the barrage according to the classification result so as to determine the disposal strategy of the user corresponding to the barrage according to the label. For example, if the user's tag is illegal, the user is restricted from speaking; if the label of the user is a business opportunity, pushing corresponding business information for the user; thereby improving the user's experience of watching live.
Compared with the solutions based on WebRTC and other audio and video processing tools, the FFmpeg-based live broadcast quality inspection system of the embodiment can support more live broadcast and audio and video protocol types, has good encapsulation and is not invasive to the plug-flow end and the client. Compared to FFmpeg commands, APIs and other writers, javaCV techniques employed in the present embodiment provide a more convenient and easy-to-use high-level API while maintaining programming flexibility. Meanwhile, the technology of the embodiment is very compatible with the ecology of most of the servers at present and is based on the powerful cloud ecology of Java, and the technology of the embodiment is convenient to expand and fuse with other systems, such as: each monitor of the live broadcast quality inspection system, such as a black screen monitor, a fuzzy detector, a flower screen monitor and a mute monitor, can be conveniently and quickly connected into the monitoring system.
In addition, the embodiment of the invention also provides a Web server, which comprises a memory, a processor and a live broadcast quality inspection program stored in the memory and capable of running on the processor, wherein the live broadcast quality inspection program realizes the steps of the live broadcast quality inspection system when being executed by the processor.
Because the live broadcast quality inspection program is executed by the processor and adopts all the technical schemes of all the embodiments, the live broadcast quality inspection program at least has all the beneficial effects brought by all the technical schemes of all the embodiments and is not described in detail herein.
In addition, the embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a live broadcast quality inspection program, and the live broadcast quality inspection program realizes the steps of the live broadcast quality inspection system when being executed by a processor.
Because the live broadcast quality inspection program is executed by the processor and adopts all the technical schemes of all the embodiments, the live broadcast quality inspection program at least has all the beneficial effects brought by all the technical schemes of all the embodiments and is not described in detail herein.
Compared with the prior art, the live broadcast quality inspection system, the live broadcast quality inspection method, the Web server and the storage medium provided by the invention pass through the quality inspection and checking module, the instant messaging module and the data display module; the quality inspection auditing module is respectively connected with the instant communication module and the data display module, the instant communication module is connected with external instant communication software, and the data display module is connected with the Web front end; the quality inspection auditing module is used for acquiring and monitoring the live broadcast data stream; the instant communication module is used for acquiring abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and outputting the abnormal data; the data display module is used for acquiring the abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and displaying the abnormal data. The invention aims to solve the problem of insufficient flexibility of live broadcast quality inspection, realize real-time communication in the live broadcast quality inspection process and improve quality inspection efficiency.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or method that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk, U-disk, mobile hard disk) as above, comprising several instructions for causing a Web server (which may be a computer, a server, a controlled terminal, or a network device, etc.) to perform the method of each embodiment of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (9)
1. A live quality inspection system, wherein the live quality inspection system is developed based on JavaCV, FFmpeg, openCV, the live quality inspection system comprising:
The system comprises a quality inspection and checking module, an instant communication module and a data display module; the quality inspection auditing module is respectively connected with the instant communication module and the data display module, the instant communication module is connected with external instant communication software, and the data display module is connected with the Web front end;
the quality inspection auditing module is used for acquiring and monitoring the live broadcast data stream;
the instant communication module is used for acquiring abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and outputting the abnormal data;
The data display module is used for acquiring abnormal data contained in the live broadcast data stream monitored by the quality inspection auditing module and displaying the abnormal data;
the quality inspection auditing module comprises a monitor abstract layer, an asynchronous frame distribution queue, a visual frame monitor abstract layer, a composite monitor abstract layer, a sound frame monitor abstract layer, an image distribution queue, a composite frame distribution queue, an audio distribution queue, a resource recovery layer, a black screen monitor, a fuzzy monitor, a static field monitor, a brightness monitor, a two-dimensional code monitor, an image auditing monitor, a flower screen monitor, a solidification monitor, a silence monitor and a voice auditing monitor;
the monitor abstract layer is connected with the asynchronous frame distribution queue, the asynchronous frame distribution queue is respectively connected with the visual frame monitor abstract layer, the composite monitor abstract layer and the audio frame monitor abstract layer, the visual frame abstract layer is connected with the image distribution queue, the image distribution queue is respectively connected with the black screen monitor, the fuzzy monitor, the static field monitor, the brightness monitor, the two-dimensional code monitor and the image auditing monitor, the composite monitor abstract layer is connected with the composite frame distribution queue, the composite frame distributing queue is connected with the screen monitor and the solidification monitor, the sound frame monitor is connected with the audio distributing queue, the audio distributing queue is connected with the mute monitor and the voice checking monitor, the black screen monitor, the fuzzy monitor, the mute monitor, the brightness monitor and the two-dimensional code monitor are connected with the image checking monitor, the screen monitor, the solidification monitor, the mute monitor and the voice checking monitor respectively and connected with the resource recycling layer.
2. The live quality inspection system of claim 1, wherein the data presentation module is further configured to:
acquiring the live broadcast data stream and displaying a live broadcast state corresponding to the live broadcast data stream;
Displaying live comments contained in the live data stream;
and summarizing the abnormal data contained in the live data stream to obtain an analysis report, and displaying the analysis report.
3. The live broadcast quality inspection method is characterized by comprising the following steps of:
acquiring a data stream through a quality inspection and checking module and monitoring the data stream in real time;
If the quality inspection auditing module monitors that the live broadcast data stream contains abnormal data, outputting the abnormal data through the instant communication module and displaying the abnormal data through the data display module;
the quality inspection auditing module comprises a monitor abstract layer, an asynchronous frame distribution queue, a visual frame monitor abstract layer, a composite monitor abstract layer, a sound frame monitor abstract layer, an image distribution queue, a composite frame distribution queue, an audio distribution queue, a resource recovery layer, a black screen monitor, a fuzzy monitor, a static field monitor, a brightness monitor, a two-dimensional code monitor, an image auditing monitor, a flower screen monitor, a solidification monitor, a silence monitor and a voice auditing monitor;
the monitor abstract layer is connected with the asynchronous frame distribution queue, the asynchronous frame distribution queue is respectively connected with the visual frame monitor abstract layer, the composite monitor abstract layer and the audio frame monitor abstract layer, the visual frame abstract layer is connected with the image distribution queue, the image distribution queue is respectively connected with the black screen monitor, the fuzzy monitor, the static field monitor, the brightness monitor, the two-dimensional code monitor and the image auditing monitor, the composite monitor abstract layer is connected with the composite frame distribution queue, the composite frame distributing queue is connected with the screen monitor and the solidification monitor, the sound frame monitor is connected with the audio distributing queue, the audio distributing queue is connected with the mute monitor and the voice checking monitor, the black screen monitor, the fuzzy monitor, the mute monitor, the brightness monitor and the two-dimensional code monitor are connected with the image checking monitor, the screen monitor, the solidification monitor, the mute monitor and the voice checking monitor respectively and connected with the resource recycling layer.
4. A live quality inspection method according to claim 3, wherein the quality inspection auditing module includes a monitor abstraction layer, an asynchronous frame distribution queue, and the step of acquiring the data stream by the quality inspection auditing module and monitoring in real time includes:
Acquiring the live data stream through the monitor abstraction layer, and writing the live data stream into the asynchronous frame distribution queue after unpacking;
The live broadcast data stream is distributed to a corresponding monitor after being decoded through the asynchronous frame distribution queue;
And monitoring the live data stream through a monitor corresponding to the live data stream.
5. A live quality inspection method according to claim 3, wherein the step of acquiring the data stream by the quality inspection auditing module and monitoring in real time comprises:
The abnormal data is converted into communication information through the instant communication module;
And outputting the communication information to external instant communication software through the instant communication module.
6. A live quality inspection method according to claim 3, wherein the step of acquiring the data stream by the quality inspection auditing module and monitoring in real time comprises:
if the quality inspection auditing module monitors that the live broadcast corresponding to the live broadcast data stream is not timely started, the quality inspection auditing module judges whether the live broadcast corresponding to the live broadcast data stream is open broadcast or not;
If the quality inspection auditing module monitors that the live broadcast corresponding to the live broadcast data stream is not open broadcast, determining that the live broadcast corresponding to the live broadcast data stream is slow broadcast, and outputting the information of the live broadcast corresponding to the live broadcast data stream through the instant messaging module and the data display module.
7. A live quality inspection method according to claim 3, wherein the step of acquiring the data stream by the quality inspection auditing module and monitoring in real time comprises:
judging whether the live data stream is continuous or not through the quality inspection auditing module;
if the live data stream is continuous, continuously monitoring the live data stream through the quality inspection auditing module;
and if the quality inspection auditing module detects that the live broadcast data flow is stopped, stopping monitoring the live broadcast data flow through the quality inspection auditing module.
8. A Web server comprising a memory, a processor and a live quality inspection method stored on the memory and executable on the processor, the program of live quality inspection implementing the steps of the live quality inspection method of any of claims 3-7 when executed by the processor.
9. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a program for live quality inspection, which when executed by a processor implements the steps of the live quality inspection method according to any of claims 3-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210682572.7A CN115134617B (en) | 2022-06-16 | 2022-06-16 | Live broadcast quality inspection system, live broadcast quality inspection method, web server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210682572.7A CN115134617B (en) | 2022-06-16 | 2022-06-16 | Live broadcast quality inspection system, live broadcast quality inspection method, web server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115134617A CN115134617A (en) | 2022-09-30 |
| CN115134617B true CN115134617B (en) | 2024-04-26 |
Family
ID=83377875
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210682572.7A Active CN115134617B (en) | 2022-06-16 | 2022-06-16 | Live broadcast quality inspection system, live broadcast quality inspection method, web server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115134617B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116128453B (en) * | 2023-02-18 | 2024-05-03 | 广州市点易资讯科技有限公司 | Online course inspection method, system, equipment and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111464819A (en) * | 2020-03-30 | 2020-07-28 | 腾讯音乐娱乐科技（深圳）有限公司 | Live image detection method, device, equipment and storage medium |
| CN112055230A (en) * | 2020-09-03 | 2020-12-08 | 北京中润互联信息技术有限公司 | Live broadcast monitoring method and device, computer equipment and readable storage medium |
| CN114257828A (en) * | 2021-12-20 | 2022-03-29 | 上海哔哩哔哩科技有限公司 | Live broadcast audit content processing method and system |
| CN114466210A (en) * | 2022-01-27 | 2022-05-10 | 广州方硅信息技术有限公司 | Live broadcast quality detection processing method and device, equipment, medium and product thereof |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11057134B2 (en) * | 2018-12-20 | 2021-07-06 | Viamedia, Inc. | Integrating digital advertising with cable tv network and broadcast advertising |
-
2022
- 2022-06-16 CN CN202210682572.7A patent/CN115134617B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111464819A (en) * | 2020-03-30 | 2020-07-28 | 腾讯音乐娱乐科技（深圳）有限公司 | Live image detection method, device, equipment and storage medium |
| CN112055230A (en) * | 2020-09-03 | 2020-12-08 | 北京中润互联信息技术有限公司 | Live broadcast monitoring method and device, computer equipment and readable storage medium |
| CN114257828A (en) * | 2021-12-20 | 2022-03-29 | 上海哔哩哔哩科技有限公司 | Live broadcast audit content processing method and system |
| CN114466210A (en) * | 2022-01-27 | 2022-05-10 | 广州方硅信息技术有限公司 | Live broadcast quality detection processing method and device, equipment, medium and product thereof |
Non-Patent Citations (2)
| Title |
|---|
| 基于Live555的手机实时直播系统设计与实现;魏崇毓;张宏琳;;计算机工程与设计;20160516(05);1157-1160页 * |
| 视频监控画面质量检测研究;侯忠辉;;计算机安全;20130515(05);55-59页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115134617A (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11363091B2 (en) | System and method for capturing interaction data relating to a host application | |
| US10721282B2 (en) | Media acceleration for virtual computing services | |
| CN101505365B (en) | Real-time video monitoring system implementing method based on network television set-top box | |
| US9100716B2 (en) | Augmenting client-server architectures and methods with personal computers to support media applications | |
| CN115134617B (en) | Live broadcast quality inspection system, live broadcast quality inspection method, web server and storage medium | |
| CN115103215B (en) | Live broadcast quality inspection method, system, web server and storage medium | |
| CN103186780A (en) | Video caption identifying method and device | |
| CN111343503B (en) | Video transcoding method and device, electronic equipment and storage medium | |
| CN107276990B (en) | Streaming media live broadcasting method and device | |
| CN111107394A (en) | System and method for integrating video streams across platforms | |
| CN114173087A (en) | Video data acquisition and processing method, edge gateway and storage medium | |
| CN113674188A (en) | Video analysis method and device, electronic equipment and readable storage medium | |
| CN111093091A (en) | Video processing method, server and system | |
| US9026983B2 (en) | Flexible and scalable software system architecture for implementing multimedia applications | |
| CN113542765B (en) | Media data jump continuous playing method and display device | |
| US10547878B2 (en) | Hybrid transmission protocol | |
| EP1505501A1 (en) | Demultiplexer application programming interface | |
| CN112866332A (en) | System, method and device for realizing emergency recognition and early warning based on cloud edge fusion, processor and storage medium thereof | |
| CN111147928A (en) | Video processing method, server, terminal and system | |
| CN110572693A (en) | Media asset transcoding method based on artificial intelligence | |
| CN115243102B (en) | Video playing method, device, equipment and storage medium based on Web technology | |
| CN114401254B (en) | Streaming media service processing method and device, electronic equipment and storage medium | |
| CN114390313B (en) | Method and system for redirecting audio and Video of browser HTML5 Video tag | |
| CN104010221A (en) | Digital billboard playing system, instant monitoring system and instant monitoring method thereof | |
| Desurmont et al. | A generic flexible and robust approach for intelligent real-time video-surveillance systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |