CN112911321A - Real Time Clock (RTC) -based audio and video live broadcast control method and system - Google Patents

Real Time Clock (RTC) -based audio and video live broadcast control method and system Download PDF

Info

Publication number
CN112911321A
CN112911321A CN202110086425.9A CN202110086425A CN112911321A CN 112911321 A CN112911321 A CN 112911321A CN 202110086425 A CN202110086425 A CN 202110086425A CN 112911321 A CN112911321 A CN 112911321A
Authority
CN
China
Prior art keywords
audio
video
video data
data
illegal behavior
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110086425.9A
Other languages
Chinese (zh)
Inventor
董晓宏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Qiniu Information Technology Co ltd
Original Assignee
Shanghai Qiniu Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Qiniu Information Technology Co ltd filed Critical Shanghai Qiniu Information Technology Co ltd
Priority to CN202110086425.9A priority Critical patent/CN112911321A/en
Publication of CN112911321A publication Critical patent/CN112911321A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/2187Live feed
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/53Recognition of crowd images, e.g. recognition of crowd congestion
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention discloses an audio and video live broadcast control method based on an RTC (real time clock), which comprises the following steps of: acquiring identity data of an audio and video data publisher, and inserting the identity data code into a data packet of the audio and video data; identifying whether an illegal behavior or an illegal behavior risk exists in the audio and video data; based on the result of the last step of recognition, further determining a processing mode of the audio and video data, wherein the processing mode comprises early warning processing and broadcast prohibition; and informing the real-time audio and video service of prohibiting the broadcasting of the publisher of the audio and video data to the audio and video needing to be prohibited. The invention also discloses an RTC-based audio and video live broadcast control system, and by utilizing the method and the system, the problem of accurate sealing is solved, meanwhile, the accuracy of live broadcast control is improved and the use experience of customers is improved by combining various modes such as risk early warning, long preview identification, manual auditing and the like, so that the method and the system have obvious technical advantages and beneficial effects.

Description

Real Time Clock (RTC) -based audio and video live broadcast control method and system
Technical Field
The invention relates to an audio and video content control technology, in particular to an audio and video live broadcast control method and system based on an RTC (real time clock).
Background
Real Time Communication (RTC) is a technology for enabling two or more persons to perform remote Real-Time interactive audio/video Communication, and is widely applied to the fields of audio/video Communication, teleconferencing, remote teaching, entertainment and social interaction, and the like. In the field of entertainment and social interaction, whether the anchor people are involved in terrorism, yellow and violence and other illegal behaviors needs to be supervised. The current general technologies of identifying terrorism, yellow and violence (hereinafter referred to as three technologies) are recording and intercepting audio and video data, carrying out image recognition and voice recognition, and sealing the live broadcast when illegal behaviors occur.
In the actual use process, especially for a large number of clients of individual creation type of UGC (user level anchor), it is difficult for a live broadcast company to effectively restrict and predict illegal behaviors of the anchor. When a plurality of people are in a real-time conversation, the illegal action of a certain anchor can cause the whole live broadcast to be forbidden. How to accurately block and predict customer illegal activities as early as possible is an important ring for processing the illegal activities.
Disclosure of Invention
In order to solve the above problems in the prior art, an object of the present invention is to provide a solution for accurate blocking and early prediction in an RTC-based audio/video live broadcast process.
The invention discloses an audio and video live broadcast control method based on an RTC (real time clock), which comprises the following steps of: s1: acquiring identity data of an audio and video data publisher, and inserting the identity data code into a data packet of the audio and video data; s2: identifying whether an illegal behavior or an illegal behavior risk exists in the audio and video data; s3: based on the result of the last step of recognition, further determining a processing mode of the audio and video data, wherein the processing mode comprises early warning processing and broadcast prohibition; s4: and informing the real-time audio and video service of prohibiting the broadcasting of the publisher of the audio and video data to the audio and video needing to be prohibited.
Further, the identity information of the audio and video data publisher includes: ID information, image position information.
Further, the inserting the identity data into the data packet of the audio/video data is specifically inserting the identity data into an SEI field of the audio/video data.
Further, after step S4, the method further includes: and acquiring the characteristic information of the audio and video needing to be prohibited, and recording the characteristic information into an illegal behavior database, wherein the illegal behavior database can be used for further identifying illegal behaviors.
Further, the identifying of the illegal behavior in the audio and video data specifically comprises the following steps: acquiring characteristic information of audio and video data; inquiring an illegal behavior database to obtain an illegal behavior record related to the characteristic information; and analyzing the audio and video data in real time to identify whether illegal behaviors exist.
Further, the feature information includes: APPID, USERID, device information (model, etc.) of the terminal, IP, and operator.
Further, the early warning processing comprises early warning prompting, long preview identification processing and manual auditing.
Based on the same inventive concept, the second aspect of the present invention also discloses an audio/video live broadcast control system based on RTC, the system comprising: a confluence extension module: the system comprises a data packet, a data processing module and a data processing module, wherein the data packet is used for acquiring identity data of an audio and video data publisher and inserting the identity data code into the data packet of the audio and video data; an illegal action identification module: identifying whether an illegal behavior or an illegal behavior risk exists in the audio and video data; and an illegal behavior processing module: the method is used for further determining a processing mode of the audio and video data, wherein the processing mode comprises early warning processing and broadcast prohibition; a broadcast forbidding processing module: and the real-time audio and video service is informed of the broadcasting prohibition of the audio and video data to the publisher.
Furthermore, the system also comprises a historical record analysis module which is used for carrying out illegal behavior identification on the behaviors related to the keywords by combining the historical illegal behavior records.
Further, the system also comprises an illegal behavior database, and the illegal behavior database is used for storing historical illegal behavior records.
The invention also discloses an electronic device, which comprises a processor and a memory, wherein the memory is used for storing the executable program; the processor is used for executing any one of the audio and video live broadcast control methods based on the RTC disclosed by the first aspect of the invention.
Compared with the prior art, the method and the system have at least the following technical advantages:
(1) the problem of accurate banning is solved: by inserting the audio and video data publisher into the data packet before distributing the audio and video data, accurate prohibition for the audio and video data publisher can be realized in the stages of illegal behavior identification and broadcast prohibition, and the problem that the whole live broadcast activity cannot be carried out due to the fact that a certain illegal main broadcast is prohibited in the prior art is solved;
(2) early warning of risk: early warning possible illegal behaviors of a client in advance in a historical illegal behavior record and big data early warning mode;
(3) the accuracy of live broadcast control is improved: the frequency and the range of recognition are increased during live broadcast distribution, and meanwhile, a manual auditing mechanism is combined, so that recognition mistakes and omissions are prevented, and the accuracy of live broadcast control is obviously improved.
In conclusion, the method and the system of the invention solve the problem of accurate prohibition, early warn risks, improve the accuracy of live broadcast control and improve the use experience of customers, thereby having obvious technical advantages and beneficial effects.
In order that the invention may be more clearly and fully understood, specific embodiments thereof are described in detail below with reference to the accompanying drawings.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly introduced below. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a schematic flowchart of an RTC-based audio and video live control method in an embodiment.
Fig. 2 is a schematic flow chart of identifying illegal activities in audio and video data in one embodiment.
Fig. 3 is a schematic structural diagram of an audio and video live control system based on an RTC in an embodiment.
Fig. 4 is a schematic workflow diagram of an RTC-based audio and video live control system in an embodiment.
Fig. 5 is a schematic diagram of a no-play in one embodiment.
Detailed Description
Referring to fig. 1, fig. 1 is a schematic flowchart of an audio/video live broadcast control method based on an RTC in an embodiment, and the steps are specifically described as follows with reference to fig. 1:
step S11: acquiring identity data of an audio and video data publisher, and inserting the identity data code into a data packet of the audio and video data;
in one embodiment, the identity information of the audio-video data publisher includes: after the ID information and the image position information acquire the identity information of the audio and video data publisher, inserting the identity data into a data packet of the audio and video data, specifically inserting the identity data into an SEI field of the audio and video data.
Step S12: before audio and video distribution, identifying whether illegal behaviors or illegal behavior risks exist in audio and video data;
in this embodiment, before audio and video distribution, audio and video data is analyzed in real time to identify whether there is an illegal action, and in one embodiment, the illegal action may also be identified in combination with an illegal action database, and a specific flow refers to the flow diagram shown in fig. 2.
Step S13: based on the result of the last step of recognition, further determining a processing mode of the audio and video data, wherein the processing mode comprises early warning processing and broadcast prohibition;
in one embodiment, for the condition that the audio and video data has the risk of illegal behavior, a processing mode further adopted for the audio and video data is determined according to the illegal risk index, if the risk index is lower, early warning processing is adopted, and if the risk index is higher, broadcast prohibition processing is adopted.
The early warning processing comprises early warning prompting, long preview identification processing and manual auditing, the frequency and the range of identification are increased during live broadcast distribution through the early warning processing, the success rate of illegal behaviors is reduced, and meanwhile, the manual auditing is carried out according to the warning duration through a manual auditing mechanism, so that mistakes and omissions are prevented.
Step S14: and informing the real-time audio and video service of prohibiting the broadcasting of the publisher of the audio and video data to the audio and video needing to be prohibited.
In one embodiment, for the audio and video needing no broadcasting, the characteristic information of the audio and video needing no broadcasting is further acquired, and the characteristic information is recorded into an illegal behavior database, which needs to establish and maintain an illegal behavior database, wherein the illegal behavior database can be used for further illegal behavior identification and big data analysis.
Referring to fig. 2, fig. 2 is a schematic flow chart of identifying an illegal action in audio/video data in an embodiment, and the following steps are specifically described with reference to fig. 2:
step S21: acquiring characteristic information of audio and video data;
in one embodiment, the characteristic information of the audio-video data includes: APPID, USERID, device information (model, etc.) of the terminal, IP, and operator.
Step S22: inquiring an illegal behavior database to obtain an illegal behavior record related to the characteristic information;
and comparing the acquired characteristic information of the audio image with records in an illegal behavior database, and searching related illegal behavior records to determine whether the identification time needs to be prolonged.
Step S23: and analyzing the audio and video data in real time to identify whether illegal behaviors exist.
Usually, methods such as image recognition and voice recognition are adopted to analyze whether audio and video data have illegal behaviors in real time.
Referring to fig. 3, fig. 3 is a schematic structural diagram of an audio/video live broadcast control system based on an RTC in an embodiment, as shown in the figure, the embodiment 10 includes a confluence extension module 101, an illegal behavior identification module 102, an illegal behavior processing module 103, and a broadcast forbidding processing module 104, where:
the confluence expansion module 101: the system comprises a data packet, a data processing module and a data processing module, wherein the data packet is used for acquiring identity data of an audio and video data publisher and inserting the identity data code into the data packet of the audio and video data;
the illegal action recognition module 102: identifying whether an illegal behavior or an illegal behavior risk exists in the audio and video data;
the illegal action processing module 103: the method is used for further determining a processing mode of the audio and video data, wherein the processing mode comprises early warning processing and broadcast prohibition;
the broadcast forbidding processing module 104: and the real-time audio and video service is informed of the broadcasting prohibition of the audio and video data to the publisher.
Referring to fig. 4, fig. 4 is a schematic workflow diagram of an audio and video live control system based on an RTC in an embodiment, as shown in the figure, the workflow diagram includes the following processes:
step S41: receiving a real-time audio and video stream by the audio and video service;
step S42: the audio and video service performs confluence processing on the received audio and video stream and forwards the received audio and video stream to a confluence extension module 101 for confluence extension processing, wherein the confluence extension processing comprises the steps of acquiring identity data of an audio and video data publisher and inserting the identity data code into a data packet of the audio and video data;
step S43: after the confluence expansion processing, the audio and video data carries the identity data of the audio and video data publisher and is sent to the audio and video distribution module;
step S44: before the audio and video distribution module distributes, the RTC-based audio and video live control system 10 performs a series of processing on the audio and video data, including the processing of the illegal behavior recognition module 102, the illegal behavior processing module 103, and the broadcast prohibition processing module 104, and completes three monitoring checks on the audio and video data.
Step S45: and for the audio and video data needing to be prohibited, informing the real-time audio and video service of prohibiting the broadcasting of the publisher of the audio and video data.
Referring to fig. 5, fig. 5 is a diagram of an embodiment of an audio/video live broadcast control system and an audio/video live broadcast control system method according to an embodiment of the present invention, after a anchor performs streaming and distribution through an RTC network, a location and ID information of the anchor are added in a streaming process, and a plurality of anchors such as A, B, C perform live broadcast, and if an anchor has inappropriate content, an SFU is notified to stop forwarding audio/video data of the anchor, so that playback prohibition is implemented.
In practical applications, the modules described in the method and system disclosed by the present invention may be deployed on one server, or each module may be deployed on a different server independently, and particularly, the modules may be deployed on a cluster server as needed.
An embodiment of the present application further provides an electronic device, where the electronic device includes a processor and a memory, where the memory stores an executable program, and when the executable program runs on a computer, the computer executes the RTC-based audio and video live broadcast control method according to any of the above embodiments.
It should be noted that, all or part of the steps in the methods of the above embodiments may be implemented by hardware related to instructions of a computer program, which may be stored in a computer-readable storage medium, which may include, but is not limited to: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. An audio and video live broadcast control method based on RTC is characterized by comprising the following steps:
s1: acquiring identity data of an audio and video data publisher, and inserting the identity data code into a data packet of the audio and video data;
s2: identifying whether an illegal behavior or an illegal behavior risk exists in the audio and video data;
s3: based on the result of the last step of recognition, further determining a processing mode of the audio and video data, wherein the processing mode comprises early warning processing and broadcast prohibition;
s4: and informing the real-time audio and video service of prohibiting the broadcasting of the publisher of the audio and video data to the audio and video needing to be prohibited.
2. The audio-video live broadcast control method according to claim 1, wherein the identity information of the audio-video data publisher includes: ID information, image position information.
3. The audio/video live broadcast control method according to claim 1, wherein inserting the identity data into a packet of audio/video data specifically inserts the identity data into an SEI field of the audio/video data.
4. The audio-video live broadcast control method according to claim 1, further comprising, after step S4: the method comprises the steps of obtaining characteristic information of the audio and video needing to be prohibited from being played, and recording the characteristic information into an illegal behavior database, wherein the illegal behavior database can be used for further illegal behavior identification and big data analysis.
5. The audio and video live broadcast control method according to claim 4, wherein the identifying of the illegal behavior in the audio and video data specifically comprises the following steps:
acquiring characteristic information of audio and video data;
inquiring an illegal behavior database to obtain an illegal behavior record related to the characteristic information;
and analyzing the audio and video data in real time to identify whether illegal behaviors exist.
6. The audio-video live broadcast control method according to claim 4, wherein the feature information includes: APPID, USERID, device information (model, etc.) of the terminal, IP, and operator.
7. The audio and video live broadcast control method according to claim 1, wherein the early warning processing includes early warning prompting, long preview recognition processing, and manual review.
8. An audio and video live control system based on an RTC, the system comprising:
a confluence extension module: the system comprises a data packet, a data processing module and a data processing module, wherein the data packet is used for acquiring identity data of an audio and video data publisher and inserting the identity data code into the data packet of the audio and video data;
an illegal action identification module: identifying whether an illegal behavior or an illegal behavior risk exists in the audio and video data;
and an illegal behavior processing module: the method is used for further determining a processing mode of the audio and video data, wherein the processing mode comprises early warning processing and broadcast prohibition;
a broadcast forbidding processing module: and the real-time audio and video service is informed of the broadcasting prohibition of the audio and video data to the publisher.
9. The audio-visual live control system of claim 8, further comprising: and the historical record analysis module is used for carrying out illegal behavior identification on the behaviors related to the keywords by combining the historical illegal behavior records.
10. The audio-visual live control system of claim 8, further comprising: and the illegal behavior database is used for storing historical illegal behavior records.
CN202110086425.9A 2021-01-22 2021-01-22 Real Time Clock (RTC) -based audio and video live broadcast control method and system Pending CN112911321A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110086425.9A CN112911321A (en) 2021-01-22 2021-01-22 Real Time Clock (RTC) -based audio and video live broadcast control method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110086425.9A CN112911321A (en) 2021-01-22 2021-01-22 Real Time Clock (RTC) -based audio and video live broadcast control method and system

Publications (1)

Publication Number Publication Date
CN112911321A true CN112911321A (en) 2021-06-04

Family

ID=76118430

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110086425.9A Pending CN112911321A (en) 2021-01-22 2021-01-22 Real Time Clock (RTC) -based audio and video live broadcast control method and system

Country Status (1)

Country Link
CN (1) CN112911321A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140373036A1 (en) * 2013-06-14 2014-12-18 Telefonaktiebolaget L M Ericsson (Publ) Hybrid video recognition system based on audio and subtitle data
CN109831696A (en) * 2018-12-28 2019-05-31 广州华多网络科技有限公司 Handle method, apparatus, electronic equipment and the storage medium of violation video content
CN110365996A (en) * 2019-07-25 2019-10-22 深圳市元征科技股份有限公司 Management method, live streaming management platform, electronic equipment and storage medium is broadcast live
CN111586421A (en) * 2020-01-20 2020-08-25 全息空间(深圳)智能科技有限公司 Method, system and storage medium for auditing live broadcast platform information
CN111770365A (en) * 2020-07-03 2020-10-13 广州酷狗计算机科技有限公司 Anchor recommendation method and device, computer equipment and computer-readable storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140373036A1 (en) * 2013-06-14 2014-12-18 Telefonaktiebolaget L M Ericsson (Publ) Hybrid video recognition system based on audio and subtitle data
CN109831696A (en) * 2018-12-28 2019-05-31 广州华多网络科技有限公司 Handle method, apparatus, electronic equipment and the storage medium of violation video content
CN110365996A (en) * 2019-07-25 2019-10-22 深圳市元征科技股份有限公司 Management method, live streaming management platform, electronic equipment and storage medium is broadcast live
CN111586421A (en) * 2020-01-20 2020-08-25 全息空间(深圳)智能科技有限公司 Method, system and storage medium for auditing live broadcast platform information
CN111770365A (en) * 2020-07-03 2020-10-13 广州酷狗计算机科技有限公司 Anchor recommendation method and device, computer equipment and computer-readable storage medium

Similar Documents

Publication Publication Date Title
CN110365996B (en) Live broadcast management method, live broadcast management platform, electronic device and storage medium
CN108353208B (en) Optimizing media fingerprint retention to improve system resource utilization
US20080144893A1 (en) Apparatus and method for selecting key frames of clear faces through a sequence of images
JP2018530080A (en) System and method for partitioning search indexes for improved media segment identification efficiency
CA2992529A1 (en) Prediction of future views of video segments to optimize system resource utilization
CN108293140B (en) Detection of common media segments
CN109660823A (en) Video distribution method, apparatus, electronic equipment and storage medium
US11037604B2 (en) Method for video investigation
US20140167963A1 (en) System and method for monitoring an area using nfc tags
CN104836781A (en) Method distinguishing identities of access users, and device
US10129592B2 (en) Audience measurement and feedback system
US20220114826A1 (en) Method for identifying potential associates of at least one target person, and an identification device
WO2009017995A1 (en) Digital video recorder collaboration and similar media segment determination
CN110543584B (en) Method, device, processing server and storage medium for establishing face index
CN112217841A (en) Live broadcast room management method and device, computer equipment and storage medium
CN114666554A (en) Edge gateway cloud service management system
CN114786038A (en) Low-custom live broadcast behavior monitoring method based on deep learning
US9462238B1 (en) Remote agent capture and monitoring
WO2008062145A1 (en) Creating fingerprints
CN112784069A (en) IPTV content intelligent recommendation system and method
CN112911321A (en) Real Time Clock (RTC) -based audio and video live broadcast control method and system
CN112148920A (en) Data management method
GB2444094A (en) Identifying repeating video sections by comparing video fingerprints from detected candidate video sequences
CN109101964A (en) Determine the method, equipment and storage medium in head and the tail region in multimedia file
US10885343B1 (en) Repairing missing frames in recorded video with machine learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210604