CN114979787A - Live broadcast playback management method, device, equipment and storage medium - Google Patents
Live broadcast playback management method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114979787A CN114979787A CN202210535703.9A CN202210535703A CN114979787A CN 114979787 A CN114979787 A CN 114979787A CN 202210535703 A CN202210535703 A CN 202210535703A CN 114979787 A CN114979787 A CN 114979787A
- Authority
- CN
- China
- Prior art keywords
- information
- live
- playback
- video
- audio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims abstract description 34
- 238000003860 storage Methods 0.000 title claims abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 46
- 238000012545 processing Methods 0.000 claims abstract description 28
- 230000003993 interaction Effects 0.000 claims abstract description 27
- 230000002452 interceptive effect Effects 0.000 claims description 29
- 238000001514 detection method Methods 0.000 claims description 13
- 230000006399 behavior Effects 0.000 claims description 11
- 238000012216 screening Methods 0.000 claims description 11
- 230000008859 change Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000010365 information processing Effects 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 15
- 238000004891 communication Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 208000001613 Gambling Diseases 0.000 description 2
- 230000002860 competitive effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000007115 recruitment Effects 0.000 description 2
- 230000000391 smoking effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000007943 implant Substances 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
- H04N21/4355—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream involving reformatting operations of additional data, e.g. HTML pages on a television screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4398—Processing of audio elementary streams involving reformatting operations of audio signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440245—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Acoustics & Sound (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Medical Informatics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The present disclosure relates to the field of computer technologies, and in particular, to a live playback management method, apparatus, device, and storage medium. The method comprises the steps of acquiring live course data, wherein the live course data comprises courseware video information, portrait video information, audio information and interaction information; respectively carrying out playback processing on courseware video information, portrait video information, audio information and interaction information, and aligning according to the timestamps to generate initial playback information; and according to the service requirement, carrying out external display setting on the initial playback information. By the embodiment, the live video content of the course is supervised in different areas and different time periods, and the supervision and video editing efficiency is improved.
Description
Technical Field
The present disclosure relates to the field of computers, and in particular, to a live playback management method, apparatus, device, and storage medium.
Background
The webpage live broadcast has the advantages of being high in instantaneity and interactivity, low in cost, free of software installation and the like, and is widely applied to the field of online education. The web live broadcast integrates rich elements such as images, characters, voice and the like, has a luxuriant sound and an excellent effect, and gradually becomes a mainstream expression mode of course live broadcast. Although the web-based live curriculum draws and extends the advantages of the internet, live curriculum often lacks complete management due to openness and interactivity, and has some bugs and disadvantages, such as: the live broadcast process involves sensitive content, disseminated or exaggerated advertisements, illegal sales of products, etc.
Live video is unlikely to be recorded repeatedly and needs to be viewed repeatedly by the student. Therefore, background personnel are required to monitor the live recorded video of the curriculum in violation or compliance through various means. The existing course live broadcast supervision mode is low in efficiency and time-consuming. Especially, compared with common live broadcast, the live broadcast video of the course not only relates to the supervision of live broadcast personnel, but also comprises live broadcast courseware, interactive content and the like.
How to manage the vulnerability and the defect of the recorded live curriculum video becomes a current problem.
Disclosure of Invention
In order to solve the problem of difficulty in playback management of live videos in the prior art, embodiments herein provide a live playback management method, apparatus, device, and storage medium, which are used to solve the problems of low supervision efficiency, unchanged clips, and the like caused by dispersed content and asynchronous time zones of course live video supervision.
Embodiments herein provide a live playback management method, the method comprising,
acquiring live course data, wherein the live course data comprises courseware video information, portrait video information, audio information and interaction information;
respectively carrying out playback processing on courseware video information, portrait video information, audio information and interaction information, and aligning according to the timestamps to generate initial playback information;
and according to the service requirement, carrying out external display setting on the initial playback information.
Further, before acquiring the live course data, the method further comprises,
sending a request for acquiring live course data to a live broadcast server, wherein the request for the live course data carries a live broadcast identifier;
and the live broadcast server verifies and returns the live broadcast course data according to the course identification in the request of the live broadcast course data.
Further, the courseware video information playback processing comprises,
acquiring the time information of the change of the courseware video information;
capturing pictures of the image information according to the time information;
extracting the features of the captured pictures, and generating an image identification result according to feature comparison;
and fuzzifying the page containing the image recognition result in the courseware video information to generate a courseware displaying video.
Further, the portrait video information playback processing includes,
extracting frames from the portrait video information to obtain one or more video frames;
and sequentially inputting one or more video frames into a video detection model, and blurring the video frames with illegal behaviors to generate a portrait displaying video.
Further, the audio information playback processing includes,
performing character conversion on the audio information to obtain an audio text file with a time mark;
generating an audio recognition result for the audio text file according to a preset screening word bank;
and fuzzifying the audio frame containing the audio recognition result in the audio information to generate the courseware displaying audio.
Further, the interactive information playing process comprises,
acquiring an interactive text file with a time mark according to the interactive information;
generating an interaction recognition result for the interaction text file according to a preset screening word bank;
and deleting the characters containing the interactive recognition result in the interactive information to generate display interactive characters.
Further, the outward display arrangement includes,
setting whether playback information is displayed and played back externally by self-definition;
setting playback information display duration in a user-defined manner;
setting a playback information display segment in a user-defined manner;
and self-defining whether the playback information opens the red packet popup window.
Embodiments herein also provide a live playback management apparatus, including,
an acquisition module for acquiring live course data, wherein the live course data comprises courseware video information, portrait video information, audio information and interaction information,
the playback information processing module is used for respectively performing playback processing on the courseware video information, the portrait video information, the audio information and the interaction information, aligning according to the timestamps and generating initial playback information;
and the external display setting module is used for carrying out external display setting on the initial playback information according to the service requirement.
Embodiments herein also provide a computer device comprising a memory, a processor, and a computer program stored on the memory, the processor implementing the above-described method when executing the computer program.
Embodiments herein also provide a computer storage medium having a computer program stored thereon, the computer program, when executed by a processor of a computer device, performing the above-described method.
By using the embodiment, the contents are respectively processed according to the courseware video information, the portrait video information, the audio information and the interaction information which are acquired from the server, and then the contents are aligned according to the timestamps. The method and the device realize the supervision of the live video content of the course in different areas and different time, and improve the supervision and video editing efficiency. Meanwhile, the problem that the whole video page is completely shielded due to the fact that part of information is in trouble is avoided.
In addition, the embodiment of the invention has high flexibility by setting the external display and customizing various items of information of the playback video.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart illustrating a live playback management method according to an embodiment of the present disclosure;
fig. 2 is a schematic block diagram illustrating a device for managing live playback according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present disclosure.
[ description of reference ]:
201. an acquisition module;
202. a playback information processing module;
203. an external display setting module;
302. a computer device;
304. a processing device;
306. a storage resource;
308. a drive mechanism;
310. an input/output module;
312. an input device;
314. an output device;
316. a presentation device;
318. a graphical user interface;
320. a network interface;
322. a communication link;
324. a communication bus.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments herein without making any creative effort, shall fall within the scope of protection.
It should be noted that the terms "first," "second," and the like in the description and claims herein and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments herein described are capable of operation in sequences other than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, apparatus, article, or device that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or device.
The rapid development of the internet technology enables a live course which is a new teaching mode to be developed, and the difference from the traditional teaching mode is that live course videos can be played back repeatedly for users to learn. Meanwhile, the course live broadcast video often lacks complete management due to openness and interactivity, and is easy to have banned content, advertisements, infringement content and value-related content. For example: the forbidden content comprises special identification, clothes, scenes and vulgars; infringement contents comprise infringement trademarks, other person works and the like; the related value items include smoking, improper speech, improper limb movements, and the like. However, since the live curriculum video cannot be repeatedly recorded regularly and the violation degree of part of live curriculum video is low, the violation content in the live curriculum video needs to be supervised. In view of the lower degree of automatic supervision of live video of course, still need the input of a large amount of manpower, material resources and time, the cost of manufacture is higher.
In order to solve the above problems, embodiments herein provide a live playback management method, which is used to solve the problems of low supervision efficiency, unchanged editing, and the like caused by content dispersion (courseware area, portrait area, and interactive area) and time zone asynchronization (different time points where problems occur in different areas) of live video supervision of a curriculum. Fig. 1 is a schematic diagram illustrating steps of a live playback management method provided in an embodiment of the present disclosure, and the present specification provides the method operation steps as described in the embodiment or the flowchart, but more or less operation steps may be included based on conventional or non-creative labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an actual system or apparatus product executes, it can execute sequentially or in parallel according to the method shown in the embodiment or the figures. Specifically, as shown in fig. 1, the method may include:
s101: acquiring live course data, wherein the live course data comprises courseware video information, portrait video information, audio information and interaction information;
in the embodiment of the present specification, the management terminal obtains relevant data of live courses, including but not limited to courseware video information, portrait video information, audio information, and interaction information, by sending a request to the server. In one example, the courseware video information may be PPT courseware in a courseware area when the courseware is live broadcast on a split screen; the portrait video information can be a portrait and a background of a live broadcast area of a live broadcast teacher; the audio information can be voice of a live teacher for lecturing, voice of student answering or communication, course teaching music and the like; the interactive information can be characters which are mutually communicated among students in an interactive frame, characters which are communicated between a live teacher and the students, and the like.
S102: respectively carrying out playback processing on courseware video information, portrait video information, audio information and interaction information, and aligning according to timestamps to generate initial playback information;
in the embodiment of the specification, the management terminal respectively identifies, converts or screens courseware video information, portrait video information, audio information and interaction information, deletes and fuzzes the content which does not meet the requirement, aligns the processed courseware video information, portrait video information, audio information and interaction information according to the timestamp, and combines the curriculum playback videos which can be played externally. Unsatisfactory content includes, but is not limited to, illicit content, advertising, infringing content, value-related content, and the like.
S103: and according to the service requirement, carrying out external display setting on the initial playback information.
In this embodiment of the specification, the management end sets the playback video that is externally displayed, and the setting may be whether to externally display, the time length of the external display, the segment of the external display, whether to implant a red packet popup window in the playback video during the external display, and the like.
Specifically, before acquiring live course data, the method further comprises,
sending a request for acquiring live course data to a live broadcast server, wherein the request for the live course data carries a live broadcast identifier;
in the embodiment of the specification, a management terminal sends a request for acquiring live course data to a server, wherein the request carries live identification, and the identification includes but is not limited to one or more of a paper cup course number, a live address, a secret key, a verification code and the like;
and the live broadcast server verifies and returns the live broadcast course data according to the course identification in the request of the live broadcast course data.
In the embodiment of the specification, the live broadcast server verifies the course identifier sent by the request terminal, and sends the request data to the request address after the verification is passed.
Specifically, the courseware video information playback processing comprises,
acquiring the time information of the change of the courseware video information;
in the embodiment of the present specification, the courseware video information is mainly an image presented by a PPT courseware, and therefore the time information of change of the courseware video information includes time information of page turning of the PPT courseware in a teaching process (for example, in the 5 th minute of the teaching process, if the PPT courseware is turned from a first page to a second page, the change time is recorded); in addition, in order to improve the interest of classroom teaching and mobilize the enthusiasm of student learning, text elements, table elements and the like in the PPT courseware are often presented in an animation insertion manner, so that the time information of change of the courseware video information also includes the time when the content displayed on the page in the PPT courseware changes (for example, at the 4 th minute of the teaching process, the page of the current PPT courseware is not switched, but the page of the PPT courseware is inserted into the table, the courseware video information displayed on the page is changed, and therefore the change time is recorded).
It should be noted that the time information of the courseware video information change includes an absolute time and a relative time; the absolute time refers to the time when the current PPT courseware image changes, and the relative time refers to the time sequence relation between two changing times.
Capturing pictures of the image information according to the time information;
in the embodiment of the present specification, for example, a picture of a PPT courseware on a second page is grabbed at the 5 th minute; and grabbing the picture of the PPT courseware on the first page after the table is inserted according to the 4 th minute, and the like.
In the embodiment of the specification, the image information is captured through the time information, so that dynamic courseware demonstration can be converted into static pictures, display contents of courseware cannot be missed, and the pictures are different from each other.
Extracting the features of the captured pictures, and generating an image recognition result according to feature comparison;
in the embodiments of the present specification, the pictures may be understood as a batch of images to be processed. And processing the picture by using an image recognition method of deep learning, a convolutional neural network and the like to obtain the feature points. In one example, the image is identified by using an image identification method of deep learning, in order to efficiently obtain the feature points of the image, the gray values corresponding to the frames 1, 5, 9 and 13 can be respectively compared with the gray value of the frame corresponding to the circle center to obtain 4 gray value comparison results, if at least 3 gray value comparison results are different in the 4 gray value comparison results (i.e., greater than or less than the gray value of the frame corresponding to the circle center), the pixel point corresponding to the circle center can be determined as the feature point, and when the pixel point corresponding to the circle center is the feature point, the feature point is marked; in another implementation manner, in order to achieve efficient acquisition of the image feature points, whether the gray values corresponding to a pair of continuous frames of the frames 1, 5, 9, 13 are greater than or less than the gray value of the pixel point corresponding to the center of a circle may be detected, if at least the gray value corresponding to the pair of continuous frames is greater than or less than the gray value of the pixel point corresponding to the center of a circle, the pixel point corresponding to the center of a circle may be used as the feature point, the feature point may be marked, and the feature point may be searched by comparing the gray values of all circumferential pixel points corresponding to all pixel points in the whole image to be processed.
And (4) according to the obtained picture characteristic points and the trained model (picture recognition model), or sending the picture characteristic points and the trained model (picture recognition model) to a third-party platform for processing, and screening out contents such as advertisements, logos, two-dimensional codes, popular pictures and the like.
And fuzzifying the page containing the image recognition result in the courseware video information to generate a courseware displaying video.
In the embodiment of the description, according to the processing result output by the trained picture recognition model or the third-party platform picture recognition model, the courseware image in the time range of the PPT courseware image with the problem is identified and is subjected to fuzzification, wherein the fuzzification can be mosaic, and finally, a courseware displaying video is formed.
According to the characteristics of live courses, PPT courseware pictures in courseware video information are captured through time information, and then recognition is carried out. Compared with the method that the whole live course video is identified completely, the identification efficiency is greatly improved, and the method is flexible and convenient.
Specifically, the portrait video information playback processing includes,
extracting frames from the portrait video information to obtain one or more video frames;
in the embodiment of the present specification, the extracting frames of the portrait video information is to extract a single picture from one or more portrait videos to obtain a video frame. The purpose here is to convert the portrait video into images (video frames) that the video detection model can process. In one example, since the violation is not a video action with high-speed change, the frequency may be 1fps or set to 2fps when performing frame extraction on a video to be detected, in a specific implementation process, the frame extraction frequency is not suitable to be set too large, if the frame extraction frequency is set too large, the violation is directly missed to be detected, and the specifically set frame extraction frequency may be flexibly set according to different application scenarios.
And sequentially inputting one or more video frames into a video detection model, and blurring the video frames with illegal behaviors to generate a portrait displaying video.
In this embodiment of the present description, the video detection model may be a majority of target detection networks, and in the specific implementation, the video detection model is improved and deployed based on yolov3 to adapt to a detection task of an illegal behavior in a live broadcast scene, it should be noted that this description mode is not intended to limit a specific type of the video detection model, and all models with detection functions belong to the protection scope of the present application.
The violation behaviors described in this application include, but are not limited to, any one or a combination of behaviors including: smoking behavior, advertising behavior, clothes exposure behavior, filth behavior, etc. When the video detection model detects a plurality of video frames, if the video frames are detected to have illegal behaviors, the video frames are marked, and a detection result with an abnormal mark is output.
And according to the video frame range output by the detection result, blurring the portrait video in the time range in which the problematic video frame is identified, wherein the blurring can be mosaic, and finally forming the display portrait video.
Specifically, the audio information playback processing includes,
performing character conversion on the audio information to obtain an audio text file with a time mark;
in the embodiment of the present specification, the audio information is subjected to word conversion, and an audio text file with a time stamp, such as an lrc format file, is generated.
Generating an audio recognition result for the audio text file according to a preset screening word bank;
in the embodiment of the specification, the audio text file is input into a preset screening word bank for retrieval. The preset filtering lexicon described in the present application includes, but is not limited to, any one or any combination of texts, including: advertisement text, illicit text, \35881, cursive text, riot text, etc. In one example, the advertisement text includes: advertising address, contact information, investment loan, competitive product drainage, recruitment information and the like; the banned text comprises: illegal education, spam answers, gambling, etc. And identifying the content of the words or word combinations in the preset screening word stock, and outputting the content to form an audio identification result.
And fuzzifying the audio frame containing the audio recognition result in the audio information to generate the courseware displaying audio.
In the embodiment of the description, according to the audio time range output by the audio recognition result, the audio sound in the time range in which the audio with the problem is recognized is fuzzified, wherein the fuzzification can be mute, busy tone and the like, and finally the courseware showing audio is formed.
Specifically, the interactive information playing process comprises,
acquiring an interactive text file with a time mark according to the interactive information;
in the embodiment of the specification, the interactive information is the text content of communication between teachers and students and between students in the video live broadcast text interactive frame.
Generating an interaction recognition result for the interaction text file according to a preset screening word bank;
in the embodiment of the specification, the interactive information text file is input into a preset screening word bank for retrieval. The preset filter lexicon described in the present application includes, but is not limited to, any one or any combination of texts, including: advertisement text, illicit text, \35881, cursive text, riot text, etc. In one example, the advertisement text includes: advertising address, contact information, investment loan, competitive product drainage, recruitment information and the like; the banned text comprises: illegal education, spam answers, gambling, etc. And identifying the content of the words or word combinations in the preset screening word stock, and outputting an interactive identification result.
And deleting the characters containing the interactive recognition result in the interactive information to generate display interactive characters.
In the embodiment of the specification, the illegal words are deleted according to the interactive recognition result, and finally, the displayed interactive words are formed.
In one example, the illegal word may be processed by replacing the illegal word with a symbol such as an "x".
Specifically, the outward display setting includes,
setting whether playback information is displayed and played back externally by self-definition;
in the embodiment of the specification, the management terminal sets, by user, whether playback videos subjected to piece video information, portrait video information, audio information and interaction information violation processing are displayed externally.
Setting playback information display duration in a user-defined manner;
in the embodiment of the present specification, the management terminal sets, by user, playback videos on which piece video information, portrait video information, audio information, and interaction information are processed in an illegal manner, and the time for displaying the playback videos is long.
Setting a playback information display segment in a user-defined manner;
in the embodiment of the description, the management terminal sets the playback video subjected to the piece video information, the portrait video information, the audio information and the interaction information violation processing in a self-defined manner, and displays one or more video clips externally.
And self-defining whether the playback information opens the red packet popup window.
In the embodiment of the description, the management end sets the playback video subjected to the piece video information, the portrait video information, the audio information and the interactive information violation processing in a self-defined manner, and whether a red packet popup window is added in the playback video or not.
In one example, the red packet popup setting further includes the opening and closing time of the red packet popup in the playback video, the pop frequency of the red packet popup in the playback video, and the like.
Based on the same inventive concept, embodiments of the present specification further provide a live playback management apparatus, and the live playback management is implemented by using the live playback management method provided above. Fig. 2 is a schematic diagram of a live playback management apparatus provided in an embodiment herein, and the present specification provides the operation steps of the method according to the embodiment or the flowchart, but more or less operation steps may be included based on conventional or non-creative labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of sequences, and does not represent a unique order of performance. In practice, the system or apparatus may be implemented in accordance with the embodiments or with the methods illustrated in the figures. The method specifically comprises the following steps:
an obtaining module 201, configured to obtain live-broadcast course data, where the live-broadcast course data includes courseware video information, portrait video information, audio information, and interaction information,
the playback information processing module 202 is configured to perform playback processing on the courseware video information, the portrait video information, the audio information, and the interaction information, and generate initial playback information according to alignment of the timestamps;
and an external display setting module 203, configured to perform external display setting on the initial playback information according to a service requirement.
The advantages achieved by the device provided by the embodiment of the specification are consistent with those achieved by the method, and are not described in detail herein.
As shown in fig. 3, which is a schematic structural diagram of a computer device in this embodiment, the live playback management apparatus in this embodiment may be a computer device in this embodiment, and execute the method in this embodiment. Computer device 302 may include one or more processing devices 304, such as one or more Central Processing Units (CPUs), each of which may implement one or more hardware threads. The computer device 302 may also include any storage resource 306 for storing any kind of information, such as code, settings, data, etc. For example, and without limitation, storage resources 306 may include any one or more of the following in combination: any type of RAM, any type of ROM, flash memory devices, hard disks, optical disks, etc. More generally, any storage resource may use any technology to store information. Further, any storage resource may provide volatile or non-volatile reservation of information. Further, any storage resource may represent a fixed or removable component of computer device 302. In one case, when processing device 304 executes associated instructions stored in any storage resource or combination of storage resources, computer device 302 may perform any of the operations of the associated instructions. The computer device 302 also includes one or more drive mechanisms 808, such as a hard disk drive mechanism, an optical disk drive mechanism, etc., for interacting with any storage resource.
Corresponding to the method in fig. 1, the embodiments herein also provide a computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, performs the steps of the optimization method of the decision engine.
Embodiments herein also provide computer readable instructions, wherein a program therein causes a processor to perform the method as shown in fig. 1 when the instructions are executed by the processor.
It should be understood that, in various embodiments herein, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments herein.
It should also be understood that, in the embodiments herein, the term "and/or" is only one kind of association relation describing an associated object, meaning that three kinds of relations may exist. For example, a and/or B, may represent: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided herein, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the elements may be selected according to actual needs to achieve the objectives of the embodiments herein.
In addition, functional units in the embodiments herein may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present invention may be implemented in a form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The principles and embodiments of this document are explained herein using specific examples, which are presented only to aid in understanding the methods and their core concepts; meanwhile, for the general technical personnel in the field, according to the idea of this document, there may be changes in the concrete implementation and the application scope, in summary, this description should not be understood as the limitation of this document.
Claims (10)
1. A live playback management method, characterized in that the method comprises,
acquiring live course data, wherein the live course data comprises courseware video information, portrait video information, audio information and interaction information;
respectively carrying out playback processing on courseware video information, portrait video information, audio information and interaction information, and aligning according to the timestamps to generate initial playback information;
and according to the service requirement, carrying out external display setting on the initial playback information.
2. The live playback management method of claim 1, wherein prior to obtaining the live lesson data, further comprising,
sending a request for acquiring live course data to a live broadcast server, wherein the request for the live course data carries a live broadcast identifier;
and the live broadcast server verifies and returns the live broadcast course data according to the course identification in the request of the live broadcast course data.
3. The live playback management method of claim 1, wherein the courseware video information playback processing comprises,
acquiring the time information of the change of the courseware video information;
capturing pictures of the image information according to the time information;
extracting the features of the captured pictures, and generating an image identification result according to feature comparison;
and fuzzifying a page containing an image recognition result in the courseware video information to generate a courseware displaying video.
4. The live playback management method according to claim 1, wherein the portrait video information playback processing includes,
performing frame extraction on the portrait video information to obtain one or more video frames;
and sequentially inputting one or more video frames into a video detection model, and blurring the video frames with illegal behaviors to generate a portrait displaying video.
5. The live playback management method according to claim 1, wherein the audio information playback processing includes,
performing character conversion on the audio information to obtain an audio text file with a time mark;
generating an audio recognition result for the audio text file according to a preset screening word bank;
and fuzzifying the audio frame containing the audio recognition result in the audio information to generate the courseware displaying audio.
6. The live playback management method according to claim 1, wherein the interactive information playback processing includes,
acquiring an interactive text file with a time mark according to the interactive information;
generating an interaction identification result for the interaction text file according to a preset screening word bank;
and deleting the characters containing the interactive recognition result in the interactive information to generate display interactive characters.
7. The live playback management method of claim 1, wherein the outward show setting comprises,
setting whether playback information is displayed and played back externally by self-definition;
setting playback information display duration in a user-defined manner;
setting a playback information display segment in a user-defined manner;
and self-defining whether the playback information opens the red packet popup window.
8. A live playback management apparatus, characterized by comprising,
an acquisition module for acquiring live course data, wherein the live course data comprises courseware video information, portrait video information, audio information and interaction information,
the playback information processing module is used for respectively performing playback processing on the courseware video information, the portrait video information, the audio information and the interaction information, aligning according to the timestamps and generating initial playback information;
and the external display setting module is used for carrying out external display setting on the initial playback information according to the service requirement.
9. A computer device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, performs the instructions of the method of any one of claims 1-7.
10. A computer storage medium on which a computer program is stored, characterized in that the computer program, when being executed by a processor of a computer device, executes instructions of a method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210535703.9A CN114979787A (en) | 2022-05-17 | 2022-05-17 | Live broadcast playback management method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210535703.9A CN114979787A (en) | 2022-05-17 | 2022-05-17 | Live broadcast playback management method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114979787A true CN114979787A (en) | 2022-08-30 |
Family
ID=82983289
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210535703.9A Pending CN114979787A (en) | 2022-05-17 | 2022-05-17 | Live broadcast playback management method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114979787A (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040006767A1 (en) * | 2002-07-02 | 2004-01-08 | Robson Gary D. | System, method, and computer program product for selective filtering of objectionable content from a program |
| CN1871856A (en) * | 2003-08-26 | 2006-11-29 | 克里尔普雷有限公司 | Method and apparatus for controlling play of an audio signal |
| CN106101819A (en) * | 2016-06-21 | 2016-11-09 | 武汉斗鱼网络科技有限公司 | A kind of live video sensitive content filter method based on speech recognition and device |
| CN106803930A (en) * | 2017-02-10 | 2017-06-06 | 上海斐讯数据通信技术有限公司 | A kind of intelligent video monitoring method and intelligent router based on router |
| CN111797752A (en) * | 2020-06-29 | 2020-10-20 | 广州市百果园信息技术有限公司 | Illegal video detection method, device, equipment and storage medium |
| CN112634892A (en) * | 2020-12-17 | 2021-04-09 | 北京大米科技有限公司 | Voice processing method and device, readable storage medium and electronic equipment |
| CN113095178A (en) * | 2021-03-30 | 2021-07-09 | 北京大米科技有限公司 | Bad information detection method, system, electronic device and readable storage medium |
| CN113613035A (en) * | 2021-07-30 | 2021-11-05 | 广州繁星互娱信息科技有限公司 | Sensitive information processing method and device, electronic equipment and storage medium |
| US20210352359A1 (en) * | 2020-05-08 | 2021-11-11 | Facebook, Inc. | Synchronizing streams of co-watching digital video content while providing live digital video chat streams across multiple client devices |
| CN113938707A (en) * | 2021-10-12 | 2022-01-14 | 深圳创维－Rgb电子有限公司 | Video processing method, recording and playing box and computer readable storage medium |
| CN114171065A (en) * | 2021-11-29 | 2022-03-11 | 重庆长安汽车股份有限公司 | Audio acquisition and comparison method and system and vehicle |
| US20220208014A1 (en) * | 2019-07-03 | 2022-06-30 | Beijing Yizhen Xuesi Education Technology Co., Ltd. | Curriculum optimisation method, apparatus, and system |
-
2022
- 2022-05-17 CN CN202210535703.9A patent/CN114979787A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040006767A1 (en) * | 2002-07-02 | 2004-01-08 | Robson Gary D. | System, method, and computer program product for selective filtering of objectionable content from a program |
| CN1871856A (en) * | 2003-08-26 | 2006-11-29 | 克里尔普雷有限公司 | Method and apparatus for controlling play of an audio signal |
| CN106101819A (en) * | 2016-06-21 | 2016-11-09 | 武汉斗鱼网络科技有限公司 | A kind of live video sensitive content filter method based on speech recognition and device |
| CN106803930A (en) * | 2017-02-10 | 2017-06-06 | 上海斐讯数据通信技术有限公司 | A kind of intelligent video monitoring method and intelligent router based on router |
| US20220208014A1 (en) * | 2019-07-03 | 2022-06-30 | Beijing Yizhen Xuesi Education Technology Co., Ltd. | Curriculum optimisation method, apparatus, and system |
| US20210352359A1 (en) * | 2020-05-08 | 2021-11-11 | Facebook, Inc. | Synchronizing streams of co-watching digital video content while providing live digital video chat streams across multiple client devices |
| CN111797752A (en) * | 2020-06-29 | 2020-10-20 | 广州市百果园信息技术有限公司 | Illegal video detection method, device, equipment and storage medium |
| CN112634892A (en) * | 2020-12-17 | 2021-04-09 | 北京大米科技有限公司 | Voice processing method and device, readable storage medium and electronic equipment |
| CN113095178A (en) * | 2021-03-30 | 2021-07-09 | 北京大米科技有限公司 | Bad information detection method, system, electronic device and readable storage medium |
| CN113613035A (en) * | 2021-07-30 | 2021-11-05 | 广州繁星互娱信息科技有限公司 | Sensitive information processing method and device, electronic equipment and storage medium |
| CN113938707A (en) * | 2021-10-12 | 2022-01-14 | 深圳创维－Rgb电子有限公司 | Video processing method, recording and playing box and computer readable storage medium |
| CN114171065A (en) * | 2021-11-29 | 2022-03-11 | 重庆长安汽车股份有限公司 | Audio acquisition and comparison method and system and vehicle |
Non-Patent Citations (1)
| Title |
|---|
| 李霖清;: ""监控直播"的侵权风险与规避――以360"水滴直播"为例", 东南传播, no. 01 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110570698B (en) | Online teaching control method and device, storage medium and terminal | |
| CN110796098B (en) | Method, device, equipment and storage medium for training and auditing content auditing model | |
| CN106303658A (en) | It is applied to exchange method and the device of net cast | |
| CN107316520B (en) | Video teaching interaction method, device, equipment and storage medium | |
| US11113472B2 (en) | Content curation for course generation | |
| CN110673777A (en) | Online teaching method and device, storage medium and terminal equipment | |
| CN104463423A (en) | Formative video resume collection method and system | |
| CN103327278A (en) | Real time sharing method and device of images | |
| CN113038185B (en) | Bullet screen processing method and device | |
| CN114339285B (en) | Knowledge point processing method, video processing method, device and electronic equipment | |
| CN110505498A (en) | Processing, playback method, device and the computer-readable medium of video | |
| DE102021125184A1 (en) | PERSONAL TALK RECOMMENDATIONS USING LISTENER RESPONSES | |
| CN111276018A (en) | Network course recording method and device and terminal | |
| CN109862375B (en) | Cloud recording and broadcasting system | |
| CN114979787A (en) | Live broadcast playback management method, device, equipment and storage medium | |
| CN112040277B (en) | Video-based data processing method and device, computer and readable storage medium | |
| US10593366B2 (en) | Substitution method and device for replacing a part of a video sequence | |
| CN108667891A (en) | Separate unit combined type multimedia information dissemination method and system | |
| CN111327943B (en) | Information management method, device, system, computer equipment and storage medium | |
| US20200026535A1 (en) | Converting Presentations into and Making Presentations from a Universal Presentation Experience | |
| CN110781322B (en) | Multimedia courseware generation method and device, storage medium and terminal equipment | |
| Shapsough et al. | Using machine learning to automate classroom observation for low-resource environments | |
| CN113515670A (en) | Method, device and storage medium for identifying state of movie and television resource | |
| CN113554904A (en) | Intelligent processing method and system for multi-mode collaborative education | |
| CN115086761B (en) | Interaction method and system for pull-tab information of audio and video works |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |