CN113271479A - Playing processing method, device and related equipment - Google Patents
Playing processing method, device and related equipment Download PDFInfo
- Publication number
- CN113271479A CN113271479A CN202110534089.XA CN202110534089A CN113271479A CN 113271479 A CN113271479 A CN 113271479A CN 202110534089 A CN202110534089 A CN 202110534089A CN 113271479 A CN113271479 A CN 113271479A
- Authority
- CN
- China
- Prior art keywords
- audio
- information
- video
- video information
- channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 19
- 238000004806 packaging method and process Methods 0.000 claims abstract description 14
- 238000004891 communication Methods 0.000 claims abstract description 9
- 238000000034 method Methods 0.000 claims description 40
- 238000004458 analytical method Methods 0.000 claims description 14
- 230000000694 effects Effects 0.000 abstract description 14
- 238000010586 diagram Methods 0.000 description 13
- 238000012544 monitoring process Methods 0.000 description 13
- 238000009877 rendering Methods 0.000 description 7
- 230000009286 beneficial effect Effects 0.000 description 5
- 206010000117 Abnormal behaviour Diseases 0.000 description 4
- 238000005538 encapsulation Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012938 design process Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/23608—Remultiplexing multiplex streams, e.g. involving modifying time stamps or remapping the packet identifiers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4344—Remultiplexing of multiplex streams, e.g. by modifying time stamps or remapping the packet identifiers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2624—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of whole input images, e.g. splitscreen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides a play processing method, a play processing device and related equipment, and belongs to the technical field of communication. The playing processing method comprises the following steps: acquiring first audio and video information pushed by electronic equipment, wherein the first audio and video information is a set of audio and video information acquired by a plurality of channels of the electronic equipment; analyzing the first audio and video information to obtain channel information, video bare code stream information and audio information of each channel in the plurality of channels; packaging the channel information, the video bare bit stream information and the audio information of each channel into audio and video information in a target format, and obtaining second audio and video information which is a set of audio and video information in a plurality of target formats corresponding to a plurality of channels; and sending second audio and video information to the browser end so that the browser end simultaneously plays the audio and video information corresponding to the multiple channels based on the second audio and video information. The playing processing method can improve the playing effect of simultaneously playing the audio and video contents corresponding to a plurality of channels.
Description
Technical Field
The embodiment of the invention relates to the technical field of communication, in particular to a playing processing method, a playing processing device and related equipment.
Background
With the development of technology, the concept of intelligent traffic is widely applied, for example, by installing an electronic recording device such as a vehicle data recorder on a vehicle, state information such as driving information of the vehicle and abnormal behavior of a driver is recorded and stored, and service support for vehicle real-time monitoring, abnormal behavior monitoring of the driver and the like is provided for a background.
Currently, the real-time monitoring of the driving of the vehicle and the behavior of the driver is generally performed in a live broadcast manner. However, when the electronic recording device includes a plurality of channels, the playing end needs to switch to play the monitoring content of each channel to realize the playing of the monitoring content of each channel, which causes a delay in the live broadcast process, and makes it difficult for the background to monitor the driving of the vehicle and the behavior of the driver in real time.
As can be seen, in the related art, when the electronic recording apparatus includes a plurality of channels, the monitored contents corresponding to the plurality of channels have a problem of poor playing effect.
Disclosure of Invention
The embodiment of the invention provides a playing processing method, a playing processing device and related equipment, which can solve the problem that in the related art, under the condition that an electronic recording device comprises a plurality of channels, the playing effect of monitoring contents corresponding to the channels is poor.
In order to solve the problems, the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a play processing method, which is applied to a server, and the method includes:
acquiring first audio and video information pushed by electronic equipment, wherein the first audio and video information is a set of audio and video information acquired by a plurality of channels of the electronic equipment;
analyzing the first audio and video information to obtain channel information, video bare code stream information and audio information of each channel in the plurality of channels;
packaging the channel information, the video bare bit stream information and the audio information of each channel into audio and video information in a target format, and obtaining second audio and video information, wherein the second audio and video information is a set of a plurality of audio and video information in the target format corresponding to the plurality of channels;
and sending the second audio and video information to a browser end so that the browser end simultaneously plays the audio and video information corresponding to the multiple channels based on the second audio and video information.
In a second aspect, an embodiment of the present invention provides a play processing method, which is applied to a browser end, and the method includes:
receiving second audio and video information sent by a server, wherein the second audio and video information is obtained by converting the server based on first audio and video information pushed by electronic equipment, and the first audio and video information is a set of audio and video information acquired by a plurality of channels of the electronic equipment;
analyzing the second audio and video information to obtain channel information, video bare code stream information and audio information of each channel in the plurality of channels;
and simultaneously playing the audio and video information corresponding to the plurality of channels based on the channel information, the video bare bit stream information and the audio information of each channel.
In a third aspect, an embodiment of the present invention further provides a play processing apparatus, including a processor and a transceiver, where the play processing apparatus further includes:
the acquisition module is used for acquiring first audio and video information pushed by electronic equipment, wherein the first audio and video information is a set of audio and video information acquired by a plurality of channels of the electronic equipment;
the analysis module is used for analyzing the first audio and video information to obtain channel information, video bare code stream information and audio information of each channel in the plurality of channels;
the packaging module is used for packaging the channel information, the video bare bit stream information and the audio information of each channel into audio and video information in a target format and obtaining second audio and video information, wherein the second audio and video information is a set of a plurality of audio and video information in the target format corresponding to the channels;
and the first sending module is used for sending the second audio and video information to a browser end so that the browser end simultaneously plays the audio and video information corresponding to the plurality of channels based on the second audio and video information.
In a fourth aspect, an embodiment of the present invention further provides a playback processing apparatus, including a processor and a transceiver, where the apparatus further includes:
the receiving module is used for receiving second audio and video information sent by a server, wherein the second audio and video information is obtained by converting the server based on first audio and video information pushed by electronic equipment, and the first audio and video information is a set of audio and video information acquired by a plurality of channels of the electronic equipment;
the analysis module is used for analyzing the second audio and video information to obtain channel information, video bare code stream information and audio information of each channel in the plurality of channels;
and the playing module is used for simultaneously playing the audio and video information corresponding to the channels based on the channel information, the video bare bit stream information and the audio information of each channel.
In a fifth aspect, an embodiment of the present invention further provides a communication device, including: a transceiver, a memory, a processor, and a program stored on the memory and executable on the processor; the processor, configured to read a program in the memory to implement the steps of the method according to the first aspect; or, a step in a method as described in the second aspect above.
In a sixth aspect, the embodiments of the present invention further provide a readable storage medium, for storing a program, where the program, when executed by a processor, implements the steps in the method according to the first aspect, or implements the steps in the method according to the second aspect.
In the embodiment of the invention, the compatibility of audio and video information in different audio and video coding formats can be realized, and the purpose that the audio and video information corresponding to a plurality of channels of the electronic equipment is simultaneously played in the same player (namely, a browser end) can be realized, namely, the browser end simultaneously plays videos of different channels of the electronic equipment, and the effect of simultaneously playing the audio and video contents corresponding to the plurality of channels is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a schematic structural diagram of a network system to which an embodiment of the present invention is applicable;
fig. 2 is one of the structural diagrams of a video playing system according to an embodiment of the present invention;
fig. 3 is a second block diagram of a video playing system according to an embodiment of the present invention;
fig. 4 is a flowchart illustrating a playing processing method according to an embodiment of the present invention;
fig. 5 is a second flowchart illustrating a playing processing method according to a second embodiment of the present invention;
FIG. 6 is a schematic diagram of a tone-picture synchronization provided by an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of a playback processing apparatus according to an embodiment of the present invention;
fig. 8 is a second schematic structural diagram of a playback processing apparatus according to the present invention;
fig. 9 is a schematic structural diagram of a communication device provided in the implementation of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," and the like in the embodiments of the present invention are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Further, as used herein, "and/or" means at least one of the connected objects, e.g., a and/or B and/or C, means 7 cases including a alone, B alone, C alone, and both a and B present, B and C present, both a and C present, and A, B and C present.
Referring to fig. 1, fig. 1 is a block diagram of a network system to which an embodiment of the present invention is applicable, and as shown in fig. 1, the network system includes a data transmitting device 11 and a data receiving device 12.
Wherein, the data transmitting device 11 and the data receiving device 12 can communicate with each other. The data transmission apparatus 11 transmits Ciphertext information (Ciphertext Block) to the data reception apparatus 12.
In practical applications, the data sending device 11 may be a terminal (also referred to as User Equipment (UE)), and the data receiving device 12 may be a network side device; alternatively, the data transmitting device 11 may be a network side device, and the data receiving device 12 may be a terminal, but is not limited thereto.
The terminal may be a Mobile phone, a Tablet Personal Computer (Tablet Personal Computer), a Laptop Computer (Laptop Computer), a Personal Digital Assistant (PDA), a Mobile Internet Device (MID), a Wearable Device (Wearable Device), or a vehicle-mounted Device. The network side device may be a base station, an Access and Mobility Management Function (AMF), a relay, an Access point, or other network elements.
The following describes a playback processing method provided in an embodiment of the present invention.
The playing processing method provided by the embodiment of the invention can be applied to a video playing system, the video playing system comprises electronic equipment, a server and a browser end, and the server is used for realizing information interaction between the electronic equipment and the browser end and can also process the information of the electronic equipment and the browser end.
The electronic device may be a driving recorder, such as a ministerial mark device, which is used for recording and storing state information of a vehicle, such as a driving speed, driving information, and abnormal behavior of a driver, and further providing service support for vehicle real-time monitoring, abnormal behavior monitoring of a driver, and the like for a background (such as a browser end).
Referring to fig. 2 and fig. 3, fig. 2 is a first structural diagram of a video playing system to which an embodiment of the present invention is applicable, and fig. 3 is a second structural diagram of a video playing system to which an embodiment of the present invention is applicable. As shown in fig. 2 and 3, the video playing system includes an electronic device 210, a server 220, and a browser end 230.
The electronic device 210 includes a plurality of channels, such as channel 1, channel 2, and channel … …, each of which can collect audio-video information of different directions or areas; for example, in the case that the electronic device 210 is a vehicle event data recorder on a vehicle, the channel 1 may collect audio and video information in front of the vehicle, the channel 2 may adopt audio and video information in a cabin of the vehicle, and the channel 3 may collect audio and video information behind the vehicle.
The server 220 includes an analysis module 221, an encapsulation module 222, and a sending module 223, where the analysis module 221 may analyze audio and video information corresponding to multiple channels pushed by the electronic device 210 to obtain corresponding audio and video bare stream information, media information, and channel information, where the channel information generally refers to a channel number corresponding to the channel, the media information may generally refer to media information associated with the channel, and the audio and video bare stream information may generally refer to audio and video information acquired by the channel; the encapsulation module 222 is configured to encapsulate the parsed audio/video bare stream information, media information, and channel information to form audio/video information in a target format, where the target format may be a format defined by a user, so as to convert audio/video information collected by electronic devices with different coding formats into audio/video information with the same coding format, and achieve the purpose of compatibility between different audio/video codes of electronic devices of different manufacturers; the sending module 223 is configured to send a set of audio/video information in multiple target formats corresponding to multiple channels to the browser end.
The server 220 may further include an audio decoding module 224, and the audio decoding module 224 may decode the audio information obtained by analyzing by the analyzing module 221 into an audio bare code stream, so as to improve an analysis effect of the audio and video information pushed by the electronic device.
For example, after the parsing module 221 parses the original audio data, the corresponding audio coding type may be determined, and then the audio decoding library of the matching type is used for decoding, so as to obtain a corresponding PCM bare stream, that is, an audio bare stream.
In an example, in a case where the AUDIO coding types supported by the ministerial protocol include g.721, g.722, g.723, g.728, g.729, g.711a, g.711u, g.726, g.729a, LPCM, ACC, AACLC, MP3, MP4AUDIO, and the like, that is, in a case where the parsing module 221 parses the original AUDIO data into any one of the above AUDIO coding types, a corresponding Pulse Code Modulation (PCM) AUDIO bare bit stream may be obtained, so as to improve the AUDIO decoding capability of the server.
The browser 230 includes a decapsulation module 231, a rendering and playing control module 232, and a player 233, where the decapsulation module 231 is configured to decapsulate and analyze a set of audio/video information in multiple target formats corresponding to multiple channels sent by the server 220, so as to obtain audio/video bare stream information, media information, and channel information of each channel; the rendering and playing control module 232 is configured to render the audio/video bare stream information, the media information, and the channel information of each channel obtained through decapsulation and analysis, so as to achieve an effect of synchronizing audio and video; the player 233 is configured to play the rendered audio/video information of the multiple channels, so as to realize playing of the audio/video information of the multiple channels, that is, to simultaneously play the audio/video information acquired by the multiple channels of the electronic device, so that a supervisor can perform comparison and problem detection in different dimensions, and thus the purpose of improving the playing effect of simultaneously playing the audio/video content corresponding to the multiple channels is achieved.
Referring to fig. 4, fig. 4 is a flowchart illustrating a playing processing method according to an embodiment of the present invention. The play processing method shown in fig. 4 may be performed by a server.
As shown in fig. 4, the playing processing method may include the steps of:
In this step, the first audio/video information may be an audio/video file packet formed by compressing and packaging audio/video information collected by a plurality of channels, or may be an audio/video folder including audio/video information collected by a plurality of channels.
In this step, the first audio/video information may be parsed based on a coding library compiled based on an audio/video protocol standard to obtain channel information, video bare bit stream information, and audio information of each of the plurality of channels.
In some embodiments, the audio and video information acquired by the acquisition channel of the electronic device may further include media information, so that in the process of analyzing the first audio and video information, corresponding media information may also be obtained, and the media information may also be used in subsequent steps and may also be displayed or played at the browser end.
For example, the conventional video encoding methods include h.264, h.265, AVS, SVAV, etc., and the video encoding format commonly used by electronic devices such as a car recorder is h.264 or h.265. Therefore, only two video encoding modes, h.264 and h.265, can be considered in the design process.
In the step, the channel information, the video bare bit stream information and the audio information of each channel are packaged into the audio and video information in the target format, so that the purpose of compatibility of different audio and video codes of electronic equipment of different manufacturers can be achieved.
And step 404, sending the second audio and video information to a browser end so that the browser end simultaneously plays the audio and video information corresponding to the multiple channels based on the second audio and video information.
Through the steps, the compatibility of audio and video information in different audio and video coding formats can be realized, and the purpose that the audio and video information corresponding to a plurality of channels of the electronic equipment is played simultaneously in the same player (namely, the browser end) can be realized, namely, the browser end plays videos of different channels of the electronic equipment simultaneously, and the playing effect of simultaneously playing the audio and video contents corresponding to the plurality of channels is improved.
Optionally, the encapsulating the channel information, the video bare bit stream information, and the audio information of each channel into audio and video information in a target format includes:
analyzing the audio information of each channel and obtaining corresponding audio bare code stream information;
and packaging the channel information, the video bare bit stream information and the audio bare bit stream information of each channel into audio and video information in a target format.
In this embodiment, the audio information obtained by analysis may be further analyzed to obtain corresponding audio bare code stream information, that is, the audio information obtained by analysis may be analyzed into a PCM audio bare code stream, so as to improve the analysis effect on audio data of different audio coding formats.
Optionally, before the first audio-video information pushed by the electronic device is acquired, the method further includes:
receiving first play information sent by the browser end;
and sending second playing information to the electronic equipment, wherein the second playing information is obtained based on the first playing information, and the second playing information is used for the electronic equipment to push the first audio and video information to the server.
In this embodiment, the first playing information may include a playing instruction sent by the browser, and the first playing information may also carry a playing channel number, and the first playing information is used to implement information interaction between the browser and the server, and the second playing information is used to implement information interaction between the server and the electronic device.
And under the condition that the server receives the first playing information, the server can return a corresponding Websocket playing address to the browser end, so that the server and the browser end can establish a data transmission link based on the Websocket through the playing address.
According to the playing processing method provided by the embodiment of the invention, first audio and video information pushed by electronic equipment is obtained, wherein the first audio and video information is a set of audio and video information acquired by a plurality of channels of the electronic equipment; analyzing the first audio and video information to obtain channel information, video bare code stream information and audio information of each channel in the plurality of channels; packaging the channel information, the video bare bit stream information and the audio information of each channel into audio and video information in a target format, and obtaining second audio and video information, wherein the second audio and video information is a set of a plurality of audio and video information in the target format corresponding to the plurality of channels; and sending the second audio and video information to a browser end so that the browser end simultaneously plays the audio and video information corresponding to the multiple channels based on the second audio and video information. Therefore, the compatibility of audio and video information in different audio and video coding formats can be realized, and the playing effect of simultaneously playing the audio and video contents corresponding to a plurality of channels can be improved.
Referring to fig. 5, fig. 5 is a second schematic flowchart of a video playing method according to an embodiment of the present invention. The video playing method shown in fig. 5 may be executed by a browser side.
As shown in fig. 5, the video playing method may include the following steps:
and step 501, receiving second audio and video information sent by a server.
In the step, the second audio/video information is obtained by converting first audio/video information pushed by the server based on the electronic equipment, and the first audio/video information is a set of audio/video information acquired by a plurality of channels of the electronic equipment.
Through the steps, the purpose that the audio and video information corresponding to the channels of the electronic equipment is played in the browser end at the same time can be achieved, the phenomena of delay and the like in the playing process can be avoided, and the playing effect of playing at the same time is improved.
Optionally, in a case that the audio information corresponding to each channel is audio bare bit stream information, the simultaneously playing the audio and video information corresponding to the multiple channels based on the channel information, the video bare bit stream information, and the audio information of each channel includes:
analyzing the video bare code stream information of each channel, and obtaining corresponding original image data;
and simultaneously playing audio and video information corresponding to the plurality of channels based on the channel information, the original image data and the audio bare code stream information of each channel.
Optionally, before the second audio-video information sent by the server is received, the method further includes:
sending first playing information to the server;
the first playing information is used for the server to send second playing information to the electronic equipment, the second playing information is obtained by the server based on the first playing information, and the second playing information is used for the electronic equipment to push the first audio and video information to the server.
It should be noted that, this embodiment is an implementation of a browser end corresponding to the method embodiment shown in fig. 4, so that reference may be made to the relevant description in the above method embodiment, and the same beneficial effects may be achieved. To avoid repetition of the description, the description is omitted.
The various optional implementations described in the embodiments of the present invention may be implemented in combination with each other or implemented separately without conflict, and the embodiments of the present invention are not limited thereto.
For ease of understanding, examples are illustrated below:
in this example, taking a monitoring platform for a ministerial enterprise as an example, a platform video monitoring page includes a ministerial device, channel information and a live broadcast control function, and refer to fig. 2 and 3, and the specific operation steps are as follows:
firstly, in a monitoring platform of a ministerial enterprise, during initialization, a WEB end sends a live broadcast instruction of a ministerial device to a server end, the server end returns a corresponding Websocket live broadcast address, and the WEB end and the server end of the platform establish a data transmission link based on the Websocket through the address.
And secondly, operating a live broadcast control button of a department bidding device channel at a platform WEB end, sending a logic channel number of the department bidding device needing live broadcast to a server, sending a live broadcast instruction to a corresponding channel of the department bidding device after the server receives a request, and starting to push stream after the department bidding device receives the instruction.
And thirdly, after receiving the plug flow of the ministerial mark device, the platform server firstly carries out decapsulation according to a ministerial mark gateway 1078 to obtain original audio and video data, wherein the ministerial mark gateway 1078 is a decoding library written according to the ministerial mark 1078 audio and video protocol standard.
The conventional ministerial protocol includes four video encoding modes of h.264, h.265, AVS and SVAV, but the manufacturers use h.264 or h.265 for implementation, so only these two encoding modes are considered at present. But the protocol supports a plurality of audio coding modes, and manufacturers realize different modes, so that a unified audio decoding service is defined, a decoding library with various coding types is included, and the unified decoding service is decoded into a PCM audio bare code stream.
In one example, the specific process of audio decoding includes: compiling decoding libraries of various audio coding types realized by different manufacturer ministerial equipment in an audio and video processing module of a platform server; and after the ministerial video gateway analyzes to obtain original audio data, judging the corresponding audio coding type, and then decoding by using an audio decoding library of a matched type to obtain a PCM audio bare code stream.
And fourthly, transmitting information such as the audio and video bare code stream in the self-defined package mode to a WEB end of the platform.
The method specifically comprises the following steps: packaging the information of the equipment channel, the audio and video bare code stream, the equipment channel and the like into a data packet according to a format defined by the following table; and merging the encapsulated data packets into a live link and pushing the data packets to a platform WEB browser end.
In one example, the information such as the custom-packaged audio/video bare code stream can be referred to in the following table.
And fifthly, the platform WEB side receives the real-time data pushed by the server side through the Websocket, and then the real-time data is unpacked through the unpacking module to separate a PCM audio bare code stream, an H.264/H.265 video bare code stream, media information and equipment channel information.
And sixthly, compiling the FFmpeg library compiled by the C language into WASM through an Emscript tool, and then calling an exposed interface of the WASM by the Javascript to decode the H.264 or H.265 video bare code stream to obtain YUV image data.
And seventhly, playing audio and rendering images in the platform WEB side player after the rendering playing control module acquires the information.
The specific process comprises the following steps:
step a, after the rendering and playing control module obtains the PCM and YUV information, the information is stored in a buffer area.
The Audio data can be cached through the AudioContext of the Web Audio API, and the YUV image information can be stored through defining a buffer queue with a fixed size.
And b, playing the PCM Audio data through the Web Audio Api, and acquiring corresponding YUV image data through the recalled pts.
And c, rendering the corresponding YUV image data to a region of a corresponding channel of the WEB-side player through WebGL and Canvas according to the equipment channel number information so as to achieve the effect of sound and picture synchronization, and referring to a principle schematic diagram of sound and picture synchronization shown in FIG. 6.
Step eight, under the condition that a playing channel of the ministerial mark equipment needs to be added, a channel which is not live-broadcasted by the ministerial mark equipment can be opened in a WEB end of a monitoring platform of the ministerial mark enterprise, and the WEB end sends channel number data which needs to be added to a server end through a Websocket link; and then judging whether the channel opens the push flow through the server side.
If the channel is opened, the channel plug flow data is directly merged into a live link; if not, the flow is merged after the opening.
Step nine, under the condition that a channel in live broadcasting needs to be closed, firstly stopping video image rendering and audio playing of a corresponding channel in a player in a WEB end of a monitoring platform of a ministerial mark enterprise; then, channel number data needing to stop live broadcasting are sent to the server; and finally, the server side judges whether the channel starts the plug flow after receiving the information.
If not, ignoring; and if the channel is started, stopping converging the live broadcast data of the channel to the live broadcast link.
Therefore, the compatibility of audio and video information in different audio and video coding formats can be realized, and the purpose that the audio and video information corresponding to a plurality of channels of the electronic equipment is simultaneously played in the same player (namely, the browser end) can be realized, namely, the browser end simultaneously plays videos of different channels of the electronic equipment, and the effect of simultaneously playing the audio and video contents corresponding to the plurality of channels is improved.
Referring to fig. 7, fig. 7 is a structural diagram of a playback processing apparatus according to an embodiment of the present invention. As shown in fig. 7, the playback processing apparatus 700 includes:
the acquisition module 701 is used for acquiring first audio and video information pushed by electronic equipment, wherein the first audio and video information is a set of audio and video information acquired by a plurality of channels of the electronic equipment;
an analyzing module 702, configured to analyze the first audio/video information to obtain channel information, video bare bit stream information, and audio information of each of the multiple channels;
the encapsulation module 703 is configured to encapsulate the channel information, the video bare bit stream information, and the audio information of each channel into audio and video information in a target format, and obtain second audio and video information, where the second audio and video information is a set of multiple audio and video information in the target format corresponding to the multiple channels;
the first sending module 704 is configured to send the second audio and video information to a browser end, so that the browser end simultaneously plays the audio and video information corresponding to the multiple channels based on the second audio and video information.
Optionally, the encapsulation module 703 includes:
the analysis unit is used for analyzing the audio information of each channel and obtaining corresponding audio bare code stream information;
and the packaging unit is used for packaging the channel information, the video bare code stream information and the audio bare code stream information of each channel into audio and video information in a target format.
Optionally, the playback processing apparatus 700 further includes:
the receiving module is used for receiving first playing information sent by the browser end;
and the second sending module is used for sending second playing information to the electronic equipment, wherein the second playing information is obtained based on the first playing information, and the second playing information is used for pushing the first audio and video information to the server by the electronic equipment.
The playing processing device 700 can implement each process of the method embodiment of fig. 4 in the embodiment of the present invention, and achieve the same beneficial effects, and is not described herein again to avoid repetition.
Referring to fig. 8, fig. 8 is a second structural diagram of a playback processing apparatus according to an embodiment of the present invention. As shown in fig. 8, the playback processing apparatus 800 includes:
the receiving module 801 is configured to receive second audio and video information sent by a server, where the second audio and video information is obtained by converting first audio and video information pushed by an electronic device by the server, and the first audio and video information is a set of audio and video information acquired by multiple channels of the electronic device;
the analysis module 802 is configured to analyze the second audio/video information to obtain channel information, video bare bit stream information, and audio information of each of the multiple channels;
and the playing module 803 is configured to simultaneously play the audio and video information corresponding to the multiple channels based on the channel information, the video bare bit stream information, and the audio information of each channel.
Optionally, in a case that the audio information corresponding to each channel is audio bare bit stream information, the playing module 803 includes:
the analysis unit is used for analyzing the video bare code stream information of each channel and obtaining corresponding original image data;
and the playing unit is used for simultaneously playing the audio and video information corresponding to the channels based on the channel information, the original image data and the audio bare code stream information of each channel.
Optionally, the playback processing apparatus 800 further includes:
the sending module is used for sending first playing information to the server;
the first playing information is used for the server to send second playing information to the electronic equipment, the second playing information is obtained by the server based on the first playing information, and the second playing information is used for the electronic equipment to push the first audio and video information to the server.
The playing processing device 800 can implement each process of the method embodiment in fig. 5 in the embodiment of the present invention, and achieve the same beneficial effects, and for avoiding repetition, details are not described here.
The embodiment of the invention also provides communication equipment. Referring to fig. 9, a communication device may include a processor 901, a memory 902, and a program 9021 stored on the memory 902 and operable on the processor 901.
In the case that the communication device is a terminal, when executed by the processor 901, the program 9021 may implement any step in the method embodiment corresponding to fig. 4 and achieve the same beneficial effect, which is not described herein again.
In the case that the communication device is a network-side device, when being executed by the processor 901, the program 9021 may implement any step in the method embodiment corresponding to fig. 5 and achieve the same beneficial effect, which is not described herein again.
Those skilled in the art will appreciate that all or part of the steps of the method according to the above embodiments may be implemented by hardware associated with program instructions, and the program may be stored in a readable medium. An embodiment of the present invention further provides a readable storage medium, where a computer program is stored on the readable storage medium, and when the computer program is executed by a processor, any step in the method embodiment corresponding to fig. 4 or fig. 5 may be implemented, and the same technical effect may be achieved, and in order to avoid repetition, details are not repeated here.
The storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A playback processing method applied to a server, the method comprising:
acquiring first audio and video information pushed by electronic equipment, wherein the first audio and video information is a set of audio and video information acquired by a plurality of channels of the electronic equipment;
analyzing the first audio and video information to obtain channel information, video bare code stream information and audio information of each channel in the plurality of channels;
packaging the channel information, the video bare bit stream information and the audio information of each channel into audio and video information in a target format, and obtaining second audio and video information, wherein the second audio and video information is a set of a plurality of audio and video information in the target format corresponding to the plurality of channels;
and sending the second audio and video information to a browser end so that the browser end simultaneously plays the audio and video information corresponding to the multiple channels based on the second audio and video information.
2. The method of claim 1, wherein the encapsulating the channel information, the video bare stream information, and the audio information of each channel into the audio/video information of the target format comprises:
analyzing the audio information of each channel and obtaining corresponding audio bare code stream information;
and packaging the channel information, the video bare bit stream information and the audio bare bit stream information of each channel into audio and video information in a target format.
3. The method according to claim 1, wherein before the obtaining of the first audio-video information pushed by the electronic device, the method further comprises:
receiving first play information sent by the browser end;
and sending second playing information to the electronic equipment, wherein the second playing information is obtained based on the first playing information, and the second playing information is used for the electronic equipment to push the first audio and video information to the server.
4. A playing processing method is applied to a browser end, and comprises the following steps:
receiving second audio and video information sent by a server, wherein the second audio and video information is obtained by converting the server based on first audio and video information pushed by electronic equipment, and the first audio and video information is a set of audio and video information acquired by a plurality of channels of the electronic equipment;
analyzing the second audio and video information to obtain channel information, video bare code stream information and audio information of each channel in the plurality of channels;
and simultaneously playing the audio and video information corresponding to the plurality of channels based on the channel information, the video bare bit stream information and the audio information of each channel.
5. The method of claim 4, wherein in a case that the audio information corresponding to each channel is audio bare bit stream information, the simultaneously playing the audio and video information corresponding to the plurality of channels based on the channel information, the video bare bit stream information, and the audio information of each channel comprises:
analyzing the video bare code stream information of each channel, and obtaining corresponding original image data;
and simultaneously playing audio and video information corresponding to the plurality of channels based on the channel information, the original image data and the audio bare code stream information of each channel.
6. The method according to claim 4, wherein before the receiving the second audio-video information sent by the server, the method further comprises:
sending first playing information to the server;
the first playing information is used for the server to send second playing information to the electronic equipment, the second playing information is obtained by the server based on the first playing information, and the second playing information is used for the electronic equipment to push the first audio and video information to the server.
7. A playback processing apparatus, comprising a processor and a transceiver, and the apparatus further comprises:
the acquisition module is used for acquiring first audio and video information pushed by electronic equipment, wherein the first audio and video information is a set of audio and video information acquired by a plurality of channels of the electronic equipment;
the analysis module is used for analyzing the first audio and video information to obtain channel information, video bare code stream information and audio information of each channel in the plurality of channels;
the packaging module is used for packaging the channel information, the video bare bit stream information and the audio information of each channel into audio and video information in a target format and obtaining second audio and video information, wherein the second audio and video information is a set of a plurality of audio and video information in the target format corresponding to the channels;
and the first sending module is used for sending the second audio and video information to a browser end so that the browser end simultaneously plays the audio and video information corresponding to the plurality of channels based on the second audio and video information.
8. A playback processing apparatus, comprising a processor and a transceiver, and the apparatus further comprises:
the receiving module is used for receiving second audio and video information sent by a server, wherein the second audio and video information is obtained by converting the server based on first audio and video information pushed by electronic equipment, and the first audio and video information is a set of audio and video information acquired by a plurality of channels of the electronic equipment;
the analysis module is used for analyzing the second audio and video information to obtain channel information, video bare code stream information and audio information of each channel in the plurality of channels;
and the playing module is used for simultaneously playing the audio and video information corresponding to the channels based on the channel information, the video bare bit stream information and the audio information of each channel.
9. A communication device, comprising: a transceiver, a memory, a processor, and a program stored on the memory and executable on the processor; the processor is configured to read a program in the memory to implement the steps in the playback processing method according to any one of claims 1 to 3, or to implement the steps in the playback processing method according to any one of claims 4 to 6.
10. A readable storage medium storing a program, wherein the program, when executed by a processor, implements the steps in the playback processing method according to any one of claims 1 to 3, or implements the steps in the playback processing method according to any one of claims 4 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110534089.XA CN113271479B (en) | 2021-05-17 | 2021-05-17 | Playing processing method and device and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110534089.XA CN113271479B (en) | 2021-05-17 | 2021-05-17 | Playing processing method and device and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113271479A true CN113271479A (en) | 2021-08-17 |
| CN113271479B CN113271479B (en) | 2022-05-10 |
Family
ID=77231160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110534089.XA Active CN113271479B (en) | 2021-05-17 | 2021-05-17 | Playing processing method and device and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113271479B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113825016A (en) * | 2021-09-18 | 2021-12-21 | 北京百度网讯科技有限公司 | Video rendering method, device, equipment, storage medium and computer program product |
| CN113973222A (en) * | 2021-12-02 | 2022-01-25 | 成都统信软件技术有限公司 | Video playing method and device and computing equipment |
| CN114745599A (en) * | 2022-03-10 | 2022-07-12 | 慧之安信息技术股份有限公司 | Web end plug-in-free monitoring real-time preview method based on wasm |
| CN114827751A (en) * | 2022-03-28 | 2022-07-29 | 慧之安信息技术股份有限公司 | Web end plug-in-free monitoring video playing method based on wasm |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1933594A (en) * | 2005-09-14 | 2007-03-21 | 王世刚 | Multichannel audio-video frequency data network transmitting and synchronous playing method |
| CN106792155A (en) * | 2016-12-06 | 2017-05-31 | 天脉聚源(北京)传媒科技有限公司 | A kind of method and device of the net cast of multiple video strems |
| CN106993212A (en) * | 2017-03-20 | 2017-07-28 | 贵阳朗玛信息技术股份有限公司 | The method and device of multi-channel video is played in browser window |
| CN109640153A (en) * | 2018-12-12 | 2019-04-16 | 安徽科大国创软件科技有限公司 | A kind of method and device thereof that the browser multiwindow based on VLC plays |
| CN111064973A (en) * | 2019-11-28 | 2020-04-24 | 湖北工业大学 | Live broadcast system based on IPV9 |
| CN211352345U (en) * | 2019-10-31 | 2020-08-25 | 成都威爱新经济技术研究院有限公司 | 5G multichannel live broadcast system |
| CN112073809A (en) * | 2020-08-09 | 2020-12-11 | 富盛科技股份有限公司 | Method for supporting browser to play video with any coding format |
| CN112422508A (en) * | 2020-10-19 | 2021-02-26 | 武汉烽火众智数字技术有限责任公司 | Method and system for playing video in browser webpage |
| CN112788431A (en) * | 2020-12-24 | 2021-05-11 | 四川云从天府人工智能科技有限公司 | Video playing method, device, system, medium and browser based on HTML5 |
-
2021
- 2021-05-17 CN CN202110534089.XA patent/CN113271479B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1933594A (en) * | 2005-09-14 | 2007-03-21 | 王世刚 | Multichannel audio-video frequency data network transmitting and synchronous playing method |
| CN106792155A (en) * | 2016-12-06 | 2017-05-31 | 天脉聚源(北京)传媒科技有限公司 | A kind of method and device of the net cast of multiple video strems |
| CN106993212A (en) * | 2017-03-20 | 2017-07-28 | 贵阳朗玛信息技术股份有限公司 | The method and device of multi-channel video is played in browser window |
| CN109640153A (en) * | 2018-12-12 | 2019-04-16 | 安徽科大国创软件科技有限公司 | A kind of method and device thereof that the browser multiwindow based on VLC plays |
| CN211352345U (en) * | 2019-10-31 | 2020-08-25 | 成都威爱新经济技术研究院有限公司 | 5G multichannel live broadcast system |
| CN111064973A (en) * | 2019-11-28 | 2020-04-24 | 湖北工业大学 | Live broadcast system based on IPV9 |
| CN112073809A (en) * | 2020-08-09 | 2020-12-11 | 富盛科技股份有限公司 | Method for supporting browser to play video with any coding format |
| CN112422508A (en) * | 2020-10-19 | 2021-02-26 | 武汉烽火众智数字技术有限责任公司 | Method and system for playing video in browser webpage |
| CN112788431A (en) * | 2020-12-24 | 2021-05-11 | 四川云从天府人工智能科技有限公司 | Video playing method, device, system, medium and browser based on HTML5 |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113825016A (en) * | 2021-09-18 | 2021-12-21 | 北京百度网讯科技有限公司 | Video rendering method, device, equipment, storage medium and computer program product |
| CN113825016B (en) * | 2021-09-18 | 2024-05-07 | 北京百度网讯科技有限公司 | Video rendering method, apparatus, device, storage medium, and computer program product |
| CN113973222A (en) * | 2021-12-02 | 2022-01-25 | 成都统信软件技术有限公司 | Video playing method and device and computing equipment |
| CN114745599A (en) * | 2022-03-10 | 2022-07-12 | 慧之安信息技术股份有限公司 | Web end plug-in-free monitoring real-time preview method based on wasm |
| CN114827751A (en) * | 2022-03-28 | 2022-07-29 | 慧之安信息技术股份有限公司 | Web end plug-in-free monitoring video playing method based on wasm |
| CN114827751B (en) * | 2022-03-28 | 2023-01-17 | 慧之安信息技术股份有限公司 | Wasm-based web-end plug-in-free monitoring video playing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113271479B (en) | 2022-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113271479B (en) | Playing processing method and device and related equipment | |
| CN109327728B (en) | One-to-many same-screen method, device and system, same-screen equipment and storage medium | |
| US10869106B2 (en) | Data transmission method and apparatus, and electronic device | |
| CN112653700B (en) | Website video communication method based on WEBRTC | |
| US20100135646A1 (en) | Storage/playback method and apparatus for mpeg-2 transport stream based on iso base media file format | |
| KR20080072019A (en) | Time-shifted presentation of media streams | |
| CN110446114B (en) | Multimedia data processing device, method, electronic equipment and storage medium | |
| US10978109B2 (en) | Synchronously playing method and device of media file, and storage medium | |
| TW201123795A (en) | System, method and apparatus for dynamic media file streaming | |
| JPH10164138A (en) | Method and device for multimedia data store-and-forward | |
| CN112752115B (en) | Live broadcast data transmission method, device, equipment and medium | |
| CN105721620A (en) | Video information push method and device as well as video information display method and device | |
| CN102118539B (en) | Multimedia playing terminal as well as playing method and device for dragging multimedia files | |
| CN110996160B (en) | Video processing method and device, electronic equipment and computer readable storage medium | |
| CN111356023B (en) | Playing mode determining method and device | |
| CN107276990B (en) | Streaming media live broadcasting method and device | |
| US9641908B2 (en) | Method and system for transferring real-time audio/video stream | |
| CN108810575B (en) | Method and device for sending target video | |
| CN106331763A (en) | Method of playing slicing media files seamlessly and device of realizing the method | |
| CN113099282A (en) | Data processing method, device and equipment | |
| CN112565224A (en) | Video processing method and device | |
| CN115134664A (en) | Real-time video stream playing method and system and nonvolatile storage medium | |
| KR101621530B1 (en) | Method and System for Providing Flash Contents via Mobile Network and Method and Apparatus for Converting Flash Contents | |
| WO2022116822A1 (en) | Data processing method and apparatus for immersive media, and computer-readable storage medium | |
| EP2260629A1 (en) | Systems and methods for determining behaviors for live and playback consumption |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |