CN112351338B - Video processing method, device, system and computer readable storage medium - Google Patents
Video processing method, device, system and computer readable storage medium Download PDFInfo
- Publication number
- CN112351338B CN112351338B CN201910733156.3A CN201910733156A CN112351338B CN 112351338 B CN112351338 B CN 112351338B CN 201910733156 A CN201910733156 A CN 201910733156A CN 112351338 B CN112351338 B CN 112351338B
- Authority
- CN
- China
- Prior art keywords
- video processing
- format
- image data
- video stream
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4782—Web browsing, e.g. WebTV
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the invention provides a video processing method, a video processing device, a video processing system and a computer readable storage medium. The video processing method comprises the following steps: receiving a video stream data packet in a first format sent by a server side in a binary form by using JavaScript on a browser page; decoding the video stream data packet using a decoder capable of decoding binary data to obtain second format image data; and displaying the second format image data using a canvas tab on a browser page. The embodiment of the invention has the advantages of simple realization, no need of Lai Chajian, good universality/cross-platform performance, low delay, small bandwidth pressure, strong flexibility and the like.
Description
Technical Field
The present invention relates to the field of video processing technologies, and in particular, to a video processing method, a video processing apparatus, a video processing system, and a computer-readable storage medium.
Background
The current Real-Time video Streaming transmission and processing scheme is usually implemented at a WEB end by using Real-Time Streaming (RTSP)/RTMP (Real Time Messaging Protocol) and other Real-Time Streaming-based schemes, which require a local plug-in at a browser end, but the adoption of the local plug-in at the browser end is not beneficial to cross-platform, breaks the sandbox environment of the browser, and has potential safety hazards; or a scheme that the progressive downloading is adopted such as HLS (HTTP Live Streaming)/DASH (Dynamic Adaptive Streaming over HTTP), and the like, although the browser is originally supported, the browser is not real-time Streaming due to the fact that the browser belongs to a progressive downloading mode and has the inevitable high-delay defect; or, the MJPEG (Motion Joint Photographic Experts Group) video stream is pushed through the video stream, and the browser extracts JPEG (Joint Photographic Experts Group) pictures from the MJPEG frame by frame for display, but the JPEG compression ratio problem causes that the bandwidth occupation is large, for example, the network pressure of a picture with the same resolution is about 10 times higher than that of an h.264 coding mode.
Disclosure of Invention
Based on the above problems, embodiments of the present invention provide a video processing method, a video processing apparatus, a video processing system, and a computer readable storage medium, which effectively solve the above problems and solve the problems of complexity, plug-in dependency, high latency, and lack of flexibility and extensibility when a real-time video stream is transmitted between a server and a browser.
Specifically, an embodiment of the present invention provides a video processing method, including: receiving a video stream data packet in a first format sent by a server side in a binary form by using JavaScript on a browser page; decoding the video stream data packet using a decoder capable of decoding binary data to obtain second format image data; and displaying the second format image data using a canvas tab on a browser page.
Compared with the prior art, the embodiment uses the decoder capable of decoding binary data to decode the video stream data packet to obtain the second format image data, the binary data is closer to machine codes, and has the advantages of low delay, small bandwidth pressure, strong flexibility and the like, and the browser uses the decoder, such as a webpage assembly language, instead of a local plug-in to decode the obtained first format video stream data packet, so that the implementation is simple, the method is not dependent on Lai Chajian, and the universality/cross-platform performance is good.
In an embodiment of the present invention, the first format video stream packet is an h.264 format video stream packet, and the second format image data is RGB format image data.
In one embodiment of the invention, the decoder of FFmpeg is a WebAssembly module or a JavaScript source code FFmpeg.
In an embodiment of the present invention, the video processing method further includes: converting the FFmpeg source code into the WebAssembly module or the JavaScript source code by a cross-coding tool to obtain the decoder.
On the other hand, an embodiment of the present invention further provides a video processing apparatus, including: the acquisition module is used for receiving a video stream data packet in a first format sent by a server side in a binary form by using JavaScript on a browser page; a processing module for decoding the video stream data packet using a decoder capable of decoding binary data to obtain second format image data; and the display module is used for displaying the second format image data by utilizing the canvas label on the browser page.
Compared with the prior art, the processing module of the embodiment decodes the video stream data packet by using a decoder capable of decoding binary data to obtain the second format image data, the binary data is closer to machine codes and has the advantages of low delay, small bandwidth pressure, strong flexibility and the like, and the browser decodes the obtained first format video stream data packet by using the decoder, such as a web page assembly language, instead of a local plug-in, so that the implementation is simple, the method is not dependent on Lai Chajian, and the universality/cross-platform performance is good.
In an embodiment of the present invention, the first format video stream data packet is a h.264 format video stream data packet, and the second format image data is RGB format image data h.264.
In one embodiment of the invention, the decoder of the FFmpeg is a WebAssembly module or a JavaScript source code WebAssembly FFmpeg.
In an embodiment of the present invention, the video processing apparatus further includes a cross-coding module, configured to convert, by a cross-coding tool, an FFmpeg source code into the WebAssembly module or the JavaScript source code to obtain the decoder.
In another aspect, an embodiment of the present invention further provides a video processing system, including: the video processing system comprises a memory and a processor connected with the memory, wherein the memory stores a computer program, and the processor executes any one of the video processing methods when running the computer program.
In another aspect, an embodiment of the present invention further provides a computer-readable storage medium, which is a non-volatile memory and stores computer-executable instructions, where the computer-executable instructions are configured to perform any one of the video processing methods described above.
One or more of the above technical solutions may have the following advantages or beneficial effects: according to the video processing method, the video processing device, the video processing system and the computer readable storage medium provided by the embodiment of the invention, the JavaScript is used on the browser page to receive the video stream data packet in the first format sent by the server side in a binary form, and the decoder capable of decoding the binary data is used for decoding the video stream data packet to obtain the image data in the second format, so that the binary data is closer to a machine code, and the method has the advantages of low delay, small bandwidth pressure, strong flexibility and the like, and the browser can be used for decoding the obtained video stream data packet in the first format by using the decoder instead of a local plug-in, so that the method is simple, does not need Lai Chajian, and is good in universality/cross-platform performance.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a video processing method according to a first embodiment of the present invention.
FIG. 2 is a schematic diagram illustrating a scenario of video streaming at a server according to the present invention.
Fig. 3 is a block diagram of a video processing apparatus according to a second embodiment of the present invention.
Fig. 4 is a block diagram of a video processing system according to a third embodiment of the present invention.
Fig. 5 is a block diagram of a computer-readable storage medium according to a fourth embodiment of the present invention.
[ brief description of the drawings ]
S1-S3: video processing method steps;
10: a video processing device; 11: an acquisition module; 12: a processing module; 13: a display module;
20: a video processing system; 21: a processor; 22: a memory;
30: a computer-readable storage medium; 31: computer-executable instructions.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
[ first embodiment ] A method for manufacturing a semiconductor device
Referring to fig. 1, fig. 1 is a flowchart of a video processing method according to a first embodiment of the invention. The video processing method comprises the following steps:
step S1, receiving a first format video stream data packet such as an H.264 format video stream data packet sent by a server side in a binary form by using JavaScript on a browser page;
step S2, decoding the video stream data packet by using a decoder capable of decoding binary data, such as a WebAssembly module or a JavaScript source code, so as to obtain second format image data, such as RGB format image data; and
and S3, displaying the image data by using the canvas element.
In a specific embodiment, the video processing method further includes converting FFmpeg source code into the WebAssembly module or the JavaScript source code by a cross-coding tool such as an emascript to obtain the decoder.
Please refer to fig. 1 and fig. 2, fig. 2 is a schematic diagram illustrating a scenario of video streaming at a server according to the present invention. Generally, video stream data transmission is performed between a video stream server and a client, where the client is, for example, a PC or a video processing device installed with a WEB browser, and the video stream data acquired by the client from the video stream server is displayed and played on the WEB browser.
In order to more clearly understand the present embodiment, the video processing method of the present embodiment is briefly described below. The video streaming server side outputs video streaming data packets in a first format, such as video streaming data packets in an H.264 format, in a websocket communication protocol. Specifically, the video streaming server side uses the multimedia framework gstreamer to continuously acquire camera image data or other image data and video data, and encodes the camera image data or other image data and video data into a video streaming data packet in an h.264 format, then acquires the h.264 video streaming data packet by using a plug-in applink of the multimedia framework gstreamer, and continuously sends out the h.264 video streaming data packet by a full-duplex communication protocol websocket based on TCP.
A full-duplex communication protocol websocket on a page based on TCP of a browser front-end code on a client uses a JavaScript format and receives an H.264 video stream data packet transmitted by a video stream server end in a websocket protocol in a binary form, a decoder capable of decoding binary data such as a WebAssembly module or a JavaScript source code is used for decoding the H.264 video stream data packet to obtain image data in a second format such as an RGB format, and finally the RGB image data is displayed by using a canvas label.
In one embodiment, the client converts the FFmpeg source code into the WebAssembly module format or JavaScript source code by a cross-compilation tool such as emscript to obtain the decoder.
In summary, the video processing method provided in the embodiment of the present invention uses a decoder capable of decoding binary data to decode the video stream packet to obtain the image data in the second format to implement the browser real-time video stream service, and has the advantages of being simple to implement, not dependent on Lai Chajian, good in universality/cross-platform performance, low in delay, low in bandwidth pressure, strong in flexibility, and the like.
[ second embodiment ]
Referring to fig. 3, fig. 3 is a block diagram of a video processing apparatus according to a second embodiment of the present invention. The video processing apparatus 10 includes:
the acquisition module 11 is configured to receive, on a browser page, a video stream packet in a first format, such as an h.264 format video stream packet, sent by a server in a binary form using JavaScript;
a processing module 12, configured to decode the video stream packet using a decoder capable of decoding binary data, such as a WebAssembly module or a JavaScript source code, to obtain second format image data, such as RGB format image data; and
a display module 13, configured to display the image data by using the canvas element.
In a specific embodiment, the video processing apparatus 10 further includes a cross-coding module, configured to convert the FFmpeg source code into the WebAssembly module or the JavaScript source code by a cross-coding tool, such as an emscript, to obtain the decoder.
The video processing method implemented by the video processing apparatus 10 according to the present embodiment is as described in the first embodiment, and therefore, detailed description thereof is omitted here. Optionally, each module, unit and the other operations or functions in the second embodiment are respectively for implementing the method in the first embodiment of the present invention, and are not described herein again for brevity.
In summary, the processing module 12 provided by the present invention decodes the video stream packet by using a decoder capable of decoding binary data, for example, a WebAssembly module or a JavaScript source code, to obtain second format image data, for example, RGB format image data, so as to implement a browser real-time video stream service, and has the beneficial effects of being simple to implement, not dependent on Lai Chajian, good in universality/cross-platform performance, low in delay, small in bandwidth pressure, strong in flexibility, and the like.
[ third embodiment ]
Referring to fig. 4, fig. 4 is a block diagram of a video processing system according to a third embodiment of the present invention. The video processing system 20 comprises a processor 21 and a memory 22 connected to the processor 21; in which the memory 22 stores a computer program. The processor 21 executes the aforementioned video processing method when running the computer program.
[ fourth embodiment ]
Referring to fig. 5, fig. 5 is a block diagram illustrating a computer-readable storage medium according to a fourth embodiment of the present invention. The computer-readable storage medium 30 is, for example, a nonvolatile memory, which is, for example: magnetic media (e.g., hard disks, floppy disks, and magnetic tape), optical media (e.g., cd ROM disks and DVDs), magneto-optical media (e.g., optical disks), and hardware devices specially constructed for storing and executing computer-executable instructions (e.g., read Only Memories (ROMs), random Access Memories (RAMs), flash memories, etc.). Computer-readable storage medium 30 has stored thereon computer-executable instructions 31. The computer-readable storage medium 30 may execute the computer-executable instructions 31 by one or more processors or processing devices to implement the video processing method in the foregoing first embodiment.
In addition, it should be understood that the foregoing embodiments are merely exemplary of the present application, and technical solutions of the embodiments may be arbitrarily combined and used in combination without conflict between technical features and structures and without departing from the purpose of the present application.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one type of logical functional division, and other divisions may be realized in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A video processing method, comprising:
receiving a video stream data packet in a first format sent by a server side in a binary form by using JavaScript on a browser page;
decoding the video stream data packet using a decoder capable of decoding binary data to obtain second format image data; and
displaying the second format image data using a canvas tab on a browser page;
wherein the first format video stream data packet is an H.264 format video stream data packet.
2. The video processing method according to claim 1, wherein the second format image data is RGB format image data.
3. The video processing method according to claim 1, wherein the decoder is a WebAssembly module or JavaScript source code.
4. The video processing method of claim 3, further comprising:
converting the FFmpeg source code into the WebAssembly module or the JavaScript source code by a cross-coding tool to obtain the decoder.
5. A video processing apparatus, comprising:
the acquisition module is used for receiving a video stream data packet in a first format sent by a server side in a binary form by using JavaScript on a browser page;
a processing module for decoding the video stream data packet using a decoder capable of decoding binary data to obtain second format image data; and
the display module is used for displaying the second format image data by utilizing a canvas label on a browser page;
wherein the first format video stream data packet is an H.264 format video stream data packet.
6. The video processing apparatus according to claim 5, wherein the second format image data is RGB format image data.
7. The video processing apparatus of claim 5, wherein the decoder is a WebAssembly module or JavaScript source code.
8. The video processing apparatus of claim 7, further comprising a cross-coding module for converting FFmpeg source code into the WebAssembly module or the JavaScript source code by a cross-coding tool to obtain the decoder.
9. A video processing system, comprising: a memory and a processor connected to the memory, the memory storing a computer program that, when executed by the processor, performs the video processing method of any of claims 1 to 4.
10. A computer-readable storage medium that is a non-volatile memory and that stores computer-executable instructions for performing the video processing method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910733156.3A CN112351338B (en) | 2019-08-09 | 2019-08-09 | Video processing method, device, system and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910733156.3A CN112351338B (en) | 2019-08-09 | 2019-08-09 | Video processing method, device, system and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112351338A CN112351338A (en) | 2021-02-09 |
| CN112351338B true CN112351338B (en) | 2023-04-07 |
Family
ID=74367556
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910733156.3A Active CN112351338B (en) | 2019-08-09 | 2019-08-09 | Video processing method, device, system and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112351338B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018197911A1 (en) * | 2017-04-28 | 2018-11-01 | Forbidden Technologies Plc | Methods, systems, processors and computer code for providing video clips |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8964830B2 (en) * | 2002-12-10 | 2015-02-24 | Ol2, Inc. | System and method for multi-stream video compression using multiple encoding formats |
| US9654809B2 (en) * | 2011-07-26 | 2017-05-16 | DMD Digital Media Distribution Limited | Method and system for providing live television and video-on-demand content to subscribers |
| CN108111904A (en) * | 2016-11-24 | 2018-06-01 | 厦门脉视数字技术有限公司 | A kind of audio/video player system and method based on Web |
| CN107277004A (en) * | 2017-06-13 | 2017-10-20 | 重庆扬讯软件技术股份有限公司 | A kind of browser is without plug-in unit net cast method |
| CN108337528B (en) * | 2018-01-17 | 2021-04-16 | 浙江大华技术股份有限公司 | Method and equipment for previewing video |
| CN108966006A (en) * | 2018-07-24 | 2018-12-07 | 上海小蚁科技有限公司 | Playback method, browser equipment and the readable storage medium storing program for executing of video |
| CN109981607B (en) * | 2019-03-07 | 2022-09-09 | 深圳市商汤科技有限公司 | Media stream processing method and device, electronic equipment and storage medium |
-
2019
- 2019-08-09 CN CN201910733156.3A patent/CN112351338B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018197911A1 (en) * | 2017-04-28 | 2018-11-01 | Forbidden Technologies Plc | Methods, systems, processors and computer code for providing video clips |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112351338A (en) | 2021-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10412130B2 (en) | Method and apparatus for playing media stream on web browser | |
| CN110784740A (en) | Video processing method, device, server and readable storage medium | |
| CN108337246B (en) | Media playback apparatus and media service apparatus preventing playback delay | |
| US11418802B2 (en) | Video decoder memory optimization | |
| CN112954457A (en) | Video playing and displaying method, device and system | |
| JP2018191269A (en) | Method and device of reconstructing image data from decoded image data | |
| CN112261377B (en) | Web edition monitoring video playing method, electronic equipment and storage medium | |
| US20140333641A1 (en) | System and method for forwarding a graphics command stream | |
| US10819951B2 (en) | Recording video from a bitstream | |
| WO2021057697A1 (en) | Video encoding and decoding methods and apparatuses, storage medium, and electronic device | |
| WO2023226915A1 (en) | Video transmission method and system, device, and storage medium | |
| CN108055595B (en) | Video image redirection method and computer-readable storage medium | |
| CN113938470A (en) | Method and device for playing RTSP data source by browser and streaming media server | |
| US20140333640A1 (en) | System and method for forwarding a graphics command stream | |
| CN112351338B (en) | Video processing method, device, system and computer readable storage medium | |
| CN111010593A (en) | Method and device for packaging H.265 video data based on FLV format | |
| US20140333639A1 (en) | System and method for forwarding a graphics command stream | |
| CN110798700B (en) | Video processing method, video processing device, storage medium and electronic equipment | |
| EP2804094A1 (en) | A system and method for forwarding a graphics command stream | |
| EP2804095A1 (en) | A system and method for forwarding a graphics command stream | |
| CN105812922A (en) | Multimedia file data processing method, system, player and client | |
| CN111669541A (en) | Pre-monitoring picture display method and display system | |
| CN113747099B (en) | Video transmission method and device | |
| US11523156B2 (en) | Method and system for distributing an audiovisual content | |
| CN115514975B (en) | Encoding and decoding method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |