CN118138832B - Network video stream display method based on GPU hard layer - Google Patents
Network video stream display method based on GPU hard layer Download PDFInfo
- Publication number
- CN118138832B CN118138832B CN202410543627.5A CN202410543627A CN118138832B CN 118138832 B CN118138832 B CN 118138832B CN 202410543627 A CN202410543627 A CN 202410543627A CN 118138832 B CN118138832 B CN 118138832B
- Authority
- CN
- China
- Prior art keywords
- gpu
- display
- frame
- decoding
- hard layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 239000000872 buffer Substances 0.000 claims abstract description 50
- 230000003139 buffering effect Effects 0.000 claims 1
- 230000001133 acceleration Effects 0.000 abstract description 7
- 238000009877 rendering Methods 0.000 abstract description 5
- 238000012545 processing Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000004075 alteration Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/4223—Cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440227—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by decomposing into layers, e.g. base layer and one or more enhancement layers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/64—Addressing
- H04N21/6402—Address allocation for clients
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a network video stream display method based on a GPU hard layer, wherein a RGBA format is directly generated by GPU decoding video without rendering by a two-dimensional and three-dimensional acceleration module; and the RGBA image is displayed by using the GPU hard layer, the generated RGBA data is not required to be copied to a display area for display, and the RGBA image corresponding to the cached frame can be displayed only by transmitting the address of the RGBA cached frame to the GPU hard layer. The GPU is used for directly decoding the video stream into the RGBA format, the hard layer displays the RGBA image by transmitting the frame buffer address instead of copying the video frame to the display area, and compared with a general mode, the method has the advantages of simpler flow and more suitability for application scenes of low delay and low power consumption of network video stream.
Description
Technical Field
The invention relates to the field of video processing, in particular to a network video stream display method based on a GPU hard layer.
Background
The coming of the 5G era, the real-time display of high-definition large-resolution network video has a great deal of demands in a plurality of scenes, such as the fields of high-definition monitoring video transmission processing, automatic driving, video conference and the like, and in the scenes, the network video stream data volume is large, the real-time requirement is high, the video is required to be decoded and displayed as soon as possible, and meanwhile, the power consumption is expected to be low.
To address the above needs, there has been a trend towards using GPUs for network video processing. The GPU has strong multipath video online processing capability, has multiple functions of intelligent computation, graphic rendering and the like, and has more advantages in the face of intelligent scenes such as automatic driving, defense, robots and the like.
The current general flow for processing network video by using a GPU mainly comprises the following steps: firstly, a GPU decoding unit receives a network video stream, the GPU decodes the network video stream into YUV pixel data, then a two-dimensional graphic acceleration module or a three-dimensional graphic acceleration module of the GPU renders the YUV pixel data into RGBA data, and finally the generated RGBA data is transmitted to a display area for display. However, when the delay is as low as possible in the case of a scene where the video zoom function is not required, for example, a high-definition monitoring video, an intelligent driving image, a reverse image, etc., the delay is still relatively high, and the performance of the GPU is not fully exerted.
Disclosure of Invention
Aiming at the technical problems in the prior art, the invention provides a network video stream display method based on a GPU hard layer, which realizes low-delay decoding display of high-definition network video stream through the GPU, and meets the scene that delay power consumption is low and video scaling is not needed.
A network video stream display method based on GPU hard layer includes:
step S1: the network camera encodes the image into a code stream, and transmits the code stream to the GPU decoding unit through a network protocol;
Step S2: the GPU decoding unit analyzes the code stream, applies for a plurality of frame caches according to the video output resolution, and is used for storing RGBA pixel images decoded by the GPU decoding unit;
Step S3: the GPU decoding unit decodes the code stream into RGBA images, and sequentially stores RGBA image frames into a multi-block frame buffer of the application;
Step S4: the display arbiter acquires the applied multi-block frame buffer address and video frame ID, establishes a corresponding relation between the frame buffer address and the video frame ID, opens a display channel, and informs the GPU of opening a hard layer;
step S5: the GPU hard layer is started after acquiring the notification signal and is initially set to be in a transparent mode;
Step S6: after the GPU decoding unit decodes the multi-frame, notifying a display arbiter that video output display is needed, and transmitting a display frame ID to the display arbiter according to the video display sequence;
Step S7: the display arbiter determines the frame buffer address of the RGBA image corresponding to the display frame ID according to the display frame ID and the corresponding relation between the frame buffer address and the video frame ID, and transmits the frame buffer address to the GPU hard layer, wherein the display frame ID is the same as the video frame ID;
Step S8: the GPU hard layer is switched to a display mode, and corresponding RGBA images are displayed according to the frame buffer addresses transmitted by the display arbiter;
Step S9: after the display is finished, the GPU hard layer sends a display finishing signal to the GPU decoding unit, the GPU decoding unit continues decoding, and decoded RGBA image frames are stored in a frame buffer which is sent to display;
Step S10: after the GPU decoding unit acquires the decoding stopping command, the frame buffer is released, the GPU hard layer is closed, and the decoding event is ended.
According to the network video stream display method based on the GPU hard layer, the RGBA format is directly generated by GPU decoding video, and rendering is not needed by a two-dimensional and three-dimensional acceleration module; and the RGBA image is displayed by using the GPU hard layer, the generated RGBA data is not required to be copied to a display area for display, and the RGBA image corresponding to the cached frame can be displayed only by transmitting the address of the RGBA cached frame to the GPU hard layer. The GPU is used for directly decoding the video stream into the RGBA format, the hard layer displays the RGBA image by transmitting the frame buffer address instead of copying the video frame to the display area, and compared with a general mode, the method has the advantages of simpler flow and more suitability for application scenes of low delay and low power consumption of network video stream.
Drawings
Fig. 1 is a flowchart of a method for displaying a network video stream based on a GPU hard layer according to the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. In addition, the technical features of each embodiment or the single embodiment provided by the invention can be combined with each other at will to form a feasible technical scheme, and the combination is not limited by the sequence of steps and/or the structural composition mode, but is necessarily based on the fact that a person of ordinary skill in the art can realize the combination, and when the technical scheme is contradictory or can not realize, the combination of the technical scheme is not considered to exist and is not within the protection scope of the invention claimed.
Based on the defects of the prior art, the invention provides a network video stream display method based on a GPU hard layer, which is used for meeting the demands of network video stream display on low delay and low power consumption. According to the method, a GPU two-dimensional or three-dimensional acceleration module is not needed to participate, the copying of RGBA data is reduced, the display effect is the same, the processing efficiency is more efficient, the time delay is greatly reduced compared with the existing scheme, meanwhile, the scheme can be completed by only needing a small number of GPU modules, the power consumption is reduced, and the display requirement of low delay and low power consumption can be better met.
Fig. 1 is a flowchart of a network video stream display method based on a GPU hard layer, where the method includes:
step S1: the network camera encodes the image into a code stream and transmits the code stream to the GPU decoding unit through a network protocol.
It can be understood that the network camera captures video images in real time, encodes the images into a code stream, and sends the code stream to the GPU decoding unit.
Step S2: and the GPU decoding unit analyzes the code stream, applies for a plurality of frame caches according to the video output resolution, and is used for storing RGBA pixel images decoded by the GPU decoding unit.
It can be understood that, when the GPU decoding unit receives the code stream, it parses the code stream, and applies for a plurality of frame buffer areas, so as to store the RGBA pixel image decoded by the GPU decoding unit.
The process of the GPU decoding unit for analyzing the code stream comprises the following steps:
Step S201: the GPU decoding unit analyzes the frame header of the code stream to obtain the coding format, resolution, level and decoding configuration information of the video, and frame header data such as a Sequence parameter set (Sequence PARAMETER SET) and an image parameter set (Picture PARAMETER SET) of H264 and the like;
Step S202: the GPU decoding unit configures decoding settings, including decoding a corresponding encoding format, decoding output size, decoding video sequence and the like, and particularly configures decoding output to be in an RGBA form;
step S203: according to the video output resolution, a plurality of frame buffers are applied, and the frame buffers can ensure that enough reference frames are stored for decoding.
Step S3: the GPU decoding unit decodes the code stream into RGBA images, and sequentially stores RGBA image frames in the applied multi-block frame buffer.
The GPU decoding unit analyzes the code stream transmitted by the network camera, decodes the code stream into RGBA images, and stores the analyzed RGBA image frames in the applied multi-block frame buffer area.
The GPU decoding unit directly decodes the video into the RGBA format without rendering by a two-dimensional and three-dimensional acceleration module, so that the method can be completed by only a small number of modules of the GPU, and reduces the power consumption.
Step S4: the display arbiter acquires the applied multi-block frame buffer address and the video frame ID, establishes a corresponding relation between the frame buffer address and the video frame ID, opens a display channel, and informs the GPU of opening the hard layer.
It can be understood that, when the GPU decoding unit stores the resolved RGBA image frames in the applied multi-block frame buffer area, the corresponding relationship between the frame buffer address and the video frame ID stored therein is obtained, and then, the corresponding video frame ID can be found according to the frame buffer address, so as to find the corresponding video frame.
When the video frame is required to be displayed, the display arbiter opens a display channel and notifies the GPU that the hard layer is opened to prepare for displaying the video.
Step S5: the GPU hard layer is started after acquiring the notification signal and is initially set to be in a transparent mode.
Step S6: and after the GPU decoding unit decodes the multi-frames, notifying the display arbiter that video output display is needed, and transmitting the display frame ID to the display arbiter according to the video display sequence.
It can be understood that, after the GPU decoding unit decodes the multi-frame code stream, if display is required, the display sequence of the multi-frame video is set, and the ID of the display frame to be displayed is transmitted to the display arbiter, so as to inform the display arbiter of the ID of the display frame to be displayed.
Step S7: and the display arbiter determines the frame buffer address of the RGBA image corresponding to the display frame ID according to the display frame ID and the corresponding relation between the frame buffer address and the video frame ID, and transmits the frame buffer address to the GPU hard layer, wherein the display frame ID is the same as the video frame ID.
It can be understood that the display arbiter finds a corresponding frame buffer address according to the display frame ID transferred by the GPU decoding unit, and transfers the frame buffer address to the GPU hard layer, so that the GPU hard layer displays the video frame.
Step S8: the GPU hard layer is switched to a display mode, and corresponding RGBA images are displayed according to the frame buffer addresses transmitted by the display arbiter.
It can be understood that when the invention displays the video frame, RGBA images in the frame buffer corresponding to the frame buffer address can be displayed only according to the frame buffer address transmitted by the display arbiter. When the RGBA image is displayed by using the GPU hard layer, the generated RGBA data is not required to be copied to a display area for display, and the RGBA image corresponding to the cache frame can be displayed by only transmitting the address of the RGBA cache frame to the hard layer.
Step S9: after the display is finished, the GPU hard layer sends a display finishing signal to the GPU decoding unit, the GPU decoding unit continues decoding, and the decoded RGBA image frames are stored in a frame buffer which is sent for display.
Wherein, step S9 comprises the following sub-steps:
step S901: after the display is finished, the GPU hard layer sends a display finishing signal to the GPU decoding unit;
step S902: the GPU decoding unit obtains the display end signal, continues decoding, waits if the display end signal is not received, and ensures that the frame buffer has a spare space, namely the frame buffer area which is already displayed has a spare area, and then decodes.
Step S903: and the GPU decoding unit stores the decoded RGBA image frames in an idle frame buffer area.
Step S10: after the GPU decoding unit acquires the decoding stopping command, the frame buffer is released, the GPU hard layer is closed, and the decoding event is ended.
Step S10: after the GPU decoding unit acquires the decoding stopping command, the frame buffer is released, the GPU hard layer is closed, and the decoding event is ended.
Wherein, step S10 comprises the following sub-steps:
step S1001: the GPU decoding unit receives a decoding stop or end command, firstly completes decoding of the current frame, and then stops decoding;
Step S1002: the GPU decoding unit resets the decoding initial value and clears the decoding state;
Step S1003: and the GPU decoding unit releases the applied frame buffer, closes the GPU hard layer and ends the decoding event.
Compared with the prior art, the network video stream display method based on the GPU hard layer has the following advantages:
The GPU decodes the video to directly generate RGBA format without rendering by a two-dimensional and three-dimensional acceleration module; and the RGBA image is displayed by using the GPU hard layer, the generated RGBA data is not required to be copied to a display area for display, and the RGBA image corresponding to the cached frame can be displayed only by transmitting the address of the RGBA cached frame to the GPU hard layer. Compared with the general mode, the method has the advantages that the flow is simplified, and the method is more suitable for application scenes of low delay and low power consumption of network video streaming.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (4)
1. The network video stream display method based on the GPU hard layer is characterized by comprising the following steps of:
step S1: the network camera encodes the image into a code stream, and transmits the code stream to the GPU decoding unit through a network protocol;
Step S2: the GPU decoding unit analyzes the code stream, applies for a plurality of frame caches according to the video output resolution, and is used for storing RGBA pixel images decoded by the GPU decoding unit;
Step S3: the GPU decoding unit decodes the code stream into RGBA images, and sequentially stores RGBA image frames into a multi-block frame buffer of the application;
Step S4: the display arbiter acquires the applied multi-block frame buffer address and video frame ID, establishes a corresponding relation between the frame buffer address and the video frame ID, opens a display channel, and informs the GPU of opening a hard layer;
step S5: the GPU hard layer is started after acquiring the notification signal and is initially set to be in a transparent mode;
Step S6: after the GPU decoding unit decodes the multi-frame, notifying a display arbiter that video output display is needed, and transmitting a display frame ID to the display arbiter according to the video display sequence;
Step S7: the display arbiter determines the frame buffer address of the RGBA image corresponding to the display frame ID according to the display frame ID and the corresponding relation between the frame buffer address and the video frame ID, and transmits the frame buffer address to the GPU hard layer, wherein the display frame ID is the same as the video frame ID;
Step S8: the GPU hard layer is switched to a display mode, and corresponding RGBA images are displayed according to the frame buffer addresses transmitted by the display arbiter;
Step S9: after the display is finished, the GPU hard layer sends a display finishing signal to the GPU decoding unit, the GPU decoding unit continues decoding, and decoded RGBA image frames are stored in a frame buffer which is sent to display;
Step S10: after the GPU decoding unit acquires the decoding stopping command, the frame buffer is released, the GPU hard layer is closed, and the decoding event is ended.
2. The GPU-hard layer-based network video stream display method according to claim 1, wherein the step S2 comprises:
Step S201: the GPU decoding unit analyzes the frame header of the code stream to obtain the coding format, resolution, grade, decoding configuration information and frame header data of the video, wherein the frame header data comprises a sequence parameter set and an image parameter set;
step S202: the GPU decoding unit configures decoding parameters including decoding a corresponding encoding format, decoding output size and decoding video sequence, and configures decoding output into RGBA form;
step S203: and applying for multi-block frame buffering according to the video output resolution.
3. The GPU-hard layer-based network video stream display method according to claim 1, wherein the step S9 comprises:
step S901: after the display is finished, the GPU hard layer sends a display finishing signal to the GPU decoding unit;
Step S902: after the GPU decoding unit obtains the display end signal, continuing to decode the code stream, and waiting if the display end signal is not received, so as to ensure that a frame buffer is free, namely a frame buffer area of a display is free;
step S903: and the GPU decoding unit stores the decoded RGBA image frames in an idle frame buffer area.
4. The GPU-hard layer-based network video stream display method according to claim 1, wherein the step S10 comprises:
Step S1001: when the GPU decoding unit receives a decoding stop or end command, the decoding of the current frame is completed, and then the decoding is stopped;
Step S1002: the GPU decoding unit resets the decoding initial value and clears the decoding state;
Step S1003: and the GPU decoding unit releases the applied frame buffer, closes the GPU hard layer and ends the decoding event.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410543627.5A CN118138832B (en) | 2024-05-06 | 2024-05-06 | Network video stream display method based on GPU hard layer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410543627.5A CN118138832B (en) | 2024-05-06 | 2024-05-06 | Network video stream display method based on GPU hard layer |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118138832A CN118138832A (en) | 2024-06-04 |
| CN118138832B true CN118138832B (en) | 2024-07-26 |
Family
ID=91236121
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410543627.5A Active CN118138832B (en) | 2024-05-06 | 2024-05-06 | Network video stream display method based on GPU hard layer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118138832B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113055681A (en) * | 2021-03-02 | 2021-06-29 | 长沙景嘉微电子股份有限公司 | Video decoding display method, device, electronic equipment and storage medium |
| CN117499755A (en) * | 2023-11-09 | 2024-02-02 | 武汉凌久微电子有限公司 | Method for embedding watermark or subtitle into hard decoding video based on GPU hard layer |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10402937B2 (en) * | 2017-12-28 | 2019-09-03 | Nvidia Corporation | Multi-GPU frame rendering |
| CN108366288A (en) * | 2018-03-01 | 2018-08-03 | 厦门普杰信息科技有限公司 | A kind of efficient decoding and playback method and system for HD video |
| CN115335866A (en) * | 2020-02-03 | 2022-11-11 | 索尼互动娱乐股份有限公司 | System and method for efficient multi-GPU rendering of geometry through geometry analysis at rendering |
| CN112954233B (en) * | 2021-01-29 | 2022-11-18 | 稿定（厦门）科技有限公司 | Video synthesis system and method based on GPU |
| CN113015003B (en) * | 2021-03-08 | 2022-11-25 | 展讯通信（上海）有限公司 | Video frame caching method and device |
| CN113625983B (en) * | 2021-08-10 | 2024-08-27 | Oppo广东移动通信有限公司 | Image display method, device, computer equipment and storage medium |
| CN116737248B (en) * | 2023-08-09 | 2023-11-14 | 武汉凌久微电子有限公司 | Dormancy and awakening method of hard decoder |
-
2024
- 2024-05-06 CN CN202410543627.5A patent/CN118138832B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113055681A (en) * | 2021-03-02 | 2021-06-29 | 长沙景嘉微电子股份有限公司 | Video decoding display method, device, electronic equipment and storage medium |
| CN117499755A (en) * | 2023-11-09 | 2024-02-02 | 武汉凌久微电子有限公司 | Method for embedding watermark or subtitle into hard decoding video based on GPU hard layer |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118138832A (en) | 2024-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111357289B (en) | Method, apparatus and system for a game engine for video encoder rendering | |
| CN109600666B (en) | Video playing method, device, medium and electronic equipment in game scene | |
| WO2017161907A1 (en) | Video coding method and apparatus, device, and storage medium | |
| US20180341323A1 (en) | Methods and apparatuses for handling virtual reality content | |
| CN108063976B (en) | Video processing method and device | |
| US11528538B2 (en) | Streaming volumetric and non-volumetric video | |
| CN109168014A (en) | A kind of live broadcasting method, device, equipment and storage medium | |
| RU2733218C2 (en) | Method, apparatus and a stream for formatting an immersive video image for traditional and immersive playback devices | |
| US11924442B2 (en) | Generating and displaying a video stream by omitting or replacing an occluded part | |
| CN112511896A (en) | Video rendering method and device | |
| WO2023040825A1 (en) | Media information transmission method, computing device and storage medium | |
| CN110049347B (en) | Method, system, terminal and device for configuring images on live interface | |
| CN104980798B (en) | A kind of far-end video display methods and device | |
| CN107580228B (en) | Monitoring video processing method, device and equipment | |
| CN118138832B (en) | Network video stream display method based on GPU hard layer | |
| CN117499755A (en) | Method for embedding watermark or subtitle into hard decoding video based on GPU hard layer | |
| US20230025664A1 (en) | Data processing method and apparatus for immersive media, and computer-readable storage medium | |
| CN110798715A (en) | Video playing method and system based on image string | |
| WO2024183387A1 (en) | Image coding method and apparatus, image decoding method and apparatus, and system | |
| JP2001128194A (en) | Stereoscopic video processor, method for processing stereoscopic video and storing medium | |
| CN115665461B (en) | Video recording method and virtual reality device | |
| US12132995B2 (en) | Enhancing 360-degree video using convolutional neural network (CNN)-based filter | |
| CN114827638B (en) | VR video cloud live broadcast method, device and equipment | |
| US20230054523A1 (en) | Enhancing 360-degree video using convolutional neural network (cnn)-based filter | |
| CN116016969A (en) | Method and system for online transcoding multiple-output shared filter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |