CN118138832A - Network video stream display method based on GPU hard layer - Google Patents

Network video stream display method based on GPU hard layer Download PDF

Info

Publication number
CN118138832A
CN118138832A CN202410543627.5A CN202410543627A CN118138832A CN 118138832 A CN118138832 A CN 118138832A CN 202410543627 A CN202410543627 A CN 202410543627A CN 118138832 A CN118138832 A CN 118138832A
Authority
CN
China
Prior art keywords
gpu
display
frame
decoding
hard layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202410543627.5A
Other languages
Chinese (zh)
Other versions
CN118138832B (en
Inventor
张坤
陈斌彬
胡艳明
翟一夫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Lingjiu Microelectronics Co ltd
Original Assignee
Wuhan Lingjiu Microelectronics Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Lingjiu Microelectronics Co ltd filed Critical Wuhan Lingjiu Microelectronics Co ltd
Priority to CN202410543627.5A priority Critical patent/CN118138832B/en
Publication of CN118138832A publication Critical patent/CN118138832A/en
Application granted granted Critical
Publication of CN118138832B publication Critical patent/CN118138832B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/4223—Cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440227—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by decomposing into layers, e.g. base layer and one or more enhancement layers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/64—Addressing
    • H04N21/6402—Address allocation for clients

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention provides a network video stream display method based on a GPU hard layer, wherein a RGBA format is directly generated by GPU decoding video without rendering by a two-dimensional and three-dimensional acceleration module; and the RGBA image is displayed by using the GPU hard layer, the generated RGBA data is not required to be copied to a display area for display, and the RGBA image corresponding to the cached frame can be displayed only by transmitting the address of the RGBA cached frame to the GPU hard layer. The GPU is used for directly decoding the video stream into the RGBA format, the hard layer displays the RGBA image by transmitting the frame buffer address instead of copying the video frame to the display area, and compared with a general mode, the method has the advantages of simpler flow and more suitability for application scenes of low delay and low power consumption of network video stream.

Description

Network video stream display method based on GPU hard layer
Technical Field
The invention relates to the field of video processing, in particular to a network video stream display method based on a GPU hard layer.
Background
The coming of the 5G era, the real-time display of high-definition large-resolution network video has a great deal of demands in a plurality of scenes, such as the fields of high-definition monitoring video transmission processing, automatic driving, video conference and the like, and in the scenes, the network video stream data volume is large, the real-time requirement is high, the video is required to be decoded and displayed as soon as possible, and meanwhile, the power consumption is expected to be low.
To address the above needs, there has been a trend towards using GPUs for network video processing. The GPU has strong multipath video online processing capability, has multiple functions of intelligent computation, graphic rendering and the like, and has more advantages in the face of intelligent scenes such as automatic driving, defense, robots and the like.
The current general flow for processing network video by using a GPU mainly comprises the following steps: firstly, a GPU decoding unit receives a network video stream, the GPU decodes the network video stream into YUV pixel data, then a two-dimensional graphic acceleration module or a three-dimensional graphic acceleration module of the GPU renders the YUV pixel data into RGBA data, and finally the generated RGBA data is transmitted to a display area for display. However, when the delay is as low as possible in the case of a scene where the video zoom function is not required, for example, a high-definition monitoring video, an intelligent driving image, a reverse image, etc., the delay is still relatively high, and the performance of the GPU is not fully exerted.
Disclosure of Invention
Aiming at the technical problems in the prior art, the invention provides a network video stream display method based on a GPU hard layer, which realizes low-delay decoding display of high-definition network video stream through the GPU, and meets the scene that delay power consumption is low and video scaling is not needed.
A network video stream display method based on GPU hard layer includes:
Step S1: the network camera encodes the image into a code stream, and transmits the code stream to the GPU decoding unit through a network protocol;
step S2: the GPU decoding unit analyzes the code stream, applies for a plurality of frame caches according to the video output resolution, and is used for storing RGBA pixel images decoded by the GPU decoding unit;
Step S3: the GPU decoding unit decodes the code stream into RGBA images, and sequentially stores RGBA image frames into a multi-block frame buffer of the application;
step S4: the display arbiter acquires the applied multi-block frame buffer address and video frame ID, establishes a corresponding relation between the frame buffer address and the video frame ID, opens a display channel, and informs the GPU of opening a hard layer;
Step S5: the GPU hard layer is started after acquiring the notification signal and is initially set to be in a transparent mode;
step S6: after the GPU decoding unit decodes the multi-frame, notifying a display arbiter that video output display is needed, and transmitting a display frame ID to the display arbiter according to the video display sequence;
Step S7: the display arbiter determines the frame buffer address of the RGBA image corresponding to the display frame ID according to the display frame ID and the corresponding relation between the frame buffer address and the video frame ID, and transmits the frame buffer address to the GPU hard layer, wherein the display frame ID is the same as the video frame ID;
step S8: the GPU hard layer is switched to a display mode, and corresponding RGBA images are displayed according to the frame buffer addresses transmitted by the display arbiter;
Step S9: after the display is finished, the GPU hard layer sends a display finishing signal to the GPU decoding unit, the GPU decoding unit continues decoding, and decoded RGBA image frames are stored in a frame buffer which is sent to display;
step S10: after the GPU decoding unit acquires the decoding stopping command, the frame buffer is released, the GPU hard layer is closed, and the decoding event is ended.
According to the network video stream display method based on the GPU hard layer, the RGBA format is directly generated by GPU decoding video, and rendering is not needed by a two-dimensional and three-dimensional acceleration module; and the RGBA image is displayed by using the GPU hard layer, the generated RGBA data is not required to be copied to a display area for display, and the RGBA image corresponding to the cached frame can be displayed only by transmitting the address of the RGBA cached frame to the GPU hard layer. The GPU is used for directly decoding the video stream into the RGBA format, the hard layer displays the RGBA image by transmitting the frame buffer address instead of copying the video frame to the display area, and compared with a general mode, the method has the advantages of simpler flow and more suitability for application scenes of low delay and low power consumption of network video stream.
Drawings
Fig. 1 is a flowchart of a method for displaying a network video stream based on a GPU hard layer according to the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. In addition, the technical features of each embodiment or the single embodiment provided by the invention can be combined with each other at will to form a feasible technical scheme, and the combination is not limited by the sequence of steps and/or the structural composition mode, but is necessarily based on the fact that a person of ordinary skill in the art can realize the combination, and when the technical scheme is contradictory or can not realize, the combination of the technical scheme is not considered to exist and is not within the protection scope of the invention claimed.
Based on the defects of the prior art, the invention provides a network video stream display method based on a GPU hard layer, which is used for meeting the demands of network video stream display on low delay and low power consumption. According to the method, a GPU two-dimensional or three-dimensional acceleration module is not needed to participate, the copying of RGBA data is reduced, the display effect is the same, the processing efficiency is more efficient, the time delay is greatly reduced compared with the existing scheme, meanwhile, the scheme can be completed by only needing a small number of GPU modules, the power consumption is reduced, and the display requirement of low delay and low power consumption can be better met.
Fig. 1 is a flowchart of a network video stream display method based on a GPU hard layer, where the method includes:
Step S1: the network camera encodes the image into a code stream and transmits the code stream to the GPU decoding unit through a network protocol.
It can be understood that the network camera captures video images in real time, encodes the images into a code stream, and sends the code stream to the GPU decoding unit.
Step S2: and the GPU decoding unit analyzes the code stream, applies for a plurality of frame caches according to the video output resolution, and is used for storing RGBA pixel images decoded by the GPU decoding unit.
It can be understood that, when the GPU decoding unit receives the code stream, it parses the code stream, and applies for a plurality of frame buffer areas, so as to store the RGBA pixel image decoded by the GPU decoding unit.
The process of the GPU decoding unit for analyzing the code stream comprises the following steps:
Step S201: the GPU decoding unit analyzes the frame header of the code stream to obtain the coding format, resolution, level and decoding configuration information of the video, and frame header data such as a Sequence parameter set (Sequence PARAMETER SET) and an image parameter set (Picture PARAMETER SET) of H264 and the like;
Step S202: the GPU decoding unit configures decoding settings, including decoding a corresponding encoding format, decoding output size, decoding video sequence and the like, and particularly configures decoding output to be in an RGBA form;
step S203: according to the video output resolution, a plurality of frame buffers are applied, and the frame buffers can ensure that enough reference frames are stored for decoding.
Step S3: the GPU decoding unit decodes the code stream into RGBA images, and sequentially stores RGBA image frames in the applied multi-block frame buffer.
The GPU decoding unit analyzes the code stream transmitted by the network camera, decodes the code stream into RGBA images, and stores the analyzed RGBA image frames in the applied multi-block frame buffer area.
The GPU decoding unit directly decodes the video into the RGBA format without rendering by a two-dimensional and three-dimensional acceleration module, so that the method can be completed by only a small number of modules of the GPU, and reduces the power consumption.
Step S4: the display arbiter acquires the applied multi-block frame buffer address and the video frame ID, establishes a corresponding relation between the frame buffer address and the video frame ID, opens a display channel, and informs the GPU of opening the hard layer.
It can be understood that, when the GPU decoding unit stores the resolved RGBA image frames in the applied multi-block frame buffer area, the corresponding relationship between the frame buffer address and the video frame ID stored therein is obtained, and then, the corresponding video frame ID can be found according to the frame buffer address, so as to find the corresponding video frame.
When the video frame is required to be displayed, the display arbiter opens a display channel and notifies the GPU that the hard layer is opened to prepare for displaying the video.
Step S5: the GPU hard layer is started after acquiring the notification signal and is initially set to be in a transparent mode.
Step S6: and after the GPU decoding unit decodes the multi-frames, notifying the display arbiter that video output display is needed, and transmitting the display frame ID to the display arbiter according to the video display sequence.
It can be understood that, after the GPU decoding unit decodes the multi-frame code stream, if display is required, the display sequence of the multi-frame video is set, and the ID of the display frame to be displayed is transmitted to the display arbiter, so as to inform the display arbiter of the ID of the display frame to be displayed.
Step S7: and the display arbiter determines the frame buffer address of the RGBA image corresponding to the display frame ID according to the display frame ID and the corresponding relation between the frame buffer address and the video frame ID, and transmits the frame buffer address to the GPU hard layer, wherein the display frame ID is the same as the video frame ID.
It can be understood that the display arbiter finds a corresponding frame buffer address according to the display frame ID transferred by the GPU decoding unit, and transfers the frame buffer address to the GPU hard layer, so that the GPU hard layer displays the video frame.
Step S8: the GPU hard layer is switched to a display mode, and corresponding RGBA images are displayed according to the frame buffer addresses transmitted by the display arbiter.
It can be understood that when the invention displays the video frame, RGBA images in the frame buffer corresponding to the frame buffer address can be displayed only according to the frame buffer address transmitted by the display arbiter. When the RGBA image is displayed by using the GPU hard layer, the generated RGBA data is not required to be copied to a display area for display, and the RGBA image corresponding to the cache frame can be displayed by only transmitting the address of the RGBA cache frame to the hard layer.
Step S9: after the display is finished, the GPU hard layer sends a display finishing signal to the GPU decoding unit, the GPU decoding unit continues decoding, and the decoded RGBA image frames are stored in a frame buffer which is sent for display.
Wherein, step S9 comprises the following sub-steps:
step S901: after the display is finished, the GPU hard layer sends a display finishing signal to the GPU decoding unit;
Step S902: the GPU decoding unit obtains the display end signal, continues decoding, waits if the display end signal is not received, and ensures that the frame buffer has a spare space, namely the frame buffer area which is already displayed has a spare area, and then decodes.
Step S903: and the GPU decoding unit stores the decoded RGBA image frames in an idle frame buffer area.
Step S10: after the GPU decoding unit acquires the decoding stopping command, the frame buffer is released, the GPU hard layer is closed, and the decoding event is ended.
Step S10: after the GPU decoding unit acquires the decoding stopping command, the frame buffer is released, the GPU hard layer is closed, and the decoding event is ended.
Wherein, step S10 comprises the following sub-steps:
Step S1001: the GPU decoding unit receives a decoding stop or end command, firstly completes decoding of the current frame, and then stops decoding;
step S1002: the GPU decoding unit resets the decoding initial value and clears the decoding state;
step S1003: and the GPU decoding unit releases the applied frame buffer, closes the GPU hard layer and ends the decoding event.
Compared with the prior art, the network video stream display method based on the GPU hard layer has the following advantages:
The GPU decodes the video to directly generate RGBA format without rendering by a two-dimensional and three-dimensional acceleration module; and the RGBA image is displayed by using the GPU hard layer, the generated RGBA data is not required to be copied to a display area for display, and the RGBA image corresponding to the cached frame can be displayed only by transmitting the address of the RGBA cached frame to the GPU hard layer. Compared with the general mode, the method has the advantages that the flow is simplified, and the method is more suitable for application scenes of low delay and low power consumption of network video streaming.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims (4)

1. The network video stream display method based on the GPU hard layer is characterized by comprising the following steps of:
Step S1: the network camera encodes the image into a code stream, and transmits the code stream to the GPU decoding unit through a network protocol;
step S2: the GPU decoding unit analyzes the code stream, applies for a plurality of frame caches according to the video output resolution, and is used for storing RGBA pixel images decoded by the GPU decoding unit;
Step S3: the GPU decoding unit decodes the code stream into RGBA images, and sequentially stores RGBA image frames into a multi-block frame buffer of the application;
step S4: the display arbiter acquires the applied multi-block frame buffer address and video frame ID, establishes a corresponding relation between the frame buffer address and the video frame ID, opens a display channel, and informs the GPU of opening a hard layer;
Step S5: the GPU hard layer is started after acquiring the notification signal and is initially set to be in a transparent mode;
step S6: after the GPU decoding unit decodes the multi-frame, notifying a display arbiter that video output display is needed, and transmitting a display frame ID to the display arbiter according to the video display sequence;
Step S7: the display arbiter determines the frame buffer address of the RGBA image corresponding to the display frame ID according to the display frame ID and the corresponding relation between the frame buffer address and the video frame ID, and transmits the frame buffer address to the GPU hard layer, wherein the display frame ID is the same as the video frame ID;
step S8: the GPU hard layer is switched to a display mode, and corresponding RGBA images are displayed according to the frame buffer addresses transmitted by the display arbiter;
Step S9: after the display is finished, the GPU hard layer sends a display finishing signal to the GPU decoding unit, the GPU decoding unit continues decoding, and decoded RGBA image frames are stored in a frame buffer which is sent to display;
step S10: after the GPU decoding unit acquires the decoding stopping command, the frame buffer is released, the GPU hard layer is closed, and the decoding event is ended.
2. The GPU-hard layer-based network video stream display method according to claim 1, wherein the step S2 comprises:
Step S201: the GPU decoding unit analyzes the frame header of the code stream to obtain the coding format, resolution, grade, decoding configuration information and frame header data of the video, wherein the frame header data comprises a sequence parameter set and an image parameter set;
step S202: the GPU decoding unit configures decoding parameters including decoding a corresponding encoding format, decoding output size and decoding video sequence, and configures decoding output into RGBA form;
Step S203: and applying for multi-block frame buffering according to the video output resolution.
3. The GPU-hard layer-based network video stream display method according to claim 1, wherein the step S9 comprises:
step S901: after the display is finished, the GPU hard layer sends a display finishing signal to the GPU decoding unit;
step S902: after the GPU decoding unit obtains the display end signal, continuing to decode the code stream, and waiting if the display end signal is not received, so as to ensure that a frame buffer is free, namely a frame buffer area of a display is free;
step S903: and the GPU decoding unit stores the decoded RGBA image frames in an idle frame buffer area.
4. The GPU-hard layer-based network video stream display method according to claim 1, wherein the step S10 comprises:
Step S1001: when the GPU decoding unit receives a decoding stop or end command, the decoding of the current frame is completed, and then the decoding is stopped;
step S1002: the GPU decoding unit resets the decoding initial value and clears the decoding state;
step S1003: and the GPU decoding unit releases the applied frame buffer, closes the GPU hard layer and ends the decoding event.
CN202410543627.5A 2024-05-06 2024-05-06 Network video stream display method based on GPU hard layer Active CN118138832B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410543627.5A CN118138832B (en) 2024-05-06 2024-05-06 Network video stream display method based on GPU hard layer

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410543627.5A CN118138832B (en) 2024-05-06 2024-05-06 Network video stream display method based on GPU hard layer

Publications (2)

Publication Number Publication Date
CN118138832A true CN118138832A (en) 2024-06-04
CN118138832B CN118138832B (en) 2024-07-26

Family

ID=91236121

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410543627.5A Active CN118138832B (en) 2024-05-06 2024-05-06 Network video stream display method based on GPU hard layer

Country Status (1)

Country Link
CN (1) CN118138832B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108366288A (en) * 2018-03-01 2018-08-03 厦门普杰信息科技有限公司 A kind of efficient decoding and playback method and system for HD video
US20190206023A1 (en) * 2017-12-28 2019-07-04 Nvidia Corporation Multi-gpu frame rendering
CN113015003A (en) * 2021-03-08 2021-06-22 展讯通信（上海）有限公司 Video frame caching method and device
CN113055681A (en) * 2021-03-02 2021-06-29 长沙景嘉微电子股份有限公司 Video decoding display method, device, electronic equipment and storage medium
WO2021158468A1 (en) * 2020-02-03 2021-08-12 Sony Interactive Entertainment Inc. System and method for efficient multi-gpu rendering of geometry by geometry analysis while rendering
WO2022160744A1 (en) * 2021-01-29 2022-08-04 稿定（厦门）科技有限公司 Gpu-based video synthesis system and method
WO2023016191A1 (en) * 2021-08-10 2023-02-16 Oppo广东移动通信有限公司 Image display method and apparatus, computer device, and storage medium
CN116737248A (en) * 2023-08-09 2023-09-12 武汉凌久微电子有限公司 Dormancy and awakening method of hard decoder
CN117499755A (en) * 2023-11-09 2024-02-02 武汉凌久微电子有限公司 Method for embedding watermark or subtitle into hard decoding video based on GPU hard layer

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190206023A1 (en) * 2017-12-28 2019-07-04 Nvidia Corporation Multi-gpu frame rendering
CN108366288A (en) * 2018-03-01 2018-08-03 厦门普杰信息科技有限公司 A kind of efficient decoding and playback method and system for HD video
WO2021158468A1 (en) * 2020-02-03 2021-08-12 Sony Interactive Entertainment Inc. System and method for efficient multi-gpu rendering of geometry by geometry analysis while rendering
WO2022160744A1 (en) * 2021-01-29 2022-08-04 稿定（厦门）科技有限公司 Gpu-based video synthesis system and method
CN113055681A (en) * 2021-03-02 2021-06-29 长沙景嘉微电子股份有限公司 Video decoding display method, device, electronic equipment and storage medium
CN113015003A (en) * 2021-03-08 2021-06-22 展讯通信（上海）有限公司 Video frame caching method and device
WO2023016191A1 (en) * 2021-08-10 2023-02-16 Oppo广东移动通信有限公司 Image display method and apparatus, computer device, and storage medium
CN116737248A (en) * 2023-08-09 2023-09-12 武汉凌久微电子有限公司 Dormancy and awakening method of hard decoder
CN117499755A (en) * 2023-11-09 2024-02-02 武汉凌久微电子有限公司 Method for embedding watermark or subtitle into hard decoding video based on GPU hard layer

Also Published As

Publication number Publication date
CN118138832B (en) 2024-07-26

Similar Documents

Publication Publication Date Title
CN111357289B (en) Method, apparatus and system for a game engine for video encoder rendering
CN109600666B (en) Video playing method, device, medium and electronic equipment in game scene
WO2017161907A1 (en) Video coding method and apparatus, device, and storage medium
US20180341323A1 (en) Methods and apparatuses for handling virtual reality content
KR102173107B1 (en) Methed, device and system for processing image based on cloud server
CN108063976B (en) Video processing method and device
CN109168014A (en) A kind of live broadcasting method, device, equipment and storage medium
CN107426491B (en) Implementation method of 360-degree panoramic video
RU2733218C2 (en) Method, apparatus and a stream for formatting an immersive video image for traditional and immersive playback devices
US20220141548A1 (en) Streaming Volumetric and Non-Volumetric Video
WO2017185761A1 (en) Method and device for playing back 2d video
US11924442B2 (en) Generating and displaying a video stream by omitting or replacing an occluded part
CN112511896A (en) Video rendering method and device
WO2023040825A1 (en) Media information transmission method, computing device and storage medium
CN110049347B (en) Method, system, terminal and device for configuring images on live interface
CN104980798B (en) A kind of far-end video display methods and device
CN107580228B (en) Monitoring video processing method, device and equipment
CN118138832B (en) Network video stream display method based on GPU hard layer
CN117499755A (en) Method for embedding watermark or subtitle into hard decoding video based on GPU hard layer
US20230025664A1 (en) Data processing method and apparatus for immersive media, and computer-readable storage medium
US20230054523A1 (en) Enhancing 360-degree video using convolutional neural network (cnn)-based filter
CN110798715A (en) Video playing method and system based on image string
WO2024183387A1 (en) Image coding method and apparatus, image decoding method and apparatus, and system
JP2001128194A (en) Stereoscopic video processor, method for processing stereoscopic video and storing medium
CN115665461B (en) Video recording method and virtual reality device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant