WO2017133481A1 - 渲染方法、解码方法、播放多媒体数据流的方法及装置 - Google Patents

渲染方法、解码方法、播放多媒体数据流的方法及装置 Download PDF

Info

Publication number
WO2017133481A1
WO2017133481A1 PCT/CN2017/071819 CN2017071819W WO2017133481A1 WO 2017133481 A1 WO2017133481 A1 WO 2017133481A1 CN 2017071819 W CN2017071819 W CN 2017071819W WO 2017133481 A1 WO2017133481 A1 WO 2017133481A1
Authority
WO
WIPO (PCT)
Prior art keywords
multimedia data
data stream
frame
image
parsed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/071819
Other languages
English (en)
French (fr)
Inventor
李江卫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Publication of WO2017133481A1 publication Critical patent/WO2017133481A1/zh
Priority to US16/047,431 priority Critical patent/US10805570B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/542—Event management; Broadcasting; Multicasting; Notifications
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/764—Media network packet handling at the destination 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
    • H04N21/4438—Window management, e.g. event handling following interaction with the user interface
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
    • H04N5/445—Receiver circuitry for the reception of television signals according to analogue transmission standards for displaying additional information
    • H04N5/45—Picture in picture, e.g. displaying simultaneously another television channel in a region of the screen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder

Definitions

  • a playback technology involving a multimedia data stream in particular, a rendering method, a decoding method of a multimedia data stream, a method of playing a multimedia data stream, a rendering device, a decoding device of a multimedia data stream, a device for playing a multimedia data stream, and a multimedia data stream device.
  • a multimedia data playback service when playing a multimedia data stream on an interface, a multimedia data playback service is required to play a multimedia data stream, and an area is displayed on the interface as a display area of the multimedia data stream.
  • One solution for achieving this effect is to draw a transparent area on the drawing layer of the drawing interface when rendering the interface, and then draw a play window in another drawing layer to overlap the position of the playing window in the transparent area.
  • the multimedia data to be played for each frame is displayed in the play window.
  • the present application provides a rendering method, a decoding method of a multimedia data stream, a method for playing a multimedia data stream, a rendering device, a decoding device for a multimedia data stream, a device and a device for playing a multimedia data stream, and can solve the multimedia data through the drawing.
  • a rendering device When the transparent area starts to play, it is easy to produce a "flower screen" phenomenon.
  • a rendering method comprising the steps of:
  • the transparent area on the interface is drawn, and the playback window of the multimedia data stream and the at least one frame image of the multimedia data stream are instructed to be rendered, the transparent The area and the play window are overlaid on different drawing layers.
  • a method for decoding a multimedia data stream includes:
  • a notification message is sent, the notification message being used to notify that at least one frame of the multimedia data stream has been parsed.
  • a method for playing a multimedia data stream includes:
  • At least one frame of the rendered multimedia data stream is displayed in the play window.
  • a rendering apparatus including:
  • a detecting module configured to determine whether at least one frame of the multimedia data stream has been parsed, and notify the drawing module of the determination result
  • a drawing module configured to draw a transparent area on the interface when at least one frame of the multimedia data stream has been parsed
  • a notification module configured to indicate, when the at least one frame image of the multimedia data stream has been parsed, to perform a drawing of the play window of the multimedia data stream and at least one frame image of the multimedia data stream, the transparent area and the playing The windows overlap in different drawing layers.
  • a decoding apparatus for a multimedia data stream including:
  • a decoding module configured to parse the multimedia data stream into a frame image
  • the notification module is configured to send, after parsing the at least one frame of the multimedia data stream, a notification message, where the notification message is used to notify that at least one frame of the multimedia data stream has been parsed.
  • an apparatus for playing a multimedia data stream includes:
  • a decoding device configured to send a notification message after parsing at least one frame of the multimedia data stream
  • a rendering device configured to: when the at least one frame image of the multimedia data stream has been parsed according to the notification message, draw a transparent area on the interface, and notify the multimedia data playing device;
  • a multimedia data playing device configured to receive a notification of the rendering device, and then draw a play window of the multimedia data stream and at least one frame image of the multimedia data stream, where the transparent area and the play window overlap with different drawings Floor;
  • a display device configured to display at least one frame image of the drawn multimedia data stream in the play window.
  • an apparatus for playing a multimedia data stream includes:
  • a memory for storing processor executable instructions
  • processor is configured to:
  • At least one frame of the rendered multimedia data stream is displayed in the play window.
  • the present application expands the function of the decoding service on the one hand, that is, adds at least one frame of image parsing reminding function, so that the decoding service can be used to solve the "flower screen” phenomenon; on the other hand, the rendering service controls the drawing of the transparent area and The timing of playing the window, that is, after the image of at least one frame of the multimedia data stream has been parsed, the transparent area is started, which means that at least the first frame image has been parsed when the transparent area is drawn, thereby solving the multimedia data.
  • the "flower screen” phenomenon when streaming the first frame.
  • FIG. 1 is a schematic structural diagram of a device 100 according to an embodiment of the present application.
  • FIG. 2 is a flowchart of a rendering method in an embodiment of the present application
  • FIGS. 2a and 2b are schematic diagrams showing two processes for determining that at least one frame of image has been parsed in the embodiment of the present application;
  • FIG. 3 is a flowchart of a method for playing multimedia data in an embodiment of the present application.
  • 4a-4c are schematic diagrams of several interfaces to which the solution of the present application is applied;
  • Figure 5 is a flow chart of an application example of the present application.
  • FIG. 6 is a hardware architecture diagram of a device/rendering device/multimedia data stream decoding device for playing a multimedia data stream in an embodiment of the present application
  • FIG. 7 is a logic block diagram of a rendering apparatus in an embodiment of the present application.
  • FIG. 8 is a logic block diagram of a decoding apparatus for a multimedia data stream in an embodiment of the present application.
  • FIG. 9 is a logic block diagram of an apparatus for playing a multimedia data stream in an embodiment of the present application.
  • first, second, third, etc. may be used to describe various information in this application, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as the second information without departing from the scope of the present application.
  • second information may also be referred to as the first information.
  • word "if” as used herein may be interpreted as "when” or “when” or “in response to a determination.”
  • the device 100 has a central processing unit (CPU) 101, a graphics processing unit (GPU) 102, and the like, and an operating system, an internal or upper layer application of the operating system (for example, multimedia data playback).
  • Software such as client, browser, mobile terminal desktop application, etc.).
  • Device 100 can be a handheld device with wireless communication capabilities, an in-vehicle device, a wearable device, a television device, a computing device, or other processing device connected to a wireless modem, and the like.
  • the GPU can provide the rendering function of the interface elements, and can realize the layout of the interface elements on the interface through rendering.
  • the effect of the interface rendered by the browser while running, the desktop effect rendered by the operating system runtime, and the effect of the application client rendering a display area on the operating system desktop are the rendering results of the data to be displayed by the GPU.
  • the CPU 101 is connected to the GPU 102.
  • the operating system runs on the CPU, and the application sends the data to be rendered to the GPU through an interface provided by the operating system; the GPU renders the data to be rendered, and stores the rendered result, for example, can be stored in the display memory of the GPU ( VRAM, Video RAM).
  • the rendering function of the GPU can be implemented by calling the GPU 102 through a rendering service.
  • the rendering service may be a module provided by the application itself.
  • the rendering service may also be a module provided by an operating system or a third party in the form of a plug-in or the like.
  • the rendering service can generate a drawing layer in VRAM, the drawing layer is the drawing medium, and the interface elements are drawn on the drawing layer.
  • the drawing result of the drawing layer is obtained by the display service providing the display function in the operating system for presentation.
  • One solution for playing a multimedia data stream on the interface is to set a transparent area in one drawing layer when rendering the interface elements of the interface, and to set a multimedia data playing window in another drawing layer through the multimedia data playing service, and through the multimedia The data service draws each frame of multimedia data to be played in the playback window position.
  • the position of the play window and the position of the transparent area may overlap (in this application, overlap means that the two are completely coincident or substantially coincident).
  • the interface referred to in the present application is not limited to an interface of an application, a desktop of an operating system, or the like.
  • the multimedia data may be video data, 3D animation, game data, and the like.
  • the application can control the playing time of the multimedia data stream and the timing of drawing the transparent area and the playing window, thereby avoiding the phenomenon of “flowering screen”.
  • FIG. 2 is a flow chart of a rendering method in an embodiment of the present application.
  • the flow of Figure 2 can be performed by a rendering service.
  • the multimedia data stream needs to be parsed into image frames of one frame and one frame that can be displayed before being displayed.
  • the decoding service that implements this process may be a decoder located in the multimedia data playback service, or may be located in an operating system or implemented by a third party service.
  • At least one frame of the image in the S201 stage may refer to the first frame image; in other scenarios, at least one frame of the image may include multiple frames of the parsed image, for example, three frames, five frames, and the like.
  • the number of frames included in at least one frame of the image may vary depending on the particular application.
  • the first frame image in the present application refers to the first frame image to be played in the play window, and the first frame image may be the first frame image frame of the multimedia data stream; however, it is not excluded that the user needs to be prioritized under different playback requirements.
  • other image frames are used as the first frame image to be played.
  • the timing of drawing the transparent area and the playing window in the present application needs to ensure that the first frame image has been parsed first, and the prior art does not consider whether the first frame data has been decoded, and the transparent area is calculated in the rendering service. After the size, the transparent area is set.
  • the rendering service may instruct the decoding service to draw the playing window after drawing the transparent area; or may notify the decoding service to draw the playing window after knowing that at least one frame of image has been parsed.
  • the decoding service can start drawing the playback window immediately; it can also start drawing the playback window when other conditions are met. Therefore, it is easy to understand that drawing a transparent window and drawing a playback window can start at the same time, or there may be a sequence.
  • the alert service may be provided by the decoding service, and after the decoding service parses at least one frame of the multimedia data stream, the notification message is sent, and the rendering service listens to the notification message through a thread (herein referred to as a second thread). .
  • a preset identifier is set in the rendering service to indicate whether the state of at least one frame of the multimedia data stream has been parsed, and the rendering service determines, by another thread (herein referred to as the first thread), whether at least the multimedia data stream has been parsed.
  • One frame of image is a thread that indicates whether the state of at least one frame of the multimedia data stream has been parsed.
  • the state of the preset identifier is an initial state, indicating that at least the multimedia data stream has not been parsed
  • a frame image when the notification message is monitored, modifies the state of the preset identifier, and the modified state indicates that at least one frame of the multimedia data stream has been parsed. For example, "1" may indicate that the decoding service has parsed at least one frame of image, and "0" indicates that the decoding service has not parsed at least one frame of image. Or use "True” and "False” to distinguish different states.
  • Figure 2b is another way of knowing if at least one frame of the multimedia data stream has been parsed.
  • the rendering service actively sends a request to the decoding service, and the decoding service returns a request response to the rendering service according to whether the image has been parsed by at least one frame.
  • the rendering service can determine whether the multimedia data stream has been parsed according to the request response. At least one frame of image.
  • the playback window of the transparent area and the multimedia data stream needs to be notified at the S202 stage.
  • the drawing process is not limited.
  • the GPU process (the GPU process is a process initiated by the operating system when the GPU is called) and the multimedia data may be separately notified.
  • the playback service, the GPU process notifies the drawing module of the rendering service to draw a transparent area; and the multimedia data playing service notifies the corresponding drawing module to draw the playback window of the multimedia data stream and at least one frame of the multimedia data stream.
  • the drawing module of the rendering service may be notified to draw an image of the predetermined content on the drawn interface, and the content may be designed according to the design.
  • the needs of the person for example, posters related to the video data to be played, or advertisement information, etc.
  • the present application improves the decoding device, and provides a reminder service, and the decoding device is responsible for parsing the multimedia data stream into an image of one frame, and parsing at least one frame of the multimedia data stream. After the image, a notification message may be sent through the extension interface to notify the rendering service that at least one frame of the multimedia data stream has been parsed.
  • FIG. 3 is a process of playing multimedia stream data according to the interface rendered in FIG. 1. This process involves the decoding of the multimedia data stream, the rendering of the data to be displayed, and the display of the rendering results in several stages.
  • the S301 phase can occur during the decoding phase of the multimedia data stream and can be performed by the decoding service.
  • Stage S302 The rendering phase of the data that belongs to the display.
  • the rendering service may determine whether at least one frame of the multimedia data stream has been parsed, and the transparent area on the interface is drawn according to the determination result. The process may refer to the manner described in FIG. 2 .
  • the rendering service notifies the multimedia playback service to draw a play window and at least one frame of image.
  • the display service may be notified to present the rendering result, and the display service is notified to acquire at least one frame image of the parsed multimedia data stream to be played at the position of the playing window.
  • the drawing module of the rendering service may be notified to draw an image of the predetermined content on the drawn interface, and the content may be designed according to the design.
  • the needs of the person for example, posters related to the video data to be played, or advertisement information, etc.
  • the notification display service displays the rendered interface, and at this time, the image of the predetermined content is presented on the interface instead of the multimedia data stream. In this way, the user is seen to see an image of the predetermined content before displaying the multimedia data stream instead of displaying a black screen or a white screen to enhance the user experience.
  • Figure 4a depicts a scenario in which the browser's interface plays a multimedia data stream.
  • the transparent area in the browser interface is drawn by the browser's rendering service when the browser's interface elements are drawn; the playback window is drawn by the multimedia data playback service on another drawing layer. Since the transparent area and the play window overlap, the user sees the rendering effect of the play window.
  • Other areas of the browser's interface display other interface elements (text blocks, picture blocks, etc.).
  • Figure 5 is a flow chart of an application example of the present application. Take the video data as an example.
  • the transparent window is started as an example after the first frame image is parsed.
  • the decoding service decodes the video data by acquiring a video data packet from the video source (S501a), and then parsing the video data packet into an image frame (S501b). After the first frame image frame is parsed, the rendering service is notified through the extension interface of the multimedia playback service (S501c). To send a notification message, follow these commands:
  • the rendering service renders the interface.
  • the specific process is: after receiving the service data of the interface (S502a), if the received service data includes a video control, check whether the status of the preset identifier is an initial state (S502b), and if it is an initial state, consider If the first frame image frame has not been parsed yet, the drawing module is notified to draw a poster at the position where the video is played (S502c); if it is not the initial state, it is considered that the first frame image frame has been parsed, and then the drawing module is notified to draw a transparent area, and notify The multimedia playback service draws a play window and a first frame image (S502d). After the poster or the transparent area is drawn, the multimedia broadcast service notification display service displays the drawing result (S502e).
  • the display service displays the rendered results and the image frames of the video. After the rendering service ends, the rendering service will notify the display service to display the rendering result.
  • the multimedia playing service draws the playing window, it also notifies the display service to display the playing window, and notifies the display service to obtain the parsed first frame image frame to be played.
  • the window plays.
  • the display service acquires the interface rendering result including the poster to display; after the first frame image frame has been parsed, the display service acquires the interface rendering result including the transparent window, and superimposes the playing window to display, When displayed, each frame of image is displayed at the position of the playback window.
  • the present application also provides an embodiment of a rendering device, a device for playing a multimedia data stream, and a decoding device for a multimedia data stream.
  • Embodiments of the apparatus provided by the present application can be applied to the apparatus 100 of FIG. 1, and the kind of the apparatus 100 can be referred to the above description.
  • the device embodiment may be implemented by software, or may be implemented by hardware or a combination of hardware and software. Taking the software implementation as an example, as a logical means, the processor of the device 100 is configured to read the corresponding executable computer program instructions in the non-volatile memory into the memory. From the hardware level, as shown in FIG. 6, a hardware structure diagram of the device 100 for playing the multimedia data device or the device for rendering the interface or the decoding device for the multimedia data, except for the processor shown in FIG.
  • the device 100 in which the device is located in the embodiment may also include other hardware according to the actual functions of the smart device, such as a GPU and the like.
  • the device 100 when the device 100 (the device 100 can be referred to as a device that plays a multimedia data stream in this embodiment) is used as a rendering device, a device for playing a multimedia data stream, and a bearer entity for a decoding device of a multimedia data stream, the device The processor of 100 may be configured to: after parsing at least one frame of the multimedia data stream, sending a notification message; when determining that at least one frame of the multimedia data stream has been parsed according to the notification message, drawing a transparent area on the interface, and a play window and the multimedia number for drawing the multimedia data stream The transparent area and the play window are superimposed on different drawing layers according to at least one frame of the stream; at least one frame of the rendered multimedia data stream is displayed in the play window.
  • the rendering apparatus 700 includes:
  • the detecting module 701 is configured to determine whether at least one frame image of the multimedia data stream has been parsed, and notify the drawing module of the determination result;
  • the drawing module 702 is configured to draw a transparent area on the interface when at least one frame of the multimedia data stream has been parsed;
  • the notification module 703 is configured to: when the at least one frame image of the multimedia data stream has been parsed, indicate that the playback window of the multimedia data stream and the at least one frame image of the multimedia data stream are drawn, and the transparent area and the play window are overlapped by different Drawing layer.
  • At least one frame of image may include a first frame of image.
  • the detecting module 701 determines, according to the state of the preset identifier, whether at least one frame of the multimedia data stream has been parsed; the device further includes:
  • the monitoring module (not shown) is configured to modify the state of the preset identifier after the notification message is monitored; the notification message is sent after parsing at least one frame of the multimedia data stream.
  • the rendering module 702 is further configured to draw an image of the predetermined content at the interface when the at least one frame image of the multimedia data stream is not parsed.
  • the decoding device 800 of the multimedia data stream in FIG. 8 includes:
  • a decoding module 801 configured to parse the multimedia data stream into a frame image
  • the notification module 802 is configured to: after parsing at least one frame of the multimedia data stream, send a notification message, where the notification message is used to notify that at least one frame of the multimedia data stream has been parsed.
  • At least one frame of image may include a first frame of image.
  • the decoding device 800 is located in a multimedia data playback service.
  • the apparatus 900 for playing a multimedia data stream includes:
  • the decoding device 901 is configured to: after parsing at least one frame of the multimedia data stream, send a notification message;
  • the rendering device 902 is configured to: when the at least one frame image of the multimedia data stream has been parsed according to the notification message, draw a transparent area on the interface, and notify the multimedia data playing device;
  • the multimedia data playing device 903 is configured to: after receiving the notification of the rendering device, draw a play window of the multimedia data stream and at least one frame image of the multimedia data stream, where the transparent area and the play window overlap with different drawing layers;
  • the display device 904 is configured to display at least one frame image of the drawn multimedia data stream in the play window.
  • At least one frame of image may include a first frame of image.
  • the rendering device 902 determines that the at least one frame image of the multimedia data stream has been parsed according to the notification message, and may include:
  • the first thread is started to determine that at least one frame of the multimedia data stream has been parsed according to the state of the preset identifier.
  • the decoding device 901 can transmit the notification message through an extension interface.
  • Rendering device 902 can also be used to:
  • the rendering device 902 can be further configured to notify the display device to display an image of the predetermined content on the interface when the at least one frame image of the multimedia data stream is not parsed.
  • the device embodiment since it basically corresponds to the method embodiment, reference may be made to the partial description of the method embodiment.
  • the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the objectives of the present application. Those of ordinary skill in the art can understand and implement without any creative effort.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请公开了渲染方法、解码方法、播放多媒体数据流的方法及装置、设备。渲染方法包括：如果判断已解析完多媒体数据流的至少一帧图像，则绘制界面上的透明区域，并指示进行所述多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像的绘制，所述透明区域和所述播放窗口重叠于不同的绘图层。本申请能够解决多媒体数据透过绘制好的透明区域开始播放时容易产生的"花屏"现象。

Description

渲染方法、解码方法、播放多媒体数据流的方法及装置
本申请要求2016年02月01日递交的申请号为201610069362.5、发明名称为“渲染方法、解码方法、播放多媒体数据流的方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
申请涉及多媒体数据流的播放技术，尤其涉及渲染方法、多媒体数据流的解码方法、播放多媒体数据流的方法、渲染装置、多媒体数据流的解码装置、播放多媒体数据流的装置、播放多媒体数据流的设备。
背景技术
目前某些场景下，在界面上播放多媒体数据流时，需要借助多媒体数据播放服务来播放多媒体数据流，在界面上有一个区域作为多媒体数据流的显示区域。目前实现该效果的一种方案是在对界面渲染时，在绘制界面的绘图层绘制一个透明区域，另外，再在另一个绘图层绘制一个播放窗口，将播放窗口的位置重叠于透明区域的位置，播放窗口内显示每一帧待播放的多媒体数据。
现有技术中多媒体数据透过绘制好的透明区域开始播放时容易产生“花屏”现象。
发明内容
本申请提供一种渲染方法、多媒体数据流的解码方法、播放多媒体数据流的方法、渲染装置、多媒体数据流的解码装置、播放多媒体数据流的装置及设备，能够解决多媒体数据透过绘制好的透明区域开始播放时容易产生“花屏”现象。
根据本申请实施例的第一方面，提供一种渲染方法，该方法包括步骤：
如果判断已解析完多媒体数据流的至少一帧图像，则绘制界面上的透明区域，并指示进行所述多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像的绘制，所述透明区域和所述播放窗口重叠于不同的绘图层。
根据本申请实施例的第二方面，提供一种多媒体数据流的解码方法，包括：
将多媒体数据流解析成帧图像；
当解析完多媒体数据流的至少一帧图像后，发送通知消息，所述通知消息用于通知已解析完多媒体数据流的至少一帧图像。
根据本申请实施例的第三方面，提供一种播放多媒体数据流的方法，包括：
当解析完多媒体数据流的至少一帧图像后，发送通知消息；
当根据所述通知消息判断已解析完多媒体数据流的至少一帧图像时，绘制界面上的透明区域，以及绘制所述多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像，所述透明区域和所述播放窗口重叠于不同的绘图层；
在所述播放窗口显示所绘制的多媒体数据流的至少一帧图像。
根据本申请实施例的第四方面，提供一种渲染装置，包括：
检测模块，用于判断是否已解析完多媒体数据流的至少一帧图像，并将判断结果通知绘制模块；
绘制模块，用于在已解析完多媒体数据流的至少一帧图像时绘制界面上的透明区域；
通知模块，用于在已解析完多媒体数据流的至少一帧图像时指示进行所述多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像的绘制，所述透明区域和所述播放窗口重叠于不同的绘图层。
根据本申请实施例的第五方面，提供一种多媒体数据流的解码装置，包括：
解码模块，用于将多媒体数据流解析成帧图像；
通知模块，用于当解析完多媒体数据流的至少一帧图像后，发送通知消息，所述通知消息用于通知已解析完多媒体数据流的至少一帧图像。
根据本申请实施例的第六方面，提供一种播放多媒体数据流的装置，包括：
解码装置，用于当解析完多媒体数据流的至少一帧图像后，发送通知消息；
渲染装置，用于当根据所述通知消息判断已解析完多媒体数据流的至少一帧图像时，绘制界面上的透明区域，并通知多媒体数据播放装置；
多媒体数据播放装置，用于接收所述渲染装置的通知后绘制所述多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像，所述透明区域和所述播放窗口重叠于不同的绘图层；
显示装置，用于在所述播放窗口显示所绘制的多媒体数据流的至少一帧图像。
根据本申请实施例的第七方面，提供一种播放多媒体数据流的设备，包括：
处理器；
用于存储处理器可执行指令的存储器；
其中，所述处理器被配置为：
当解析完多媒体数据流的至少一帧图像后，发送通知消息；
当根据所述通知消息判断已解析完多媒体数据流的至少一帧图像时，绘制界面上的透明区域，以及绘制所述多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像，所述透明区域和所述播放窗口重叠于不同的绘图层；
在所述播放窗口显示所绘制的多媒体数据流的至少一帧图像。
本申请一方面扩展了解码服务的功能，即增加了至少一帧图像解析后的提醒功能，以使解码服务可以被用于解决“花屏”现象；另一方面，渲染服务通过控制绘制透明区域和播放窗口的时机，即在保证多媒体数据流的至少一帧图像已被解析完后再开始绘制透明区域，也就意味着在绘制透明区域时至少首帧图像已经被解析完，从而解决了多媒体数据流首帧播放时的“花屏”现象。
附图说明
图1为本申请实施例中设备100的结构示意图；
图2为本申请实施例中渲染方法的流程图；
图2a、图2b为本申请实施例中两种判断至少一帧图像已被解析的流程示意图；
图3为本申请实施例中播放多媒体数据的方法的流程图；
图4a-图4c为几种应用本申请方案的界面的示意图；
图5为本申请应用实例的流程图；
图6为本申请实施例中播放多媒体数据流的装置/渲染装置/多媒体数据流的解码装置的硬件架构图；
图7为本申请实施例中渲染装置的逻辑框图；
图8为本申请实施例中多媒体数据流的解码装置的逻辑框图；
图9为本申请实施例中播放多媒体数据流的装置的逻辑框图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。
在本申请使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本申请。在本申请和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包 括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本申请可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本申请范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
图1中，设备100具有中央处理器(Central Processing Unit，CPU)101、图形处理器(Graphic Processing Unit，GPU)102等硬件设备，以及操作系统、操作系统内部或上层的应用(例如多媒体数据播放客户端、浏览器、移动终端桌面应用等)等软件。
设备100可以是具有无线通信功能的手持设备、车载设备、可穿戴设备、电视设备、计算设备或连接到无线调制解调器的其它处理设备等等。
GPU可以提供界面元素的渲染功能，通过渲染可以实现对界面上的界面元素进行排版。例如，浏览器运行时所呈现的界面效果、操作系统运行时所呈现的桌面效果、应用客户端呈现在操作系统桌面上的某个显示区域的效果均是GPU对待显示的数据的渲染结果。在硬件层面，CPU 101与GPU 102连接。操作系统运行在CPU上，应用通过操作系统提供的接口将需要渲染的数据发给GPU；GPU对需要渲染的数据进行渲染，并将渲染后的结果存储，例如，可以存储在GPU的显示存储器(VRAM，Video RAM)中。
本申请中，可以通过渲染服务调用GPU 102来实现GPU的渲染功能。在某些例子中，渲染服务可以是属于应用本身所提供的模块，在某些例子中，渲染服务也可以是操作系统或第三方以插件等形式所提供的模块。在渲染时，渲染服务可以在VRAM中生成绘图层，绘制层是绘图的介质，在绘图层上绘制界面元素。通过操作系统中提供显示功能的显示服务获取绘图层的绘制结果进行呈现。
在界面上播放多媒体数据流的一种方案是，在渲染界面的界面元素时，在一个绘图层设置透明区域，可以通过多媒体数据播放服务在另一个绘图层设置多媒体数据的播放窗口，并通过多媒体数据服务将即将播放的每帧多媒体数据绘制在播放窗口位置。在一个例子中，播放窗口的位置与透明区域的位置可以重叠(本申请中，重叠是指二者完全重合或基本重合)。
本申请中所指的界面不局限于应用的界面、操作系统的桌面等。多媒体数据可以是视频数据、3D动画、游戏数据等。
本申请可以控制多媒体数据流的播放时机和绘制透明区域、播放窗口的时机，可以避免“花屏”现象。
图2为本申请一个实施例中渲染方法的流程图。
S201，判断是否已解析完多媒体数据流的至少一帧图像；
S202，如果已解析完多媒体数据流的至少一帧图像，则绘制界面上的透明区域，并指示进行多媒体数据流的播放窗口和多媒体数据流的至少一帧图像的绘制。
作为例子，图2的流程可以由渲染服务执行。
多媒体数据流在被显示前需要被解析成可以显示的一帧一帧的图像帧。实现此过程的解码服务可以是位于多媒体数据播放服务中的解码器，也可以位于操作系统中，或通过第三方服务实现。
在某些场景下，S201阶段中至少一帧图像是可以指首帧图像；在另一些场景下，至少一帧图像可以包括多帧已解析的图像，例如，可以是三帧、五帧等数量的图像，至少一帧图像中所包含的帧数可以根据具体的应用场合有所不同。本申请中首帧图像是指即将在播放窗口中播放的第一帧图像，该第一帧图像可以是多媒体数据流的首帧图像帧；但并不排除在用户的不同播放需求下，需要优先播放其他非第一帧图像帧时，将其他图像帧作为即将播放的第一帧图像的可能。
与现有技术不同，本申请中绘制透明区域和播放窗口的时机是需要先确保已经解析完首帧图像，而现有技术并不考虑是否已经解码好首帧数据，在渲染服务计算好透明区域大小后就开始设置透明区域。
当已经解析完至少一帧图像后，渲染服务可以在绘制完透明区域后再指示解码服务绘制播放窗口；也可以在获知已解析完至少一帧图像后便通知解码服务绘制播放窗口。解码服务在收到指示后，可以立刻开始绘制播放窗口；也可以在满足其他条件时开始绘制播放窗口。因此，容易理解，绘制透明窗口与绘制播放窗口可以在同一时间开始，也可以存在先后顺序。
获知是否已经解析完多媒体数据流的至少一帧图像的途径可以存在多种。例如，图2a所示，可以通过解码服务提供提醒服务，在解码服务解析完多媒体数据流的至少一帧图像后发送通知消息，渲染服务通过一个线程(本文称为第二线程)监听这个通知消息。另外，在渲染服务中设置预设标识来表示是否已解析完多媒体数据流的至少一帧图像的状态，渲染服务通过另一个线程(本文称第一线程)判断是否已解析完多媒体数据流的至少一帧图像。当预设标识的状态为初始状态时，表明尚未解析完多媒体数据流的至少 一帧图像，当监听到通知消息时，修改该预设标识的状态，修改后的状态表示已解析完多媒体数据流的至少一帧图像。举例来说，可以将“1”表示解码服务已解析完至少一帧图像，“0”表示解码服务未解析完至少一帧图像。再或者用“True”“False”等值来区分不同的状态等。
图2b是另一种获知是否已经解析完多媒体数据流的至少一帧图像的方式。如图所示，渲染服务主动向解码服务发送请求，解码服务根据自己是否已经解析完至少一帧图像来给渲染服务返回请求响应，渲染服务可以根据请求响应来判断是否已解析完多媒体数据流的至少一帧图像。
如果在S201阶段的判断结果为已解析完多媒体数据流的至少一帧图像，则在S202阶段需要通知绘制透明区域和多媒体数据流的播放窗口。绘制过程本申请不作限制，例如，可以在渲染服务获知已解析完多媒体数据流的至少一帧图像后，分别通知GPU进程(GPU进程是当GPU被调用时在操作系统启动的进程)和多媒体数据播放服务，GPU进程通知渲染服务的绘图模块绘制透明区域；而多媒体数据播放服务通知相应的绘图模块绘制多媒体数据流的播放窗口和多媒体数据流的至少一帧图像。
如果在S201阶段的判断结果为未解析完多媒体数据流的至少一帧图像，则在执行S202阶段之前，可以通知渲染服务的绘图模块在所绘制的界面上绘制预定内容的图像，内容可以根据设计者的需要制定，例如，与将要播放的视频数据相关的海报，或广告信息等。
为了配合渲染引擎的绘制过程，作为例子，本申请对解码装置做出了改进，提供了提醒服务，解码装置负责将多媒体数据流解析成一帧帧的图像，当解析完多媒体数据流的至少一帧图像后，可通过扩展接口发送通知消息，通知渲染服务已解析完多媒体数据流的至少一帧图像。
图3为根据图1所渲染的界面播放多媒体流数据的过程。此过程涉及多媒体数据流的解码、待显示的数据渲染、显示渲染结果几个阶段。
S301，当解析完多媒体数据流的至少一帧图像后，发送通知消息；
S302，当根据通知消息判断已解析完多媒体数据流的至少一帧图像时，绘制界面上的透明区域，以及多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像，透明区域和播放窗口重叠于不同的绘图层；
S303，在播放窗口显示所绘制的多媒体数据流的至少一帧图像。
S301阶段可以发生于多媒体数据流的解码阶段，可以由解码服务执行。S302阶段可 以属于对待显示的数据渲染阶段。可以通过渲染服务判断是否已解析完多媒体数据流的至少一帧图像，并根据判断结果绘制界面上的透明区域，此过程可参考图2所描述的方式。渲染服务通知多媒体播放服务绘制播放窗口和至少一帧图像。对于S303，在绘制完成后播放窗口和透明区域后，可以通知显示服务来呈现渲染结果，同时通知显示服务获取已解析好的多媒体数据流的至少一帧图像在播放窗口的位置播放。
如果在渲染服务的判断结果为未解析完多媒体数据流的至少一帧图像，则在执行S302阶段之前，可以通知渲染服务的绘图模块在所绘制的界面上绘制预定内容的图像，内容可以根据设计者的需要制定，例如，与将要播放的视频数据相关的海报，或广告信息等。在绘制完成后通知显示服务显示渲染好的界面，此时在界面上呈现的是预定内容的图像而不是多媒体数据流。这样，使用户在未看到多媒体数据流之前，看到的是预定内容的图像而不是显示黑屏或白屏，增强用户体验。
图4a-图4c列举了几种播放多媒体数据流的应用场景，值得注意，本申请所适用的应用场景不局限于所列举的实例。
图4a描述了浏览器的界面播放多媒体数据流的场景。浏览器界面中的透明区域是在绘制浏览器的界面元素时，由浏览器的渲染服务所绘制；播放窗口是由多媒体数据播放服务在另一个绘图层绘制。由于透明区域和播放窗口重叠，因此用户看到的是播放窗口的呈现效果。浏览器的界面中其他区域显示有其他的界面元素(图中的文字块、图片块等)。
图4b中，在渲染IPTV的操作系统的界面时，通过操作系统的渲染服务绘制透明区域，通过多媒体数据播放服务绘制播放窗口；播放窗口和透明区域布满整个界面区域。
图4c中，平板电脑的桌面上放置有游戏App，游戏App的播放窗口重叠于桌面上的透明区域。
对于其他可播放多媒体的设备的场景描述不再一一列举。
图5是本申请一个应用实例的流程图。以播放视频数据为例说明。本应用实例中以首帧图像被解析完后开始绘制透明窗口为例。
在S501阶段，解码服务对视频数据进行解码，具体过程为：从视频源获取视频数据包(S501a)，然后将视频数据包解析成图像帧(S501b)。当解析出第一帧图像帧后，通过多媒体播放服务的扩展接口通知渲染服务(S501c)。发送通知消息可以按照以下命令：
message type：MSG_INFO_EXT
param1：306
“306”表示第一帧图像帧已经解析完成。
在S502阶段，渲染服务对界面进行渲染。具体过程为：当收到界面的业务数据(S502a)后，如果所收到的业务数据中包含视频控件，则查看预设标识的状态是否为初始状态(S502b)，如果是初始状态，则认为目前首帧图像帧尚未被解析完成，则通知绘图模块在播放视频的位置绘制海报(S502c)；如果不是初始状态，则认为已解析完成首帧图像帧，于是通知绘图模块绘制透明区域，并通知多媒体播放服务绘制播放窗口和首帧图像(S502d)。在绘制完海报或者透明区域后，多媒体播放服务通知显示服务显示绘制结果(S502e)。
在显示阶段，显示服务对渲染结果和视频的图像帧进行显示。渲染服务在渲染结束后会通知显示服务显示渲染结果，另外，当多媒体播放服务在绘制好播放窗口后，也会通知显示服务显示播放窗口，并通知显示服务获取已解析的首帧图像帧在播放窗口播放。在还未开始播放视频图像帧时，显示服务获取包含海报的界面渲染结果来显示；在首帧图像帧已解析完成后，显示服务获取包含透明窗口的界面渲染结果，并叠加播放窗口来显示，显示时将每帧图像帧在播放窗口的位置显示出来。
与前述渲染方法和播放多媒体数据流的方法的实施例相对应，本申请还提供了渲染装置、播放多媒体数据流的装置、多媒体数据流的解码装置的实施例。
本申请所提供的装置的实施例可以应用在图1的设备100上，设备100的种类可参考上文描述。装置实施例可以通过软件实现，也可以通过硬件或者软硬件结合的方式实现。以软件实现为例，作为一个逻辑意义上的装置，是通过其所在设备100的处理器将非易失性存储器中对应的可执行计算机程序指令读取到内存中运行形成的。从硬件层面而言，如图6所示，为本申请播放多媒体数据的装置或界面渲染的装置或多媒体数据的解码装置所在设备100的一种硬件结构图，除了图6所示的处理器、内存、网络接口、以及非易失性存储器、显示存储器之外，实施例中装置所在的设备100通常根据该智能设备的实际功能，还可以包括其他硬件，例如GPU等对此不再赘述。
作为一种实施例，当设备100(本实施例中可将设备100称为播放多媒体数据流的设备)作为渲染装置、播放多媒体数据流的装置、多媒体数据流的解码装置的承载实体时，设备100的处理器可以被配置为执行以下操作：当解析完多媒体数据流的至少一帧图像后，发送通知消息；当根据所述通知消息判断已解析完多媒体数据流的至少一帧图像时，绘制界面上的透明区域，以及绘制所述多媒体数据流的播放窗口和所述多媒体数 据流的至少一帧图像，所述透明区域和所述播放窗口重叠于不同的绘图层；在所述播放窗口显示所绘制的多媒体数据流的至少一帧图像。
请参考图7，渲染装置700，包括：
检测模块701，用于判断是否已解析完多媒体数据流的至少一帧图像，并将判断结果通知绘制模块；
绘制模块702，用于在已解析完多媒体数据流的至少一帧图像时绘制界面上的透明区域；
通知模块703，用于在已解析完多媒体数据流的至少一帧图像时指示进行多媒体数据流的播放窗口和多媒体数据流的至少一帧图像的绘制，透明区域和所述播放窗口重叠于不同的绘图层。
至少一帧图像可以包括首帧图像。
检测模块701根据预设标识的状态判断是否已解析完多媒体数据流的至少一帧图像；所述装置还包括：
监听模块(图中未示出)，用于在监听到通知消息后修改预设标识的状态；通知消息在解析完多媒体数据流的至少一帧图像后被发送。
绘制模块702还用于在未解析完多媒体数据流的至少一帧图像时在界面绘制预定内容的图像。
图8中多媒体数据流的解码装置800，包括：
解码模块801，用于将多媒体数据流解析成帧图像；
通知模块802，用于当解析完多媒体数据流的至少一帧图像后，发送通知消息，所述通知消息用于通知已解析完多媒体数据流的至少一帧图像。
至少一帧图像可以包括首帧图像。
解码装置800位于多媒体数据播放服务中。
图9中，播放多媒体数据流的装置900，包括：
解码装置901，用于当解析完多媒体数据流的至少一帧图像后，发送通知消息；
渲染装置902，用于当根据通知消息判断已解析完多媒体数据流的至少一帧图像时，绘制界面上的透明区域，并通知多媒体数据播放装置；
多媒体数据播放装置903，用于接收渲染装置的通知后绘制所述多媒体数据流的播放窗口和多媒体数据流的至少一帧图像，透明区域和播放窗口重叠于不同的绘图层；
显示装置904，用于在播放窗口显示所绘制的多媒体数据流的至少一帧图像。
至少一帧图像可以包括首帧图像。
渲染装置902根据通知消息判断已解析完多媒体数据流的至少一帧图像可以包括：
启动第二线程监听到通知消息后修改预设标识的状态；
启动第一线程根据预设标识的状态判断已解析完多媒体数据流的至少一帧图像。
解码装置901可以通过扩展接口发送所述通知消息。
渲染装置902还可以用于：
如果未解析完多媒体数据流的至少一帧图像，则在界面绘制预定内容的图像。
渲染装置902还可以用于当未解析完所述多媒体数据流的至少一帧图像时，通知显示装置在所述界面上显示所述预定内容的图像。
上述装置中各个单元的功能和作用的实现过程具体详见上述方法中对应步骤的实现过程，在此不再赘述。
对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本申请方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。

Claims (27)

  1. 一种渲染方法，其特征在于，该方法包括步骤：
    如果已解析完多媒体数据流的至少一帧图像，则绘制界面上的透明区域，并指示进行所述多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像的绘制，所述透明区域和所述播放窗口重叠于不同的绘图层。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    第一线程根据预设标识的状态判断是否已解析完多媒体数据流的至少一帧图像；以及
    通过第二线程监听到通知消息后修改所述预设标识的状态，所述通知消息在解析完多媒体数据流的至少一帧图像后被发送。
  3. 根据权利要求1所述的方法，其特征在于，所述至少一帧图像包括首帧图像。
  4. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    如果未解析完多媒体数据流的至少一帧图像，则在所述界面绘制预定内容的图像。
  5. 一种多媒体数据流的解码方法，其特征在于，包括：
    将多媒体数据流解析成帧图像；
    当解析完多媒体数据流的至少一帧图像后，发送通知消息，所述通知消息用于通知已解析完多媒体数据流的至少一帧图像。
  6. 根据权利要求5所述的方法，其特征在于，所述至少一帧图像包括首帧图像。
  7. 根据权利要求5所述的方法，其特征在于，通过扩展接口发送所述通知消息。
  8. 一种播放多媒体数据流的方法，其特征在于，包括：
    当解析完多媒体数据流的至少一帧图像后，发送通知消息；
    当根据所述通知消息判断已解析完多媒体数据流的至少一帧图像时，绘制界面上的透明区域，以及绘制所述多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像，所述透明区域和所述播放窗口重叠于不同的绘图层；
    在所述播放窗口显示所绘制的多媒体数据流的至少一帧图像。
  9. 根据权利要求8所述的方法，其特征在于，所述至少一帧图像包括首帧图像。
  10. 根据权利要求8所述的方法，其特征在于，根据所述通知消息判断已解析完多媒体数据流的至少一帧图像包括：
    第二线程监听到通知消息后修改预设标识的状态；
    第一线程根据预设标识的状态判断已解析完多媒体数据流的所述至少一帧图像。
  11. 根据权利要求8所述的方法，其特征在于，通过扩展接口发送所述通知消息。
  12. 根据权利要求8所述的方法，其特征在于，所述方法还包括：
    如果未解析完多媒体数据流的所述至少一帧图像，则在所述界面绘制预定内容的图像。
  13. 根据权利要求12所述的方法，其特征在于，所述方法还包括：
    当未解析完所述多媒体数据流的所述至少一帧图像时，在所述界面上显示所述预定内容的图像。
  14. 一种渲染装置，其特征在于，包括：
    检测模块，用于判断是否已解析完多媒体数据流的至少一帧图像，并将判断结果通知绘制模块；
    绘制模块，用于在已解析完多媒体数据流的至少一帧图像时绘制界面上的透明区域；
    通知模块，用于在已解析完多媒体数据流的至少一帧图像时指示进行所述多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像的绘制，所述透明区域和所述播放窗口重叠于不同的绘图层。
  15. 根据权利要求14所述的装置，其特征在于，
    所述检测模块根据预设标识的状态判断是否已解析完多媒体数据流的至少一帧图像；所述装置还包括：
    监听模块，用于在监听到通知消息后修改所述预设标识的状态；所述通知消息在解析完多媒体数据流的至少一帧图像后被发送。
  16. 根据权利要求14所述的装置，其特征在于，所述至少一帧图像包括首帧图像。
  17. 根据权利要求14所述的装置，其特征在于，所述绘制模块还用于在未解析完多媒体数据流的至少一帧图像时在所述界面绘制预定内容的图像。
  18. 一种多媒体数据流的解码装置，其特征在于，包括：
    解码模块，用于将多媒体数据流解析成帧图像；
    通知模块，用于当解析完多媒体数据流的至少一帧图像后，发送通知消息，所述通知消息用于通知已解析完多媒体数据流的至少一帧图像。
  19. 根据权利要求18所述的装置，其特征在于，所述至少一帧图像包括首帧图像。
  20. 根据权利要求18所述的装置，其特征在于，所述解码装置位于多媒体数据播放服务中。
  21. 一种播放多媒体数据流的装置，其特征在于，包括：
    解码装置，用于当解析完多媒体数据流的至少一帧图像后，发送通知消息；
    渲染装置，用于当根据所述通知消息判断已解析完多媒体数据流的至少一帧图像时，绘制界面上的透明区域，并通知多媒体数据播放装置；
    多媒体数据播放装置，用于接收所述渲染装置的通知后绘制所述多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像，所述透明区域和所述播放窗口重叠于不同的绘图层；
    显示装置，用于在所述播放窗口显示所绘制的多媒体数据流的至少一帧图像。
  22. 根据权利要求21所述的装置，其特征在于，所述至少一帧图像包括首帧图像。
  23. 根据权利要求21所述的装置，其特征在于，所述渲染装置根据所述通知消息判断已解析完多媒体数据流的至少一帧图像包括：
    启动第二线程监听到通知消息后修改预设标识的状态；
    启动第一线程根据预设标识的状态判断已解析完多媒体数据流的至少一帧图像。
  24. 根据权利要求21所述的装置，其特征在于，所述解码装置通过扩展接口发送所述通知消息。
  25. 根据权利要求21所述的装置，其特征在于，所述渲染装置还用于：
    如果未解析完多媒体数据流的至少一帧图像，则在所述界面绘制预定内容的图像。
  26. 根据权利要求25所述的装置，其特征在于，
    所述渲染装置还用于当未解析完所述多媒体数据流的至少一帧图像时，通知显示装置在所述界面上显示所述预定内容的图像。
  27. 一种播放多媒体数据流的设备，其特征在于，包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为：
    当解析完多媒体数据流的至少一帧图像后，发送通知消息；
    当根据所述通知消息判断已解析完多媒体数据流的至少一帧图像时，绘制界面上的透明区域，以及绘制所述多媒体数据流的播放窗口和所述多媒体数据流的至少一帧图像，所述透明区域和所述播放窗口重叠于不同的绘图层；
    在所述播放窗口显示所绘制的多媒体数据流的至少一帧图像。
PCT/CN2017/071819 2016-02-01 2017-01-20 渲染方法、解码方法、播放多媒体数据流的方法及装置 Ceased WO2017133481A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/047,431 US10805570B2 (en) 2016-02-01 2018-07-27 System and method for streaming multimedia data

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610069362.5 2016-02-01
CN201610069362.5A CN107027068B (zh) 2016-02-01 2016-02-01 渲染方法、解码方法、播放多媒体数据流的方法及装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/047,431 Continuation-In-Part US10805570B2 (en) 2016-02-01 2018-07-27 System and method for streaming multimedia data

Publications (1)

Publication Number Publication Date
WO2017133481A1 true WO2017133481A1 (zh) 2017-08-10

Family

ID=59500531

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/071819 Ceased WO2017133481A1 (zh) 2016-02-01 2017-01-20 渲染方法、解码方法、播放多媒体数据流的方法及装置

Country Status (3)

Country Link
US (1) US10805570B2 (zh)
CN (1) CN107027068B (zh)
WO (1) WO2017133481A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110018864A (zh) * 2018-01-09 2019-07-16 阿里巴巴集团控股有限公司 页面资源投放方法及装置

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110020359B (zh) * 2017-11-08 2024-04-05 亿阳信通股份有限公司 应用在网页前端的数据处理方法、装置及存储介质
CN112035771A (zh) * 2019-06-03 2020-12-04 阿里巴巴集团控股有限公司 基于web的摄像头数据绘制方法、装置及电子设备
CN110989948B (zh) * 2019-10-31 2023-03-21 中国航空工业集团公司洛阳电光设备研究所 一种基于ufcp绘图帧率提升方法
CN111526425B (zh) * 2020-04-26 2022-08-09 北京字节跳动网络技术有限公司 视频播放方法、装置、可读介质及电子设备
US12367004B2 (en) * 2021-07-27 2025-07-22 App-Pop-Up Inc. System and method for simultaneously displaying multiple GUIs via the same display
CN114374867B (zh) * 2022-01-19 2024-03-15 平安国际智慧城市科技股份有限公司 多媒体数据的处理方法、装置及介质
CN115329225B (zh) * 2022-10-18 2023-03-24 中孚信息股份有限公司 基于服务端浏览器的页面预渲染方法、系统及设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101937416A (zh) * 2010-09-09 2011-01-05 苏州华芯微电子股份有限公司 基于fpga实现的ps2协议逻辑分析仪
CN102739983A (zh) * 2011-04-11 2012-10-17 腾讯科技（深圳）有限公司 一种实现半透明效果的方法及系统
US8724029B2 (en) * 2011-05-26 2014-05-13 Adobe Systems Incorporated Accelerating video from an arbitrary graphical layer
CN104407865A (zh) * 2014-11-25 2015-03-11 广州酷狗计算机科技有限公司 窗口的显示方法及装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6658056B1 (en) 1999-03-30 2003-12-02 Sony Corporation Digital video decoding, buffering and frame-rate converting method and apparatus
EP1323308B1 (en) 2000-08-15 2014-08-20 Polycom Israel Ltd. Delay reduction for transmission and processing of video data
US7885338B1 (en) 2005-04-25 2011-02-08 Apple Inc. Decoding interdependent frames of a video for display
US9197685B2 (en) 2012-06-28 2015-11-24 Sonic Ip, Inc. Systems and methods for fast video startup using trick play streams
US9106942B2 (en) * 2013-07-22 2015-08-11 Archana Vidya Menon Method and system for managing display of personalized advertisements in a user interface (UI) of an on-screen interactive program (IPG)
US9641867B2 (en) * 2014-10-24 2017-05-02 Megachips Corporation Image processor

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101937416A (zh) * 2010-09-09 2011-01-05 苏州华芯微电子股份有限公司 基于fpga实现的ps2协议逻辑分析仪
CN102739983A (zh) * 2011-04-11 2012-10-17 腾讯科技（深圳）有限公司 一种实现半透明效果的方法及系统
US8724029B2 (en) * 2011-05-26 2014-05-13 Adobe Systems Incorporated Accelerating video from an arbitrary graphical layer
CN104407865A (zh) * 2014-11-25 2015-03-11 广州酷狗计算机科技有限公司 窗口的显示方法及装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110018864A (zh) * 2018-01-09 2019-07-16 阿里巴巴集团控股有限公司 页面资源投放方法及装置

Also Published As

Publication number Publication date
US20190028668A1 (en) 2019-01-24
CN107027068B (zh) 2021-04-13
US10805570B2 (en) 2020-10-13
CN107027068A (zh) 2017-08-08

Similar Documents

Publication Publication Date Title
WO2017133481A1 (zh) 渲染方法、解码方法、播放多媒体数据流的方法及装置
US20250056068A1 (en) Live broadcasting comment presentation method and apparatus, and device, program product and medium
CN107018370B (zh) 用于视频墙的显示方法及其系统
KR101571754B1 (ko) 그래픽들에 대한 로우 레이턴시 무선 디스플레이
CN108632674B (zh) 一种全景视频的播放方法和客户端
CN110324671B (zh) 网页视频播放方法及装置、电子设备及存储介质
US11179634B2 (en) Content presenting method, user equipment and system
US8887178B1 (en) Method to customize native operating system views using web technologies in mobile devices
CN108124173A (zh) 一种一对多投屏显示方法、系统及存储介质
CN102685417B (zh) 电视机换台的方法、装置及电视机
US20170171277A1 (en) Method and electronic device for multimedia recommendation based on android platform
WO2016188276A1 (zh) 视频播放方法、客户端和计算机存储介质
CN112243137A (zh) 一种直播间界面更新方法、装置、服务器以及系统
CN104811785A (zh) 智能终端的显示图形用户界面的控制方法及装置
KR20180069576A (ko) 영상제공장치, 그의 제어 방법 및 영상제공시스템
WO2014089793A1 (zh) 一种视频重定向的方法、装置、系统及计算机可读介质
WO2019233314A1 (zh) 一种电视墙图像回显方法、服务器件及电视墙系统
CN114040251A (zh) 音视频播放方法、系统、存储介质及计算机程序产品
CN103458310A (zh) 一种信息展示方法及装置
CN113778360A (zh) 投屏方法和电子设备
WO2017133475A1 (zh) 播放多媒体数据的方法、界面渲染方法及装置、设备
CN110012336B (zh) 直播界面的图片配置方法、终端及装置
CN114630184A (zh) 视频的渲染方法、装置、设备和计算机可读存储介质
CN109842792B (zh) 视频播放方法、装置、系统及存储介质
CN104135686B (zh) 一种拼接墙信号的分享方法及用户终端

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17746801

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17746801

Country of ref document: EP

Kind code of ref document: A1