WO2017133475A1 - 播放多媒体数据的方法、界面渲染方法及装置、设备 - Google Patents

播放多媒体数据的方法、界面渲染方法及装置、设备 Download PDF

Info

Publication number
WO2017133475A1
WO2017133475A1 PCT/CN2017/071716 CN2017071716W WO2017133475A1 WO 2017133475 A1 WO2017133475 A1 WO 2017133475A1 CN 2017071716 W CN2017071716 W CN 2017071716W WO 2017133475 A1 WO2017133475 A1 WO 2017133475A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
multimedia data
transparent area
window
playing
Prior art date
Application number
PCT/CN2017/071716
Other languages
English (en)
French (fr)
Inventor
李江卫
钟学书
刘永卿
Original Assignee
阿里巴巴集团控股有限公司
李江卫
钟学书
刘永卿
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴集团控股有限公司, 李江卫, 钟学书, 刘永卿 filed Critical 阿里巴巴集团控股有限公司
Publication of WO2017133475A1 publication Critical patent/WO2017133475A1/zh
Priority to US16/044,088 priority Critical patent/US10957285B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/14—Display of multiple viewports
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/37—Details of the operation on graphic patterns
    • G09G5/377—Details of the operation on graphic patterns for mixing or overlaying two or more graphic patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04804—Transparency, e.g. transparent or translucent windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0485—Scrolling or panning
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/12—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
    • G09G2340/125—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels wherein one of the images is motion video
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2354/00—Aspects of interface with display user

Definitions

  • the present application relates to a technology for playing multimedia data, and particularly relates to a method for playing multimedia data, a method for rendering an interface, a device for playing multimedia data, a device for rendering an interface, and a device for playing multimedia data.
  • multimedia data playing services are required to play multimedia data, and an area on the interface of the application is used as a display area of multimedia data.
  • One solution for achieving this effect is to draw a transparent area on the drawing layer of the drawing interface when rendering the interface of the application, and then draw a play window in another drawing layer to overlap the position of the playing window with transparency. The location of the area, the multimedia data to be played for each frame is displayed in the playback window.
  • the position of the transparent area changes, and the position of the playback window needs to be adjusted accordingly.
  • the current process of adjusting the transparent area and the playback window is likely to cause white edges or data jitter on the interface.
  • the application provides a method for playing multimedia data, a method for rendering an interface, a device for playing multimedia data, and an interface rendering device, which can avoid interface white edge or data jitter caused by interface scrolling.
  • a method for playing multimedia data comprising the steps of:
  • the multimedia data is played in a window.
  • a method for interface rendering including the steps of:
  • the first information is used to indicate that a transparent area is drawn
  • the second information is used to indicate a play window for drawing multimedia data
  • the transparent area is drawn in the drawing layer according to the first information.
  • an apparatus for playing multimedia data includes:
  • a drawing control module configured to notify to draw a transparent area on the first drawing layer; to notify a playing window for drawing multimedia data on the second drawing layer, wherein a playing window of the multimedia data overlaps the transparent area;
  • a display control module configured to notify the synchronous display of the transparent area and the play window, and notify that the multimedia data is played in the play window.
  • an apparatus for interface rendering including:
  • a notification module configured to send the first information and the second information to the image processing unit GPU process; the first information is used to indicate that a transparent area is drawn, and the second information is used to indicate a play window for drawing multimedia data;
  • a drawing module configured to draw the transparent area in the drawing layer according to the first information when receiving the notification of the GPU process.
  • an apparatus for playing multimedia data includes:
  • a memory for storing processor executable instructions
  • processor is configured to:
  • the first information is used to indicate that the transparent area is drawn
  • the second information is used to indicate a play window for drawing multimedia data
  • the notification synchronously displays the transparent area and the play window, and notifies that the multimedia data is played in the play window.
  • FIG. 1 is a hardware block diagram of a device 100 in an embodiment of the present application.
  • FIG. 2 is a flowchart of a method for playing multimedia data in an embodiment of the present application
  • FIG. 3 is a flowchart of a method for rendering an interface in an embodiment of the present application.
  • 3a, 3b, and 3c are three scene diagrams in the embodiment of the present application.
  • FIG. 5 is a hardware architecture diagram of an apparatus for playing multimedia data or an apparatus for rendering an interface in an embodiment of the present application
  • FIG. 6 is a logic block diagram of an apparatus for playing multimedia data in an embodiment of the present application.
  • FIG. 7 is a logic block diagram of an apparatus for rendering an interface in an embodiment of the present application.
  • first, second, third, etc. may be used to describe various information in this application, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as the second information without departing from the scope of the present application.
  • second information may also be referred to as the first information.
  • word "if” as used herein may be interpreted as "when” or “when” or “in response to a determination.”
  • the device 100 has a central processing unit (CPU) 101, a graphics processing unit (GPU) 102, and the like, and an operating system, an internal or upper layer application of the operating system (for example, multimedia data playback).
  • an operating system for example, multimedia data playback
  • Software such as client, browser, mobile terminal desktop application, etc., the operating system can provide a variety of services, such as multimedia data playback services.
  • Device 100 can be a handheld device with wireless communication capabilities, an in-vehicle device, a wearable device, a television device, a computing device, or other processing device connected to a wireless modem, and the like.
  • the GPU 102 can provide a rendering function of the interface elements, and can perform layout on the interface elements on the interface through rendering.
  • the effect of the interface rendered by the browser while running, the desktop effect rendered by the operating system runtime, and the effect of the application client rendering a display area on the operating system desktop are the rendering results of the data to be displayed by the GPU.
  • the CPU 101 is connected to the GPU 102.
  • the operating system runs on the CPU 101, and the application sends the data to be rendered to the GPU through an interface provided by the operating system; the GPU renders the data to be rendered, and stores the rendered result, for example, can be stored in the display of the GPU 102.
  • Memory VRAM, Video RAM
  • the rendering function of the GPU can be implemented by calling the GPU 102 through a rendering engine.
  • the rendering engine may be a module provided by the application itself.
  • the rendering engine may also be a module provided by an operating system or a third party in the form of a plug-in or the like.
  • the rendering engine can generate a drawing layer in VRAM, the drawing layer is the drawing medium, and the interface elements are drawn on the drawing layer.
  • the drawing result of the drawing layer is obtained by the display application providing the display function in the operating system for presentation.
  • One solution for playing multimedia data on the interface is to set a transparent area in one drawing layer when rendering the interface elements of the interface, and the multimedia data playing window can be set in another drawing layer through the multimedia data playing service, in an example.
  • the position of the play window and the position of the transparent area may overlap (in the present application, the overlap means that the two completely coincide or substantially coincide).
  • the technical solution for playing multimedia data provided by the present application can solve the interface white edge or data jitter caused when the interface scrolls.
  • S201 notify to draw a transparent area on the first drawing layer; notify a playing window for drawing multimedia data on the second drawing layer, and the playing window of the multimedia data overlaps with the transparent area;
  • the notification synchronously displays the transparent area and the play window, and notifies that the multimedia data is played in the play window.
  • S202 may be executed after the execution of S201 is completed, and the main bodies of S201 and S202 may be the same or different.
  • both S201 and S202 can be executed by the GPU process (the GPU process is a process started by the operating system when the GPU is called); S201 can also be executed by other subjects, and S202 and the like are executed by the GPU process. If the execution bodies of S201 and S202 are the same, it will be more advantageous to control the synchronous display process of the transparent area and the play window.
  • the present application controls the display service to synchronously display the transparent area and the play window by executing S202 to avoid interface white edges and data jitter.
  • notification can be understood as indicating (or controlling) the notified subject to perform an action by transmitting an instruction, which can be carried by a computer to execute an instruction. For example, Sending an instruction through the GPU process notifying the recipient of the instruction to draw a transparent area on the first drawing layer; or sending an instruction to notify the recipient of the instruction to draw a playback window of the multimedia data on the second drawing layer, and the like.
  • the sending of the notification of the transparent area and the sending of the notification of the playing window may be performed after the first information and the second information are obtained.
  • the first information in the application is used to indicate the transparent area
  • the second information is used to indicate the drawing.
  • the play window as an example, the first information and the second information may be information such as coordinates or size of the transparent area, or may be information capable of calculating the coordinates or size of the transparent area.
  • the process shown in FIG. 2 is executed by the GPU process.
  • the first information and the second information may be obtained by the GPU process from other entities, for example, receiving from a rendering thread, or may be received by the GPU process from other subjects.
  • a message is then calculated based on the first information to obtain the second information.
  • the timing at which the rendering thread sends the first information or the second information may be when the interface element is first rendered, or when the interface element needs to be re-typed (for example, the user issues a trigger instruction, the interface downloads new business data from the server, etc.).
  • the process of transmitting the first information and the second information to the GPU for other entities may be described with reference to the relevant portions of this document in conjunction with FIG.
  • the GPU process may notify the entity that receives the transparent area and the play window that the first information and the second information may be carried, and may also notify the entity that receives the message to obtain the first information and the second information from other paths.
  • the GPU process may notify the rendering engine to send the first information and the second information to the module responsible for drawing the transparent area and the module for drawing the playing window, respectively, or the GPU may notify the module responsible for drawing the transparent area and the module for drawing the playing window respectively.
  • the rendering engine requests the first information and the second information.
  • the GPU process can start two parallel threads, one thread is responsible for notifying the position of the transparent area, and the other thread is responsible for notifying the position of the drawing window. .
  • the GPU process may start a thread to serially send a notification to draw a transparent area and a play window, and may first notify the drawing of the transparent area, and then notify the drawing of the playing window; or notify the playing window first, and then notify the drawing of the transparent area.
  • the GPU process can issue commands to the multimedia playback service through the extended interface to set the location of the playback window. For example, the following can be The command informs the multimedia playback service to draw a play window in the drawing layer, where SurfaceX represents the drawing layer where the playback window is located.
  • the display service of the operating system can be notified by the GPU process to display the interface of the re-formatted application and the play window of the multimedia data. Since the display service is uniformly notified by the GPU process, the transparent area and the playback are eliminated. The window is out of sync to display problems, avoiding white edges and jitter.
  • the GPU process can also notify the display service to acquire the multimedia data to be displayed in the next frame.
  • the process described in FIG. 3 can be performed by a rendering engine.
  • the object that the rendering engine sends the second information is a GPU process instead of a multimedia data playing service; the GPU process is used herein to refer to the execution body of S202.
  • the method for obtaining the first information may be obtained according to a preset parameter, or obtained according to service data delivered by the server, or obtained according to a trigger instruction of the user.
  • the first information may be obtained according to the preset parameter, or the first information may be obtained after the service data sent by the server is generated in the rendering tree, or the first information may be obtained according to the trigger instruction of the user.
  • the path for obtaining the second information may be the same as the first information, or the second information may be calculated according to the first information after obtaining the first information.
  • the user issues a trigger command to the interface that causes the displacement of the interface element to change.
  • the interface can notify the rendering engine to re-render and trigger.
  • the command may be a button message, a touch message, etc.
  • the trigger instruction may include information such as "up”, “down”, “left”, “right”, and moving distance.
  • the manner in which the interface notifies the rendering engine may be to obtain information related to the change in position of the interface element from the trigger instruction (eg, a parameter representing the direction, a parameter of the offset, etc.) to the rendering engine.
  • the rendering engine may calculate new location information of the interface element according to the location change information of the interface element, where the new location information includes the first information and the second information.
  • the first information or the second information may be sent together with the location information of the other interface elements.
  • the various information that needs to be sent may be sent through a message, or may be sent through different messages, which may be determined according to design requirements. limit.
  • the first information and the second information may be notified to the rendering thread through the extended interface; the first information and the second information are respectively sent to the GPU process by the rendering thread to trigger the GPU process to perform the S202 phase. action.
  • the interface scrolling may be that when the multimedia data is played, the user may perform some operation on the interface of the application, causing the interface elements to be displaced (for example, the user triggers scrolling, page turning, etc.).
  • the interface element on the interface where the transparent area is located is displaced, the interface layout needs to be re-executed, and the position of the transparent area and the position of the play window of the multimedia data may need to be adjusted accordingly.
  • the interface referred to in the present application is not limited to an interface of an application, a desktop of an operating system, or the like.
  • the multimedia data may be video data, 3D animation, game data, and the like.
  • FIG. 3a - FIG. 3c illustrate an application scenario in which a user command triggers several kinds of interface scrolling phenomena. It is noted that the application scenario to which the present application is applied is not limited to the enumerated examples.
  • Figure 3a depicts a scenario where the browser's interface scrolls.
  • the transparent area in the browser interface is drawn by the browser's rendering engine when the browser's interface elements are drawn; the playback window is drawn by the multimedia data playback service on another drawing layer. Since the transparent area and the play window overlap, the user sees the rendering effect of the play window. Other areas of the browser's interface display other interface elements (text blocks, picture blocks, etc.).
  • the user issues a trigger command through a scroll bar on the interface, and after receiving the trigger command, the browser 300 notifies the browser's rendering engine to re-type the interface.
  • the transparent area is drawn by the rendering engine of the operating system, and the playing window is drawn by the multimedia data playing service; the playing window The mouth and transparent areas fill the entire interface area.
  • the user sends a trigger command such as moving up, down, or turning the page through the remote controller.
  • the IPTV interface After receiving the trigger command, the IPTV interface notifies the rendering engine of the operating system to re-type the interface.
  • a game app is placed on the desktop of the tablet, and the play window of the game app is superimposed on the transparent area on the desktop.
  • the user issues a trigger command by swiping the screen left and right by the finger to change the position of the desktop element, and the desktop responds to the trigger instruction of the user, and notifies the rendering engine of the operating system to re-type the desktop element.
  • the communicating entity includes a rendering engine 401 responsible for rendering the interface 400 of the application, a GPU process 402, a multimedia data playback service 403, a display service 404, and a drawing module 405 that implements the drawing function in the rendering engine.
  • the user sends a trigger command to the interface 400, and the trigger instruction carries the moving direction and offset information of the interface element.
  • the interface 400 sends a rendering request to the rendering engine 401 in response to the triggering instruction, where the rendering request carries the moving direction and offset information of the interface element;
  • the rendering engine 401 calculates new location information of the interface element, including the first information, and obtains the second information according to the first information (the definition of the first/second information is referred to the above description), and two instructions are executed by the rendering thread. Sent to the GPU process 402;
  • the GPU process 402 carries the new location information (including the first information) of the interface element to the drawing module 405 in a drawing notification, and carries the second information to the multimedia data playing service 403 in another drawing notification.
  • the drawing module 405 draws the interface element
  • the new location of the transparent area is drawn according to the first information
  • the multimedia playing service 403 notifies the drawing information corresponding to the service to the drawing module corresponding to the service, and draws the playing window.
  • the GPU process 403 sends a notification to the display service 404 to display the transparent display window and the multimedia data.
  • the display service 404 obtains the rendering result, presents the interface, and displays the multimedia playback data to be displayed in the next frame in the playback window.
  • the present application also provides an embodiment of an apparatus for playing multimedia data and an apparatus for interface rendering.
  • the device embodiment may be implemented by software, or may be implemented by hardware or a combination of hardware and software. Taking the software implementation as an example, as a logical means, the processor of the device 100 is configured to read the corresponding computer program instructions in the non-volatile memory into the memory. From a hardware level, as shown in FIG. 5, a hardware structure diagram of the device 100 in which the device for playing multimedia data or the interface rendering device is located, except for the processor, the memory, the network interface, and the device shown in FIG. In addition to the non-volatile memory and the display memory, the device 100 in which the device is located in the embodiment may also include other hardware according to the actual functions of the smart device, such as a GPU and the like.
  • the device 100 may load a device for playing multimedia data and a device for rendering an interface into the device.
  • executable instructions of the multimedia playback service may also be loaded into the device 100.
  • the processor of device 100 can be configured to:
  • the notification synchronously displays the transparent area and the play window, and notifies that the multimedia data is played in the play window.
  • the apparatus 600 for playing multimedia data includes:
  • a drawing control module 601 configured to notify to draw a transparent area on the first drawing layer; to notify a playing window for drawing multimedia data on the second drawing layer, the playing window of the multimedia data The mouth overlaps the transparent area;
  • the display control module 602 is configured to notify the synchronous display of the transparent area and the play window, and notify that the multimedia data is played in the play window.
  • Device 600 can be located in an image processing unit GPU process.
  • the apparatus 600 may further include:
  • a receiving module (not shown), configured to receive the first information and the second information, and notify the drawing control module 601 of the first information and the second information, where the first information is used to indicate that the drawing is a transparent area, the second information is used to indicate that the play window is drawn.
  • the receiving module receives the first information and the second information from a rendering thread when the interface element is first rendered or when the user issues a triggering instruction; the triggering instruction includes a user's scrolling instruction or a page turning instruction.
  • the apparatus 600 further includes:
  • a receiving module (not shown), configured to receive first information, where the first information is used to indicate that the transparent area is drawn;
  • a calculation module (not shown), configured to obtain second information according to the first information, where the second information is used to indicate that the play window is drawn.
  • the drawing control module 601 notifies that the playing window for drawing the multimedia data on the second drawing layer comprises:
  • the drawing control module 601 notifies the multimedia data playing service, and the multimedia data playing service draws a playing window of the multimedia data in the second drawing layer according to the second information; the second information is used to indicate that the playing window is drawn.
  • the multimedia data may include video data, 3D animation data, game data.
  • the device 700 for interface rendering includes:
  • the notification module 701 is configured to send the first information and the second information to the image processing unit GPU process; the first information is used to indicate that a transparent area is drawn, and the second information is used to indicate a play window for drawing multimedia data;
  • the drawing module 702 is configured to, when receiving the notification of the GPU process, according to the The first information draws the transparent area in the drawing layer.
  • it can also include:
  • a calculation module for obtaining the first information according to a preset parameter when the interface element is first rendered or a trigger instruction of the user;
  • the trigger instruction of the user includes a scroll instruction or a page turning instruction.
  • the calculation module is further configured to obtain the second information according to the first information.
  • the sending, by the notification module 701, the first information and the second information to the GPU process may include:
  • the rendering thread notifies the GPU process of the first information and the second information, respectively.
  • the device embodiment since it basically corresponds to the method embodiment, reference may be made to the partial description of the method embodiment.
  • the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the objectives of the present application. Those of ordinary skill in the art can understand and implement without any creative effort.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请公开了播放多媒体数据的方法、界面渲染方法及装置、设备，播放多媒体数据的方法包括：通知在第一绘图层上绘制透明区域；通知在第二绘图层上绘制多媒体数据的播放窗口，所述多媒体数据的播放窗口与所述透明区域重叠；通知同步显示所述透明区域和所述播放窗口，以及通知在所述播放窗口中播放所述多媒体数据。本申请能够避免界面滚动引起的界面白边或数据抖动。

Description

播放多媒体数据的方法、界面渲染方法及装置、设备 技术领域
本申请涉及播放多媒体数据的技术，尤其涉及播放多媒体数据的方法、界面渲染的方法及播放多媒体数据的装置、界面渲染的装置、播放多媒体数据的设备。
背景技术
目前某些场景下，在应用的界面上播放多媒体数据时，需要借助多媒体数据播放服务来播放多媒体数据，在应用的界面上有一个区域作为多媒体数据的显示区域。目前实现该效果的一种方案是在对该应用的界面渲染时，在绘制界面的绘图层绘制一个透明区域，另外，再在另一个绘图层绘制一个播放窗口，将播放窗口的位置重叠于透明区域的位置，播放窗口内显示每一帧待播放的多媒体数据。
当用户执行滚动或翻页等操作时，透明区域的位置发生改变，需要相应的调整播放窗口的位置。但目前对透明区域和播放窗口调整的过程容易造成界面出现白边或数据抖动现象。
发明内容
本申请提供播放多媒体数据的方法、界面渲染的方法及播放多媒体数据的装置、界面渲染的装置，能够避免界面滚动引起的界面白边或数据抖动。
根据本申请实施例的第一方面，提供一种播放多媒体数据的方法，所述方法包括步骤：
通知在第一绘图层上绘制透明区域；通知在第二绘图层上绘制多媒体数据的播放窗口，所述多媒体数据的播放窗口与所述透明区域重叠；
通知同步显示所述透明区域和所述播放窗口，以及通知在所述播放 窗口中播放所述多媒体数据。
根据本申请实施例的第二方面，提供一种界面渲染的方法，包括步骤：
将第一信息和第二信息发给图像处理单元GPU进程；所述第一信息用于指示绘制透明区域，所述第二信息用于指示绘制多媒体数据的播放窗口；
当接收到所述GPU进程的通知时，根据所述第一信息在绘图层绘制所述透明区域。
根据本申请实施例的第三方面，提供一种播放多媒体数据的装置，包括：
绘图控制模块，用于通知在第一绘图层上绘制透明区域；通知在第二绘图层上绘制多媒体数据的播放窗口，所述多媒体数据的播放窗口与所述透明区域重叠；
显示控制模块，用于通知同步显示所述透明区域和所述播放窗口，以及通知在所述播放窗口中播放所述多媒体数据。
根据本申请实施例的第四方面，提供一种界面渲染的装置，包括：
通知模块，用于将第一信息和第二信息发给图像处理单元GPU进程；所述第一信息用于指示绘制透明区域，所述第二信息用于指示绘制多媒体数据的播放窗口；
绘图模块，用于当接收到所述GPU进程的通知时，根据所述第一信息在绘图层绘制所述透明区域。
根据本申请实施例的第五方面，提供一种播放多媒体数据的设备，包括：
处理器；
用于存储处理器可执行指令的存储器；
其中，所述处理器被配置为：
发送第一信息和第二信息；所述第一信息用于指示绘制透明区域， 所述第二信息用于指示绘制多媒体数据的播放窗口；
根据所述第一信息通知在第一绘图层上绘制透明区域；根据所述第二信息通知在第二绘图层上绘制多媒体数据的播放窗口，所述多媒体数据的播放窗口与所述透明区域重叠；
当收到所述通知后，根据所述第一信息绘制所述透明区域，根据所述第二信息绘制所述播放窗口；
通知同步显示所述透明区域和所述播放窗口，以及通知在所述播放窗口中播放所述多媒体数据。
由于本申请对透明区域和播放窗口的显示过程进行了同步控制，因此可以避免现有技术中界面滚动时引起的界面白边和数据抖动现象。
附图说明
图1为本申请实施例中设备100的硬件框图；
图2为本申请实施例中播放多媒体数据的方法的流程图；
图3为本申请实施例中界面渲染的方法的流程图；
图3a、图3b、图3c为本申请实施例中三种场景图；
图4为本申请实施例中一个应用实例中的时序图；
图5为本申请实施例中播放多媒体数据的装置或界面渲染的装置的硬件架构图；
图6为本申请实施例中播放多媒体数据的装置软件逻辑框图；
图7为本申请实施例中界面渲染的装置软件逻辑框图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申 请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。
在本申请使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本申请。在本申请和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本申请可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本申请范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
图1中，设备100具有中央处理器(Central Processing Unit，CPU)101、图形处理器(Graphic Processing Unit，GPU)102等硬件设备，以及操作系统、操作系统内部或上层的应用(例如多媒体数据播放客户端、浏览器、移动终端桌面应用等)等软件，操作系统可以提供多种服务，例如多媒体数据播放服务等。
设备100可以是具有无线通信功能的手持设备、车载设备、可穿戴设备、电视设备、计算设备或连接到无线调制解调器的其它处理设备等等。
GPU 102可以提供界面元素的渲染功能，通过渲染可以实现对界面上的界面元素进行排版。例如，浏览器运行时所呈现的界面效果、操作系统运行时所呈现的桌面效果、应用客户端呈现在操作系统桌面上的某个显示区域的效果均是GPU对待显示的数据的渲染结果。在硬件层面，CPU 101与GPU 102连接。操作系统运行在CPU 101上，应用通过操作系统提供的接口将需要渲染的数据发给GPU；GPU对需要渲染的数据进行渲染，并将渲染后的结果存储，例如，可以存储在GPU 102的显示存储器(VRAM，Video RAM)中。
可以通过渲染引擎调用GPU 102来实现GPU的渲染功能。在某些例子中，渲染引擎可以是属于应用本身所提供的模块，在某些例子中，渲染引擎也可以是操作系统或第三方以插件等形式所提供的模块。在渲染时，渲染引擎可以在VRAM中生成绘图层，绘制层是绘图的介质，在绘图层上绘制界面元素。通过操作系统中提供显示功能的显示应用获取绘图层的绘制结果进行呈现。
在界面上播放多媒体数据的一种方案是，在渲染界面的界面元素时，在一个绘图层设置透明区域，可以通过多媒体数据播放服务在另一个绘图层设置多媒体数据的播放窗口，在一个例子中，播放窗口的位置与透明区域的位置可以重叠(本申请中，重叠是指二者完全重合或基本重合)。
本申请所提供的播放多媒体数据的技术方案能够解决在界面滚动时引起的界面白边或数据抖动。
本申请提供的播放多媒体数据的方法的流程可参考图2，详细过程如下：
S201，通知在第一绘图层上绘制透明区域；通知在第二绘图层上绘制多媒体数据的播放窗口，多媒体数据的播放窗口与透明区域重叠；
S202，通知同步显示透明区域和播放窗口，以及通知在播放窗口中播放所述多媒体数据。
作为例子，S202可以在S201执行完成后运行，执行S201和S202的主体可以相同，也可以不同。例如可以S201和S202均通过GPU进程来执行(GPU进程是当GPU被调用时在操作系统启动的进程)；也可以通过其他主体来执行S201，通过GPU进程来执行S202等。如果S201和S202的执行主体相同，将更有利于控制透明区域和播放窗口的同步显示过程。
本申请通过执行S202来控制显示服务同步显示透明区域和播放窗口，以避免界面白边和数据抖动。
本申请中，“通知”可以理解为通过发送指令指示(或控制)被通知的主体执行某个动作，该通知可以通过计算机执行指令承载。例如，可 以通过GPU进程发送一条指令，通知该指令的接收者在第一绘图层上绘制透明区域；或者发送一条指令通知该指令的接收者在第二绘图层上绘制多媒体数据的播放窗口等等。
在S201阶段，绘制透明区域的通知和绘制播放窗口的通知的发送时机可以是获得第一信息和第二信息后，本申请中第一信息用来指示绘制透明区域，第二信息用来指示绘制播放窗口，作为示例，第一信息和第二信息可以是诸如透明区域的坐标或尺寸等信息，也可以是能够计算出透明区域坐标或尺寸的信息。
以下以GPU进程执行图2所示的流程为例，第一信息和第二信息的获取方式可以是GPU进程从其他主体接收，例如，从渲染线程接收，还可以是GPU进程从其他主体接收第一信息，然后根据第一信息计算获得第二信息。
渲染线程发送第一信息或第二信息的时机可以是当初次渲染界面元素时，或者需要重新对界面元素排版时(例如，用户发出触发指令、界面从服务端下载新的业务数据等情况)。对于其他实体将第一信息和第二信息发送给GPU的过程可结合图3参考本文的相关部分描述。
GPU进程通知绘制透明区域和播放窗口的消息中可以携带第一信息和第二信息，也可以通知接收消息的实体从其他途径获取第一信息和第二信息。例如，GPU进程可以通知渲染引擎将第一信息和第二信息分别发给负责绘制透明区域的模块和绘制播放窗口的模块，或者GPU可以通知负责绘制透明区域的模块和绘制播放窗口的模块分别向渲染引擎请求第一信息和第二信息。
对于GPU进程来通知绘制透明区域和播放窗口的方式可以不作限制，例如，GPU进程可以分别启动两个并行的线程，一个线程负责通知绘制透明区域的位置，另一个线程负责通知绘制播放窗口的位置。再例如，GPU进程可以启动一个线程来串行发送绘制透明区域和播放窗口的通知，可以先通知绘制透明区域，再通知绘制播放窗口；也可以先通知播放窗口，再通知绘制透明区域。GPU进程可以通过扩展接口向多媒体播放服务发命令，以便设置播放窗口的位置。例如，可通过以下 命令通知多媒体播放服务在绘制层绘制播放窗口，其中SurfaceX表示播放窗口所在的绘制层。
GL_APICALL void GL_APIENTRY glSetSurfaceX(GLint xoffset,GLint yoffset,GLint width,GLint height)
在本申请的S202阶段，可以由GPU进程通知操作系统的显示服务来显示重新排版后的应用的界面以及多媒体数据的播放窗口，由于统一由GPU进程来通知显示服务，因此消除了透明区域和播放窗口的不同步显示问题，避免了白边和抖动现象。GPU进程还可以通知显示服务获取下一帧待显示的多媒体数据。
对于多媒体播放服务绘制播放窗口的方案可参考现有技术的实现方式，以下结合图3描述本申请中界面渲染的详细流程。
S301，将第一信息和第二信息发给GPU进程；第一信息和第二信息的定义参考前文描述；
S302，当接收到GPU进程的通知时，根据第一信息在绘图层绘制透明区域。
图3所述的流程可通过渲染引擎执行，不同于现有技术，渲染引擎发送第二信息的对象是GPU进程而不是多媒体数据播放服务；这里用GPU进程指代S202的执行主体。
获得第一信息的途径可以是根据预设参数获得，或者根据服务器下发的业务数据获得，或者根据用户的触发指令获得。例如，在初次渲染界面元素时，可以根据预设参数获得第一信息，或者将服务器下发的业务数据生成渲染树后获得第一信息，也可以是根据用户的触发指令获得第一信息。获得第二信息的途径可以与第一信息相同，也可以是先获得第一信息后，根据第一信息计算得到第二信息。
根据用户的触发指令生成第一信息和第二信息的过程可以参照以下描述：
用户向界面发出触发指令，该触发指令导致界面元素的位移发生改变。界面接收到用户的触发指令后，可以通知渲染引擎重新渲染，触发 指令可以是按键消息、触摸消息等，触发指令中可以包括“上”、“下”、“左”、“右”以及移动距离等信息。界面通知渲染引擎的方式可以是从触发指令中获取与界面元素的位置变化有关的信息(例如代表方向的参数、偏移量的参数等)发给渲染引擎。渲染引擎可以根据界面元素的位置变化信息计算出界面元素新的位置信息，新的位置信息中包括第一信息和第二信息。
第一信息或第二信息可以与设置其他界面元素的位置信息一同发送，需要发送的各种信息可以通过一条消息发送，也可以通过不同的消息发送，具体可根据设计需求确定，本申请不做限制。作为示例，可以通过扩展接口将第一信息和第二信息通知渲染线程；由渲染线程将所述第一信息和第二信息分别通过不同的命令发给GPU进程，以触发GPU进程执行S202阶段的动作。
本申请中，界面滚动可以是在多媒体数据播放时，用户可能会对应用的界面进行某种操作，导致界面元素发生位移(例如用户触发滚动、翻页等动作)。当透明区域所在的界面上界面元素发生位移时，需要重新进行界面排版，透明区域的位置和多媒体数据的播放窗口的位置可能均需要相应调整。本申请中所指的界面不局限于应用的界面、操作系统的桌面等。多媒体数据可以是视频数据、3D动画、游戏数据等。
图3a-图3c列举了用户指令触发发生几种界面滚动现象的应用场景，值得注意，本申请所适用的应用场景不局限于所列举的实例。
图3a描述了浏览器的界面滚动的场景。浏览器界面中的透明区域是在绘制浏览器的界面元素时，由浏览器的渲染引擎所绘制；播放窗口是由多媒体数据播放服务在另一个绘图层绘制。由于透明区域和播放窗口重叠，因此用户看到的是播放窗口的呈现效果。浏览器的界面中其他区域显示有其他的界面元素(图中的文字块、图片块等)。用户通过界面上的滚动条发出触发指令，浏览器300在收到触发指令后，通知浏览器的渲染引擎对界面重新排版。
图3b中，在渲染IPTV的操作系统的界面时，通过操作系统的渲染引擎绘制透明区域，通过多媒体数据播放服务绘制播放窗口；播放窗 口和透明区域布满整个界面区域。用户通过遥控器发出上移、下移或翻页等触发指令，IPTV的界面收到触发指令后，通知操作系统的渲染引擎对界面重新排版。
图3c中，平板电脑的桌面上放置有游戏App，游戏App的播放窗口重叠于桌面上的透明区域。用户通过手指左右滑动屏幕发出触发指令，使桌面元素的位置发生改变，桌面响应用户的触发指令，通知操作系统的渲染引擎对桌面元素重新排版。
对于其他可播放多媒体的设备的场景描述不再一一列举。
以下结合图4描述一个应用实例，当用户触发界面滚动时播放多媒体数据的时序图。通信的实体包括负责对应用的界面400进行渲染的渲染引擎401、GPU进程402、多媒体数据播放服务403，显示服务404、渲染引擎中实现绘图功能的绘图模块405。
用户向界面400发出触发指令，触发指令中携带有界面元素的移动方向和偏移量信息。
界面400响应触发指令，生成渲染请求发给渲染引擎401，渲染请求中携带有界面元素的移动方向和偏移量信息；
渲染引擎401计算获得界面元素新的位置信息，其中包括第一信息，并根据第一信息获得第二信息(第一/第二信息的定义参考上文描述)，并通过渲染线程以两条指令分别发给GPU进程402；
GPU进程402将界面元素新的位置信息(包括第一信息)携带在一条绘图通知中发给绘图模块405，将第二信息携带在另一条绘图通知中发给多媒体数据播放服务403。
绘图模块405绘制界面元素时根据第一信息绘制透明区域的新位置，多媒体播放服务403将第二信息通知给该服务所对应的绘制模块，绘制播放窗口。
GPU进程403向显示服务404发送同步显示透明窗口和多媒体数据播放的通知，显示服务404获取绘制结果，呈现于界面，并将下一帧待显示的多媒体播放数据显示在播放窗口。
与前述播放多媒体数据的方法和界面渲染的方法的实施例相对应，本申请还提供了播放多媒体数据的装置和界面渲染的装置的实施例。
本申请播放多媒体数据的装置或界面渲染的装置的实施例可以应用在图1的设备100上，设备100的种类可参考上文描述。装置实施例可以通过软件实现，也可以通过硬件或者软硬件结合的方式实现。以软件实现为例，作为一个逻辑意义上的装置，是通过其所在设备100的处理器将非易失性存储器中对应的计算机程序指令读取到内存中运行形成的。从硬件层面而言，如图5所示，为本申请播放多媒体数据的装置或界面渲染的装置所在设备100的一种硬件结构图，除了图5所示的处理器、内存、网络接口、以及非易失性存储器、显示存储器之外，实施例中装置所在的设备100通常根据该智能设备的实际功能，还可以包括其他硬件，例如GPU等对此不再赘述。
一个实施例中，设备100作为播放多媒体数据的设备，可以将播放多媒体数据的装置和界面渲染的装置装载于该设备中，另外，多媒体播放服务的可执行指令也可以被装载于设备100中。此时设备100的处理器可以被配置为：
发送第一信息和第二信息；第一信息和第二信息的定义可以参见前文描述；
根据第一信息通知在第一绘图层上绘制透明区域；根据第二信息通知在第二绘图层上绘制多媒体数据的播放窗口，多媒体数据的播放窗口与透明区域可以重叠；
当收到通知后，根据第一信息绘制所述透明区域，根据第二信息绘制所述播放窗口；
通知同步显示所述透明区域和所述播放窗口，以及通知在播放窗口中播放所述多媒体数据。
请参考图6，播放多媒体数据的装置600，包括：
绘图控制模块601，用于通知在第一绘图层上绘制透明区域；通知在第二绘图层上绘制多媒体数据的播放窗口，所述多媒体数据的播放窗 口与所述透明区域重叠；
显示控制模块602，用于通知同步显示所述透明区域和所述播放窗口，以及通知在所述播放窗口中播放所述多媒体数据。
装置600可以位于图像处理单元GPU进程中。
在一个实施例中，装置600还可以包括：
接收模块(图未示出)，用于接收第一信息和第二信息，并将所述第一信息和第二信息通知所述绘图控制模块601，所述第一信息用于指示绘制所述透明区域，所述第二信息用于指示绘制所述播放窗口。接收模块在初次渲染界面元素时或用户发出触发指令时，从渲染线程接收所述第一信息和所述第二信息；所述触发指令包括用户的滚动指令或翻页指令。
在另一个实施例中，装置600还包括：
接收模块(图未示出)，用于接收第一信息，所述第一信息用于指示绘制所述透明区域；以及
计算模块(图未示出)，用于根据所述第一信息获得第二信息，所述第二信息用于指示绘制所述播放窗口。
绘图控制模块601通知在第二绘图层上绘制多媒体数据的播放窗口包括：
绘图控制模块601通知多媒体数据播放服务，多媒体数据播放服务根据第二信息在第二绘图层绘制多媒体数据的播放窗口；第二信息用于指示绘制所述播放窗口。
多媒体数据可以包括视频数据、3D动画数据、游戏数据。
请参考图7，界面渲染的装置700包括：
通知模块701，用于将第一信息和第二信息发给图像处理单元GPU进程；所述第一信息用于指示绘制透明区域，所述第二信息用于指示绘制多媒体数据的播放窗口；
绘图模块702，用于当接收到所述GPU进程的通知时，根据所述 第一信息在绘图层绘制所述透明区域。
另外，还可以包括：
计算模块(图中未示出)，用于根据在初次渲染界面元素时的预设参数或用户的触发指令获得所述第一信息；所述用户的触发指令包括滚动指令或翻页指令。
计算模块还可以用于根据所述第一信息获得所述第二信息。
通知模块701将所述第一信息和所述第二信息发给所述GPU进程可以包括：
通过扩展接口将所述第一信息和所述第二信息通知渲染线程；
所述渲染线程将所述第一信息和所述第二信息分别通知所述GPU进程。
上述装置中各个单元的功能和作用的实现过程具体详见上述方法中对应步骤的实现过程，在此不再赘述。
对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本申请方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。

Claims (23)

  1. 一种播放多媒体数据的方法，其特征在于，所述方法包括步骤：
    通知在第一绘图层上绘制透明区域；通知在第二绘图层上绘制多媒体数据的播放窗口，所述多媒体数据的播放窗口与所述透明区域重叠；
    通知同步显示所述透明区域和所述播放窗口，以及通知在所述播放窗口中播放所述多媒体数据。
  2. 根据权利要求1所述的方法，其特征在于，所述方法运行于图像处理单元GPU进程中。
  3. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    接收第一信息和第二信息，所述第一信息用于指示绘制所述透明区域，所述第二信息用于指示绘制所述播放窗口。
  4. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    接收第一信息，所述第一信息用于指示绘制所述透明区域；
    根据所述第一信息获得第二信息，所述第二信息用于指示绘制所述播放窗口。
  5. 根据权利要求3所述的方法，其特征在于，接收第一信息和第二信息包括：
    在初次渲染界面元素时或用户发出触发指令时，从渲染线程接收所述第一信息和所述第二信息；所述触发指令包括用户的滚动指令或翻页指令。
  6. 根据权利要求1所述的方法，其特征在于，通知在第二绘图层上绘制多媒体数据的播放窗口包括：
    图像处理单元GPU进程通知多媒体数据播放服务，所述多媒体数据播放服务根据第二信息在第二绘图层绘制多媒体数据的播放窗口；所述第二信息用于指示绘制所述播放窗口。
  7. 根据权利要求1所述的方法，其特征在于，所述多媒体数据包括视频数据、3D动画数据、游戏数据。
  8. 一种界面渲染的方法，其特征在于，包括步骤：
    将第一信息和第二信息发给图像处理单元GPU进程；所述第一信息用于指示绘制透明区域，所述第二信息用于指示绘制多媒体数据的播放窗口；
    当接收到所述GPU进程的通知时，根据所述第一信息在绘图层绘制所述透明区域。
  9. 根据权利要求8所述的方法，其特征在于，所述方法还包括步骤：
    根据在初次渲染界面元素时的预设参数或用户的触发指令获得所述第一信息；所述用户的触发指令包括滚动指令或翻页指令。
  10. 根据权利要求8所述的方法，其特征在于，所述方法还包括步骤：根据所述第一信息获得所述第二信息。
  11. 根据权利要求8所述的方法，其特征在于，将所述第一信息和所述第二信息发给所述GPU进程包括：
    通过扩展接口将所述第一信息和所述第二信息通知渲染线程；
    所述渲染线程将所述第一信息和所述第二信息分别通知所述GPU进程。
  12. 一种播放多媒体数据的装置，其特征在于，包括：
    绘图控制模块，用于通知在第一绘图层上绘制透明区域；通知在第二绘图层上绘制多媒体数据的播放窗口，所述多媒体数据的播放窗口与所述透明区域重叠；
    显示控制模块，用于通知同步显示所述透明区域和所述播放窗口，以及通知在所述播放窗口中播放所述多媒体数据。
  13. 根据权利要求12所述的装置，其特征在于，所述装置位于图像处理单元GPU进程中。
  14. 根据权利要求12所述的装置，其特征在于，所述装置还包括：
    接收模块，用于接收第一信息和第二信息，并将所述第一信息和第 二信息通知所述绘图控制模块，所述第一信息用于指示绘制所述透明区域，所述第二信息用于指示绘制所述播放窗口。
  15. 根据权利要求12所述的装置，其特征在于，所述装置还包括：
    接收模块，用于接收第一信息，所述第一信息用于指示绘制所述透明区域；
    计算模块，用于根据所述第一信息获得第二信息，所述第二信息用于指示绘制所述播放窗口。
  16. 根据权利要求14所述的装置，其特征在于，接收模块接收第一信息和第二信息包括：
    在初次渲染界面元素时或用户发出触发指令时，从渲染线程接收所述第一信息和所述第二信息；所述触发指令包括用户的滚动指令或翻页指令。
  17. 根据权利要求12所述的装置，其特征在于，所述绘图控制模块通知在第二绘图层上绘制多媒体数据的播放窗口包括：
    所述绘图控制模块通知多媒体数据播放服务，所述多媒体数据播放服务根据第二信息在第二绘图层绘制多媒体数据的播放窗口；所述第二信息用于指示绘制所述播放窗口。
  18. 根据权利要求12所述的装置，其特征在于，所述多媒体数据包括视频数据、3D动画数据、游戏数据。
  19. 一种界面渲染的装置，其特征在于，包括：
    通知模块，用于将第一信息和第二信息发给图像处理单元GPU进程；所述第一信息用于指示绘制透明区域，所述第二信息用于指示绘制多媒体数据的播放窗口；
    绘图模块，用于当接收到所述GPU进程的通知时，根据所述第一信息在绘图层绘制所述透明区域。
  20. 根据权利要求19所述的装置，其特征在于，还包括：
    计算模块，用于根据在初次渲染界面元素时的预设参数或用户的触 发指令获得所述第一信息；所述用户的触发指令包括滚动指令或翻页指令。
  21. 根据权利要求20所述的装置，其特征在于，所述计算模块还用于根据所述第一信息获得所述第二信息。
  22. 根据权利要求19所述的装置，其特征在于，所述通知模块将所述第一信息和所述第二信息发给所述GPU进程包括：
    通过扩展接口将所述第一信息和所述第二信息通知渲染线程；
    所述渲染线程将所述第一信息和所述第二信息分别通知所述GPU进程。
  23. 一种播放多媒体数据的设备，其特征在于，包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为：
    发送第一信息和第二信息；所述第一信息用于指示绘制透明区域，所述第二信息用于指示绘制多媒体数据的播放窗口；
    根据所述第一信息通知在第一绘图层上绘制透明区域；根据所述第二信息通知在第二绘图层上绘制多媒体数据的播放窗口，所述多媒体数据的播放窗口与所述透明区域重叠；
    当收到所述通知后，根据所述第一信息绘制所述透明区域，根据所述第二信息绘制所述播放窗口；
    通知同步显示所述透明区域和所述播放窗口，以及通知在所述播放窗口中播放所述多媒体数据。
PCT/CN2017/071716 2016-02-01 2017-01-19 播放多媒体数据的方法、界面渲染方法及装置、设备 WO2017133475A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/044,088 US10957285B2 (en) 2016-02-01 2018-07-24 Method and system for playing multimedia data

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610070464.9A CN107025100A (zh) 2016-02-01 2016-02-01 播放多媒体数据的方法、界面渲染方法及装置、设备
CN201610070464.9 2016-02-01

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/044,088 Continuation-In-Part US10957285B2 (en) 2016-02-01 2018-07-24 Method and system for playing multimedia data

Publications (1)

Publication Number Publication Date
WO2017133475A1 true WO2017133475A1 (zh) 2017-08-10

Family

ID=59500574

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/071716 WO2017133475A1 (zh) 2016-02-01 2017-01-19 播放多媒体数据的方法、界面渲染方法及装置、设备

Country Status (3)

Country Link
US (1) US10957285B2 (zh)
CN (1) CN107025100A (zh)
WO (1) WO2017133475A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109523609A (zh) * 2018-10-16 2019-03-26 华为技术有限公司 一种内容编辑的方法及终端
CN112035771A (zh) * 2019-06-03 2020-12-04 阿里巴巴集团控股有限公司 基于web的摄像头数据绘制方法、装置及电子设备

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112581557A (zh) * 2019-09-30 2021-03-30 Oppo广东移动通信有限公司 图层绘制方法及电子设备
CN113126855B (zh) * 2021-05-14 2023-05-12 广州虎牙科技有限公司 客户端交互方法、装置、电子设备和存储介质
CN116166259A (zh) * 2021-11-25 2023-05-26 华为技术有限公司 界面生成方法及电子设备
CN114466235B (zh) * 2021-12-31 2023-09-15 福建星网视易信息系统有限公司 一种播控设备及其控制灯效的方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8065614B2 (en) * 2003-04-09 2011-11-22 Ati Technologies, Inc. System for displaying video and method thereof
CN102739983A (zh) * 2011-04-11 2012-10-17 腾讯科技（深圳）有限公司 一种实现半透明效果的方法及系统
CN104685460A (zh) * 2012-08-01 2015-06-03 三星电子株式会社 透明显示装置及其显示方法
CN104954848A (zh) * 2015-05-12 2015-09-30 乐视致新电子科技（天津）有限公司 智能终端的显示图形用户界面的控制方法及装置

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE69917489T2 (de) * 1998-11-09 2005-06-02 Broadcom Corp., Irvine Anzeigesystem zur mischung von graphischen daten und videodaten
US20030170006A1 (en) 2002-03-08 2003-09-11 Bogda Peter B. Versatile video player
US7876333B2 (en) * 2006-03-30 2011-01-25 Smart Technologies Ulc Method and graphical interface for embedding animated content into a computer application
WO2010127043A1 (en) 2009-04-28 2010-11-04 Rey Flemings System and method for annotating multimedia objects
US20150310833A1 (en) 2011-05-24 2015-10-29 Intel Corporation Displaying Hardware Accelerated Video on X Window Systems
CN102665130B (zh) * 2012-05-14 2014-10-15 青岛海信传媒网络技术有限公司 应用程序窗口与视频播放窗口的显示方法及装置
EP2883193A4 (en) * 2012-08-10 2016-07-13 Transaxy Inc SYSTEM FOR ENTERING DATA IN A DATA PROCESSING SYSTEM
CN104615418B (zh) * 2014-07-25 2019-04-16 腾讯科技（北京）有限公司 一种可滑动播放窗口的实现方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8065614B2 (en) * 2003-04-09 2011-11-22 Ati Technologies, Inc. System for displaying video and method thereof
CN102739983A (zh) * 2011-04-11 2012-10-17 腾讯科技（深圳）有限公司 一种实现半透明效果的方法及系统
CN104685460A (zh) * 2012-08-01 2015-06-03 三星电子株式会社 透明显示装置及其显示方法
CN104954848A (zh) * 2015-05-12 2015-09-30 乐视致新电子科技（天津）有限公司 智能终端的显示图形用户界面的控制方法及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109523609A (zh) * 2018-10-16 2019-03-26 华为技术有限公司 一种内容编辑的方法及终端
CN112035771A (zh) * 2019-06-03 2020-12-04 阿里巴巴集团控股有限公司 基于web的摄像头数据绘制方法、装置及电子设备

Also Published As

Publication number Publication date
US20190012987A1 (en) 2019-01-10
US10957285B2 (en) 2021-03-23
CN107025100A (zh) 2017-08-08

Similar Documents

Publication Publication Date Title
WO2017133475A1 (zh) 播放多媒体数据的方法、界面渲染方法及装置、设备
KR102649254B1 (ko) 디스플레이 제어 방법, 저장 매체 및 전자 장치
US11941762B2 (en) System and method for augmented reality scenes
WO2022052772A1 (zh) 多窗口投屏场景下的应用界面显示方法及电子设备
WO2019047748A9 (zh) 分屏显示的方法、装置、终端及存储介质
US9996220B2 (en) Multi-zone interface switching method and device
US20130346865A1 (en) Dynamic wallpaper of mobile systems
US10209945B2 (en) Information processing apparatus capable of quickly updating a display in accordance with an operation for changing a display appearance and control method thereof
US20180349001A1 (en) Automatic Home Screen Determination Based on Display Device
WO2016095429A1 (zh) 一种移动终端屏幕截图方法及装置
KR20160061133A (ko) 이미지 표시 방법 및 그 전자 장치
CN108027707B (zh) 用户终端设备、电子设备以及控制用户终端设备和电子设备的方法
KR102498815B1 (ko) 전자 장치 및 전자 장치의 비디오 편집 방법
WO2018010440A1 (zh) 一种投影画面调整方法、装置和投影终端
WO2020073334A1 (zh) 扩展内容显示方法、装置、系统及存储介质
US11024257B2 (en) Android platform based display device and image display method thereof
CN113282262B (zh) 投屏显示画面的控制方法、装置、移动终端及存储介质
US9495064B2 (en) Information processing method and electronic device
US20140229823A1 (en) Display apparatus and control method thereof
KR20140086979A (ko) 인터넷 브라우저-기반 원격 사용자 인터페이스 가상 마우스 커서 위치결정 방법
US20210067476A1 (en) Intelligent chat object sending animations
WO2019105062A1 (zh) 一种内容显示方法、装置和终端设备
EP3048524B1 (en) Document display support device, terminal, document display method, and computer-readable storage medium for computer program
KR102223554B1 (ko) 단말기, 이의 제어 방법 및 상기 방법을 구현하기 위한 프로그램을 기록한 기록 매체
WO2016033934A1 (zh) 一种投影方法、投影装置及计算机存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17746795

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17746795

Country of ref document: EP

Kind code of ref document: A1