WO2018121557A1 - 直播数据显示方法、装置、程序及介质 - Google Patents

直播数据显示方法、装置、程序及介质 Download PDF

Info

Publication number
WO2018121557A1
WO2018121557A1 PCT/CN2017/118840 CN2017118840W WO2018121557A1 WO 2018121557 A1 WO2018121557 A1 WO 2018121557A1 CN 2017118840 W CN2017118840 W CN 2017118840W WO 2018121557 A1 WO2018121557 A1 WO 2018121557A1
Authority
WO
WIPO (PCT)
Prior art keywords
application
data
display
display image
live broadcast
Prior art date
Application number
PCT/CN2017/118840
Other languages
English (en)
French (fr)
Inventor
葛山
董晶阳
Original Assignee
北京奇虎科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京奇虎科技有限公司 filed Critical 北京奇虎科技有限公司
Publication of WO2018121557A1 publication Critical patent/WO2018121557A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
    • H04N21/4756—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data for rating content, e.g. scoring a recommended movie
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting

Definitions

  • the present invention relates to the field of computer technologies, and in particular, to a live data display method, a live broadcast data display device, a computer program, and a computer readable medium.
  • the webcasting is a multimedia interactive mode in which the live broadcast video is used by the signal collecting device, and then the live video is uploaded to the server, and then fed back to the user terminal by the server, and the multimedia interactive mode also supports the user to interact with the anchor of the webcast. That is, the user's message can be fed back to the anchor, so that the anchor interacts based on the message.
  • the anchor broadcasts the live video added with the interactive message, which may cause the network bandwidth to be occupied and affect the normal upload of the live video.
  • the present invention has been made in order to provide a live broadcast data display method and corresponding live broadcast data display device, computer program and computer readable medium that overcome the above problems or at least partially solve the above problems.
  • a live data display method which includes: acquiring live broadcast return information in a process of performing live broadcast based on a target application, wherein the live broadcast return information is used to feed back information other than video data.
  • the live broadcast return information includes the barrage data; the first display image corresponding to the live broadcast return information is drawn, and is superimposed on the second display image corresponding to the target application for display, wherein the second display image is used for generating Live video data.
  • the first display image corresponding to the live broadcast return information is drawn and superimposed on the second display image corresponding to the target application for display, including: generating a sub-window and displaying the live broadcast in the sub-window drawing sequence Returning a first display image of the information; setting the sub-window to display on the second display image corresponding to the target application.
  • the first display image corresponding to the live broadcast return information is drawn and superimposed on the second display image corresponding to the target application for display, including: generating a transparent window and drawing a dynamic display live return on the transparent window a first display image of the information; superimposing the transparent window onto the second display image corresponding to the target application for display.
  • the first display image corresponding to the live broadcast return information is generated, including: calling the target interface according to the application mapping architecture data, and drawing a first display image corresponding to the live broadcast return information, where the application mapping architecture The data is determined based on the application source framework data of the target application.
  • superimposing on the target application corresponding to the second display image for display comprises: acquiring a second display image created based on the application source frame data, and superimposing and displaying the first display image and the second display image.
  • the first display image corresponding to the live broadcast return information is displayed and superimposed on the second display image corresponding to the target application for display, including: calling the target interface according to the application mapping architecture data, and drawing the target The superimposed image data corresponding to the first display image corresponding to the second display image and the live broadcast return information is applied.
  • the method further includes: initializing the application mapping architecture data and the application source architecture data by initializing the target application by using the suspended target object; determining at least one target interface according to the application mapping architecture data.
  • the creating, by the suspended target object, the application mapping architecture data and the application source architecture data in the initialization target application including: monitoring a creation function in the system, and suspending when the creation function creates the target object The target object; when the target object initializes the target application, determines the mapping object and the interface call according to the application information, builds the application mapping architecture data, determines the source object and the interface call according to the application information, and builds the application source architecture. data.
  • the method further includes: when the object is created, creating a mapping object according to the application information and pointing to the source memory address, and then creating the source object and pointing to the source memory address; when the interface is invoked, corresponding to the application information A set of hook programs is set at each interface, and the interface information of each interface is determined.
  • determining the at least one target interface according to the application mapping architecture data includes: determining, according to the application mapping schema data, at least one target interface for drawing a display image according to the interface information.
  • the method further includes: generating live video data according to the second display image, and uploading the live video data.
  • the live broadcast return information further includes: service data and/or non-public information.
  • a live data display device which specifically includes: an obtaining module, configured to acquire live broadcast return information during a live broadcast based on a target application, where the live broadcast return information is used for feedback other than video
  • the information other than the data, the live broadcast return information includes the barrage data
  • the drawing display module is configured to draw the first display image corresponding to the live broadcast return information, and superimposed on the second display image corresponding to the target application for display
  • the second display image is used to generate live video data.
  • the drawing display module includes: a drawing module, configured to generate a sub-window and display a first display image of the live broadcast return information in the sub-window drawing sequence; a display module, configured to display the sub-window The setting is performed on the second display image corresponding to the target application.
  • the drawing display module includes: a drawing module, configured to generate a transparent window, and draw a first display image that dynamically displays live broadcast return information on the transparent window; and a display module, configured to superimpose the transparent window Displaying is performed on the second display image corresponding to the target application.
  • the drawing display module includes: a drawing module, configured to invoke the target interface according to the application mapping architecture data, and draw a first display image corresponding to the live broadcast return information, where the application mapping architecture data is Determined according to the application source framework data of the target application.
  • the drawing display module further includes: a display module, configured to acquire a second display image created based on the application source frame data, and superimpose and display the first display image and the second display image.
  • a display module configured to acquire a second display image created based on the application source frame data, and superimpose and display the first display image and the second display image.
  • the method further includes: an initialization module, configured to create application mapping architecture data and application source architecture data in the initialization target application by using the suspended target object in advance; and determining at least one target interface according to the application mapping architecture data.
  • an initialization module configured to create application mapping architecture data and application source architecture data in the initialization target application by using the suspended target object in advance; and determining at least one target interface according to the application mapping architecture data.
  • the initialization module includes: a suspending module, configured to monitor a creation function in the system, suspending the target object when the creation function creates a target object; and an architecture building module, configured to be in the target object
  • the target application is initialized to create an application, and the mapping object and the interface call are determined according to the application information, the application mapping architecture data is set up, and the source object and the interface call are determined according to the application information, and the application source architecture data is constructed.
  • the architecture building module is further configured to: when the object is created, create a mapping object according to the application information and point to a source memory address, and then create a source object and point to the source memory address; when the interface is invoked, according to the The application information sets a set of hook programs at corresponding interfaces, and determines interface information of each interface.
  • the initialization module further includes: an interface analysis module, configured to determine, according to the application mapping architecture data, at least one target interface for drawing a display image according to the interface information.
  • an interface analysis module configured to determine, according to the application mapping architecture data, at least one target interface for drawing a display image according to the interface information.
  • the method further includes: a live broadcast module, configured to generate live video data according to the second display image, and upload the live video data.
  • a live broadcast module configured to generate live video data according to the second display image, and upload the live video data.
  • the live broadcast return information further includes: service data and/or non-public information.
  • a computer program comprising computer readable code, when the readable code is run on a computing device, causes the computing device to perform any of the embodiments of the present invention
  • Embodiments of the present invention also provide a computer readable medium storing a program as described in an embodiment of the present invention.
  • the live broadcast return information that does not carry the video data may be obtained, where the live broadcast return information includes the barrage data, and the first display image corresponding to the live broadcast return information is drawn.
  • the first display image is superimposed on the second display image corresponding to the target application for display, so that the interactive message obtained by the anchor acquisition facilitates interaction with the viewing user, and the live broadcast return information does not carry live video data, which can effectively reduce bandwidth resources. Occupied to ensure the stability of the live broadcast.
  • FIG. 1 is a flow chart showing the steps of an embodiment of a live data display method according to an embodiment of the present invention
  • FIG. 2 is a flow chart showing the steps of an embodiment of a live broadcast data display method according to another embodiment of the present invention.
  • FIG. 3 is a flow chart showing the steps of another embodiment of a live broadcast data display method according to another embodiment of the present invention.
  • FIG. 4 is a flow chart showing the steps of another embodiment of a live data display method according to another embodiment of the present invention.
  • FIG. 5 is a flow chart showing the steps of an embodiment of a drawing display method based on an application mapping architecture according to another embodiment of the present invention
  • FIG. 6 is a block diagram showing the structure of an embodiment of a live broadcast data display device according to an embodiment of the present invention.
  • FIG. 7 is a structural block diagram of an embodiment of a live broadcast data display device according to another embodiment of the present invention.
  • Figure 8 shows a block diagram of a computing device for performing a live data display method in accordance with the present invention
  • Figure 9 illustrates a storage unit for holding or carrying program code that implements live data display in accordance with the present invention.
  • FIG. 1 is a flow chart showing the steps of an embodiment of a live data display method according to an embodiment of the present invention. Specifically, the method may include the following steps:
  • Step 102 Acquire live broadcast return information in the process of performing live broadcast based on the target application, where the live broadcast return information is used to feed back information other than video data, where the live broadcast return information includes barrage data.
  • Step 104 Draw a first display image corresponding to the live broadcast return information, and superimpose it on the second display image corresponding to the target application for display, wherein the second display image is used to generate live video data.
  • the live broadcast of the present embodiment is based on the live broadcast of the target application, that is, the data of the target application is used as the live broadcast data.
  • the live broadcast of the game is a process in which the user uses the game application to play the game, and the data of the corresponding image of the game application is used as the live broadcast data.
  • the anchor is the user who performs the live broadcast, such as the game player in the game live broadcast, that is, the anchor.
  • the user can send the barrage data, or send a gift to the host broadcast, etc., wherein the barrage refers to a large number of display modes in the video that float through the screen in the form of subtitle pop-ups.
  • the live broadcast return information may be obtained during the live broadcast based on the target application.
  • the live broadcast information of the embodiment is used to feed back information other than the video data, that is, the live broadcast information is an interactive message that does not carry live video data, and may include barrage data, and may also include Gifts and other data.
  • the interface and the window displayed in the system can be regarded as one frame image, so the interactive message image, that is, the first display image can be drawn based on the live broadcast return information, so that the first display image
  • the interactive message can be displayed, and the corresponding display interface and the display content of the target application are referred to as a second display image, and the target application can be displayed in full screen, so that the first display image can be superimposed on the second display image for display. That is, each frame image displayed on the screen is generated by superimposing the first display image and the second display image drawn in the frame, thereby obtaining the image by drawing the first display image and the second display image for each frame.
  • the live broadcast return information that does not carry the video data may be acquired, where the live broadcast return information includes the barrage data, and the first display corresponding to the live broadcast return information is drawn. And displaying the first display image on the second display image corresponding to the target application for display, so that the interactive message obtained by the anchor acquisition is convenient for interaction with the viewing user, and the live broadcast return information does not carry the live video data, which can effectively reduce the The occupation of bandwidth resources ensures the stability of live broadcast.
  • the live broadcast return information includes: barrage data, business data, and/or non-public information, wherein the service data is determined according to a specific service, for example, the game service is business data as a gift and other electronic item data, and the non-public information is sent to the anchor.
  • the service data is determined according to a specific service, for example, the game service is business data as a gift and other electronic item data, and the non-public information is sent to the anchor.
  • Private information, permission issues and other data, the permission question can be the question data after the purchase permission.
  • the first display image generated according to the live broadcast return information may be displayed in various manners for convenient anchor viewing.
  • One way is to use a separate sub-window display.
  • FIG. 2 a flow chart of a method for displaying a live data display method according to another embodiment of the present invention is shown, which may specifically include the following steps:
  • Step 202 Generate live video data according to the second display image of the target application, and upload the live video data.
  • Step 204 Acquire live broadcast return information during the live broadcast based on the target application.
  • Step 206 Generate a child window and display a first display image of the live broadcast return information in the sub-window drawing order.
  • Step 208 Set the sub-window to display on the second display image corresponding to the target application.
  • the display data of the target application is used as live data, wherein the display data includes a display interface and an interface content, and the second display image corresponding to each frame is drawn according to the data of the target application, and the display data of the target application is formed by the second display image of each frame, thereby The second display image of each frame is obtained and combined according to time stamps to form corresponding live data, thereby realizing live broadcast of the display screen of the target application.
  • the user can send live broadcast information such as barrage data, private message data, and service data, and the user terminal of the anchor can receive the live broadcast return information.
  • the live return information is then processed and displayed.
  • a sub-window For displaying the live return information in the sub-window order, a sub-window may be generated, that is, a sub-window is drawn at the preset position, and the live broadcast return information is sequentially displayed in the drawn sub-window, thereby obtaining the first display image.
  • the first display image of each frame is drawn in the above manner to form a live display content of the dynamic display, such as live broadcast barrage data.
  • a second display image based on the target application, and superimposing the drawn sub-window on the preset position of the second display image for each frame image of the screen, thereby performing drawing by each frame image Display of screen images and generation of live data.
  • Another way is to use a transparent sub-window display.
  • FIG. 3 a flow chart of steps of another embodiment of a live broadcast data display method according to another embodiment of the present invention is shown. Specifically, the method may include the following steps:
  • Step 302 Generate live video data according to the second display image of the target application, and upload the live video data.
  • Step 304 Acquire live broadcast return information during the live broadcast based on the target application.
  • Step 306 Generate a transparent window and draw a first display image of the dynamic display live return information on the transparent window.
  • Step 308 superimposing the transparent window on the second display image corresponding to the target application for display.
  • the display data of the target application is used as live data, wherein the display data includes a display interface and an interface content, and the second display image corresponding to each frame is drawn according to the data of the target application, and the display data of the target application is formed by the second display image of each frame, thereby The second display image of each frame is obtained and combined according to time stamps to form corresponding live data, thereby realizing live broadcast of the display screen of the target application.
  • the user can send live broadcast information such as barrage data, private message data, and service data, and the user terminal of the anchor can receive the live broadcast return information.
  • the live return information is processed and displayed, and a similar viewing user mode can be displayed, for example, the barrage data is displayed by using a barrage method.
  • the live return information can be displayed through a transparent window (or transparent mask). That is, a transparent window is generated, that is, a transparent window is drawn at a preset position, and the live broadcast return information is sequentially displayed in the drawn transparent window, thereby obtaining a first display image.
  • the first display image of each frame is drawn in the above manner to form a live display content of the dynamic display, such as live broadcast barrage data. And correspondingly drawing a second display image based on the target application, and superimposing the drawn sub-window on the preset position of the second display image for each frame image of the screen, thereby performing drawing by each frame image Display of screen images and generation of live data.
  • FIG. 4 a flow chart of steps of an embodiment of a live data display method according to another embodiment of the present invention is shown. Specifically, the method may include the following steps:
  • Step 402 Create application mapping architecture data and application source architecture data in the initialization target application by using the suspended target object in advance.
  • the system is monitored in advance, and the system creation function such as the crate function is used.
  • the creation function is called to create the target object
  • the target object can be suspended, that is, the first hook program is set when the target object is initialized, thereby at the source of the target object. Set the hook program.
  • This target object is an important component object of the system and is used to perform the operations required by various applications.
  • the creation, operation, etc. of some applications in the system require the participation of the target object. Therefore, by setting a hook at the source of the target object, each call to the target object can be intercepted to determine the required information.
  • the application needs to invoke the target object to perform the operation, so the application information when the target object is invoked may be intercepted by the first hook program, and the application information is used to indicate the application source architecture data of the target application. That is, when the target application is created, it is required to call a series of interfaces corresponding to the target to acquire data, and create a required function object, etc., through the interface call and the function object, the application source schema data of the target application can be determined, thereby starting and running the target application. Target application.
  • the application first constructs the application mapping architecture data of the target application by using the application information, and then constructs the application source architecture data, that is, determines the required operation according to the application information, first creates a mapping information, and then creates the source information, and the mapping information is
  • the source information points to the same content address, that is, the mapping architecture data is used to construct a shell with the same source data as the application source architecture, but the substance content is still provided by the application source architecture data, so that the memory information is consumed in a small amount, and the target application can be learned.
  • a hook program can be set in various target objects of various systems.
  • a component object model (COM) is taken as an example object, and when a function is initialized, a COM component can be initialized.
  • the first hook program is set for the COM component, so that the hook is set at the source of the COM component to implement the takeover of the COM component.
  • the COM component is a software component technology of Microsoft's interaction between the web server and the client. It is an object-oriented programming model that defines how the object behaves within a single application or between multiple applications. .
  • COM is implemented on multiple platforms, not limited to the Windows operating system.
  • the game client uses DirectX (Direct eXtension, DX) 3D engine, which is a multimedia programming interface created by Microsoft Corporation, implemented by the C++ programming language, following COM components. Therefore, when the game client is started to run, when the 3D engine operation is involved, the COM component needs to be called, and the application information of the calling COM component can be obtained through the first hook engine set in the COM component, thereby determining the 3D engine of the game client.
  • the source architecture data that is, the various interfaces and function objects required for the 3D engine to run.
  • Step 404 Determine at least one target interface according to the application mapping architecture data.
  • the application mapping architecture data When the application mapping architecture data is set up, various interfaces called by the target application and functions of each interface can be determined, so that at least one target interface required can be determined, and the target interface includes, for drawing, rendering, Various interfaces for displaying images.
  • Step 406 Perform live broadcast based on the target application, and obtain live broadcast return information.
  • Step 408 The target interface is invoked according to the application mapping architecture data, and the first display image corresponding to the live broadcast return information is drawn.
  • Step 410 Acquire a second display image created based on the application source frame data, and superimpose and display the first display image and the second display image.
  • the full screen mode can be adopted.
  • the display interface of the target application that is, the interface image
  • the interface and window displayed in the system can be regarded as one frame of image.
  • the first display image is drawn based on the development technology principle of the target application.
  • the live data can be generated based on the second display image drawn by the application source frame data, and after the live return information is obtained, the pre-injected hook program can be used to call the target interface to draw, render, display, etc. according to the application mapping architecture data. And acquiring, by the first display image corresponding to the live broadcast information, a second display image created based on the application source frame data, and superimposing the first display image on the second display image, that is, displaying the second display image of the target application. Based on the superimposed first display image for display.
  • the second display image of the game client created by the application source frame data may be used, and the live data is generated based on the second display image; Applying the mapping architecture data to invoke the target interface, drawing a first display image corresponding to the live broadcast return information, and superimposing the first display image and the second display image, so that the user appears to be in a full-screen game, displaying the live broadcast Data such as barrage, gifts, and private messages enhance user experience and the flexibility of live streaming.
  • the mapping architecture data to invoke the target interface, drawing a first display image corresponding to the live broadcast return information, and superimposing the first display image and the second display image, so that the user appears to be in a full-screen game, displaying the live broadcast Data such as barrage, gifts, and private messages enhance user experience and the flexibility of live streaming.
  • Step 412 Call the target interface according to the application mapping architecture data, and draw the superimposed image data corresponding to the second display image and the live broadcast return information corresponding to the first display image.
  • the second display image and the first display image are respectively displayed in a superimposed manner.
  • the target interface may be invoked according to the application mapping architecture data, and the superimposed image data is directly drawn, that is, the second display image is directly drawn and The superimposed image data in which the first display image has been superimposed. Therefore, in the subsequent live broadcast process, the live broadcast data based on the superimposed image data can directly return the live broadcast data of the data such as the barrage and the gift, and the data of the live broadcast data is not superimposed on the service side.
  • At least one target interface can be determined, and the target interface is used to draw image data based on the application mapping schema data, which can be based on the target application.
  • the implementation of the architecture principle is more suitable for applications, reducing the impact on live broadcast and application operations, and improving the user experience.
  • the image rendering described above is performed based on the architectural principle of the target application. Therefore, the architecture of the target application and the interfaces, functions, and the like under the architecture may be determined in advance, and then the image rendering is provided in the full screen mode.
  • the architecture of the target application can be determined.
  • the game client is used as a target application, and the game client can adopt a DX 3D engine.
  • the 3D engine is based on a hardware graphics processor (GPU) acceleration in the Windows operating system, directly from the memory. Read and write can circumvent the message mechanism.
  • GPU hardware graphics processor
  • FIG. 5 a flow chart of steps of an embodiment of a method for drawing and displaying based on an application mapping architecture according to another embodiment of the present invention is shown, which may specifically include the following steps:
  • Step 502 Monitor a creation function in the system, and suspend the target object when the creation function creates a target object.
  • the system is monitored in advance, and the system creation function such as the crate function is used.
  • the creation function is called to create the target object
  • the target object can be suspended, that is, the first hook program is set when the target object is initialized, thereby at the source of the target object. Set the hook program.
  • This target object is an important component object of the system and is used to perform the operations required by various applications.
  • a hook program can be set in various target objects of various systems.
  • a component object model (COM) is taken as an example object, and when a function is initialized, a COM component can be initialized.
  • the first hook program is set for the COM component, so that the hook is set at the source of the COM component to implement the takeover of the COM component.
  • COM component object model
  • Step 504 Intercept the application information when the target object is created by calling the target object.
  • the creation, operation, etc. of some applications in the system require the participation of the target object. Therefore, by setting a hook at the source of the target object, each call to the target object can be intercepted to determine the required information.
  • the application needs to invoke the target object to perform the operation, so the application information when the target object is invoked may be intercepted by the first hook program, and the application information is used to indicate the application source architecture data of the target application. That is, when the target application is created, it is required to call a series of interfaces corresponding to the target to acquire data, and create a required function object, etc., through the interface call and the function object, the application source schema data of the target application can be determined, thereby starting and running the target application. Target application.
  • the game client uses DirectX (Direct eXtension, DX) 3D engine, which is a multimedia programming interface created by Microsoft Corporation, implemented by the C++ programming language, following COM components. Therefore, when the game client is started to run, when the 3D engine operation is involved, the COM component needs to be called, and the application information of the calling COM component can be obtained through the first hook engine set in the COM component, thereby determining the 3D engine of the game client.
  • the source architecture data that is, the various interfaces and function objects required for the 3D engine to run.
  • Step 506 When creating an object, create a mapping object according to the application information and point to a source memory address, and then create a source object and point to the source memory address.
  • Step 508 respectively setting a hook program on each interface corresponding to each source object.
  • Step 510 Build application mapping architecture data according to the mapping object and the interface call.
  • Step 512 Build the application source architecture data according to the source object and the interface call.
  • the information about the interface to be called may be created according to the determined object.
  • the mapping architecture data is used to construct a shell with the same application source architecture data, but The substance is still provided by the application source schema data. Therefore, when the application information needs to create an object, it is possible to determine related information of the object to be created, create a mapping object to point to the corresponding source memory address, and then create a source object and point to the source memory address.
  • the mapping object and the interface call are used to build the application mapping architecture data, and the source object and the interface call are used to construct the application source architecture data, thereby obtaining the same shell as the application source architecture data, that is, the application mapping architecture data, and the application mapping architecture data is actually defined.
  • the called content can correspond to the application source schema data, and can also be mapped to the application source schema data for processing.
  • the call information is intercepted, for example, the call information indicates that the function A is created, the function A calls the interface B and the function C, and the function C calls the interfaces D and E, and can create the mapping function A' and C', and at the interfaces B, D, E respectively set the hook program and establish the correspondence relationship of the mapping function, and then create the source functions A and C and the corresponding relationship with the interfaces B, D, E, thereby building a game client
  • the shell with the same source schema data maps the schema data through which the schema schema data can be mapped to the source schema data.
  • Step 514 Determine, according to the application mapping architecture data, at least one target interface for drawing a display image according to the interface information.
  • various interfaces called by the target application and functions of each interface can be determined, so that at least one target interface required can be determined, and the target interface includes, for drawing, rendering, Various interfaces for displaying images.
  • a series of target interfaces related to the image can be determined, and each target interface is injected with a hook program, so that the target interface can be directly called through the hook program.
  • the application information can be intercepted during the target application at the time of initialization, and the application mapping architecture data and the application source architecture data of the target application are constructed, that is, the architecture of the target application is analyzed, so that the subsequent operations based on the architecture can be performed.
  • Step 516 Perform live broadcast based on the target application, and obtain live broadcast return information.
  • Step 518 The target interface is invoked according to the application mapping architecture data, and the first display image corresponding to the live broadcast return information is drawn.
  • Step 520 Acquire a second display image created based on the application source frame data, and superimpose and display the first display image and the second display image.
  • Step 522 Call the target interface according to the application mapping architecture data, and draw the superimposed image data corresponding to the second display image and the live broadcast return information corresponding to the first display image.
  • the application mapping architecture data and the application source architecture data are set up when the target application is initialized in the terminal, and at least one target interface is determined. Then complete initialization to start the target application, and the target application enters full screen mode after startup. In this mode, the application mapping schema data can be used to invoke the application source schema data for response.
  • the live broadcast data is generated according to the second display image of the target application, and the live broadcast return information may be received during the live broadcast process, and then the first display image is drawn and rendered based on the broadcast return information by calling the target interface based on the architectural principle determined by the application mapping architecture data.
  • the drawing data may also be determined based on the live broadcast return information and the data of the target application, and the target interface is called to render and render the superimposed image data of the first display image and the second display image based on the drawing data.
  • the corresponding source architecture data and the mapping schema data are determined according to the target interface required by the game client to draw, render, and display the image.
  • the user plays the game live during the game playing.
  • the first display image can be drawn based on the image rendering principle of the 3D engine and the target interface is called, and the first image is superimposed on the game client. Display on the interface image, or directly draw the superimposed image instead of the game client display interface image.
  • the interface image of the target application that is, the second display image may be drawn and displayed every frame, and the second display image that responds to the live broadcast return information is also superimposed and displayed on the second display image after each frame is drawn. Therefore, even if the user angle image is still, the corresponding image is drawn every frame at the system angle.
  • the displayed game interface image is drawn and rendered every frame, so the first display image displayed on the game interface image is also superimposed on the game interface image of the corresponding position after each frame is rendered and rendered. to show.
  • the communication mechanism of the system (such as the message mechanism of Windows) can be bypassed, so that the display and manipulation of the live return information can be performed normally without exiting the full screen, and the user is guaranteed. Improve the user experience based on normal operation.
  • the embodiment of the present invention further provides a live data display device.
  • FIG. 6 is a structural block diagram of an embodiment of a live broadcast data display device according to an embodiment of the present invention, which may specifically include the following modules:
  • the obtaining module 602 is configured to obtain live broadcast return information during the live broadcast based on the target application, where the live broadcast return information is used to feed back information other than the video data, where the live broadcast return information includes the barrage data.
  • a drawing display module 604 configured to display a first display image corresponding to the live broadcast return information, and superimposed on the second display image corresponding to the target application for display, wherein the second display image is used to generate live video data .
  • the live broadcast return information that does not carry the video data may be obtained, where the live broadcast return information includes the barrage data, and the first display corresponding to the live broadcast return information is drawn. And displaying the first display image on the second display image corresponding to the target application for display, so that the interactive message obtained by the anchor acquisition is convenient for interaction with the viewing user, and the live broadcast return information does not carry the live video data, which can effectively reduce the The occupation of bandwidth resources ensures the stability of live broadcast.
  • FIG. 7 is a structural block diagram of an embodiment of a live broadcast data display device according to another embodiment of the present invention, which may specifically include the following modules:
  • the initialization module 606 is configured to create application mapping architecture data and application source architecture data in the initialization target application by using the suspended target object in advance; and determining at least one target interface according to the application mapping architecture data.
  • the obtaining module 602 is configured to obtain live broadcast return information during the live broadcast based on the target application, where the live broadcast return information is used to feed back information other than the video data, where the live broadcast return information includes the barrage data.
  • a drawing display module 604 configured to display a first display image corresponding to the live broadcast return information, and superimposed on the second display image corresponding to the target application for display, wherein the second display image is used to generate live video data .
  • the live broadcast module 608 is configured to generate live video data according to the second display image, and upload the live video data.
  • the live broadcast return information further includes: service data and/or non-public information.
  • the drawing display module 604 includes: a drawing module 6042 and a display module 6044.
  • the drawing module 6042 is configured to generate a child window and display a first display image of the live broadcast return information in the sub-window drawing order; a display module 6044, configured to set the sub-window in the The target application is displayed on the second display image.
  • the drawing module 6042 is configured to generate a transparent window and draw a first display image that dynamically displays live broadcast return information on the transparent window; a display module 6044 is configured to superimpose the transparent window to the The target application is displayed on the second display image.
  • the drawing module 6042 is configured to invoke the target interface according to the application mapping architecture data, and draw a first display image corresponding to the live broadcast return information, where the application mapping architecture data is according to the The target application's application source framework data is determined.
  • the display module 6044 is configured to acquire a second display image created based on the application source frame data, and display the first display image and the second display image in a superimposed manner.
  • the drawing display module 604 is configured to invoke the target interface according to the application mapping architecture data, and draw the superimposed image data corresponding to the second display image and the live broadcast return information corresponding to the first display image.
  • the initialization module 606 includes:
  • the suspending module 6062 is configured to monitor a creation function in the system, and suspend the target object when the creation function creates a target object.
  • the architecture building module 6064 is configured to create an application in the target object initialization target application, determine a mapping object and an interface call according to the application information, build an application mapping architecture data, and then determine a source object and an interface call according to the application information, and build Apply source schema data.
  • the architecture building module 6064 is further configured to: when creating an object, create a mapping object according to the application information and point to a source memory address, and then create a source object and point to the source memory address; when the interface is invoked, according to the application The information sets a set of hook programs at the corresponding interfaces, and determines the interface information of each interface.
  • the initialization module 606 further includes an interface analysis module 6066, configured to determine, according to the application mapping architecture data, at least one target interface for drawing a display image according to the interface information.
  • the communication mechanism of the system (such as the message mechanism of Windows) can be bypassed, so that the display and manipulation of the live return information can be performed normally without exiting the full screen, and the user is guaranteed. Improve the user experience based on normal operation.
  • the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.
  • modules in the devices of the embodiments can be adaptively changed and placed in one or more devices different from the embodiment.
  • the modules or units or components of the embodiments may be combined into one module or unit or component, and further they may be divided into a plurality of sub-modules or sub-units or sub-components.
  • any combination of the features disclosed in the specification, including the accompanying claims, the abstract and the drawings, and any methods so disclosed, or All processes or units of the device are combined.
  • Each feature disclosed in this specification (including the accompanying claims, the abstract and the drawings) may be replaced by alternative features that provide the same, equivalent or similar purpose.
  • the various component embodiments of the present invention may be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some of the live data display method and a live data display device device according to an embodiment of the present invention or Some or all of the features of all components.
  • DSP digital signal processor
  • the invention can also be implemented as a device or device program (e.g., a computer program and a computer program product) for performing some or all of the methods described herein.
  • Such a program implementing the invention may be stored on a computer readable medium or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
  • Figure 8 illustrates a computing device that can implement a video push method implemented using a parasitic toolkit in accordance with the present invention.
  • the computing device conventionally includes a processor 810 and a program product or readable medium in the form of a memory 820.
  • Memory 820 can be an electronic memory such as a flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, or ROM.
  • Memory 820 has a memory space 830 for program code 831 for performing any of the method steps described above.
  • storage space 830 for program code may include various program code 831 for implementing various steps in the above methods, respectively.
  • These program codes can be read from or written to one or more program products.
  • These program products include program code carriers such as memory cards.
  • Such a program product is typically a portable or fixed storage unit as described with reference to FIG.
  • the storage unit may have storage segments, storage spaces, and the like that are similar to the storage 820 in the computing device of FIG.
  • the program code can be compressed, for example, in an appropriate form.
  • the storage unit includes readable code 831', ie, code readable by a processor, such as 810, that when executed by a computing device causes the computing device to perform various steps in the methods described above .

Abstract

一种直播数据显示方法、装置、程序及介质，所述方法包括：基于目标应用进行直播的过程中，获取直播返回信息，其中，所述直播返回信息用于反馈除视频数据之外的信息，所述直播返回信息包括弹幕数据；绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，其中，所述第二显示图像用于生成直播视频数据；从而便于主播获取返回的互动消息与观看用户进行互动，并且直播返回信息不携带直播视频数据，能够有效减少对带宽资源的占用，保证直播的稳定性。

Description

直播数据显示方法、装置、程序及介质 技术领域
本发明涉及计算机技术领域，特别是涉及一种直播数据显示方法、一种直播数据显示装置、一种计算机程序及一种计算机可读介质。
背景技术
随着网络和终端技术的发展，用户可以通过终端执行各种网络操作，如上网、玩游戏、观看视频等，因此也兴起了一种新的多媒体互动方式，即网络直播。
网络直播是通过信号采集设备采用直播视频，然后将直播视频上传到服务器，再由服务器反馈给各用户终端的播放的多媒体互动方式，该多媒体互动方式还支持用户和该网络直播的主播进行互动，即用户的消息可以反馈给主播，从而主播基于该消息进行互动。
但是，在网络直播过程中主播除了上传直播视频外，还要下载添加了互动消息的直播视频，这可能导致网络带宽被占用，影响直播视频的正常上传。
发明内容
鉴于上述问题，提出了本发明以便提供一种克服上述问题或者至少部分地解决上述问题的直播数据显示方法和相应的直播数据显示装置、计算机程序及计算机可读介质。
依据本发明的一个方面，提供了直播数据显示方法，具体包括：在基于目标应用进行直播的过程中，获取直播返回信息，其中，所述直播返回信息用于反馈除视频数据之外的信息，所述直播返回信息包括弹幕数据；绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，其中，所述第二显示图像用于生成直播视频数据。
可选地，绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，包括：生成子窗口并在所述子窗口绘制顺序展示所述直播返回信息的第一显示图像；将所述子窗口设置在所述目标应用对应第二显示图像上进行显示。
可选地，绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，包括：生成透明窗口并在所述透明窗口上绘制动态展示直播返回信息的第一显示图像；将所述透明窗口叠加到所述目标应用对应第二显示图像上进行显示。
可选地，绘制所述直播返回信息对应的第一显示图像，包括：依据应用映射架构数据调用所述目标接口，绘制所述直播返回信息对应的第一显示图像，其中，所述应用映射架构数据是依据所述目标应用的应用源框架数据确定的。
可选地，叠加到所述目标应用对应第二显示图像上进行显示，包括：获取基于应用源框架数据创建的第二显示图像，将所述第一显示图像和第二显示图像进行叠加显示。
可选地，绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，包括：依据应用映射架构数据调用所述目标接口，绘制所述目标应用对应第二显示图像和所述直播返回信息对应第一显示图像的叠加图像数据。
可选地，还包括：预先通过挂起的目标对象，在初始化所述目标应用创建应用映射架构数据和应用源架构数据；依据所述应用映射架构数据确定至少一个目标接口。
可选地，所述预先通过挂起的目标对象，在初始化目标应用创建应用映射架构数据和应用源架构数据，包括：对系统中创建函数进行监控，在所述创建函数创建目标对象时挂起所述目标对象；在所述目标对象初始化目标应用时，依据所述应用信息确定映射对象和接口调用，搭建应用映射架构数据，再依据所述应用信息确定源对象和接口调用，搭建应用源架构数据。
可选地，还包括：在创建对象时，依据所述应用信息创建映射对象并指向源内存地址，再创建源对象并指向所述源内存地址；在调用接口时，依据所述应用信息在对应各接口处设置一组钩子程序，并确定各接口的接口信息。
可选地，依据所述应用映射架构数据确定至少一个目标接口，包括：基于所述应用映射架构数据，依据所述接口信息确定用于绘制显示图像的至少一个目标接口。
可选地，还包括：依据所述第二显示图像生成直播视频数据，上传所述直播视频数据。
可选地，所述直播返回信息还包括：业务数据和/或非公开信息。
根据本发明的另一方面，提供了直播数据显示装置，具体包括：获取模块，用于在基于目标应用进行直播的过程中，获取直播返回信息，其中，所述直播返回信息用于反馈除视频数据之外的信息，所述直播返回信息包括弹幕数据；绘制显示模块，用于绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，其中，所述第二显示 图像用于生成直播视频数据。
可选地，所述绘制显示模块，包括：绘制模块，用于生成子窗口并在所述子窗口绘制顺序展示所述直播返回信息的第一显示图像；显示模块，用于将所述子窗口设置在所述目标应用对应第二显示图像上进行显示。
可选地，所述绘制显示模块，包括：绘制模块，用于生成透明窗口并在所述透明窗口上绘制动态展示直播返回信息的第一显示图像；显示模块，用于将所述透明窗口叠加到所述目标应用对应第二显示图像上进行显示。
可选地，所述绘制显示模块，包括：绘制模块，用于依据应用映射架构数据调用所述目标接口，绘制所述直播返回信息对应的第一显示图像，其中，所述应用映射架构数据是依据所述目标应用的应用源框架数据确定的。
可选地，所述绘制显示模块，还包括：显示模块，用于获取基于应用源框架数据创建的第二显示图像，将所述第一显示图像和第二显示图像进行叠加显示。
可选地，还包括：初始化模块，用于预先通过挂起的目标对象，在初始化目标应用创建应用映射架构数据和应用源架构数据；依据所述应用映射架构数据确定至少一个目标接口。
可选地，初始化模块，包括：挂起模块，用于对系统中创建函数进行监控，在所述创建函数创建目标对象时挂起所述目标对象；架构搭建模块，用于在所述目标对象初始化目标应用创建应用，依据所述应用信息确定映射对象和接口调用，搭建应用映射架构数据，再依据所述应用信息确定源对象和接口调用，搭建应用源架构数据。
可选地，架构搭建模块，还用于在创建对象时，依据所述应用信息创建映射对象并指向源内存地址，再创建源对象并指向所述源内存地址；在调用接口时，依据所述应用信息在对应各接口处设置一组钩子程序，并确定各接口的接口信息。
可选地，初始化模块，还包括：接口分析模块，用于基于所述应用映射架构数据，依据所述接口信息确定用于绘制显示图像的至少一个目标接口。
可选地，还包括：直播模块，用于依据所述第二显示图像生成直播视频数据，上传所述直播视频数据。
可选地，所述直播返回信息还包括：业务数据和/或非公开信息。
根据本发明的又一方面，提供了一种计算机程序，包括计算机可读代码，当所述可读代码在计算设备上运行时，导致所述计算设备执行根据本发明实施例中的任一所述的直播数据显示方法。
本发明实施例还提供了一种计算机可读介质，其中存储了如本发明实施 例所述的程序。
在采用目标应用的第二显示图像生成直播视频数据进行直播时，可以获取不携带视频数据的直播返回信息，该直播返回信息包括弹幕数据，绘制该直播返回信息对应的第一显示图像，将第一显示图像叠加到所述目标应用对应第二显示图像上进行显示，从而便于主播获取返回的互动消息与观看用户进行互动，并且直播返回信息不携带直播视频数据，能够有效减少对带宽资源的占用，保证直播的稳定性。
上述说明仅是本发明技术方案的概述，为了能够更清楚了解本发明的技术手段，而可依照说明书的内容予以实施，并且为了让本发明的上述和其它目的、特征和优点能够更明显易懂，以下特举本发明的具体实施方式。
附图说明
通过阅读下文优选实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的，而并不认为是对本发明的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：
图1示出了根据本发明一个实施例的一种直播数据显示方法实施例的步骤流程图；
图2示出了根据本发明另一个实施例的一种直播数据显示方法实施例的步骤流程图；
图3示出了根据本发明另一个实施例的另一种直播数据显示方法实施例的步骤流程图；
图4示出了根据本发明另一个实施例的又一种直播数据显示方法实施例的步骤流程图；
图5示出了根据本发明另一个实施例提供的基于应用映射架构的绘制显示方法实施例的步骤流程图；
图6示出了根据本发明一个实施例的一种直播数据显示装置实施例的结构框图；
图7示出了根据本发明另一个实施例的一种直播数据显示装置实施例的结构框图；
图8示出了用于执行根据本发明直播数据显示方法的计算设备的框图；
图9示出了用于保持或者携带实现根据本发明的直播数据显示的程序代码的存储单元。
具体实施方式
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。
参照图1，示出了根据本发明一个实施例的一种直播数据显示方法实施例的步骤流程图，具体可以包括如下步骤：
步骤102，在基于目标应用进行直播的过程中，获取直播返回信息，其中，所述直播返回信息用于反馈除视频数据之外的信息，所述直播返回信息包括弹幕数据。
步骤104，绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，其中，所述第二显示图像用于生成直播视频数据。
本实施例的直播是基于目标应用的直播，即将目标应用的数据作为直播数据，例如游戏直播是用户在使用游戏应用玩游戏的过程中，将游戏应用对应画面的数据作为直播数据。直播过程中观看直播的用户和主播可以进行互动，主播即进行该直播的用户，如游戏直播中游戏玩家即主播。用户可以发送弹幕数据，也可以给主播送礼物等，其中，弹幕指的是视频中大量以字幕弹出形式从屏幕飘过的显示方式。
本实施例应用于直播过程的主播端，为了便于主播与用户互动，在基于目标应用进行直播的过程中，可以获取直播返回信息。为了减少对主播带宽的占用，本实施例的直播返回信息用于反馈除视频数据之外的信息，即该直播返回信息为不携带直播视频数据的互动消息，可以包括弹幕数据，也可以包括礼物等数据。
其中，在系统底层以及屏幕显示的角度，在系统中显示的界面、窗口都可以看作是一帧图像，因此可以基于该直播返回信息绘制互动消息图像即第一显示图像，从而第一显示图像能够显示互动消息，将目标应用对应显示界面及显示内容称为第二显示图像，目标应用可以全屏显示，因此可以将第一显示图像叠加到第二显示图像上进行显示。即在屏幕上显示的每一帧图像都是在该帧绘制的第一显示图像和第二显示图像叠加生成的，从而通过对每一帧第一显示图像和第二显示图像的绘制，得到在屏幕上动态显示的图像。由于第二显示图像是目标应用对应显示界面及显示内容，因此可以获取各帧对应第二显示图像，依据各帧的时间戳等信息生成直播视频数据。
综上，在采用目标应用的第二显示图像生成直播视频数据进行直播时， 可以获取不携带视频数据的直播返回信息，该直播返回信息包括弹幕数据，绘制该直播返回信息对应的第一显示图像，将第一显示图像叠加到所述目标应用对应第二显示图像上进行显示，从而便于主播获取返回的互动消息与观看用户进行互动，并且直播返回信息不携带直播视频数据，能够有效减少对带宽资源的占用，保证直播的稳定性。
其中，直播返回信息包括：弹幕数据、业务数据和/或非公开信息，其中，业务数据依据具体业务确定，例如游戏业务则业务数据为礼物等电子物品数据，非公开信息包括给主播发送的私信、权限问题等数据，该权限问题可以是购买的权限后的提问数据。
本实施例在直播互动过程中，仅给主播返回互动消息而不携带视频数据，从而减少对带宽的占用。其中，依据直播返回信息生成的第一显示图像可以通过各种方式显示，便于主播观看。
一种方式是采用独立的子窗口显示。
参照图2，示出了根据本发明另一个实施例的一种直播数据显示方法实施例的步骤流程图，具体可以包括如下步骤：
步骤202，依据目标应用的第二显示图像生成直播视频数据，上传所述直播视频数据。
步骤204，在基于目标应用进行直播的过程中，获取直播返回信息。
步骤206，生成子窗口并在所述子窗口绘制顺序展示所述直播返回信息的第一显示图像。
步骤208，将所述子窗口设置在所述目标应用对应第二显示图像上进行显示。
将目标应用的显示数据作为直播数据，其中，显示数据包括显示界面和界面内容，依据目标应用的数据绘制各帧对应第二显示图像，通过各帧第二显示图像构成目标应用的显示数据，从而获取每一帧第二显示图像并按照时间戳进行组合，构成相应的直播数据，从而实现对目标应用的显示画面的直播。在基于目标应用进行直播的过程中，用户可以发送弹幕数据、私信数据以及业务数据等直播返回信息，主播的用户终端可以接收直播返回信息。然后对直播返回信息进行处理并显示。其中，针对采用子窗口顺序显示直播返回信息的方式，可以生成子窗口即在预设位置绘制一子窗口，并在绘制的子窗口内顺序展示所述直播返回信息，从而得到第一显示图像。对每帧第一显示图像采用上述方式绘制，构成动态显示的直播显示内容，如直播的弹幕数据。在该过程中基于目标应用对应绘制第二显示图像，针对屏幕的每一帧图像，将该绘制的子窗口叠加到所述第二显示图像的预设位置处，从而通过各 帧图像的绘制进行屏幕图像的显示以及直播数据的生成。
另一种方式是采用透明子窗口显示。
参照图3，示出了根据本发明另一个实施例的另一种直播数据显示方法实施例的步骤流程图，具体可以包括如下步骤：
步骤302，依据目标应用的第二显示图像生成直播视频数据，上传所述直播视频数据。
步骤304，在基于目标应用进行直播的过程中，获取直播返回信息。
步骤306，生成透明窗口并在所述透明窗口上绘制动态展示直播返回信息的第一显示图像。
步骤308，将所述透明窗口叠加到所述目标应用对应第二显示图像上进行显示。
将目标应用的显示数据作为直播数据，其中，显示数据包括显示界面和界面内容，依据目标应用的数据绘制各帧对应第二显示图像，通过各帧第二显示图像构成目标应用的显示数据，从而获取每一帧第二显示图像并按照时间戳进行组合，构成相应的直播数据，从而实现对目标应用的显示画面的直播。在基于目标应用进行直播的过程中，用户可以发送弹幕数据、私信数据以及业务数据等直播返回信息，主播的用户终端可以接收直播返回信息。然后对直播返回信息进行处理并显示，其中可以显示类似观看用户方式，如弹幕数据采用弹幕方式显示。则可以通过透明窗口(或透明遮罩)显示直播返回信息。即生成透明窗口即在预设位置绘制一透明窗口，并在绘制的透明窗口内顺序展示所述直播返回信息，从而得到第一显示图像。对每帧第一显示图像采用上述方式绘制，构成动态显示的直播显示内容，如直播的弹幕数据。在该过程中基于目标应用对应绘制第二显示图像，针对屏幕的每一帧图像，将该绘制的子窗口叠加到所述第二显示图像的预设位置处，从而通过各帧图像的绘制进行屏幕图像的显示以及直播数据的生成。
又一种方式是基于应用映射架构的绘制显示方式。
参照图4，示出了根据本发明另一个实施例的又一种直播数据显示方法实施例的步骤流程图，具体可以包括如下步骤：
步骤402，预先通过挂起的目标对象，在初始化目标应用创建应用映射架构数据和应用源架构数据。
预先对系统进行监控，监控系统的创建函数如crate函数，当调用该创建函数创建目标对象时，可以挂起该目标对象，即在初始化目标对象时设置第一钩子程序，从而在目标对象的源头设置钩子程序。该目标对象为系统的重要组件对象，用于执行各种应用所需的操作。
系统中部分应用的创建、运行等均需要目标对象的参与，因此在目标对象的源头设置钩子，就可以截获各个针对目标对象的调用，从而确定所需的信息。本实施例中，应用在创建需要调用目标对象执行操作，因此可以通过第一钩子程序截获调用所述目标对象时的应用信息，该应用信息用于指示创建该目标应用的应用源架构数据。即在目标应用创建时需要调用目标对应的一系列接口获取数据，并创建所需的函数对象等，通过该接口调用和函数对象可以确定出该目标应用的应用源架构数据，从而启动并运行该目标应用。
因此本实施例先采用应用信息搭建该目标应用的应用映射架构数据，再搭建应用源架构数据，即依据该应用信息确定所需操作，先创建一个映射信息再创建源信息，且该映射信息与源信息指向相同的内容地址，即采用映射架构数据搭建一个与应用源架构数据相同的外壳，但其实质内容仍然由应用源架构数据提供，从而极少量的消耗内存信息，且能够获知该目标应用的架构，并控制应用所需数据的返回。
本发明实施例中，可以在各种系统的各种目标对象设置钩子程序，本实施例中以组件对象模型(Component Object Model，COM)作为目标对象为例，可以在创建函数初始化COM组件时，为COM组件设置第一钩子程序，从而在COM组件的源头设置钩子，实现对COM组件的接管。其中，COM组件是微软对于网页服务器与客户端之间交互的一项软件组件技术，是一种面向对象的编程模式，它定义了对象在单个应用程序内部或多个应用程序之间的行为方式。并且COM被实现于多个平台之上，并不限于Windows操作系统。例如，游戏客户端采用DirectX(Direct eXtension，DX)的3D引擎，该DirectX是由微软公司创建的多媒体编程接口，由C++编程语言实现，遵循COM组件。因此在启动运行该游戏客户端，涉及到3D引擎操作时，需要调用COM组件，通过在COM组件设置的第一钩子引擎，可以获取调用COM组件的应用信息，从而确定出该游戏客户端的3D引擎的源架构数据，即获知3D引擎运行所需的各种接口以及函数对象等。
步骤404，依据所述应用映射架构数据确定至少一个目标接口。
在搭建应用映射架构数据时可以确定该目标应用所需调用的各种接口，以及各接口的功能等信息，因此可以确定出所需的至少一个目标接口，该目标接口包括用于绘制、渲染、显示图像的各种接口。
步骤406，基于目标应用进行直播，及获取直播返回信息。
步骤408，依据应用映射架构数据调用所述目标接口，绘制所述直播返回信息对应的第一显示图像。
步骤410，获取基于应用源框架数据创建的第二显示图像，将所述第一 显示图像和第二显示图像进行叠加显示。
目标应用运行后可以采用全屏模式，全屏模式下该目标应用的显示界面即界面图像会铺满整个窗口，且焦点位于该目标应用中。其中，在系统底层以及屏幕显示的角度，在系统中显示的界面、窗口都可以看作是一帧图像。本实施例依据上述应用映射架构数据，基于该目标应用的开发技术原理来执行第一显示图像的绘制。
因此可以基于应用源框架数据绘制的第二显示图像生成直播数据，以及获取直播返回信息后，可以采用预先注入的钩子程序调用目标接口依据应用映射架构数据对应图像绘制、渲染、显示等方式，绘制所述直播返回信息对应的第一显示图像，获取基于应用源框架数据创建的第二显示图像，将该第一显示图像叠加到第二显示图像上显示，即在显示目标应用的第二显示图像的基础上，叠加第一显示图像进行显示。
例如针对游戏客户端，用户在玩客户端游戏时通常是采用全屏模式并进行游戏直播，则可以采用应用源框架数据创建的游戏客户端的第二显示图像，基于第二显示图像生成直播数据；依据应用映射架构数据调用所述目标接口，绘制所述直播返回信息对应的第一显示图像，将第一显示图像和第二显示图像进行叠加显示，从而用户看起来就是在全屏游戏下，显示了直播弹幕、礼物以及私信等数据，提高用户体验以及直播的灵活性。
步骤412，依据应用映射架构数据调用所述目标接口，绘制所述目标应用对应第二显示图像和直播返回信息对应第一显示图像的叠加图像数据。
上述采用分别绘制第二显示图像和第一显示图像后叠加显示的方式，实际处理中，还可以依据应用映射架构数据调用所述目标接口，直接绘制叠加图像数据，即直接绘制第二显示图像和第一显示图像已叠加的叠加图像数据。从而后续直播过程中基于叠加图像数据进行直播，可以直接返回已添加弹幕、礼物等数据的直播数据，不用在服务侧为直播数据叠加弹幕等数据。
综上，通过在初始化时创建应用映射架构数据和应用源架构数据确定目标应用的架构，从而可以确定出至少一个目标接口，基于应用映射架构数据调用所述目标接口绘制图像数据，能够基于目标应用的架构原理执行，更加贴合应用，减少对直播以及应用操作的影响，提高用户体验。
本实施例中，上述图像绘制是基于目标应用的架构原理执行的，因此，可以预先确定目标应用的架构以及在架构下的接口、函数等信息，然后再全屏模式下提供图像绘制。
一个可选实施例中，可以确定目标应用的架构。
本实施例以游戏客户端作为目标应用为例，该游戏客户端可以采用DX 的3D引擎，该3D引擎在Windows操作系统中是基于硬件图形处理器(Graphics Processing Unit，GPU)加速，直接从内存读写，可规避消息机制。
参照图5，示出了根据本发明另一个实施例提供的基于应用映射架构的绘制显示方法实施例的步骤流程图，具体可以包括如下步骤：
步骤502，对系统中创建函数进行监控，在所述创建函数创建目标对象时挂起所述目标对象。
预先对系统进行监控，监控系统的创建函数如crate函数，当调用该创建函数创建目标对象时，可以挂起该目标对象，即在初始化目标对象时设置第一钩子程序，从而在目标对象的源头设置钩子程序。该目标对象为系统的重要组件对象，用于执行各种应用所需的操作。
本发明实施例中，可以在各种系统的各种目标对象设置钩子程序，本实施例中以组件对象模型(Component Object Model，COM)作为目标对象为例，可以在创建函数初始化COM组件时，为COM组件设置第一钩子程序，从而在COM组件的源头设置钩子，实现对COM组件的接管。
步骤504，在调用所述目标对象创建目标应用时截获应用信息。
系统中部分应用的创建、运行等均需要目标对象的参与，因此在目标对象的源头设置钩子，就可以截获各个针对目标对象的调用，从而确定所需的信息。本实施例中，应用在创建需要调用目标对象执行操作，因此可以通过第一钩子程序截获调用所述目标对象时的应用信息，该应用信息用于指示创建该目标应用的应用源架构数据。即在目标应用创建时需要调用目标对应的一系列接口获取数据，并创建所需的函数对象等，通过该接口调用和函数对象可以确定出该目标应用的应用源架构数据，从而启动并运行该目标应用。
例如，游戏客户端采用DirectX(Direct eXtension，DX)的3D引擎，该DirectX是由微软公司创建的多媒体编程接口，由C++编程语言实现，遵循COM组件。因此在启动运行该游戏客户端，涉及到3D引擎操作时，需要调用COM组件，通过在COM组件设置的第一钩子引擎，可以获取调用COM组件的应用信息，从而确定出该游戏客户端的3D引擎的源架构数据，即获知3D引擎运行所需的各种接口以及函数对象等。
步骤506，在创建对象时，依据所述应用信息创建映射对象并指向源内存地址，再创建源对象并指向所述源内存地址。
步骤508，在各源对象对应的各接口分别设置钩子程序。
步骤510，依据映射对象和接口调用，搭建应用映射架构数据。
步骤512，依据源对象和接口调用，搭建应用源架构数据。
本实施例中，目标对象创建目标应用相关内容时，可以创建所需对象依 据确定需要调用的接口等信息，本实施例中，采用映射架构数据搭建一个与应用源架构数据相同的外壳，但其实质内容仍然由应用源架构数据提供。因此，可以在该应用信息需要创建对象时，确定所需创建对象的相关信息，创建一个映射对象指向对应的源内存地址，然后创建源对象并指向所述源内存地址。即依据该应用信息确定所需定义的内容，定义一个具有含义的映射对象，再定义原本需要定义的源对象，并且在各源对象所需的接口处分别设置钩子程序，从而针对该目标应用可以设置一组钩子程序，实现在入口级挂钩子的调用方式。从而采用映射对象和接口调用搭建应用映射架构数据，采用源对象和接口调用搭建应用源架构数据，从而得到与该应用源架构数据相同的外壳即应用映射架构数据，而该应用映射架构数据实际定义、调用的内容可以与应用源架构数据对应，还可以映射到应用源架构数据中处理。
例如，游戏客户端的3D引擎调用COM组件时，截获调用信息，例如调用信息指示创建函数A，该函数A调用接口B和函数C，函数C调用接口D、E，可以向创建映射函数A′和C′，并且在接口B、D、E处分别设置钩子程序以及建立映射函数的对应关系，再创建源函数A和C以及与接口B、D、E的对应关系，从而搭建一个与游戏客户端的源架构数据相同的外壳即映射架构数据，通过该映射架构数据可以映射到源架构数据。
步骤514，基于所述应用映射架构数据，依据所述接口信息确定用于绘制显示图像的至少一个目标接口。
在搭建应用映射架构数据时可以确定该目标应用所需调用的各种接口，以及各接口的功能等信息，因此可以确定出所需的至少一个目标接口，该目标接口包括用于绘制、渲染、显示图像的各种接口。即可以确定出与图像相关的一系列目标接口，且各目标接口均注入了钩子程序，从而后续可以直接通过钩子程序调用目标接口。
从而能够在初始化时目标应用时截获应用信息，搭建所述目标应用的应用映射架构数据和应用源架构数据，即分析出了目标应用的架构，便于后续基于该架构执行所需操作。
步骤516，基于目标应用进行直播，获取直播返回信息。
步骤518，依据应用映射架构数据调用所述目标接口，绘制所述直播返回信息对应的第一显示图像。
步骤520，获取基于应用源框架数据创建的第二显示图像，将所述第一显示图像和第二显示图像进行叠加显示。
步骤522，依据应用映射架构数据调用所述目标接口，绘制所述目标应用对应第二显示图像和所述直播返回信息对应第一显示图像的叠加图像数 据。
在终端中初始化目标应用时搭建应用映射架构数据和应用源架构数据，并确定至少一个目标接口。然后完成初始化可以启动该目标应用，目标应用在启动后进入全屏模式。在该模式下可以采用应用映射架构数据调用应用源架构数据进行响应。基于该目标应用的第二显示图像生成直播数据，直播过程中可接收直播返回信息，然后基于应用映射架构数据所确定的架构原理，基于直播返回信息调用目标接口绘制、渲染第一显示图像。并且可以确定出该第一图像的显示位置，在目标应用的第二显示图像对应位置叠加该第一图像，从而实现在第二显示图像上叠加第一显示图像的显示。还可以基于直播返回信息和目标应用的数据确定绘制数据，基于该绘制数据调用目标接口绘制、渲染第一显示图像和第二显示图像的叠加图像数据。
例如通过COM组件初始化游戏客户端时，确定对应源架构数据和映射架构数据，依据该游戏客户端在绘制、渲染、显示图像所需的目标接口。初始化完成后可以运行游戏客户端并进入全屏模式。用户在玩游戏的过程中进行游戏直播，对于直播的弹幕等数据，可以基于3D引擎的图像绘制原理、调用目标接口绘制相应的第一显示图像，将该第一图像叠加到游戏客户端显示界面图像上显示，或直接绘制叠加的图像替代游戏客户端显示界面图像。
实际上在目标应用运行过程中，目标应用的界面图像即第二显示图像可以是每帧都绘制显示，响应直播返回信息的第二显示图像也是每帧绘制后叠加显示在第二显示图像上的，因此即使用户角度图像是静止的，而在系统角度是每帧均绘制相应的图像。以游戏客户端为例，所显示的游戏界面图像是每帧绘制、渲染显示的，因此该游戏界面图像上显示的第一显示图像也是每帧绘制渲染后，叠加到相应位置的游戏界面图像上显示的。
从而通过调用底层窗口直接绘制图像并显示的方式，可绕过系统的通信机制(如Windows的消息机制)，从而对于直播返回信息的显示以及操纵也可正常执行且不会退出全屏，在保证用户正常操作的基础上，提高用户体验。
对于方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明实施例并不受所描述的动作顺序的限制，因为依据本发明实施例，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作并不一定是本发明实施例所必须的。
在上述实施例基础上，本发明实施例还提供了一种直播数据显示装置。
参照图6，示出了根据本发明一个实施例的一种直播数据显示装置实施例的结构框图，具体可以包括如下模块：
获取模块602，用于在基于目标应用进行直播的过程中，获取直播返回信息，其中，所述直播返回信息用于反馈除视频数据之外的信息，所述直播返回信息包括弹幕数据。
绘制显示模块604，用于绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，其中，所述第二显示图像用于生成直播视频数据。
综上，在采用目标应用的第二显示图像生成直播视频数据进行直播时，可以获取不携带视频数据的直播返回信息，该直播返回信息包括弹幕数据，绘制该直播返回信息对应的第一显示图像，将第一显示图像叠加到所述目标应用对应第二显示图像上进行显示，从而便于主播获取返回的互动消息与观看用户进行互动，并且直播返回信息不携带直播视频数据，能够有效减少对带宽资源的占用，保证直播的稳定性。
参照图7，示出了根据本发明另一个实施例的一种直播数据显示装置实施例的结构框图，具体可以包括如下模块：
初始化模块606，用于预先通过挂起的目标对象，在初始化目标应用创建应用映射架构数据和应用源架构数据；依据所述应用映射架构数据确定至少一个目标接口。
获取模块602，用于在基于目标应用进行直播的过程中，获取直播返回信息，其中，所述直播返回信息用于反馈除视频数据之外的信息，所述直播返回信息包括弹幕数据。
绘制显示模块604，用于绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，其中，所述第二显示图像用于生成直播视频数据。
直播模块608，用于依据所述第二显示图像生成直播视频数据，上传所述直播视频数据。所述直播返回信息还包括：业务数据和/或非公开信息。
其中，绘制显示模块604包括：绘制模块6042和显示模块6044。
一个可选实施例中，绘制模块6042用于生成子窗口并在所述子窗口绘制顺序展示所述直播返回信息的第一显示图像；显示模块6044，用于将所述子窗口设置在所述目标应用对应第二显示图像上进行显示。
另一个可选实施例中，绘制模块6042，用于生成透明窗口并在所述透明窗口上绘制动态展示直播返回信息的第一显示图像；显示模块6044，用于将所述透明窗口叠加到所述目标应用对应第二显示图像上进行显示。
又一个可选实施例中，绘制模块6042，用于依据应用映射架构数据调用所述目标接口，绘制所述直播返回信息对应的第一显示图像，其中，所述应 用映射架构数据是依据所述目标应用的应用源框架数据确定的。显示模块6044，用于获取基于应用源框架数据创建的第二显示图像，将所述第一显示图像和第二显示图像进行叠加显示。
其中，绘制显示模块604，用于依据应用映射架构数据调用所述目标接口，绘制所述目标应用对应第二显示图像和所述直播返回信息对应第一显示图像的叠加图像数据。
其中，初始化模块606，包括：
挂起模块6062，用于对系统中创建函数进行监控，在所述创建函数创建目标对象时挂起所述目标对象。
架构搭建模块6064，用于在所述目标对象初始化目标应用创建应用，依据所述应用信息确定映射对象和接口调用，搭建应用映射架构数据，再依据所述应用信息确定源对象和接口调用，搭建应用源架构数据。
其中，架构搭建模块6064，还用于在创建对象时，依据所述应用信息创建映射对象并指向源内存地址，再创建源对象并指向所述源内存地址；在调用接口时，依据所述应用信息在对应各接口处设置一组钩子程序，并确定各接口的接口信息。
初始化模块606，还包括：接口分析模块6066，用于基于所述应用映射架构数据，依据所述接口信息确定用于绘制显示图像的至少一个目标接口。
从而通过调用底层窗口直接绘制图像并显示的方式，可绕过系统的通信机制(如Windows的消息机制)，从而对于直播返回信息的显示以及操纵也可正常执行且不会退出全屏，在保证用户正常操作的基础上，提高用户体验。
对于装置实施例而言，由于其与方法实施例基本相似，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
在此提供的算法和显示不与任何特定计算机、虚拟系统或者其它设备固有相关。各种通用系统也可以与基于在此的示教一起使用。根据上面的描述，构造这类系统所要求的结构是显而易见的。此外，本发明也不针对任何特定编程语言。应当明白，可以利用各种编程语言实现在此描述的本发明的内容，并且上面对特定语言所做的描述是为了披露本发明的最佳实施方式。
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本发明的实施例可以在没有这些具体细节的情况下实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。
类似地，应当理解，为了精简本公开并帮助理解各个发明方面中的一个或多个，在上面对本发明的示例性实施例的描述中，本发明的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开 的方法解释成反映如下意图：即所要求保护的本发明要求比在每个权利要求中所明确记载的特征更多的特征。更确切地说，如下面的权利要求书所反映的那样，发明方面在于少于前面公开的单个实施例的所有特征。因此，遵循具体实施方式的权利要求书由此明确地并入该具体实施方式，其中每个权利要求本身都作为本发明的单独实施例。
本领域那些技术人员可以理解，可以对实施例中的设备中的模块进行自适应性地改变并且把它们设置在与该实施例不同的一个或多个设备中。可以把实施例中的模块或单元或组件组合成一个模块或单元或组件，以及此外可以把它们分成多个子模块或子单元或子组件。除了这样的特征和/或过程或者单元中的至少一些是相互排斥之外，可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述，本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的的替代特征来代替。
此外，本领域的技术人员能够理解，尽管在此所述的一些实施例包括其它实施例中所包括的某些特征而不是其它特征，但是不同实施例特征的组合意味着处于本发明的范围之内并且形成不同的实施例。例如，在下面权利要求书中，所要求保护的实施例的任意之一都可以以任意的组合方式来使用。
本发明的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的软件模块实现，或者以它们的组合实现。本领域的技术人员应当理解，可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据本发明实施例的一种直播数据显示方法和一种直播数据显示装置设备中的一些或者全部部件的一些或者全部功能。本发明还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如，计算机程序和计算机程序产品)。这样的实现本发明的程序可以存储在计算机可读介质上，或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到，或者在载体信号上提供，或者以任何其他形式提供。
例如，图8示出了可以实现根据本发明的利用寄生工具包实现的视频推送方法的计算设备。该计算设备传统上包括处理器810和以存储器820形式的程序产品或者可读介质。存储器820可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM或者ROM之类的电子存储器。存储器820具有用于执行上述方法中的任何方法步骤的程序代码831的存储空间830。例如，用于程序代码的存储空间830可以包括分别用于实现上面的方法中的各种步骤的各个程序代码831。这些程序代码可以从一个或者多个程序产品 中读出或者写入到这一个或者多个程序产品中。这些程序产品包括诸如存储卡之类的程序代码载体。这样的程序产品通常为如参考图9所述的便携式或者固定存储单元。该存储单元可以具有与图8的计算设备中的存储器820类似布置的存储段、存储空间等。程序代码可以例如以适当形式进行压缩。通常，存储单元包括可读代码831’，即可以由例如诸如810之类的处理器读取的代码，这些代码当由计算设备运行时，导致该计算设备执行上面所描述的方法中的各个步骤。
应该注意的是上述实施例对本发明进行说明而不是对本发明进行限制，并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本发明可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。

Claims (26)

  1. 一种直播数据显示方法，包括：
    在基于目标应用进行直播的过程中，获取直播返回信息，其中，所述直播返回信息用于反馈除视频数据之外的信息，所述直播返回信息包括弹幕数据；
    绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，其中，所述第二显示图像用于生成直播视频数据。
  2. 如权利要求1所述的方法，其特征在于，绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，包括：
    生成子窗口并在所述子窗口绘制顺序展示所述直播返回信息的第一显示图像；
    将所述子窗口设置在所述目标应用对应第二显示图像上进行显示。
  3. 如权利要求1所述的方法，其特征在于，绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，包括：
    生成透明窗口并在所述透明窗口上绘制动态展示直播返回信息的第一显示图像；
    将所述透明窗口叠加到所述目标应用对应第二显示图像上进行显示。
  4. 如权利要求1所述的方法，其特征在于，绘制所述直播返回信息对应的第一显示图像，包括：
    依据应用映射架构数据调用所述目标接口，绘制所述直播返回信息对应的第一显示图像，其中，所述应用映射架构数据是依据所述目标应用的应用源框架数据确定的。
  5. 如权利要求4所述的方法，其特征在于，叠加到所述目标应用对应第二显示图像上进行显示，包括：
    获取基于应用源框架数据创建的第二显示图像，将所述第一显示图像和第二显示图像进行叠加显示。
  6. 如权利要求1所述的方法，其特征在于，绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，包括：
    依据应用映射架构数据调用所述目标接口，绘制所述目标应用对应第二显示图像和所述直播返回信息对应第一显示图像的叠加图像数据。
  7. 如权利要求4-6任一所述的方法，其特征在于，还包括：
    预先通过挂起的目标对象，在初始化所述目标应用创建应用映射架构数据和应用源架构数据；
    依据所述应用映射架构数据确定至少一个目标接口。
  8. 如权利要求7所述的方法，其特征在于，所述预先通过挂起的目标对象，在初始化目标应用创建应用映射架构数据和应用源架构数据，包括：
    对系统中创建函数进行监控，在所述创建函数创建目标对象时挂起所述目标对象；
    在所述目标对象初始化目标应用时，依据所述应用信息确定映射对象和接口调用，搭建应用映射架构数据，再依据所述应用信息确定源对象和接口调用，搭建应用源架构数据。
  9. 如权利要求8所述的方法，其特征在于，还包括：
    在创建对象时，依据所述应用信息创建映射对象并指向源内存地址，再创建源对象并指向所述源内存地址；
    在调用接口时，依据所述应用信息在对应各接口处设置一组钩子程序，并确定各接口的接口信息。
  10. 如权利要求9所述的方法，其特征在于，依据所述应用映射架构数据确定至少一个目标接口，包括：
    基于所述应用映射架构数据，依据所述接口信息确定用于绘制显示图像的至少一个目标接口。
  11. 如权利要求1所述的方法，其特征在于，还包括：
    依据所述第二显示图像生成直播视频数据，上传所述直播视频数据。
  12. 如权利要求1-6任一所述的方法，其特征在于，所述直播返回信息还包括：业务数据和/或非公开信息。
  13. 一种直播数据显示装置，包括：
    获取模块，用于在基于目标应用进行直播的过程中，获取直播返回信息，其中，所述直播返回信息用于反馈除视频数据之外的信息，所述直播返回信息包括弹幕数据；
    绘制显示模块，用于绘制所述直播返回信息对应的第一显示图像，并叠加到所述目标应用对应第二显示图像上进行显示，其中，所述第二显示图像用于生成直播视频数据。
  14. 如权利要求13所述的装置，其特征在于，所述绘制显示模块，包括：
    绘制模块，用于生成子窗口并在所述子窗口绘制顺序展示所述直播返回 信息的第一显示图像；
    显示模块，用于将所述子窗口设置在所述目标应用对应第二显示图像上进行显示。
  15. 如权利要求13所述的装置，其特征在于，所述绘制显示模块，包括：
    绘制模块，用于生成透明窗口并在所述透明窗口上绘制动态展示直播返回信息的第一显示图像；
    显示模块，用于将所述透明窗口叠加到所述目标应用对应第二显示图像上进行显示。
  16. 如权利要求13所述的装置，其特征在于，所述绘制显示模块，包括：
    绘制模块，用于依据应用映射架构数据调用所述目标接口，绘制所述直播返回信息对应的第一显示图像，其中，所述应用映射架构数据是依据所述目标应用的应用源框架数据确定的。
  17. 如权利要求16所述的装置，其特征在于，所述绘制显示模块，还包括：
    显示模块，用于获取基于应用源框架数据创建的第二显示图像，将所述第一显示图像和第二显示图像进行叠加显示。
  18. 如权利要求13所述的装置，其特征在于，
    所述绘制显示模块，用于依据应用映射架构数据调用所述目标接口，绘制所述目标应用对应第二显示图像和所述直播返回信息对应第一显示图像的叠加图像数据。
  19. 如权利要求16或18所述的装置，其特征在于，还包括：
    初始化模块，用于预先通过挂起的目标对象，在初始化目标应用创建应用映射架构数据和应用源架构数据；依据所述应用映射架构数据确定至少一个目标接口。
  20. 如权利要求19所述的装置，其特征在于，初始化模块，包括：
    挂起模块，用于对系统中创建函数进行监控，在所述创建函数创建目标对象时挂起所述目标对象；
    架构搭建模块，用于在所述目标对象初始化目标应用创建应用，依据所述应用信息确定映射对象和接口调用，搭建应用映射架构数据，再依据所述应用信息确定源对象和接口调用，搭建应用源架构数据。
  21. 如权利要求20所述的装置，其特征在于，
    架构搭建模块，还用于在创建对象时，依据所述应用信息创建映射对象 并指向源内存地址，再创建源对象并指向所述源内存地址；在调用接口时，依据所述应用信息在对应各接口处设置一组钩子程序，并确定各接口的接口信息。
  22. 如权利要求21所述的装置，其特征在于，初始化模块，还包括：
    接口分析模块，用于基于所述应用映射架构数据，依据所述接口信息确定用于绘制显示图像的至少一个目标接口。
  23. 如权利要求13所述的装置，其特征在于，还包括：
    直播模块，用于依据所述第二显示图像生成直播视频数据，上传所述直播视频数据。
  24. 如权利要求13-18任一所述的方法，其特征在于，所述直播返回信息还包括：业务数据和/或非公开信息。
  25. 一种计算机程序，包括计算机可读代码，当所述可读代码在计算设备上运行时，导致所述计算设备执行根据权利要求1-12中的任一所述的直播数据显示方法。
  26. 一种计算机可读介质，其中存储了如权利要求25所述的程序。
PCT/CN2017/118840 2016-12-27 2017-12-27 直播数据显示方法、装置、程序及介质 WO2018121557A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611229470.0 2016-12-27
CN201611229470.0A CN106713968B (zh) 2016-12-27 2016-12-27 一种直播数据显示方法和装置

Publications (1)

Publication Number Publication Date
WO2018121557A1 true WO2018121557A1 (zh) 2018-07-05

Family

ID=58895498

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/118840 WO2018121557A1 (zh) 2016-12-27 2017-12-27 直播数据显示方法、装置、程序及介质

Country Status (2)

Country Link
CN (1) CN106713968B (zh)
WO (1) WO2018121557A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112911329A (zh) * 2021-02-03 2021-06-04 广州虎牙科技有限公司 窗口直播方法、装置、电子设备和计算机可读存储介质
CN113225606A (zh) * 2021-04-30 2021-08-06 上海哔哩哔哩科技有限公司 视频弹幕处理方法及装置
CN113938753A (zh) * 2021-12-20 2022-01-14 北京搜狐新动力信息技术有限公司 直播数据处理方法、装置、存储介质和设备
CN116170611A (zh) * 2023-02-13 2023-05-26 百果园技术(新加坡)有限公司 基于直播的游戏管理方法、装置、设备、存储介质及产品

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106713968B (zh) * 2016-12-27 2020-04-24 北京奇虎科技有限公司 一种直播数据显示方法和装置
CN110147185B (zh) * 2018-11-16 2021-02-26 腾讯科技（深圳）有限公司 消息提示方法、装置、电子装置及存储介质
CN110536094A (zh) * 2019-08-27 2019-12-03 上海盛付通电子支付服务有限公司 一种在视频通话过程中传递信息的方法与设备

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101078982A (zh) * 2006-05-24 2007-11-28 北京壁虎科技有限公司 基于绘图引擎的屏幕显示方法
CN101227421A (zh) * 2007-01-16 2008-07-23 沃天醒石（北京）科技有限公司 全屏图形模式下的即时通讯方法和装置
CN101500125A (zh) * 2008-02-03 2009-08-05 突触计算机系统（上海）有限公司 用户终端上用于在视频显示时提供用户交互的方法及装置
US20130198774A1 (en) * 2012-01-30 2013-08-01 Consultants Net Creation Inc. Live broadcasting of dynamically generated content
CN105597321A (zh) * 2015-12-18 2016-05-25 武汉斗鱼网络科技有限公司 一种全屏游戏状态下的弹幕显示方法与系统
WO2016154149A1 (en) * 2015-03-20 2016-09-29 Twitter, Inc. Live video stream sharing
CN106162230A (zh) * 2016-07-28 2016-11-23 北京小米移动软件有限公司 直播信息的处理方法、装置、主播端、服务器及系统
CN106713968A (zh) * 2016-12-27 2017-05-24 北京奇虎科技有限公司 一种直播数据显示方法和装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104902318B (zh) * 2015-04-29 2018-09-18 小米科技有限责任公司 播放控制方法及终端设备

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101078982A (zh) * 2006-05-24 2007-11-28 北京壁虎科技有限公司 基于绘图引擎的屏幕显示方法
CN101227421A (zh) * 2007-01-16 2008-07-23 沃天醒石（北京）科技有限公司 全屏图形模式下的即时通讯方法和装置
CN101500125A (zh) * 2008-02-03 2009-08-05 突触计算机系统（上海）有限公司 用户终端上用于在视频显示时提供用户交互的方法及装置
US20130198774A1 (en) * 2012-01-30 2013-08-01 Consultants Net Creation Inc. Live broadcasting of dynamically generated content
WO2016154149A1 (en) * 2015-03-20 2016-09-29 Twitter, Inc. Live video stream sharing
CN105597321A (zh) * 2015-12-18 2016-05-25 武汉斗鱼网络科技有限公司 一种全屏游戏状态下的弹幕显示方法与系统
CN106162230A (zh) * 2016-07-28 2016-11-23 北京小米移动软件有限公司 直播信息的处理方法、装置、主播端、服务器及系统
CN106713968A (zh) * 2016-12-27 2017-05-24 北京奇虎科技有限公司 一种直播数据显示方法和装置

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112911329A (zh) * 2021-02-03 2021-06-04 广州虎牙科技有限公司 窗口直播方法、装置、电子设备和计算机可读存储介质
CN112911329B (zh) * 2021-02-03 2023-08-25 广州虎牙科技有限公司 窗口直播方法、装置、电子设备和计算机可读存储介质
CN113225606A (zh) * 2021-04-30 2021-08-06 上海哔哩哔哩科技有限公司 视频弹幕处理方法及装置
CN113938753A (zh) * 2021-12-20 2022-01-14 北京搜狐新动力信息技术有限公司 直播数据处理方法、装置、存储介质和设备
CN113938753B (zh) * 2021-12-20 2022-04-26 北京搜狐新动力信息技术有限公司 直播数据处理方法、装置、存储介质和设备
CN116170611A (zh) * 2023-02-13 2023-05-26 百果园技术(新加坡)有限公司 基于直播的游戏管理方法、装置、设备、存储介质及产品

Also Published As

Publication number Publication date
CN106713968A (zh) 2017-05-24
CN106713968B (zh) 2020-04-24

Similar Documents

Publication Publication Date Title
WO2018121556A1 (zh) 直播数据处理方法、装置、程序及介质
WO2018121557A1 (zh) 直播数据显示方法、装置、程序及介质
US11617947B2 (en) Video game overlay
CN111729293B (zh) 一种数据处理方法、装置及存储介质
CN109525853B (zh) 直播间封面展示方法、装置、终端、服务器及可读介质
CN109032793B (zh) 资源配置的方法、装置、终端及存储介质
US9442897B2 (en) Screenshot method and device
US9937423B2 (en) Voice overlay
US10972511B2 (en) Streaming relay for digital signage
CN113840154B (zh) 基于虚拟礼物的直播互动方法、系统及计算机设备
CN111476851B (zh) 图像处理方法、装置、电子设备及存储介质
CN115065684B (zh) 数据处理方法、装置、设备以及介质
US20230290043A1 (en) Picture generation method and apparatus, device, and medium
CN105847907B (zh) 更换电视机开机动画的方法和装置
CN108845741B (zh) 一种ar表情的生成方法、客户端、终端和存储介质
WO2016066056A1 (zh) 图像远程投射方法、服务器和客户端
US20190215581A1 (en) A method and system for delivering an interactive video
JP2023055615A (ja) 自然言語処理を使用するゲーム・ログからのイベント情報抽出
CN113609194A (zh) 一种数据处理方法、装置、存储介质及电子设备
CN111107425B (zh) 基于渲染组件元素获取计算资源的方法、系统和存储介质
JP7475610B2 (ja) クラウドネイティブによる３ｄ場面のゲーム方法及びシステム
CN113660500A (zh) 直播房间的展示方法、装置、存储介质及电子设备
CN112019911A (zh) 一种网页动画的显示方法、装置及电子设备
CN117493263A (zh) 多媒体资源的生成方法、装置、计算机设备及存储介质
CN115168759A (zh) 图像数据展示方法、装置、存储介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17889112

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17889112

Country of ref document: EP

Kind code of ref document: A1