CN115175002A

CN115175002A - Video playing method and device

Info

Publication number: CN115175002A
Application number: CN202210746742.3A
Authority: CN
Inventors: 王鹏; 贾玉辉
Original assignee: Hisense Mobile Communications Technology Co Ltd
Current assignee: Hisense Mobile Communications Technology Co Ltd
Priority date: 2022-06-28
Filing date: 2022-06-28
Publication date: 2022-10-11
Anticipated expiration: 2042-06-28
Also published as: CN115175002B

Abstract

The application discloses a video playing method and video playing equipment, which are used for solving the problem that user experience is influenced due to the fact that a browser page is jammed during moving. The method provided by the application comprises the following steps: responding to the starting operation of a tab page of the browser; determining the title of a label page and at least one video address of a video to be played on a video plug-in; acquiring a window handle of a label page according to the title of the label page; acquiring a window handle of a video plug-in, and embedding a window of the video plug-in into a window of a tab page as a child window of the tab page of the browser according to the relationship between the window handle of the tab page and a parent-child window of the window handle of the video plug-in; and displaying the label page, and playing the video to be played corresponding to the video address of at least one video to be played in the window of the video plug-in unit through the video plug-in unit.

Description

Video playing method and device

Technical Field

The present application relates to the field of internet technologies, and in particular, to a video playing method and device.

Background

In the command and dispatch industry, a user does not independently install a client for simple operation, and generally adopts a browser to perform command and dispatch operation and video watching. Currently, a user can use an IE browser to process audio and video by calling a system interface by using an Active X control technology, but the Active X control technology is only supported by a Microsoft browser and does not support other browsers. In the prior art, a video window can be pasted into a browser through a plug-in, but the playing plug-in is floating on the browser, when a user drags or moves the browser window, because the size of a player needs to be calculated in real time, the plug-in cannot move synchronously with the browser page due to calculation delay, so that the browser page moves and is blocked, and the experience of the user is influenced.

Disclosure of Invention

The application discloses a video playing method and video playing equipment, which are used for solving the problem that user experience is influenced due to the fact that a browser page moves and is jammed.

In a first aspect, an embodiment of the present application discloses a video playing method, which is applied to a terminal device, and includes:

responding to the starting operation of a tab page of the browser; determining the title of the label page and the video address of at least one video to be played on the video plug-in; the video plug-in is used for playing a video on a tab page of the browser; acquiring a window handle of the label page according to the title of the label page; acquiring a window handle of the video plug-in, and embedding a window of the video plug-in into a window of a tab page of the browser as a child window of the tab page according to the relationship between the window handle of the tab page and a parent child window of the window handle of the video plug-in; and displaying the label page, and playing the video to be played corresponding to the video address of the at least one video to be played in the window of the video plug-in through the video plug-in.

Based on the scheme, the video plug-in window is embedded into the browser tab window, and any operation of the browser tab window is directly transmitted to the child window through a system layer protocol method. When the browser tab is dragged or zoomed, the position of the video plug-in window in the browser tab window is fixed and unchanged, the position of the plug-in window does not need to be recalculated, and the video plug-in can simultaneously acquire the action of the parent window, so that the video plug-in does not need to communicate with the browser tab when the browser tab is dragged or zoomed, the video plug-in window can synchronously act along with the action of the browser tab window, and the user experience is improved.

In one possible implementation manner, obtaining a window handle of the tab page according to the title of the tab page includes: determining at least one process of the browser according to all processes running in the terminal equipment; determining a process of a browser tab corresponding to the browser tab title in at least one process of the browser; and determining the window handle of the browser tab page according to the process of the browser tab page.

Based on the scheme, the window handle corresponding to the label page title in the terminal equipment can be determined so as to be in a binding relationship with the window handle according to the label page and the window handle of the video plug-in.

In a possible implementation manner, playing, by the video plug-in, a to-be-played video corresponding to a video address of the at least one to-be-played video includes: the video plug-in obtains the coded video to be played through the playing address; the video plug-in performs hard decoding on the coded video to be played through decoding equipment to obtain the decoded video to be played; and the video plug-in plays the decoded video to be played.

By the scheme, the coded video is decoded in a hard decoding mode, multi-thread decoding can be realized, decoding efficiency is improved, and multi-path playing of the video can be supported.

In a possible implementation manner, in the process of playing the decoded video to be played by the video plug-in, the video to be played by the video plug-in is rendered to the tab page through a rendering window, and a processing level of the rendering window is higher than a processing level of a window of the video plug-in.

In one possible implementation manner, the embedding the window of the video plug-in into the window of the tab page as a child window of the tab page of the browser includes: determining the position of the window of the video plug-in the window of the browser; and embedding the window of the video plug-in into the window of the tab page as a sub-window of the tab page of the browser according to the position of the window of the video plug-in the window of the browser.

Based on the scheme, the video plug-in can be embedded into the specified position of the browser tab, and the relative position of the video plug-in the browser tab page is fixed. Therefore, when the browser zooms and drags, the position of the video plug-in the browser tab does not need to be recalculated, the calculation amount is reduced, and the response speed is improved.

In a possible implementation manner, the number of the video addresses of the at least one video to be played is N, where N is an integer greater than 1; playing the video to be played corresponding to the video address of the at least one video to be played in the window of the video plug-in through the video plug-in, wherein the playing comprises the following steps: dividing the window of the video plug-in into M sub-windows; m is a positive integer greater than or equal to N; and playing videos to be played corresponding to the N video addresses respectively in the M sub-windows through the video plug-in.

In a possible implementation manner, when N is an even number, M is equal to N, and the sizes of the N sub-windows are the same; or, when N is an odd number, M is equal to N +1, and the sizes of the N +1 sub-windows are the same; or when N is an odd number, M is equal to N, where the size of one of the M sub-windows is 2 times that of the other sub-windows.

Based on the scheme, the window of one video plug-in can be divided into a plurality of sub-windows, and each sub-window supports playing one path of video to be played, so that the requirement of multi-path playing is met.

In a possible implementation manner, the at least one video to be played is N, where N is a positive integer, and the method further includes: responding to the operation of adding S videos to be played in the browser tab, and acquiring playing addresses corresponding to the S videos to be played; when the N + S is larger than M, dividing the window of the video plug-in into Q sub-windows, wherein Q is a positive integer larger than or equal to the N + S; the N + S is the number of videos to be played, which are required to be played by the video plug-in after the S videos to be played are newly added; playing videos to be played corresponding to the video addresses of the N + S videos to be played through the video plug-in Q sub-windows of the window of the video plug-in; or when the N + S is smaller than or equal to M, playing the video to be played corresponding to the video addresses of the S videos to be played through the video plug-in M-N sub-windows except the N sub-windows in the M sub-windows.

Based on the scheme, when the video is played through the video plug-in, a new video to be played can be added, and the video plug-in window is re-divided according to the number of the newly added played videos.

In a second aspect, an embodiment of the present application provides a terminal device, including:

a memory for storing program instructions;

a processor, configured to call a program instruction stored in the memory, and execute the method according to the first aspect and different implementation manners of the first aspect according to an obtained program;

and the display screen is used for displaying the browser window.

In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions that, when executed on a computer, cause the computer to perform the method according to the first aspect and different implementations of the first aspect.

Technical effects brought by any one implementation manner of the second aspect to the third aspect may be referred to technical effects brought by the first aspect and different implementation manners of the first aspect, and are not described herein again.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.

Fig. 1 is a schematic hardware diagram of a terminal device according to an embodiment of the present application;

fig. 2 is a schematic software diagram of a terminal device according to an embodiment of the present application;

fig. 3 is a schematic flowchart of a video playing method according to an embodiment of the present application;

FIG. 4 is a diagram illustrating a relationship of a window handle according to an embodiment of the present disclosure;

fig. 5 is a flowchart illustrating a browser establishing communication with a video plug-in according to an embodiment of the present application;

FIG. 6 is a flowchart illustrating embedding a video plug-in into a browser according to an embodiment of the present application;

fig. 7 is a schematic view of a window of a browser tab according to an embodiment of the present application;

FIG. 8 is a schematic view of a window of a video plug-in provided in an embodiment of the present application;

FIG. 9 is a schematic view of a window of another video plug-in provided in an embodiment of the present application;

FIG. 10 is a schematic view of a window of another video plug-in provided by an embodiment of the present application;

fig. 11 is a schematic display diagram of a tab page window of a browser provided in an embodiment of the present application;

FIG. 12 is a schematic diagram illustrating a display of another tab page window of a browser according to an embodiment of the present application;

fig. 13 is a schematic view of a tab page window of a browser provided in an embodiment of the present application;

fig. 14 is a schematic diagram of a tab window after a video to be played is newly added according to an embodiment of the present application;

fig. 15 is a schematic diagram of another tab page window after a video to be played is newly added according to an embodiment of the present application;

FIG. 16 is a schematic view of another browser tab window provided in an embodiment of the present application;

fig. 17 is a schematic view of a tab window after a video to be played stops being played according to an embodiment of the present application;

fig. 18 is a schematic diagram of another tab page window after the video to be played stops playing according to an embodiment of the present application;

fig. 19 is a schematic diagram of a video playback device according to an embodiment of the present application;

fig. 20 is a schematic diagram of another video playing apparatus according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present application, as presented in the figures, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

It is noted that relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.

The following schemes are generally used for playing videos in multiple paths in video command scheduling management:

1. when the client device uses the video playing service provided by the video service platform, desktop software (EXE) is installed on the client device. When the client device needs the video service platform to provide the video playing service, the client device can use a system interface provided by the video service platform to obtain the played video from the video service platform. In the scheme, multi-path playing can be supported, but the corresponding desktop software is required to be installed on the client. Different video service platforms may correspond to different desktop software. Therefore, when using the video playing service provided by the video service platform, desktop software corresponding to each video service platform needs to be installed, and the memory of the client is occupied.

2. The video service platform is developed by adopting pure web, and provides a web website for the client equipment. When the client device needs to play a video, a video (video) label of a browser is used for obtaining the video from the video service platform and rendering and playing the video, decoding depends on a JavaScript pure application algorithm, a system interface cannot be called, multiple paths of playing are blocked, delay is serious, and the delay is more obvious along with the increase of playing time.

3. The video service platform is developed by adopting pure web. The client device plays the video by calling a video plug-in. The video plug-in is pasted in the browser, namely the playing plug-in floats on the upper layer of the browser. However, in this solution, when the web page is dragged or zoomed, the size of the browser window occupied by the video plug-in window needs to be calculated in real time. Due to the calculated time delay, the video plug-in window cannot be completely attached to the browser window in the zooming or dragging process, and the user experience is poor.

The embodiment of the application provides a video playing method and video playing equipment, wherein a video plug-in window is set as a child window of a browser tab window, and the browser tab window is set as a parent window, so that the video plug-in is embedded into a browser tab. Since the browser tab window and the plug-in window are parent and child windows, the position of the plug-in window in the browser tab is fixed. Therefore, when the browser tab page window receives the instruction to drag or zoom, the position of the dragged or zoomed plug-in window in the browser tab page does not need to be recalculated, the plug-in window and the browser tab page window can be changed synchronously, and the experience of a user is improved.

In addition, in the video playing method provided by the embodiment of the application, the video decoding uses hardware decoding of a machine instead of soft decoding of a webpage video tag. The video rendering uses the rendering of the development tool, so that the multi-path video playing is supported, and the decoding efficiency is improved.

The video playing method provided by the embodiment of the application can be executed through the terminal equipment. The following describes a terminal device according to an embodiment of the present application. It should be understood that the terminal device 100 shown in fig. 1 is merely an example, and the terminal device 100 may have more or fewer components than shown in fig. 1, may combine two or more components, or may have a different configuration of components. The various components shown in fig. 1 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

A hardware configuration block diagram of the terminal device 100 is exemplarily shown in fig. 1. As shown in fig. 1, the terminal device 100 includes: radio Frequency (RF) circuit 110, memory 120, display unit 130, camera 140, sensor 150, audio circuit 160, wireless Fidelity (Wi-Fi) module 170, processor 180, bluetooth module 181, and power supply 190.

The RF circuit 110 may be used for receiving and transmitting signals during information transmission and reception or during a call, and may receive downlink data of a base station and then send the downlink data to the processor 180 for processing; the uplink data may be transmitted to the base station. Typically, the RF circuitry includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.

The memory 120 may be used to store software programs and data. The processor 580 performs various functions of the terminal device 100 and data processing by executing software programs or data stored in the memory 120. The memory 120 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. The memory 120 stores an operating system that enables the terminal device 100 to operate. The memory 120 may store an operating system and various application programs, and may also store codes for performing the methods described in the embodiments of the present application.

The display unit 130 may be used to receive input numeric or character information and generate signal input related to user settings and function control of the terminal device 100, and particularly, the display unit 130 may include a touch screen 131 disposed on the front surface of the terminal device 100 and may collect touch operations of a user thereon or nearby, such as clicking a button, dragging a scroll box, and the like.

The display unit 130 may also be used to display a Graphical User Interface (GUI) of information input by or provided to the user and various menus of the terminal apparatus 100. Specifically, the display unit 130 may include a display screen 132 disposed on the front surface of the terminal device 100. The display panel 132 may be a color liquid crystal panel, and may be configured in the form of a liquid crystal display, a light emitting diode, or the like.

The touch screen 131 may cover the display screen 132, or the touch screen 131 and the display screen 132 may be integrated to implement the input and output functions of the terminal device 100, and after the integration, the touch screen may be referred to as a touch display screen for short. The display unit 130 in this application can be used to display the video to be played in this application.

The camera 140 may be used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing elements convert the optical signals into electrical signals which are then passed to processor 190 for conversion into digital image signals.

The terminal device 100 may further comprise at least one sensor 150, such as an acceleration sensor 151, a distance sensor 152, a fingerprint sensor 153, a temperature sensor 154. The terminal device 100 may also be configured with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, motion sensor, and the like.

The audio circuitry 160, speaker 161, microphone 162 may provide an audio interface between the user and the terminal device 100. The audio circuit 160 may transmit the electrical signal converted from the received audio data to the speaker 161, and convert the electrical signal into a sound signal for output by the speaker 161. The terminal device 100 may also be provided with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 162 converts the collected sound signal into an electrical signal, converts the electrical signal into audio data after being received by the audio circuit 160, and outputs the audio data to the RF circuit 110 to be transmitted to, for example, another terminal or outputs the audio data to the memory 120 for further processing.

Wi-Fi belongs to short-distance wireless transmission technology, and the terminal device 100 can help the user to send and receive e-mails, browse web pages, access streaming media and the like through the Wi-Fi module 170, and provides wireless broadband Internet access for the user.

The processor 180 is a control center of the terminal device 100, connects various parts of the entire terminal using various interfaces and lines, performs various functions of the terminal device 100 and processes data by running or executing software programs stored in the memory 120 and calling data stored in the memory 120. In some embodiments, processor 180 may include one or more processing units; the processor 180 may also integrate an application processor, which mainly handles operating systems, user interfaces, applications, etc., and a baseband processor, which mainly handles wireless communications. It will be appreciated that the baseband processor described above may not be integrated into the processor 180. In the present application, the processor 180 may run an operating system, an application program, a user interface display, and a touch response, and the processing method described in the embodiments of the present application. Further, the processor 180 is coupled with the display unit 130.

And the bluetooth module 181 is configured to perform information interaction with other bluetooth devices having a bluetooth module through a bluetooth protocol. For example, the terminal device 100 may establish a bluetooth connection with a wearable electronic device (e.g., a smart watch) that is also equipped with a bluetooth module through the bluetooth module 181, so as to perform data interaction.

The terminal device 100 also includes a power supply 190 (such as a battery) for powering the various components. The power supply may be logically connected to the processor 180 through a power management system to manage charging, discharging, power consumption, etc. through the power management system. The terminal device 100 may also be configured with power buttons for powering on and off the terminal, and for locking the screen.

Fig. 2 is a block diagram of a software configuration of the terminal device 100 according to the embodiment of the present application.

The layered architecture divides the software into several layers, each layer having a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the Android System is divided into four layers, which are an application layer (System Apps), an application Framework layer (Java API Framework), a System runtime layer (Native), and a Kernel layer (Linux Kernel), from top to bottom.

The application layer may include a series of application packages.

As shown in fig. 2, the application package may include applications such as camera, gallery, calendar, phone call, map, navigation, WLAN, bluetooth, music, video, short message, etc.

The application framework layer provides an Application Programming Interface (API) and a programming framework for the application program of the application layer. The application framework layer includes a number of predefined functions.

As shown in FIG. 2, the application framework layers may include a window manager, content provider, view system, phone manager, resource manager, notification manager, and the like.

The window manager is used for managing window programs. The window manager can obtain the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.

The content provider is used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and answered, browsing history and bookmarks, phone books, etc.

The view system includes visual controls such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide the communication function of the terminal device 100. Such as management of call status (including on, off, etc.).

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and the like.

The notification manager enables the application to display notification information in the status bar, can be used to convey notification-type messages, can disappear automatically after a short dwell, and does not require user interaction. Such as a notification manager used to notify download completion, message alerts, etc. The notification manager may also be a notification that appears in the form of a chart or scrollbar text in a status bar at the top of the system, such as a notification of a running application in the background, or a notification that appears on the screen in the form of a dialog window. For example, text information is prompted in the status bar, a prompt tone is given, the mobile terminal vibrates, an indicator light flashes, and the like.

The system operation library layer is divided into two parts: android Runtime (Android Runtime) and system libraries. The Android Runtime comprises a core library and a virtual machine. The Android Runtime is responsible for scheduling and managing an Android system.

The core library comprises two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer and the application framework layer run in a virtual machine. The virtual machine executes java files of the application layer and the application framework layer as binary files. The virtual machine is used for performing the functions of object life cycle management, stack management, thread management, safety and exception management, garbage collection and the like.

The system library may include a plurality of functional modules. For example: surface Manager (Surface Manager), media Libraries (Media Libraries), three-dimensional graphics processing Libraries (e.g., openGL ES), 2D graphics engines (e.g., SGL), and the like.

The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

The media library supports a variety of commonly used audio, video format playback and recording, and still image files, among others. The media library may support a variety of audio-video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The kernel layer is a layer between hardware and software. The kernel layer provides core system services such as security, memory management, process management, network protocol stacks, and driver models. The inner core layer at least comprises a display driver, a camera driver, an audio driver and a sensor driver.

It should be noted that the configuration of different terminal devices may be different, and therefore fig. 1 to fig. 2 are only used as an exemplary description, and the present application is not limited thereto in particular.

Fig. 3 exemplarily shows a flow of the video playing method, where the flow may be executed by a video playing apparatus, and the apparatus may be located in the terminal device 100 shown in fig. 1, such as the processor 180, or the terminal device 100. Taking the terminal device as an example, the specific process is as follows:

301, in response to a launch operation of a tab page of the browser.

In some embodiments, the user may launch the browser through the terminal device. When the user inputs the website in the browser, the terminal device detects the operation of the user and responds to the operation of the user.

302, determining the title of the label page and the video address of at least one video to be played on the video plug-in.

In some embodiments, the video plug-in is a plug-in for playing a video on a tab page of a browser. When a video to be played is played through a video plug-in, it is necessary to determine on which tab of a browser the video to be played is played by the video plug-in. Therefore, when the video to be played is played through the video plug-in, the title of the browser tab needs to be determined.

In some embodiments, after determining the tab title, the terminal device may determine, through the video address of the at least one video to be played, which is input by the user in the browser, the video address of the at least one video to be played, which needs to be played through the browser.

303, acquiring a window handle of the tab page according to the title of the tab page.

In some embodiments, at least one process of the browser may be determined according to all processes running in the terminal device, and a process of the browser tab corresponding to the tab title of the browser may be determined in the at least one process of the browser. Specifically, if the browser is a google browser, a Chrome browser program named Chrome. Further, a window handle for a browser tab page may be determined based on the progress of the browser tab page. For example, the browser is a google browser, the progress of the google browser includes the progresses of a plurality of browser tab pages, and the window handle of the tab page corresponding to the google browser tab page can be determined according to the title of the browser tab page.

And 304, acquiring a window handle of the video plug-in, and embedding the window of the video plug-in as a child window of the tab page of the browser into the window of the tab page according to the relationship between the window handle of the tab page and a parent child window of the window handle of the video plug-in.

In some embodiments, after obtaining the window handle of the video plug-in, the tab page window handle and the window handle of the browser tab page may be bound in a parent-child relationship according to a system layer protocol. Further, according to the parent-child relationship of the two windows, the window of the video plug-in can be embedded into the tab window as the child window of the browser tab.

In some embodiments, the position of the window of the video plug-in the tab page of the browser may be determined before embedding the window of the video plug-in the tab page window as a child of the window of the tab page of the browser. Further, the window of the video plug-in can be embedded into the window of the tab page of the browser as a sub-window of the tab page according to the position of the window of the video plug-in the window of the browser.

And 305, displaying the tab page, and playing the video to be played corresponding to the video address of at least one video to be played in the window of the video plug-in through the video plug-in.

In some embodiments, after the encoded video to be played is acquired through the playing address, the encoded video to be played may be hard-decoded by the decoding device to obtain the decoded video to be played. Furthermore, the decoded video to be played can be played through the video plug-in. As an example, the encoded video to be played may be decoded by system hardware. For example, hardware decoding may be performed by the GPU. Specifically, the system hardware attribute of the device where the video plug-in is located may be obtained first, and different interfaces may be called according to different names of the graphics cards to decode the encoded video. For example, the GPU may be invoked via the FFMpeg interface to decode the encoded video and synchronize the audio and video.

In some embodiments, in the process of playing the decoded video to be played by the video plug-in, the video to be played by the video plug-in may be rendered to the tab page through the rendering window. Wherein a processing level of the render window is higher than a processing level of the window of the video plug-in. Specifically, after the FFMpeg interface calls the CPU to decode the encoded video, the decoded video frame and audio frame may be output. Further, the video plug-in may call an interface to render the decoded video. Specifically, the interface for the video plug-in call may be determined by the development platform of the video plug-in. Different development platforms have different built-in functions and different use modes. Therefore, the calling mode and the rendering mode of the video plug-in are different. As an example, after receiving the video frame data, the video plug-in may call an Open Graphics Library (OpenGL) interface to perform video frame rendering (image drawing) on the video frame data. Further, the rendered video frame can be sent to a window of the video plug-in, and the video frame can be played through the video plug-in the window of the video plug-in. In some scenarios, the browser web page has different rendering mechanisms, the display of the web page needs to be refreshed continuously, and the refreshing is based on the D3D rendering window. Therefore, when the window of the video plug-in plays the video frame rendered through the OpenGL interface through the video plug-in, the window display video frame of the video plug-in needs to be rendered into the tab page through the rendering window. The browser is rendered based on the D3D window, and when the video plug-in window is at the top layer of all the child windows of the browser, the window of the video plug-in covers the D3D window. Therefore, the video plug-in window will be hidden by the browser and will not be rendered by D3D. At this time, the window of the video plug-in may flash in the tab page of the browser and cannot be completely played. Based on this, the processing level of the rendering window (D3D window) may be set higher than the processing level of the window of the video plug-in so that the D3D window can render the video played by the video plug-in window. As shown in fig. 4, 000671072 "local plug-in test-Google Chrome" Chrome _ widderwin _1 is the handle of the Window of the browser tab page, 00025101C "" intermediary D3D Window is the Window handle of the D3D rendering Window, 00071052 "softszzvideoplug in" qt5qwindowlcon is the handle of the Window of the video plug-in. The D3D window and the video plug-in window are both sub-windows of the browser tab, but the processing level of the D3D window is higher than that of the video plug-in window.

In some embodiments, when it is detected that the user starts the browser, the browser and the video plug-in establish a communication connection, so that the browser and the video plug-in can interact with the instruction through the established communication channel. As shown in fig. 5, after the browser is started, the browser may serve as a client, and the video plug-in may serve as a server, so that the client and the server can communicate with each other through establishing a communication connection between the client and the server. The specific process is as follows:

when the browser tab is launched, the video plug-in is launched 501.

In some embodiments, the video plug-in is launched when the terminal device detects that the user launches the tab page in the browser. In some scenarios, before the video plug-in is started, it is detected whether the video plug-in is installed. And if the video plug-in is not installed, prompting the user to install the video plug-in and then starting the plug-in. And if the video plug-in is in the installed state, directly starting the video plug-in.

In some embodiments, the browser initiates the communication connection as a client when the video plug-in is initiated. For example, the browser starts Websocket. Websocket is a new protocol of HTML5, and is a channel for establishing unlimited two-way communication between a browser and a server. For example, the server may send a message to the browser at any time. When the client starts the Websocket, whether the client is connected with the server or not can be detected circularly until the client is connected with the server and then detection is not carried out.

And 502, initializing a video plug-in as a server side to start communication connection.

In some embodiments, after the video plug-in is started, the video plug-in may be initialized. Further, websocket can be started with the video plug-in as a server to wait for the connection of the client browser. After the two parties are connected, command interaction can be carried out through the Websocket.

In some embodiments, when the communication connection between the browser and the video plug-in is successful, the server (i.e., the video plug-in) generates a unique tag ID and transmits the tag ID to the client (browser). Furthermore, the mark is carried every time the client is connected with the server, so that whether the browser request is legal or not and uniqueness can be conveniently verified. When the browser is upgraded, reinstalled or restarted when a browser is opened, and the browser is started to be connected with the server, the upgraded, reinstalled or restarted browser does not have the mark ID of the original browser, and the upgraded, reinstalled or restarted browser can be used as a new browser. When a new browser starts communication connection, the server side can detect whether the tag ID of the browser is generated or not. If not, a unique tag ID is generated and transmitted to the client. If the tag ID is generated, the communication connection with the original browser is closed, a new tag ID is generated and returned to the new browser, and the new browser carries the new tag ID when communicating with the video plug-in.

And 503, embedding the video plug-in into the browser, so that the browser controls the video plug-in to execute actions according to the communication instructions by sending the communication instructions to the video plug-in.

In some embodiments, the browser may control the video plug-in to perform the action indicated by the communication instruction by sending the communication instruction to the video plug-in. Specifically, the browser may send an instruction to the video plug-in to play a video, record a video, capture a screenshot, stop playing a video, and the like. After the video plug-in receives the communication instruction, the video plug-in can process according to the communication instruction, and the video plug-in will send an execution result to the browser after executing to inform the browser of executing the action indicated by the communication instruction.

In some embodiments, when the communication instruction is used to instruct the video plug-in to play the video, the communication instruction may further include a play address corresponding to the video to be played. The playing plug-in can obtain the video to be played according to the playing address, and play the video to be played through the video plug-in the video plug-in window. In some scenarios, when the communication instruction is used to instruct the video plug-in to stop playing the video, the communication instruction may further include a playing address corresponding to the video to be stopped playing. The playing plug-in can stop playing the video to be stopped at the video plug-in window through the video plug-in according to the playing address corresponding to the video to be stopped. In other scenarios, the communication instruction may include a video recording instruction or a screen capture instruction, so as to control the video plug-in to perform operations such as video recording or screen capture.

How to embed the video plug-in into the browser is described below, as shown in fig. 6, the specific flow is as follows:

s11, all running processes in the terminal equipment are obtained.

S12, determining at least one process of the browser according to all processes running in the terminal equipment.

As an example, when the browser is a google browser, at least one browser program named chrome.

S13, determining the process of the browser tab page corresponding to the browser tab page title in at least one process of the browser.

In some embodiments, the name of the browser tab page process includes the title of the browser tab page, so that the process corresponding to the browser tab page title can be determined according to the title of the browser tab page.

And S14, determining a window handle of the browser tab page according to the process of the browser tab page.

In some embodiments, when the browser sends a communication instruction to the video plug-in, the communication instruction includes the tag ID, the browser tab title, and the position of the video plug-in window in the browser tab. After the video plug-in receives the communication instruction sent by the browser, the window handle of the browser tab can be obtained through the browser tab title.

In some embodiments, after determining the window handle for the browser tab, the window handle for the tab may be translated to the same handle format as the video plug-in. The video plug-in can be developed through different development platforms, when the video plug-in is developed through a QT development platform, a QT built-in method is required to be adopted to convert the window handle of the tab page into a handle format required by QT, and if the video plug-in is developed through MFC, the window handle is converted into a handle required by MFC.

S15, acquiring the handle of the plug-in window, and mapping the parent-child relationship of the handle to realize the embedding of the video plug-in into the browser.

In some embodiments, the video plug-in window handles may be obtained and bound into a parent-child relationship according to a system layer protocol, so as to embed the video plug-in into the browser. Specifically, the tab window handle may be set as a parent, the video plug-in window handle may be set as a child, and the video plug-in window may be displayed at a specified position in the tab window according to the position of the video plug-in window in the browser tab in the communication instruction. As an example, as shown in fig. 7, the video plug-in window is a child window of the browser tab window.

By the above method, any operation of the parent window is directly transmitted to the child window by the system layer protocol method. At this time, when the browser tab is dragged or zoomed, the video plug-in can simultaneously obtain the action of the parent window, so that the video plug-in does not need to communicate with the browser tab again, and the video plug-in window can synchronously act along with the action of the browser tab window.

In some embodiments, since a hardware decoding manner is adopted in decoding the video, the video plug-in supports multiple video plays. When the video addresses of the video to be played are N, N is an integer greater than 1. The segmentation of the video plug-in window may be achieved by: and dividing the window of the video plug-in into M sub-windows, wherein M is a positive integer greater than or equal to N. And playing videos to be played corresponding to the N video addresses in the M sub-windows through the video plug-in. In some scenarios, when N is an even number, M equals N, and the size of the N sub-windows is the same. For example, when the play address is 4 play addresses, the window of the video plug-in may be divided into 4 sub-windows, as shown in fig. 8.

In some scenarios, when N is odd, M equals N +1, N +1 sub-windows are the same size. For example, when the play address is 5 play addresses, the window of the video plug-in may be divided into 6 sub-windows, as shown in fig. 9.

In other scenarios, when N is an odd number, M is equal to N, wherein the size of one of the M sub-windows is 2 times the size of the other sub-windows. When the play addresses are 5 play addresses, the window of the video plug-in may be divided into 5 sub-windows, and the size of one sub-window is 2 times that of the other 4 sub-windows. For example, the size of the last sub-window may be 2 times the size of the first 4 sub-windows, as shown in fig. 10. In a possible implementation manner, when the video to be played is played in the last sub-window, the video to be played may be played in the middle of the window.

In other embodiments, the splitting of the video plug-in window may be implemented by: and setting a default window lattice number for the video plug-in window, and limiting the number of playing window lattices according to the number of video addresses sent by the browser. As an example, the default window pane number for a video plug-in window may be 1, 2, 4, 9, 16, etc. And when the video addresses of the video to be played are S, S is an integer larger than 1. The video plug-in windows are divided into windows with the number of grids which is larger than and closest to the default window number of the number of videos to be played. For example, when S equals 7, the video plug-in window may be divided into windows with a window pane number of 9. In some scenarios, the video plug-in window displays 9 sub-windows, the first 7 sub-windows are used for playing the video to be played, and the last two sub-windows do not play the video, as shown in fig. 11. In other scenes, after the video plugin window is divided into windows with a window lattice number of 9, the video plugin window displays 7 sub-windows for playing the video to be played, as shown in fig. 12.

In some embodiments, when N videos to be played are played through a video plug-in a video plug-in window, in response to an operation of adding S videos to be played on a tab page of a browser, a play address corresponding to the S videos to be played that are played through the video plug-in window is determined. Further, the number of videos to be played, which need to be played by the video plug-in after the S videos to be played are newly added, can be determined to be N + S. As an example, when N videos to be played are played through Q sub-windows in a video plug-in window, after S videos to be played are newly added, the number of sub-windows that need to be played through the video plug-in should be at least N + S sub-windows. In some scenarios, when N + S is greater than M, a window of the video plug-in may be divided into Q sub-windows, where Q is a positive integer greater than or equal to N + S, and videos to be played corresponding to video addresses of N + S videos to be played are played in the Q sub-windows through the video plug-in. In other scenes, when N + S is less than or equal to M, the video to be played corresponding to the video addresses of the S videos to be played may be played in M-N sub-windows of the M sub-windows except the N sub-windows.

Specifically, the browser tab includes controls corresponding to a plurality of videos, and a user can click the controls to control the video plug-in to play the video to be played corresponding to the controls. When the terminal device detects that the user clicks the control corresponding to the video, the browser sends the video address of the video to be played corresponding to the video control to the video plug-in through the websocket communication instruction. After the video plug-in receives the communication instruction, whether a sub-window is added in a window of the video plug-in can be determined according to the number of videos played by the video plug-in, and videos corresponding to video addresses in the communication instruction are played in the sub-window of the video plug-in window through the video plug-in. For example, in a window of a current video plug-in plays 5 videos to be played through the video plug-in, and in some scenes, after detecting that a user adds 1 video to be played by clicking 1 video control in a tab page of a browser, and after responding to an operation of adding 1 video to be played in the tab page of the browser, determining that the number of the videos to be played, which are required to be played by the video plug-in after adding 1 video to be played, is 6. Furthermore, the last sub-window can play the newly added 1 video to be played through the video plug-in. In other scenes, when a user adds 2 videos to be played, the number of videos to be played, which need to be played by the video plug-in after adding 2 videos to be played, is determined to be 7, and is greater than the number of sub-windows of the window of the current video plug-in. At this time, the window of the video plug-in may be re-divided, and then, the newly added 2 videos to be played are played in the sub-windows other than the sub-window corresponding to the 5 videos to be played in the divided sub-windows.

In some embodiments, when the video plug-in window is subjected to window segmentation according to the default window pane number, after it is detected that the user clicks a control corresponding to a new video in the playing process, the number of sub-windows played by the video plug-in can be determined according to the number of the new video controls clicked by the user. Further, the window pane number may be readjusted by the number of sub-windows. In some scenarios, the number of videos to be played in the video plug-in is 8, and the number of sub-windows is 8, as shown in fig. 13. When detecting that a user clicks a video control to add a new video to be played, the number of played videos is changed into 9. If the number of the newly added played videos is less than or equal to the current grid number, a sub-window is added at the end to display the newly added played videos, as shown in fig. 14. In other scenes, if the number of originally played videos is 8, the number of played videos is changed into 10 after two new played videos are added. And when the newly increased video playing quantity is larger than the current window lattice number, readjusting the window lattice number according to the video plug-in window segmentation method. Specifically, the window of the video plug-in may be divided into 16 window lattices, and then the first 10 sub-windows are displayed and the video to be played is played, as shown in fig. 15.

In some embodiments, when the video plug-in window is divided according to the default window lattice number, and when the video to be played stops playing in the playing process, the window lattice number may be readjusted according to the number of the played videos after the video to be played stops playing. For example, if the number of originally played videos is 6, as shown in fig. 16. When one path of video to be played is stopped playing, the number of played video is changed to 5 paths. If the number of played videos after the video to be played is stopped to be played is less than or equal to the current grid number, the sub-window corresponding to the video to be played is closed, and the subsequent sub-windows are moved forward, as shown in fig. 17. If the number of the originally played videos is 6, the number of the played videos is changed into 4 after the two videos to be played are stopped to be played. When the current default window pane number is not greater than and is closest to the video playing number after the playing is stopped, determining that the latest default window pane number is 4, re-dividing the video plug-in window, and playing the video to be played through the video plug-in window after re-division, as shown in fig. 18.

Based on the same technical concept, fig. 19 exemplarily shows a video playback apparatus 1900. The apparatus 1900 may perform any step of the video playing method, and is not described in detail herein to avoid repetition. The apparatus 1900 includes a processing module 1901 and a display module 1902.

A processing module 1901, configured to respond to a starting operation of a tab page of a browser; determining the title of the label page and the video address of at least one video to be played on the video plug-in; the video plug-in is used for playing a video on a tab page of the browser; acquiring a window handle of the label page according to the title of the label page; acquiring a window handle of the video plug-in, and embedding a window of the video plug-in into a window of a tab page of the browser as a child window of the tab page according to the relationship between the window handle of the tab page and a parent child window of the window handle of the video plug-in;

a display module 1902, configured to display the tab page, and play, in the window of the video plug-in, the video to be played corresponding to the video address of the at least one video to be played through the video plug-in.

In some embodiments, the processing module 1901, when acquiring the window handle of the tab page according to the title of the tab page, is specifically configured to: determining at least one process of the browser according to all processes running in the terminal equipment; determining a process of a browser tab corresponding to the browser tab title in at least one process of the browser; and determining the window handle of the browser tab page according to the process of the browser tab page.

In some embodiments, when the video to be played corresponding to the video address of the at least one video to be played is played through the video plug-in, the display module 1902 is specifically configured to: the video plug-in obtains the coded video to be played through the playing address; the video plug-in performs hard decoding on the coded video to be played through decoding equipment to obtain the decoded video to be played; and the video plug-in plays the decoded video to be played.

In some embodiments, the processing module 1901 is further configured to: and rendering the decoded video to be played through a rendering window, wherein the processing level of the rendering window is higher than that of the window of the video plug-in.

In some embodiments, the processing module 1901, when embedding the window of the video plug-in as a child window of the tab page of the browser into the window of the tab page, is specifically configured to: determining the position of the window of the video plug-in the window of the browser; and embedding the window of the video plug-in into the window of the tab page as a sub-window of the tab page of the browser according to the position of the window of the video plug-in the window of the browser.

In some embodiments, the video addresses of the at least one video to be played are N, where N is an integer greater than 1; the display module 1902, when the video to be played corresponding to the video address of the at least one video to be played is played through the video plug-in the window of the video plug-in, is specifically configured to: dividing the window of the video plug-in into M sub-windows; m is a positive integer greater than or equal to N; and playing videos to be played corresponding to the N video addresses respectively in the M sub-windows through the video plug-in.

In some embodiments, when N is an even number, M is equal to N, and the N sub-windows have the same size; or, when N is an odd number, M is equal to N +1, and the sizes of the N +1 sub-windows are the same; or, when N is an odd number, M is equal to N, where the size of one of the M sub-windows is 2 times that of the other sub-windows.

In some embodiments, the at least one video to be played is N, where N is a positive integer, and the processing module 1901 is further configured to: responding to the operation of adding S videos to be played in the browser tab page, and acquiring playing addresses corresponding to the S videos to be played; when the N + S is larger than M, dividing the window of the video plug-in into Q sub-windows, wherein Q is a positive integer larger than or equal to the N + S; the N + S is the number of videos to be played, which need to be played by the video plug-in after the S videos to be played are newly added;

the display module 1902 is further configured to: playing videos to be played corresponding to the video addresses of the N + S videos to be played through the video plug-in Q sub-windows of the window of the video plug-in; or when the N + S is smaller than or equal to M, playing the video to be played corresponding to the video addresses of the S videos to be played through the video plug-in M-N sub-windows except the N sub-windows in the M sub-windows.

Based on the same technical concept, the present embodiment provides a video playing apparatus 2000, and the apparatus 2000 can implement any step of the video playing method discussed above, please refer to fig. 20. The apparatus includes a memory 2001 and a processor 2002.

The memory 2001 for storing program instructions;

the processor 2002 is configured to call the program instruction stored in the memory, and execute the video playing method according to the obtained program.

And a display screen 2003 for displaying a browser window.

In the embodiments of the present application, the processor 2002 may be a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, and may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in a processor.

Memory 2001, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The Memory 2001 may include at least one type of storage medium, and may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charge Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and the like. The memory 2001 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto. The memory 2001 in the embodiments of the present application may also be a circuit or any other device capable of implementing a memory function for storing program instructions and/or data.

Based on the same technical concept, embodiments of the present application provide a computer-readable storage medium, including: computer program code which, when run on a computer, causes the computer to perform a video playback method as discussed in the foregoing. Since the principle of solving the problem of the computer-readable storage medium is similar to that of the video playing method, the implementation of the computer-readable storage medium can refer to the implementation of the method, and repeated details are not repeated.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A video playing method is applied to terminal equipment and comprises the following steps:

responding to the starting operation of a tab page of the browser;

determining the title of the label page and the video address of at least one video to be played on the video plug-in; the video plug-in is used for playing a video on a tab page of the browser;

acquiring a window handle of the label page according to the title of the label page;

acquiring a window handle of the video plug-in, and embedding a window of the video plug-in into a window of a tab page of the browser as a child window of the tab page according to the relationship between the window handle of the tab page and a parent child window of the window handle of the video plug-in;

and displaying the label page, and playing the video to be played corresponding to the video address of the at least one video to be played in the window of the video plug-in through the video plug-in.

2. The method of claim 1, wherein obtaining the window handle of the tab page according to the title of the tab page comprises:

determining at least one process of the browser according to all processes running in the terminal equipment;

determining a process of a browser tab corresponding to the browser tab title in at least one process of the browser;

and determining the window handle of the browser tab according to the process of the browser tab.

3. The method of claim 1, wherein playing the video to be played corresponding to the video address of the at least one video to be played through the video plug-in comprises:

the video plug-in obtains the coded video to be played through the playing address;

the video plug-in performs hard decoding on the coded video to be played through decoding equipment to obtain the decoded video to be played;

and the video plug-in plays the decoded video to be played.

4. The method of claim 3, wherein the method further comprises:

and in the process of playing the decoded video to be played by the video plug-in, rendering the video to be played by the video plug-in to the label page through a rendering window, wherein the processing level of the rendering window is higher than that of the window of the video plug-in.

5. The method of claim 1, wherein said embedding a window of the video plug-in into a window of a tab page of the browser as a child of the window of the tab page comprises:

determining the position of the window of the video plug-in the window of the browser;

and embedding the window of the video plug-in into the window of the tab page as a sub-window of the tab page of the browser according to the position of the window of the video plug-in the window of the browser.

6. The method according to any one of claims 1 to 5, wherein the video addresses of the at least one video to be played are N, where N is an integer greater than 1;

playing the video to be played corresponding to the video address of the at least one video to be played in the window of the video plug-in through the video plug-in, wherein the playing comprises the following steps:

dividing the window of the video plug-in into M sub-windows; m is a positive integer greater than or equal to N;

and playing videos to be played corresponding to the N video addresses respectively in the M sub-windows through the video plug-in.

7. The method of claim 6, wherein when N is an even number, M is equal to N, and the N sub-windows are the same size; alternatively, the first and second electrodes may be,

when N is an odd number, M is equal to N +1, and the sizes of the N +1 sub-windows are the same; alternatively, the first and second liquid crystal display panels may be,

and when the N is an odd number, the M is equal to the N, wherein the size of one sub-window in the M sub-windows is 2 times that of other sub-windows.

8. The method of claim 6, wherein the at least one video to be played is N, N being a positive integer, the method further comprising:

responding to the operation of adding S videos to be played in the browser tab page, and acquiring playing addresses corresponding to the S videos to be played;

when the N + S is larger than M, dividing the window of the video plug-in into Q sub-windows, wherein Q is a positive integer larger than or equal to the N + S; the N + S is the number of videos to be played, which are required to be played by the video plug-in after the S videos to be played are newly added; playing videos to be played corresponding to the video addresses of the N + S videos to be played through the video plug-in Q sub-windows of the window of the video plug-in; alternatively, the first and second liquid crystal display panels may be,

and when the N + S is less than or equal to M, playing the video to be played corresponding to the video addresses of the S videos to be played through the video plug-in M-N sub-windows except the N sub-windows in the M sub-windows.

9. A terminal device, comprising:

a memory for storing program instructions;

a processor for calling program instructions stored in said memory to execute the method of any one of claims 1 to 8 in accordance with the obtained program;

and the display screen is used for displaying the browser window.

10. A computer readable storage medium having stored thereon computer instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1-8.