WO2017084306A1

WO2017084306A1 - Method and apparatus for playing key information of video in browser of mobile device

Info

Publication number: WO2017084306A1
Application number: PCT/CN2016/084654
Authority: WO
Inventors: 王璟瑶
Original assignee: 乐视控股（北京）有限公司; 乐视移动智能信息技术（北京）有限公司
Priority date: 2015-11-20
Filing date: 2016-06-03
Publication date: 2017-05-26
Also published as: CN105657514A

Abstract

The present application provides a method and apparatus for playing key information of a video in a browser of a mobile device. A specific implementation manner of the method comprises: during video loading and pause, by means of a method in which a search is made in a server by using a key frame and a current frame of a video, aggregated pushing of interesting content of a user within the latest time is implemented; by adding an interactive information policy, the effect of a balance between scenario advertisements and the efficiency is achieved on a mobile platform on the basis of not affecting normal video browsing, and accordingly the problem of poor user experience caused by spot announcement interference on the mobile platform in the prior art is resolved. Also disclosed in the present application is an apparatus for playing key information of a video in a browser of a mobile device, which can also bring the same beneficial effects. In addition, the present application is not limited to the applications on the mobile terminal device, and can also have the same effect achieved on an internet video, a smart set-top-box and other terminal devices.

Description

Method and device for playing video key information in mobile device browser

The present application claims the priority of the Chinese Patent Application Serial No. PCT Application Serial No.

Technical field

The present application relates to the field of mobile platform video playback technologies, and in particular, to a method and apparatus for playing video key information in a mobile device browser.

Background technique

When users play videos, in order to increase the click rate of information by publishers and precision advertising platforms through information push, the following three main approaches are adopted: scheme one, inserting insert advertisements in movie and TV dramas; scheme two On the existing video advertising platform, by uploading an advertisement that is not long before the broadcast, generally 15s-30s, the amount of clicks is played by playing; and scheme 3, playing the video through the barrage function on the video software A large number of scrolling user comments appear above the plug-in window to increase the interactivity of the video loading.

Although the above three methods can improve the click rate of information, there are also many problems. Among them, for the first scheme, the input cost is high, and it is necessary to plan in advance when shooting, and the time continuity is relatively poor; Simple, but because the ad position is fixed and easily blocked by plug-ins such as ad filtering, the effect is not good, which affects the viewing experience. For the third solution, although the interaction of video loading is enhanced to a certain extent, The process affects the viewing experience and the scrolling subtitles are mostly useless information, and the amount of information transmitted is small and not timely enough.

To this end, the present application provides a method and apparatus for video playback of key information in a mobile device browser, which solves the defect that the above three prior art solutions have poor user experience on the mobile platform.

Summary of the invention

The present application provides a method and a device for playing video key information in a mobile device browser, so as to implement video key information aggregation search on a mobile platform, and can smoothly push the video information or advertisement information associated with the current video of the user. .

The present application discloses a method for video playback of key information in a mobile device browser, including:

Obtaining the video key frame and the current frame currently being played by the video player when the video is loaded or paused;

Searching for the video key frame and the current frame by using a search method, searching for key information of the video key frame and the current frame, where the key information includes: video information and/or picture information;

The key information is aggregated and pushed to the video player for presentation.

The searching for the video key frame and the current frame, searching for the key information of the video key frame and the current frame, where the key information includes: the video information and/or the picture information is specifically:

Scanning the video key frame and the key block in the corresponding picture of the current frame;

Segmenting the key blocks to obtain key sections;

Sending the critical section to a video server for searching, obtaining key information related to the video key frame and the current frame, and feeding back the key information to the video server.

Performing scaling and cropping on the video key frame and the current frame;

Performing fast grayscale calculation and perceptual Phash/SIFT calculation on the information fragment obtained by scaling and cropping, and obtaining a picture fingerprint of the video key frame and the corresponding picture of the current frame;

Returning the picture fingerprint to obtain key information of the video key frame and the current frame.

The aggregating the key information to the video player for display is specifically as follows:

Determining whether to detect a CPU load on the video player for one second;

If not, detect the CPU load of the system and determine whether data synchronization is required.

If yes, reading the key information and determining whether the GPU load exceeds a threshold;

If yes, the video performs synchronization status and buffer related data, and the key information is processed by the CPU, and the key information is read back;

If not, the video performs synchronization status and buffers related data, the GPU processes the key information and reads back the key information;

Determining whether to read back the key information, and if so, to push the key information aggregate to the video player for presentation.

Preferably, after the said key information is aggregated and pushed to the video player for display, the method further comprises:

The key information is displayed in the right area of the video player shown with relevant pictures and details.

The application also discloses an apparatus for playing key information in a mobile device browser, comprising:

An acquiring unit, configured to acquire a video key frame and a current frame currently being played by the video player when the video is loaded or paused;

a search unit, configured to search for the video key frame and the current frame by using a search method, and search for key information of the video key frame and the current frame, where the key information includes: video information and/or picture information;

An aggregate push unit is configured to push the key information to the video player for display.

The search unit includes:

a scanning unit, configured to scan the video key frame and a key block in a picture corresponding to the current frame;

a dividing unit, configured to divide the key block to obtain a key segment;

a feedback unit, configured to send the key segment to a video server for searching, obtain key information related to the video key frame and the current frame, and feed back the key information to the video server.

The search unit includes:

a scaling unit for scaling and cropping the video key frame and the current frame;

a calculating unit, configured to perform fast grayscale calculation and perceptual Phash/SIFT calculation on the information fragment obtained by scaling and cropping, and obtain a picture fingerprint of the video key frame and the corresponding picture of the current frame;

The backhaul unit is configured to return the picture fingerprint to obtain key information of the video key frame and the current frame.

The aggregation push unit includes:

a first determining unit, configured to determine whether to detect a CPU load on the video player for one second;

The detecting and determining unit is configured to detect a CPU load of the system and determine whether data synchronization is required;

a second determining unit, configured to read the key information, and determine whether the GPU load exceeds a threshold;

a synchronization buffer unit for synchronizing states and buffering related data of the video;

a first processing unit, configured to process the key information by using the CPU, and read back the key information;

a second processing unit, configured to process the key information by using the GPU, and read back the key information;

a display unit, configured to determine whether to read back the key information, and if so, to push the key information to the video player for display.

Preferably, the device further comprises:

a display unit for displaying the related information and details in the right area of the video player shown.

The present application provides a method for playing video key information in a mobile device browser. When a video is loaded and paused, the video key frame and the current frame are used to search the server, and the interest content of the user in the latest time is implemented. Aggregate push, by adding an interactive information strategy, on the basis of not affecting normal video browsing, the effect of both scene advertisement and efficiency can be achieved on the mobile platform, and the interference of the interstitial advertisement on the mobile platform in the prior art is not solved, and the user experience is not A good problem; at the same time, the present application also discloses a device for playing video key information in a mobile device browser, which can also achieve the above beneficial effects. In addition, the present application can be applied not only to mobile terminal devices but also to Internet devices, smart set-top boxes and other terminal devices have the same effect.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative labor for those skilled in the art.

1 is a flowchart of Embodiment 1 of a method for video playback of key information in a mobile device browser according to the present application;

2 is a flowchart of Embodiment 2 of a method for playing video key information in a mobile device browser according to the present application;

3 is a flowchart of Embodiment 3 of a method for playing video key information in a mobile device browser according to the present application;

4 is a schematic diagram of image frame sampling using the pHash/SIFT improved algorithm in the present application;

FIG. 5 is a flowchart of specific steps of step S3 in the method embodiment of the present application;

6 is a schematic structural diagram of Embodiment 1 of an apparatus for playing video key information in a mobile device browser according to the present application;

7 is a schematic structural diagram of Embodiment 2 of an apparatus for playing video key information in a mobile device browser according to the present application;

FIG. 8 is a schematic structural diagram of Embodiment 3 of an apparatus for playing video key information in a mobile device browser according to the present application;

FIG. 9 is a schematic diagram of a specific structure of an aggregation push unit in an embodiment of the apparatus of the present application.

detailed description

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. It is a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

FIG. 1 is a flowchart of Embodiment 1 of a method for video playback of key information in a mobile device browser. As shown in FIG. 1 , the method in this embodiment may include:

Step S1: Obtain a video key frame and a current frame currently being played by the video player when the video is loaded or paused.

The present application obtains the video key frame and the current frame currently being played by the video player when the video is loaded or paused, and uses the video key frame search method at the fragmentation time to implement the aggregated push of the user's latest time interest content, and the prior art The use of video software barrage function or video advertising platform to achieve advertising push, which makes the user's experience is not good.

Step S2: Searching for the video key frame and the current frame by using a search method, and searching for key information of the video key frame and the current frame, where the key information includes: video information and/or picture information.

For example, during the video viewing process, the user obtains key frames in the video, where the key frames may be rich media information such as actor information, character pictures, clothes pictures, jewelry or related products.

Step S3: The key information is aggregated and pushed to the video player for display.

The basic idea of the present application is to obtain a video frame or a video first frame/Poster picture that is currently being played, scan a key block in the picture, and then send it to the server for searching, and then display the relevant details in the right area of the video play after successfully returning. . For example, in the right area of the video playback, rich media information such as actor information, character pictures, clothes pictures, jewelry or related products are displayed.

In this embodiment, by acquiring a video key frame and a current frame currently being played by the video player when the video is loaded or paused, and using the video key frame search method at the fragmentation time, the interest content of the user's latest time is aggregated and pushed, and now The technology adoption is the video software barrage function or the video advertisement platform to realize the advertisement push, which makes the user experience poor.

The technical solutions of the method embodiment shown in FIG. 1 are described in detail below by using several specific embodiments.

For example, if the obtained video key frame and the current frame are the character avatar frame, the character avatar frame is searched by using the search method, thereby obtaining the actor information, the character picture and the actor work related information, and then the video playing right area display.

For example, if you are currently watching TV works as "You from the Stars", you will get and collect information about the professor's coat, and connect to the online purchase path to provide users with rich media information such as certain products, which will be relevant during the user's video viewing. Relevant content of interest.

2 is a flowchart of Embodiment 2 of a method for playing video key information in a mobile device browser according to the present application. As shown in FIG. 2, the method in this embodiment may include:

Specifically, step S2 of the first embodiment may specifically be:

Step S21: Scan the video key frame and the key block in the corresponding picture of the current frame.

Step S22: segment the key block to obtain a key segment.

Step S23: Send the key section to the video server for searching, obtain key information related to the video key frame and the current frame, and feed back the key information to the video server.

In this embodiment, by acquiring a video frame or a video first frame/Poster picture currently being played by the video player when the video is loaded or paused, the key block in the picture is scanned, and the segment is sent to the server for searching, and then the video is successfully returned. Play the right area to display relevant details, such as displaying rich media information such as actor information, character pictures, clothes pictures, jewelry or related products in the right area of the video playback. However, the prior art adopts a video software barrage function or a video advertisement platform to implement advertisement push, which makes the user's experience effect poor.

FIG. 3 is a flowchart of Embodiment 3 of a method for playing video key information in a mobile device browser according to the present application. As shown in FIG. 3, the difference between this embodiment and the embodiment shown in FIG. 2 is obtained by using different methods. The key information, the method in this embodiment may include:

Specifically, step S2 of the first embodiment may specifically be:

Step S211: Perform scaling and cropping on the video key frame and the current frame.

Step S222: performing fast grayscale calculation and perceptual Phash/SIFT calculation on the information fragment obtained by scaling and cropping, and obtaining a picture fingerprint of the video key frame and the corresponding picture of the current frame.

The standard hash hash algorithm in the prior art: analyzing video key frame picture characteristics, reducing the size and simplifying the color, and calculating the gray average of 8*8 color blocks:

Key frame of the block

The picture fingerprint corresponding to the image of the block is obtained according to the key frame of the block.

This application adopts the improved hash pHash/SIFT (Scale-invariant feature transform) algorithm: it has advantages for identifying the planar image by the standard Hash algorithm, but it is poorly recognized for the deformed image. An algorithm for detecting local features using a modified pHash/SIFT algorithm, Input images in video keyframes, and establish octave images of different scales. At any scale, there can be corresponding feature points. The first sub-octave is the original image size, and each subsequent sub-layer is the result of down-sampling of the previous sub-layer. , that is, 1/4 of the original image (half the length and width), forming the next sub-octave. Specifically, as shown in FIG. 4, FIG. 4 is a schematic diagram of image frame sampling by using the pHash/SIFT improved algorithm in the present application.

The pHash/SIFT algorithm described above can recognize the deformation of the picture, and can match the original picture as long as the degree of deformation does not exceed 25%. Therefore, the algorithm is used to match the video key frames in the present application, and the original picture can be matched with a standard. .

Step S233: The image fingerprint is returned to obtain key information of the video key frame and the current frame.

Specifically, after step S3, the method further includes:

Step S4: Display the related information and details in the right area of the video player shown.

Please refer to FIG. 5, which is a flowchart of specific steps of step S3 in the method embodiment of the present application. Specifically, as shown in FIG. 5, step S3 of the first embodiment and the second embodiment may specifically be:

Step S31: determining whether to detect a CPU (Central Processing Unit) load on the video player, if not, proceed to step S32, and if yes, proceed to step S35;

Step S32: detecting the CPU load of the system;

Step S33: determining whether data synchronization is required, if yes, proceeding to step S34, if no, proceeding to step S35;

Step S34: the video performs synchronization state and buffer related data;

Step S35: reading the key information;

Step S36 determines whether the GPU (Graphic Processing Unit) load exceeds the threshold, and if so, proceeds to step S37, and if not, proceeds to step S38;

Step S37: processing the key information by the CPU, and reading back the key information;

Step S38: The GPU processes the key information and reads back the key information.

Step S39: determining whether to read back the key information, if yes, proceed to step S310, and if not, return to step S31;

Step S310: The key information is aggregated and pushed to the video player for display.

FIG. 6 is a schematic structural diagram of Embodiment 1 of a device for playing video key information in a mobile device browser according to the present application. As shown in FIG. 6 , the device in this embodiment may include: an obtaining unit 101, a searching unit 102, and an aggregation push. The unit 103 is configured to acquire a video key frame and a current frame that are currently being played by the video player when the video is loaded or paused. The searching unit 102 is configured to perform the video key frame and the current frame by using a search method. Searching for the key information of the video key frame and the current frame, the key information includes: video information and/or picture information; the aggregation push unit 103 is configured to push the key information to the video player for display .

The device in this embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 1 , and the implementation principle and technical effects are similar, and details are not described herein again.

FIG. 7 is a schematic structural diagram of Embodiment 2 of a device for playing video key information in a mobile device browser according to the present application. As shown in FIG. 7, the device structure of the device in FIG. 6 specifically includes: an acquiring unit 101. The search unit 102 is configured to acquire the video key frame and the current frame currently being played by the video player when the video is loaded or paused; the searching unit 102 is configured to use the search method to Searching for the video key frame and the current frame, searching for key information of the video key frame and the current frame, the key information includes: video information and/or picture information; the aggregation push unit 103 is configured to push the key information to the aggregated The video player is displayed. On the basis of the foregoing structure, the search unit 102 includes: a scanning unit 1021, a dividing unit 1022, and a feedback unit 1023. Specifically, the scanning unit 1021 is configured to scan the video key frame and the corresponding picture in the current frame. a key block; the splitting unit 1022 is configured to divide the key block to obtain a key segment; and the feedback unit 1023 is configured to send the key segment to a video server for searching, obtaining a context The key information related to the video key frame and the current frame is fed back to the video server.

The device in this embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 2, and the implementation principle and technical effects are similar, and details are not described herein again.

FIG. 8 is a schematic structural diagram of Embodiment 3 of a device for playing video key information in a mobile device browser according to the present application. As shown in FIG. 8 , the device structure of the device in FIG. 6 specifically includes: an acquiring unit 101. The search unit 102 is configured to acquire the video key frame and the current frame currently being played by the video player when the video is loaded or paused; the searching unit 102 is configured to use the search method to Searching for the video key frame and the current frame, searching for key information of the video key frame and the current frame, the key information includes: video information and/or picture information; the aggregation push unit 103 is configured to push the key information to the aggregated The video player is displayed. On the basis of the foregoing structure, the search unit 102 includes: a scaling and cropping unit 1024, a computing unit 1025, and a backhaul unit 1026, wherein the scaling and cropping unit 1024 is configured to scale the video key frame and the current frame. The calculating unit 1025 is configured to perform fast grayscale calculation and perceptual Phash/SIFT calculation on the information fragment obtained by scaling and cropping, and obtain a picture fingerprint of the video key frame and the current frame corresponding picture; the returning unit 1026 is configured to: The picture fingerprint is returned to obtain key information of the video key frame and the current frame. Specifically, the device of the present application further includes: a display unit 104, configured to display the related information and details in the right area of the video player shown in the video player.

Please refer to FIG. 9 , which is a schematic structural diagram of a polymerization push unit in an embodiment of the apparatus of the present application. As shown in FIG. 9, the foregoing aggregation pushing unit 103 includes: a first determining unit 1031, configured to determine whether to detect a CPU load on the video player for one second; and a detecting unit 1032 for detecting a CPU load of the system. Determining whether data synchronization is required; the second determining unit 1033 is configured to read the key information, and determine whether the GPU load exceeds a threshold; the synchronization buffer unit 1034 is configured to perform video synchronization status and buffer related data; The processing unit 1035 is configured to process the key information by using the CPU, and read back the key information. The second processing unit 1036 is configured to process the key information by using the GPU, and read back the key. Information; display unit 1037, configured to determine whether to read back the key information.

The device in this embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 3, and the implementation principle and technical effects are similar, and details are not described herein again.

One of ordinary skill in the art will appreciate that all or part of the steps to implement the various method embodiments described above may be accomplished by hardware associated with the program instructions. The aforementioned program can be stored in a computer readable storage medium. The program, when executed, performs the steps including the foregoing method embodiments; and the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

The device embodiments described above are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located in one place. Or it can be distributed to at least two network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.

Finally, it should be noted that the above embodiments are only for explaining the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that The technical solutions described in the foregoing embodiments may be modified, or some or all of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the technical solutions of the embodiments of the present application. range.

Claims

A method for playing key information in a mobile device browser, characterized in that it comprises:

Obtaining the key frame and current frame of the video currently being played by the video player when the video is loaded or paused;

Searching for the video key frame and the current frame by using a search method, searching for key information of the video key frame and the current frame, where the key information includes: video information and/or picture information;

The key information is aggregated and pushed to the video player for presentation.
The method according to claim 1, wherein the searching for the video key frame and the current frame is performed by using a search method, and searching for key information of the video key frame and the current frame, the key information includes: The video information and/or picture information is specifically:

Scanning the video key frame and the key block in the corresponding picture of the current frame;

Segmenting the key blocks to obtain key sections;

Sending the critical section to a video server for searching, obtaining key information related to the video key frame and the current frame, and feeding back the key information to the video server.
The method according to claim 1, wherein the searching for the video key frame and the current frame is performed by using a search method, and searching for key information of the video key frame and the current frame, the key information includes: The video information and/or picture information is specifically:

Performing scaling and cropping on the video key frame and the current frame;

Performing fast grayscale calculation, perceptual hash Phash/scale invariant feature transform SIFT calculation on the information fragment obtained by scaling and cropping, and obtaining a picture fingerprint of the video key frame and the corresponding picture of the current frame;

Returning the picture fingerprint to obtain key information of the video key frame and the current frame.
The method according to claim 1, wherein the pushing the key information to the video player for display is specifically:

Determining whether to detect a CPU CPU load on the video player for one second;

If not, detect the CPU load of the system and determine whether data synchronization is required.

If yes, reading the key information and determining whether the graphics processor GPU load exceeds a threshold;

If so, the video is synchronized and buffered with relevant data, through the CPU to the key Information processing and reading back the key information;

If not, the video performs synchronization status and buffers related data, the GPU processes the key information and reads back the key information;

Determining whether to read back the key information, and if so, to push the key information aggregate to the video player for presentation.
The method according to claim 1, wherein the step of pushing the key information to the video player for display further comprises:

The key information is displayed in the right area of the video player shown with relevant pictures and details.
An apparatus for playing key information in a video of a mobile device browser, comprising:

An acquiring unit, configured to acquire a video key frame and a current frame currently being played by the video player when the video is loaded or paused;

a search unit, configured to search for the video key frame and the current frame by using a search method, and search for key information of the video key frame and the current frame, where the key information includes: video information and/or picture information;

An aggregate push unit is configured to push the key information to the video player for display.
The apparatus according to claim 6, wherein said searching unit comprises:

a scanning unit, configured to scan the video key frame and a key block in a picture corresponding to the current frame;

a dividing unit, configured to divide the key block to obtain a key segment;

a feedback unit, configured to send the key segment to a video server for searching, obtain key information related to the video key frame and the current frame, and feed back the key information to the video server.
The apparatus according to claim 6, wherein said searching unit comprises:

a scaling unit for scaling and cropping the video key frame and the current frame;

a calculating unit, configured to perform fast grayscale calculation, perceptual hash Phash/scale invariant feature transform SIFT calculation on the information fragment obtained by scaling and cropping, and obtain a picture fingerprint of the video key frame and the corresponding picture of the current frame;

The backhaul unit is configured to return the picture fingerprint to obtain key information of the video key frame and the current frame.
The device according to claim 6, wherein the aggregation push unit comprises:

a first determining unit, configured to determine whether to detect a CPU load on the video player for one second;

The detecting and determining unit is configured to detect a CPU load of the system and determine whether data synchronization is required;

a second determining unit, configured to read the key information, and determine whether the GPU load of the graphics processor exceeds a threshold;

a synchronization buffer unit for synchronizing states and buffering related data of the video;

a first processing unit, configured to process the key information by using the CPU, and read back the key information;

a second processing unit, configured to process the key information by using the GPU, and read back the key information;

a display unit, configured to determine whether to read back the key information, and if so, to push the key information to the video player for display.
The device of claim 6 further comprising:

a display unit for displaying the related information and details in the right area of the video player shown.