WO2020056690A1

WO2020056690A1 - Method and apparatus for presenting interface associated with video content, and electronic device

Info

Publication number: WO2020056690A1
Application number: PCT/CN2018/106785
Authority: WO
Inventors: 菲永奥利维尔; 李建亿
Original assignee: 太平洋未来科技（深圳）有限公司
Priority date: 2018-09-20
Filing date: 2018-09-20
Publication date: 2020-03-26
Also published as: CN109218697A; CN109218697B

Abstract

Provided in the embodiments of the present invention are a method and apparatus for presenting an interface associated with video content, and an electronic device. The method comprises: acquiring a video frame image of currently played video content every preset time; extracting color information and light ray information from the video frame image; and generating, according to the color information and the light ray information, a background of an interface associated with the video content, wherein the video content and the interface associated with the video content are presented on the same interface. By means of the method and apparatus, and the electronic device mentioned above, the interface associated with the video content can be switched according to a change in the background of the currently played video content, so that the interface associated with the video content is presented in a more enriched manner, more flexibly and more conveniently, and the interactive experience of a user is improved.

Description

Video content association interface presentation method, device and electronic equipment

Technical field

The present invention relates to the field of Internet application technology, and in particular, to a method, a device, and an electronic device for presenting a video content association interface.

Background technique

At present, for some applications on the Internet, for example, video applications can be run on mobile terminals to present video content for the convenience of users to watch. At the same time, users can comment on the content of the application on the user interface associated with presenting video content. And interaction. However, the inventors discovered during the implementation of the embodiments of the present invention that the interfaces associated with video content in related technologies are usually presented in a single background. Regardless of the application, the review interface is generally presented in the form of a whiteboard. Too simple. In addition, more and more online videos are completed by taking selfies with mobile devices. However, the effect of the camera of the mobile device to obtain videos or pictures is not only affected by the surrounding ambient light, but also by the stability of the shooting. The quality is not conducive to the processing of subsequent images or videos, and the use of a mobile device's stand or selfie stick cannot meet the requirements.

Summary of the Invention

The presenting method, device, and electronic device of the video content association interface provided by the embodiments of the present invention are used to solve at least the foregoing problems in related technologies.

An aspect of the embodiments of the present invention provides a method for presenting a video content association interface, including:

Obtain a video frame image of the currently playing video content every preset time, extract color information and light angle from the video frame image; and generate a background of the video content association interface according to the color information and light angle, where The video content and the video content association interface are presented on the same interface.

Further, the extracting color information from the video frame image includes: calculating a hue average value of the video frame image; traversing pixels of the video frame image, calculating a hue value of the pixel and the hue average value. Determine whether the color difference is greater than a preset threshold; if the color difference is greater than the preset threshold, add the pixel to the main-tone pixel list, and determine all the pixels according to the tone value of the pixel in the main-tone pixel list Mentioned color information.

Further, the extracting color information from the video frame image includes: determining a weight value corresponding to the pixel according to a distance between each pixel in the video frame image and the video content association interface; and according to the pixel The hue value and the corresponding weight value to calculate the color information.

Further, the extracting the light angle from the video frame image includes: identifying and extracting a target object image in the video frame image, wherein the target object is a person or an animal; extracting a nose in the target object image The sub-image of the partial region is obtained based on the light intensity weighting center of the sub-image and the weighting center of the target object image to obtain the light angle.

Further, generating the background of the video content association interface according to the color information and the light signal angle includes: generating a background color of the video content association interface according to the color information; The brightness of the background color is adjusted to obtain the background of the video content association interface.

Another aspect of the embodiments of the present invention provides a device for presenting a video content association interface, including:

An obtaining module for obtaining a video frame image of a currently playing video content at a preset time; an extracting module for extracting color information and a light angle from the video frame image; a generating module for The light angle generates the background of the video content association interface, wherein the video content and the video content association interface are presented on the same interface.

Further, the extraction module includes: a first calculation unit for calculating an average hue of the video frame image; a traversal unit for traversing pixels of the video frame image, calculating a hue value of the pixel and the The color difference of the hue average value; a determining unit for determining whether the color difference is greater than a preset threshold; a first determining unit for adding the pixel to the main-tone pixel list if the color difference is greater than a preset threshold; Tonal values of the pixels in the main-tone pixel list determine the color information.

Further, the extraction module includes: a second determination unit, configured to determine a weight value corresponding to the pixel according to a distance between each pixel in the video frame image and the video content association interface; a second calculation unit, And is configured to calculate the color information according to a hue value of the pixel and a corresponding weight value.

Further, the extraction module includes: a recognition unit for identifying and extracting a target object image in the video frame image, wherein the target object is a person or an animal; an extraction unit for extracting the target object image The sub-image of the middle nose region is obtained based on the light-weighted center of the sub-image and the weighted center of the target object image to obtain the light angle.

Further, the generating module is configured to generate a background color of the video content association interface according to the color information; and adjust the brightness of the background color according to the light angle to obtain the background of the video content association interface. .

Another aspect of the embodiments of the present invention provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a memory that can be used by the at least one processor And an instruction that is executed by the at least one processor, so that the at least one processor can execute any one of the methods for presenting a video content association interface.

Further, the electronic device further includes an image acquisition module including a lens, an auto-focusing voice coil motor, a mechanical image stabilizer, and an image sensor, and the lens is fixed on the auto-focusing voice coil motor. The lens is used to acquire an image, the image sensor transmits the image acquired by the lens to the recognition module, the autofocus voice coil motor is mounted on the mechanical image stabilizer, and the processing module is based on the inside of the lens The feedback of the lens shake detected by the gyroscope drives the action of the mechanical image stabilizer to achieve lens shake compensation.

Further, the mechanical anti-shake device includes a movable plate, a movable frame, an elastic restoring mechanism, a base plate, and a compensation mechanism; a central portion of the movable plate is provided with a through hole through which the lens passes, and the auto-focusing voice coil motor Installed on the movable plate, the movable plate is installed in the movable frame, and the opposite sides of the movable plate are slidingly fitted with the inner walls of the opposite sides of the movable frame so that the movable plate can be moved along Reciprocating sliding in the first direction; the size of the movable frame is smaller than that of the substrate, and two opposite sides of the movable frame are connected to the substrate through two elastic restoring mechanisms, respectively. The through hole through which the lens passes; the compensation mechanism, driven by the processing module, drives the movable plate and the lens movement on the movable plate to achieve lens shake compensation; the compensation mechanism includes a drive shaft, gears, A gear track and a limit track, the drive shaft is mounted on the base plate, the drive shaft is connected with the gear drive; the gear track is provided on the movable plate, and the gear is mounted In the gear track, when the gear rotates, the gear track enables the movable plate to generate a displacement in a first direction and a displacement in a second direction, wherein the first direction is perpendicular to the second direction; The limit track is disposed on the movable plate or the base plate, and the limit track is used to prevent the gear from detaching from the gear track.

Further, a side of the movable plate is provided with a waist-shaped hole, and the waist-shaped hole is provided with a plurality of teeth that mesh with the gear along a circumferential direction thereof, and the waist-shaped hole and the plurality of teeth together constitute the A gear track, wherein the gear is located in the waist-shaped hole and meshes with the teeth; the limit track is disposed on the base plate, and a bottom of the movable plate is provided with a limit position within the limit track Piece, the limit track makes the movement track of the limit piece in a waist shape.

Further, the limiting member is a protrusion provided on the bottom surface of the movable plate.

Further, the gear track includes a plurality of cylindrical protrusions provided on the movable plate, the plurality of cylindrical protrusions are evenly spaced along the second direction, and the gear is in phase with the plurality of protrusions. Meshing; the limit track is a first arc-shaped stopper and a second arc-shaped stopper provided on the movable plate, and the first arc-shaped stopper and the second arc-shaped stopper are respectively It is arranged on two opposite sides of the gear track in the first direction, and the first arc-shaped stopper, the second arc-shaped stopper, and a plurality of the protrusions cooperate to make the moving track of the movable plate Waist-shaped.

Further, the elastic recovery mechanism includes a telescopic spring.

Further, the image acquisition module includes a mobile phone and a bracket for mounting the mobile phone.

Further, the bracket includes a mobile phone mounting base and a retractable support rod; the mobile phone mounting base includes a retractable connection plate and a folding plate group installed at opposite ends of the connection plate, and one end of the support rod is connected to the connection The middle portions of the plates are connected by a damping hinge; the folded plate group includes a first plate body, a second plate body, and a third plate body, wherein one of two opposite ends of the first plate body is in phase with the connecting plate. Hinged, the other end of the opposite ends of the first plate body is hinged to one of the opposite ends of the second plate body; the other end of the opposite ends of the second plate body is connected to the third plate One end of the two opposite ends of the body is hinged; the second plate body is provided with an opening for the corner of the mobile phone to be inserted; when the mobile phone mount is used to install the mobile phone, the first plate body, the second plate body, and the first plate body The three plates are folded in a right triangle state, the second plate is a hypotenuse of a right triangle, and the first plate and the third plate are right angles of a right triangle, wherein the third plate is One side of the side is attached to one side of the connecting plate , One of the opposite ends of the third plate member and the other end opposite ends of the first plate against body.

Further, a first connection portion is provided on one side surface of the third plate body, and a first connection portion that is matched with the first connection portion is provided on a side surface where the connection plate is in contact with the third plate body. When the mobile phone mounting base is used to install a mobile phone, the first connection portion and the first mating portion are snap-connected.

Further, a second connection portion is provided at one end of the opposite ends of the first plate body, and a second connection is provided at the other end of the opposite ends of the third plate body to cooperate with the second connection portion. When the mobile phone mounting base is used to install a mobile phone, the second connection portion and the second mating portion are snap-connected.

Further, the other end of the support rod is detachably connected to a base.

As can be seen from the above technical solutions, the method, device, and electronic device for presenting the video content association interface provided by the embodiments of the present invention. The video content association interface can be switched as the background of the currently played video content changes, making the video content association interface presentation richer , Flexible, convenient, and improve the user's interactive experience. The anti-shake hardware structure of the mobile phone camera and the mobile phone selfie stand also further enhance the shooting effect, which is more conducive to subsequent image or video processing.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly explain the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely These are some of the embodiments described in the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to these drawings.

FIG. 1 is a flowchart of a method for presenting a video content association interface according to an embodiment of the present invention; FIG.

2 is a flowchart of a method for presenting a video content association interface according to an embodiment of the present invention;

3 is a flowchart of a method for presenting a video content association interface according to an embodiment of the present invention;

4 is a structural diagram of a presentation device for a video content association interface according to an embodiment of the present invention;

5 is a structural diagram of a presentation device for a video content association interface according to an embodiment of the present invention;

6 is a structural diagram of a presentation device for a video content association interface according to an embodiment of the present invention;

7 is a schematic diagram of a hardware structure of an electronic device that executes a method for presenting a video content association interface provided by a method embodiment of the present invention;

8 is a schematic structural diagram of an image acquisition module according to an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of a first mechanical vibration stabilizer provided by an embodiment of the present invention; FIG.

10 is a schematic structural diagram of a bottom surface of a first movable board according to an embodiment of the present invention;

11 is a schematic structural diagram of a second mechanical vibration stabilizer provided by an embodiment of the present invention;

12 is a schematic structural diagram of a bottom surface of a second movable board provided by an embodiment of the present invention;

13 is a structural diagram of a stent provided by an embodiment of the present invention;

FIG. 14 is a schematic state diagram of a stent provided by an embodiment of the present invention; FIG.

15 is a schematic view of another state of a stent according to an embodiment of the present invention;

FIG. 16 is a structural state diagram when the mounting base and the mobile phone are connected according to an embodiment of the present invention.

detailed description

In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described The embodiments are only a part of the embodiments of the present invention, but not all the embodiments. Based on the embodiments in the embodiments of the present invention, all other embodiments obtained by those skilled in the art should belong to the protection scope of the embodiments of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other. FIG. 1 is a flowchart of a method for presenting a video content association interface according to an embodiment of the present invention. As shown in FIG. 1, a method for presenting a video content association interface according to an embodiment of the present invention includes:

Step S101: Obtain a video frame image of a currently playing video content at a preset time.

In the embodiment of the invention, the video content association interface may be operated by a user to perform comments and sharing on the played video content, for example, a comment on the video content and an interface for generating a comment; a sharing and interaction identifier related to the played video content The interface where it is located; recommendation interface related to the video content being played, etc. These video content association interfaces and the currently played video content are presented on the same user interface of the mobile terminal. Therefore, the embodiments of the present invention are directed to enabling the background of the video content association interface to be associated with the currently played video content, so that the video content is associated. The interface presentation is more abundant, flexible and convenient, which improves the user's interactive experience.

That is, the background of the video content association interface can change according to the background of the video content being played. When the video content changes, the background of the video content changes accordingly. Therefore, the video content association interface generated based on the background of the video content being played. The background of the video content will change accordingly. In order to reflect this change in time, when the background of the video content changes, the interface associated with the video content is switched accordingly.

In the embodiment of the present invention, as the scene changes, the background presented by the video content will also change. Generally, each scene is composed of several video frames. Therefore, the current content of the currently playing video can be obtained every preset time. A video frame image. The acquired video frame image can represent a corresponding scene background. Optionally, the preset time may be determined in advance according to the number of video frames corresponding to each scene, or may be an average time for each scene switch obtained by performing statistical analysis on a large number of videos before. The present invention here No restrictions.

In step S102, color information and light angle are extracted from the video frame image.

Specifically, the color and light angle in the extracted video frame image can be used as the background that can reflect the currently playing video content. This step can be divided into two parts, one part is to extract color information from the video frame image, and the other part is to extract the light angle from the video frame image. The execution of the two parts is not sequential.

As an optional implementation manner of the embodiment of the present invention, color information can be extracted in the following three ways.

As shown in FIG. 2, the first embodiment includes the following sub-steps:

S1021: Calculate an average hue of the video frame image.

S1022: Traverse pixels of the video frame image, and calculate a color difference between a hue value of the pixel and the hue average value.

S1023: Determine whether the color difference is greater than a preset threshold.

S1024. Add the pixel to a main-tone pixel list, and determine the color information according to a tone value of the pixel in the main-tone pixel list.

Specifically, we need to extract the color information of the video frame image. The color information is the main color tone that can best represent the video frame image. The main color tone does not refer to the most colorful color, but refers to the image in the video frame The most eye-catching shade. Optionally, the hue value may be represented by an RGB value, or may be represented by a hexadecimal color value that is more commonly applicable now. In this embodiment, each pixel in the video frame image is traversed, and the color difference between the hue value and the hue average value of each pixel is calculated. When the color difference between the hue value and the hue average value of a pixel is greater than a preset threshold, the pixel is added to the main hue pixel list. After all pixels are compared with the tone value, an average value of the tone values of the pixels included in the main-tone pixel list is calculated, and the average value is determined as the color information of the video frame image.

The second embodiment includes the steps of: determining a weight value corresponding to the pixel according to a distance between each pixel in the video frame image and the video content association interface; and according to a hue value and a corresponding weight value of the pixel To calculate the color information.

Specifically, in this embodiment, a weighted average of the hue values of all pixels in the video frame image may be obtained, where the points closer to the video content association interface in the video frame image have greater weights and the farther points The smaller the weight. According to this method, the obtained hue value is more biased to the color of the video frame image in a part close to the video content association interface, so that the transition of the color of the video content association interface and the color of the currently played video content is more natural.

As a third embodiment of color information extraction, the hue value of each pixel of a video frame image may also be directly covered on the video content association interface. If it cannot be completely covered, the weighted average of the above-mentioned video frame image tone is taken for each pixel of the uncovered video content association interface portion; if it can be completely covered, when the video content association interface is smaller than the video frame image interface, the video can be The frame image interface is reduced to the size of the video content association interface.

As an optional implementation of the embodiment of the present invention, as shown in FIG. 3, extracting a light angle from the video frame image includes the following sub-steps:

S1021 ', identifying and extracting a target object image in the video frame image, where the target object is a person or an animal.

Among all the objects in the video frame image, because a character or an animal has facial features, and the facial features of the nose are protruding from the human face, the light and darkness of the nose area are more likely to be affected by the ambient light when taking pictures. Therefore, in this embodiment, The target object is a person or an animal.

Specifically, the recognition process may use an image recognition technology to identify a target object, such as contour extraction, pattern matching, or feature comparison by using multiple types of target object images stored in a preset database library, etc. Image recognition technology is not limited. After the target object is identified, it can be matted along the outline of the identified target object, or an image keying technique, such as the related matting technology in Photoshop, can be used to extract the target object from the first picture.

S1022 ': Extract a sub-image of the nasal region in the target object image, and obtain the light angle based on the light-intensity weighting center of the sub-image and the weight center of the target object image.

Specifically, the facial features of the target object are extracted from the target object image through a human face, animal face detection algorithm, or any CNN-based human face, animal face detection algorithm, etc. that are well known to those skilled in the art. Because the nose is protruding from the face of the target object in the five senses, the light and dark of the nose area are more easily affected by the ambient light when taking pictures. Therefore, in this embodiment, the nose in the target feature is extracted to obtain a sub-image of the nose area.

In this step, the corresponding light intensity weighting center is determined according to the image moment of the sub-image. An image moment is a set of moments calculated from digital graphics. It usually describes the global features of the image and provides a lot of information about the different types of geometric features of the image, such as size, position, orientation, and shape. For example, a The order moment is related to the shape. The second order moment shows the degree of expansion of the curve around the average value of the line, and the third order moment is a measure of the symmetry of the mean value. From the second and third order moments, a group of 7 constants can be derived. Order moments and invariant order moments are statistical characteristics of images, which can be used to classify images based on this, which are common knowledge in the art, and the present invention will not repeat them here.

Optionally, after the light intensity weighting center of the sub-picture is determined, it is compared with the weighting center of the sub-image (the weighting center is the collection center of the image), and the coordinate position of the weighting center of the image is compared with that of the sub-picture. The coordinate position of the light intensity weighted center is compared. The direction from the geometric center to the light intensity weighted center is the direction of the ambient light. At the same time, we can establish the coordinate system by selecting the coordinate origin and obtain the angle between the vector and the X axis. , As the estimated light angle of the video frame image. In addition, the light estimation angle can also be calculated by other non-proprietary algorithms, which is not limited in the present invention. It should be noted that, in the embodiment of the present invention, the ambient light will be considered to be unidirectional and uniform.

As some optional implementations of the embodiment of the present invention, in order to make the light estimation angle more accurate, the sub-image may be divided into several sub-regions, and a sub-light intensity weighting center of each of the sub-regions is determined; The light intensity weighting center is compared with the weighting center of the target object image to obtain an estimated sub-ray angle of each of the sub-regions; a sub-light intensity of each of the sub-regions is calculated; A weight of the sub-ray estimation angle of the sub-area; and calculating a ray angle of the video frame image according to each of the sub-ray estimation angle and the weight of the sub-ray estimation angle.

Specifically, first, the sub-images can be divided into four equal parts to obtain four sub-regions, and the sub-light intensity weighting center of each sub-region and the sub-ray estimation angle of each sub-region are determined according to the above method. Second, for each sub-region, For the picture, the light intensity corresponding to the sub-picture is obtained according to the light and dark contrast information therein. After obtaining the sub-light intensity of each sub-region, the sub-light intensity of each sub-region is used as the weight of the sub-light estimation angle of the sub-region; finally , The sub-ray estimation angles of the four sub-areas are calculated by adding and averaging according to their corresponding weights to obtain the average ray estimation angle.

When there are multiple target object images in the video frame image, the calculation of the light angle can be performed for each target object image in the above manner, and then averaged and summed; or the position of each target object image on the video frame image can also be calculated. Determine the corresponding weights with the pose, and then perform weighted summation according to the weights to obtain the final light angle of the video frame image.

Step S103: Generate a background of the video content association interface according to the color information and the light angle, wherein the video content and the video content association interface are presented on the same interface.

In this step, the background of the video content association interface is generated according to the color information and light angle obtained in step S102, so that the video content association interface is presented with a background that is the same as or similar to the background of the video frame image.

Specifically, this step may include: generating a background color of the video content association interface according to the color information; adjusting the brightness of the background color according to the light angle to obtain the background of the video content association interface.

That is, first, use the hue value obtained in the first or second embodiment in step S102 as the background color of the video content association interface, or perform each step according to the method described in the third embodiment in step S103. Coverage of the pixel tone value; then, the brightness of the background color is adjusted according to the light angle of the video frame image obtained in step S102. Before this, the relative position relationship between the currently playing video content interface and the video content association interface can be adjusted Divide the video content association interface into several parts (for example, if the two interfaces are distributed left and right, the video content association interface is divided into three parts: left, middle, and right; if the two interfaces are distributed up and down, then The video content association interface is divided into four parts: up, down, left, and so on). Determine the positional relationship between each part and the angle of the light. If a part is along the direction of the light angle, increase the brightness of the background color of the part; if a part is against the direction of the light angle, reduce the brightness of the color of the part; If a part is just in the middle of the above two, the brightness of the background color of the part is maintained as a transition part of the brightness.

The embodiments of the present invention are described below with reference to specific application scenarios. Assuming that an application server provides video services, users can watch the video on their smartphones or tablets, and there are user comments on the video content under the playback interface. Among them, The main color of the background of the currently playing video content is green, so the background color of the comment content under the playback interface also appears green, and the brightness of the background will be adjusted according to the light angle of the currently playing video content.

A method for presenting a video content association interface according to an embodiment of the present invention. The video content association interface can be switched according to the background change of the currently played video content, making the video content association interface presentation richer, more flexible, and convenient, and improving the user's interactive experience. .

FIG. 4 is a structural diagram of a presentation device for a video content association interface according to an embodiment of the present invention. As shown in FIG. 3, the device specifically includes: an acquisition module 100, an extraction module 200, and a generation module 300. among them,

The obtaining module 100 is configured to obtain a video frame image of a currently playing video content at a preset time;

An extraction module 200 is configured to extract color information and light angle from the video frame image; a generation module 300 is configured to generate a background of the video content association interface according to the color information and light angle, wherein the video content The interface associated with the video content is presented on the same interface.

The presenting device for a video content association interface provided by an embodiment of the present invention is specifically configured to execute the method provided by the embodiment shown in FIG. 1, and its implementation principles, methods, and functional uses are similar to the embodiment shown in FIG. 1, and are not described herein. More details.

FIG. 5 is a structural diagram of a presentation device for a video content association interface according to an embodiment of the present invention. As shown in FIG. 5, the device specifically includes: an obtaining module 100, an extracting module 200, and a generating module 300. among them,

Further, the extraction module 200 includes: a first calculation unit 210, a traversal unit 220, a determination unit 230, a first determination unit 240, a recognition unit 250, and an extraction unit 260. among them,

A first calculation unit 210 is configured to calculate an average hue of the video frame image; a traversal unit 220 is configured to traverse pixels of the video frame image, and calculate a color difference between the hue value of the pixel and the average hue; A determining unit 230 is configured to determine whether the color difference is greater than a preset threshold; a first determination unit 240 is configured to add the pixels to a main color pixel list if the color difference is greater than a preset threshold, The hue value of the pixel in the list determines the color information.

A recognition unit 250 is configured to recognize and extract a target object image in the video frame image, where the target object is a person or an animal; an extraction unit 260 is configured to extract a sub-image of a nasal region in the target object image , Obtaining the light angle based on the weighted center of the light intensity of the sub-image and the weighted center of the target object image.

Optionally, the generating module 300 is specifically configured to generate a background color of the video content association interface according to the color information; and adjust the brightness of the background color according to the light angle to obtain the video content association interface. background.

The apparatus for presenting a video content association interface provided by the embodiment of the present invention is specifically configured to execute the method provided by the embodiment shown in FIG. 1 to FIG. 3, and its implementation principles, methods, and functional uses are similar to the embodiment shown in FIG. 1 to FIG. 3. Similarly, we will not repeat them here.

FIG. 6 is a structural diagram of a presentation device for a video content association interface according to an embodiment of the present invention. As shown in FIG. 6, the device specifically includes: an obtaining module 100, an extracting module 200, and a generating module 300. among them,

Further, the extraction module 200 includes: a second determination unit 270, a second calculation unit 280, a recognition unit 250, and an extraction unit 260. among them,

A second determining unit 270 is configured to determine a weight value corresponding to the pixel according to a distance between each pixel in the video frame image and the video content association interface; a second calculating unit 280 is configured to determine a weight value according to the pixel The hue value and the corresponding weight value are calculated to obtain the color information.

The apparatus for presenting a video content association interface provided by the embodiment of the present invention is specifically configured to execute the method provided by the embodiment shown in FIG. 1 and FIG. 3, and its implementation principles, methods, and functional uses are similar to the embodiments shown in FIG. 1 and FIG. Similarly, we will not repeat them here.

The above-mentioned presentation device of the video content association interface in the embodiments of the present invention can be used as one of the software or hardware functional units, which can be independently set in the above-mentioned electronic device, or can be used as one of the functional modules integrated in the processor to implement the implementation of the present invention. Presentation method of video content association interface for example.

FIG. 7 is a schematic diagram of a hardware structure of an electronic device that executes a method for presenting a video content association interface provided by a method embodiment of the present invention. According to FIG. 7, the electronic device includes:

One or more processors 710 and a memory 720. One processor 710 is taken as an example in FIG. 7. The device for performing the method for presenting a video content association interface may further include an input device 730 and an output device 730.

The processor 710, the memory 720, the input device 730, and the output device 740 may be connected through a bus or in other manners. In FIG. 7, the connection through the bus is taken as an example.

The memory 720 is a non-volatile computer-readable storage medium, and can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as the video content association interface in the embodiment of the present invention. Program instructions / modules corresponding to the rendering method. The processor 710 executes various functional applications and data processing of the server by running non-volatile software programs, instructions, and modules stored in the memory 720, that is, a method for presenting the video content association interface.

The memory 720 may include a storage program area and a storage data area, where the storage program area may store an operating system and application programs required for at least one function; the storage data area may store a presentation device for a video content association interface provided according to an embodiment of the present invention Use of created data, etc. In addition, the memory 720 may include a high-speed random access memory 720, and may further include a non-volatile memory 720, such as at least one disk memory 720, a flash memory device, or other non-volatile solid-state memory 720. In some embodiments, the memory 720 may optionally include a memory 720 remotely set relative to the processor 77, and these remote memories 720 may be connected to a presentation device of the video content association interface through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The input device 730 may receive inputted numeric or character information, and generate key signal inputs related to user settings and function control of a presentation device of a video content association interface. The input device 730 may include a device such as a pressing module.

The one or more modules are stored in the memory 720, and when executed by the one or more processors 710, perform a method of presenting the video content association interface.

The electronic devices in the embodiments of the present invention exist in various forms, including but not limited to:

(1) Mobile communication equipment: This type of equipment is characterized by mobile communication functions, and its main goal is to provide voice and data communication. Such terminals include: smart phones (such as iPhone), multimedia phones, feature phones, and low-end phones.

(2) Ultra-mobile personal computer equipment: This type of equipment belongs to the category of personal computers, has computing and processing functions, and generally has the characteristics of mobile Internet access. Such terminals include: PDA, MID and UMPC devices, such as iPad.

(3) Portable entertainment equipment: This type of equipment can display and play multimedia content. Such devices include: audio and video players (such as iPod), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

(4) The server.

(5) Other electronic devices with data interaction function.

Specifically, the electronic device includes an image acquisition module. As shown in FIG. 8, the image acquisition module in this embodiment includes a lens 1000, an autofocus voice coil motor 2000, a mechanical image stabilizer 3000, and an image sensor 4000. The lens 1000 is fixed on the auto-focusing voice coil motor 2000, and the lens 1000 is fixed on the auto-focusing voice coil motor 2000. The lens 1000 is used to acquire an image, and the image sensor 4000 sets the lens 1000 The acquired image is transmitted to the recognition module, the auto-focusing voice coil motor 2000 is mounted on the mechanical image stabilizer 3000, and the processing module drives the camera based on the feedback of the lens 1000 shake detected by the gyroscope in the lens 1000. The operation of the mechanical image stabilizer 3000 is described to realize the shake compensation of the lens 1000.

Most of the existing image stabilizers are powered by coils that generate Loren magnetic force in a magnetic field to drive the lens 1000. To achieve optical image stabilization, the lens 1000 needs to be driven in at least two directions, which means that multiple coils need to be arranged. It brings certain challenges to the miniaturization of the overall structure, and is easily affected by external magnetic fields, which affects the anti-shake effect. Therefore, the Chinese patent published as CN106131435A provides a miniature optical anti-shake camera module, which realizes memory through temperature changes. The alloy wire is stretched and shortened to pull the auto-focusing voice coil motor 2000 to achieve the shake compensation of the lens 1000. The control chip of the micro memory alloy optical anti-shake actuator can control the change of the driving signal to change the memory alloy wire. The temperature is used to control the elongation and shortening of the memory alloy wire, and the position and moving distance of the actuator are calculated based on the resistance of the memory alloy wire. When the micro memory alloy optical image stabilization actuator moves to the specified position, the resistance of the memory alloy wire at this time is fed back. By comparing the deviation of this resistance value and the target value, the movement on the micro memory alloy optical image stabilization actuator can be corrected. deviation.

However, the applicant found that due to the randomness and uncertainty of the shake, the structure of the above technical solution alone cannot achieve accurate compensation for the lens 1000 in the case of multiple shakes, which is due to the heating of the shape memory alloy It takes a certain time to cool down and cool down. When the shake occurs in the first direction, the above technical solution can achieve the compensation of the lens 1000 for the shake in the first direction, but when the subsequent shake in the second direction occurs, due to the memory alloy It is too late to deform in an instant, so it is easy to cause compensation in a timely manner, and it is impossible to accurately achieve 1000-shake compensation for a lens that has multiple shakes and continuous shakes in different directions, so it is necessary to improve its structure.

With reference to Figures 9-12, this embodiment improves the optical image stabilizer and design it as a mechanical image stabilizer 3000. The specific structure is as follows:

The mechanical image stabilizer 3000 of this embodiment includes a movable plate 3100, a movable frame 3200, an elastic restoring mechanism 3300, a base plate 3400, and a compensation mechanism 3500. The movable plate 3100 and the base plate 3400 are provided in the middle of the plate for the lens to pass through. Through the through hole 3700, the auto-focusing voice coil motor is installed on the movable plate 3100, and the movable plate 3100 is installed in the movable frame 3200. As can be seen from the figure, the movable plate 3100 of this embodiment The width in the left-to-right direction is substantially the same as the internal width of the movable frame 3200, so that the opposite sides (left and right sides) of the movable plate 3100 and the inner walls of the opposite sides (left and right sides) of the movable frame 3200 slide to fit. The movable plate 3100 can slide back and forth along the first direction within the movable frame 3200. The first direction in this embodiment is the vertical direction in the figure.

Specifically, the size of the movable frame 3200 in this embodiment is smaller than the size of the substrate 3400, and two opposite sides of the movable frame 3200 are connected to the substrate 3400 through two elastic recovery mechanisms 3300, respectively. The elastic restoring mechanism 3300 is a telescopic spring or other elastic member, and it should be noted that the elastic restoring mechanism 3300 of this embodiment only allows it to expand and retract in the left-right direction in the figure (that is, the second direction described below). The ability to move along the first direction cannot be designed. The purpose of designing the elastic recovery mechanism 3300 is also to facilitate the movable frame 3200 to reset the movable plate 3100 after the movable frame 3200 compensates for displacement. The specific action process of this embodiment will be described below. The process is described in detail.

The compensation mechanism 3500 of this embodiment drives the movable plate 3100 and the lens on the movable plate 3100 under the driving of the processing module (which may be an action instruction sent by the processing module) to implement lens shake compensation.

Specifically, the compensation mechanism 3500 in this embodiment includes a driving shaft 3510, a gear 3520, a gear track 3530, and a limit track 3540. The driving shaft 3510 is mounted on the substrate 3400, and specifically is mounted on the substrate 3400. On the surface, the drive shaft 3510 is drivingly connected to the gear 3520. The drive shaft 3510 can be driven by a structure such as a micro motor (not shown), and the micro motor is controlled by the processing module described above; the gear track 3530 is provided On the movable plate 3100, the gear 3520 is installed in the gear track 3530 and moves in a preset direction of the gear track 3530. When the gear 3520 rotates, the gear track 3530 makes the movement The plate 3100 can generate a displacement in a first direction and a displacement in a second direction, wherein the first direction is perpendicular to the second direction; the limit track 3540 is disposed on the movable plate 3100 or the base plate 3400 In the above, the limit track 3540 is used to prevent the gear 3520 from leaving the gear track 3530.

Specifically, the gear track 3530 and the limit track 3540 of this embodiment have the following two structural forms:

As shown in FIGS. 8-10, a waist-shaped hole 3550 is provided on the lower side of the movable plate 3100 in this embodiment, and a plurality of waist-shaped holes 3550 are provided along the circumferential direction (that is, the surrounding direction of the waist-shaped hole 3550). A tooth 3560 meshing with the gear 3520, the waist-shaped hole 3550 and a plurality of the teeth 3560 together form the gear track 3530, and the gear 3520 is located in the waist-shaped hole 3550 and communicates with the tooth 3560. The meshing makes the gear 3520 move the gear track 3530 when it rotates, and then directly drives the movement of the movable plate 3100. In order to ensure that the gear 3520 keeps meshing with the gear track 3530 at all times during the rotation, this embodiment describes A limiting rail 3540 is provided on the base plate 3400, and a bottom of the movable plate 3100 is provided with a limiting member 3570 installed in the limiting rail 3540. The limiting rail 3540 enables the limiting member 3570 to be located therein. The movement trajectory is waist-shaped, that is, the movement trajectory of the limiter 3570 in the current track is the same as the movement trajectory of the movable plate 3100. Specifically, the limiter 3570 of this embodiment is provided on the movable plate 3100. Bulge on the bottom.

In combination with FIGS. 11 and 12, the gear track 3530 of this embodiment may also be composed of a plurality of cylindrical protrusions 3580 provided on the movable plate 3100. The gears 3520 are arranged at regular intervals in the second direction, and the gears 3520 are engaged with a plurality of the protrusions; and the limit track 3540 is a first arc-shaped stopper 3590 and a second arc provided on the movable plate 3100. Shape limiting member 3600, the first arc-shaped limiting member 3590 and the second arc-shaped limiting member 3600 are respectively disposed on opposite sides of the gear rail 3530 in the first direction to prevent movement on the movable plate 3100 When the preset position is reached, the gear 3520 is located on one side of the gear rail 3530. The gear 3520 is easily separated from the gear rail 3530 formed by the cylindrical protrusion 3580. Therefore, the first arc-shaped stopper 3590 or the second arc-shaped stopper 3600 can Plays a guiding role, so that the movable plate 3100 can move in a preset direction of the gear track 3530, that is, the first arc-shaped limiting member 3590, the second arc-shaped limiting member 3600, and a plurality of the protrusions cooperate The motion trajectory of the movable board 3100 is waist-shaped.

The following describes the working process of the mechanical image stabilizer 3000 of this embodiment in detail with reference to the above structure. Taking the lens 1000 as an example of two shakes, the two shake directions are opposite, and the movable plate 3100 needs to be compensated once in the first direction. And then motion compensation once in the second direction. When the movable plate 3100 needs motion compensation in the first direction, the gyroscope feeds back the detected lens 1000 shake direction and distance to the processing module in advance, and the processing module calculates the required moving distance of the movable plate 3100, so that the driving shaft 3510 drives The gear 3520 rotates. The gear 3520 cooperates with the gear track 3530 and the limit track 3540, and the processing module wirelessly sends a driving signal, thereby driving the movable plate 3100 to move to the compensation position in the first direction. After compensation, the movable plate is driven again by the driving shaft 3510. 3100 reset. During the reset process, the elastic recovery mechanism 3300 also provides a reset force for resetting the movable plate 3100, which is more convenient for the movable plate 3100 to return to the initial position. When the movable plate 3100 needs to be compensated for motion in the second direction, the processing method is the same as the compensation steps in the first direction described above.

Of course, the above is just two simple jitters. When multiple jitters occur or the direction of the jitter is not reciprocating, you can drive multiple compensation components to compensate for the jitter. The basic working process is the same as the principle described above. To repeat, the detection feedback from the gyroscope and the control module sending control instructions to the drive shaft 3510 are all existing technologies, and are not described here too.

It can be known from the above description that the mechanical compensator provided in this embodiment not only does not receive interference from external magnetic fields and has a good anti-shake effect, but also can accurately compensate the lens 1000 in the event of multiple shakes, and the compensation is timely and accurate. In addition, the mechanical anti-shake device using this embodiment is not only simple in structure, but also requires small installation space for each component, which facilitates the integration of the entire anti-shake device and has higher compensation accuracy.

Specifically, the electronic device in this embodiment includes a mobile phone and a bracket for mounting the mobile phone. The purpose of the electronic device including a bracket is to support and fix the electronic device due to the uncertainty of the image acquisition environment.

In addition, the applicant found that the existing mobile phone holder only has the function of supporting the mobile phone, but does not have the function of a selfie stick, so the applicant made the first step improvement to the holder, combining the mobile phone holder 5000 and the support rod 5200, As shown in FIG. 13, the bracket 5000 in this embodiment includes a mobile phone mounting base 5100 and a retractable supporting rod 5200. The supporting rod 5200 and the middle portion of the mobile phone mounting base 5100 (specifically, the middle portion of the substrate 3200 described below) pass through a damping hinge. When the supporting rod 5200 is rotated to the state of FIG. 13, the bracket 5000 may form a selfie stick structure, and when the supporting rod 5200 is rotated to the state of FIG. 14, the bracket 5000 may form a mobile phone bracket 5000 structure.

In combination with the above bracket structure, the applicant also found that the mobile phone mount 5100 combined with the support pole 5200 occupies a large space. Even if the support pole 5200 is retractable, the mobile phone mount 5100 cannot undergo structural changes and the volume will not be further reduced. Putting it in a pocket or a small bag causes the inconvenience of carrying the stent 5000. Therefore, in this embodiment, a second step improvement is made to the stent 5000, so that the overall storability of the stent 5000 is further improved.

As shown in FIGS. 13-16, the mobile phone mounting base 5100 of this embodiment includes a retractable connecting plate 5110 and a folding plate group 5120 installed at opposite ends of the connecting plate 5110. The support rod 5200 and the connecting plate 5110 The middle part is connected by a damping hinge; the folding plate group 5120 includes a first plate body 5121, a second plate body 5122, and a third plate body 5123, wherein one of the two opposite ends of the first plate body 5121 is connected to the first plate body 5121. The connecting plate 5110 is hinged, the other end of the opposite ends of the first plate body 5121 is hinged with one of the opposite ends of the second plate body 5122, and the opposite ends of the second plate body 5122 The other end is hinged to one of opposite ends of the third plate body 5123; the second plate body 5122 is provided with an opening 5130 for a corner of the mobile phone to be inserted.

As shown in FIG. 16, when the mobile phone mounting base 5100 is used to install a mobile phone, the first plate 5121, the second plate 5122, and the third plate 5123 are folded into a right triangle state, and the second plate 5122 is a hypotenuse of a right-angled triangle, and the first plate body 5121 and the third plate 5123 are right-angled sides of a right triangle, wherein one side surface of the third plate body 5123 and one of the connection plate 5110 are The side is attached side by side, and the other end of the opposite ends of the third plate body 5123 and the one of the opposite ends of the first plate body 5121 are against each other. This structure can make the three folding plates in a self-locking state, and When the two lower corners of the mobile phone are inserted into the two openings 5130 on both sides, the lower sides of the mobile phone 6000 are located in two right-angled triangles. The mobile phone 6000 can be completed through the joint work of the mobile phone, the connecting plate 5110, and the folding plate group 5120. The triangle state cannot be opened under external force. The triangle state of 5120 pieces of folding plate group can only be released after the mobile phone is pulled out from the opening 5130.

When the mobile phone mounting base 5100 is not in working state, the connecting plate 5110 is reduced to a minimum length, and the folding plate group 5120 and the connecting plate 5110 are folded to each other. The user can fold the mobile phone mounting base 5100 to a minimum volume. Due to the scalability of the lever 5200, the entire bracket 5000 can be accommodated in the smallest state, which improves the collection of the bracket 5000. The user can even put the bracket 5000 directly into the pocket or small handbag, which is very convenient.

Preferably, in this embodiment, a first connection portion is also provided on one side of the third plate body 5123, and a side surface where the connection plate 5110 is in contact with the third plate body 5123 is provided with the first connection portion. A first mating portion that mates with the connecting portion. When the bracket 5000 mobile phone mounting base 5100 is used to install a mobile phone, the first connecting portion and the first mating portion are snap-connected. Specifically, the first connecting portion of this embodiment is a convex strip or protrusion (not shown in the figure), and the first matching portion is a card slot (not shown in the figure) opened on the connecting plate 5110. This structure not only improves the stability when the 5120 pieces of the folding plate group are in a triangular state, but also facilitates the connection of the 5120 pieces of the folding plate group and the connecting plate 5110 when the mobile phone mounting base 5100 needs to be folded to a minimum state.

Preferably, in this embodiment, a second connection portion is also provided at one end of the opposite ends of the first plate body 5121, and the other end of the opposite ends of the third plate body 5123 is provided with the second connection portion. When the mobile phone mounting base 5100 of the bracket 5000 is used for mounting a mobile phone, the second connecting portion is engaged with the second matching portion. The second connection portion may be a protrusion (not shown in the figure), and the second mating portion is an opening 5130 or a card slot (not shown in the figure) that cooperates with the protrusion. This structure improves the stability when the laminated board assembly is in a triangular state

In addition, in this embodiment, a base (not shown in the figure) can be detachably connected to the other end of the support rod 5200. When the mobile phone needs to be fixed and the mobile phone 6000 has a certain height, the support rod 5200 can be stretched to A certain length, put the bracket 5000 on a plane through the base, and then place the mobile phone in the mobile phone mount 5100 to complete the fixing of the mobile phone; and the detachable connection of the support rod 5200 and the base can make the two can be carried separately, further The accommodating and carrying convenience of the bracket 5000 are improved.

The device embodiments described above are only schematic, and the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, which may be located in One place, or can be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the objective of the solution of this embodiment. Those of ordinary skill in the art can understand and implement without creative labor.

An embodiment of the present invention provides a non-transitory computer-readable storage storage medium, where the computer storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by an electronic device, the electronic device is caused The method for presenting a video content association interface in any of the foregoing method embodiments is performed on the above.

An embodiment of the present invention provides a computer program product, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, and when the program instructions When executed by an electronic device, the electronic device is caused to execute the method for presenting a video content association interface in any of the foregoing method embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary universal hardware platform, and of course, also by hardware. Based on such an understanding, the above-mentioned technical solution in essence or a part that contributes to the existing technology may be embodied in the form of a software product, and the computer software product may be stored in a computer-readable storage medium, the computer-readable record A medium includes any mechanism for storing or transmitting information in a form readable by a computer (eg, a computer). For example, machine-readable media include read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash storage media, electrical, optical, acoustic, or other forms of propagation signals (e.g., carrier waves , Infrared signals, digital signals, etc.), the computer software product includes a number of instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute various embodiments or certain parts of the embodiments Methods.

In the end, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than limiting them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and range.

Claims

A method for presenting a video content association interface, comprising:

Acquiring video frame images of currently playing video content every preset time;

Extracting color information and light angle from the video frame image;

The background of the video content association interface is generated according to the color information and the light angle, wherein the video content and the video content association interface are presented on the same interface.
The method according to claim 1, wherein the extracting color information from the video frame image comprises:

Calculating an average hue of the video frame image;

Traverse the pixels of the video frame image, and calculate the color difference between the hue value of the pixel and the hue average value;

Determining whether the color difference is greater than a preset threshold;

If the color difference is greater than a preset threshold, the pixel is added to a main-tone pixel list, and the color information is determined according to a tone value of the pixel in the main-tone pixel list.
The method according to claim 1, wherein the extracting color information from the video frame image comprises:

Determining a weight value corresponding to the pixel according to a distance between each pixel in the video frame image and the video content association interface;

The color information is calculated and obtained according to the hue value of the pixel and the corresponding weight value.
The method according to any one of claims 1 to 3, wherein the extracting a light angle from the video frame image comprises:

Identifying and extracting a target object image in the video frame image, wherein the target object is a person or an animal;

The sub-image of the nasal region in the target object image is extracted, and the light angle is obtained based on the light intensity weighting center of the sub-image and the weight center of the target object image.
The method according to claim 1, wherein the generating a background of the video content association interface according to the color information and the light signal angle comprises:

Generating a background color of the video content association interface according to the color information;

The brightness of the background color is adjusted according to the light angle to obtain the background of the video content association interface.
A device for presenting a video content association interface is characterized in that it includes:

An acquisition module, configured to acquire a video frame image of a currently playing video content at a preset time;

An extraction module, configured to extract color information and light angles from the video frame image;

A generating module is configured to generate a background of the video content association interface according to the color information and light angle, wherein the video content and the video content association interface are presented on a same interface.
The apparatus according to claim 6, wherein the extraction module comprises:

A first calculation unit, configured to calculate an average hue of the video frame image;

A traversal unit, configured to traverse pixels of the video frame image, and calculate a color difference between a hue value of the pixel and the hue average value;

A judging unit, configured to judge whether the color difference is greater than a preset threshold;

A first determining unit is configured to add the pixel to a main-tone pixel list if the color difference is greater than a preset threshold, and determine the color information according to a tone value of the pixel in the main-tone pixel list.
The apparatus according to claim 6, wherein the extraction module comprises:

A second determining unit, configured to determine a weight value corresponding to the pixel according to a distance between each pixel in the video frame image and the video content association interface;

A second calculation unit is configured to calculate the color information according to a hue value of the pixel and a corresponding weight value.
The apparatus according to any one of claims 6 to 8, wherein the extraction module includes:

A recognition unit, configured to recognize and extract a target object image in the video frame image, where the target object is a person or an animal;

An extraction unit is configured to extract a sub-image of a nasal region in the target object image, and obtain the light angle based on a light intensity weighting center of the sub-image and a weight center of the target object image.
An electronic device, comprising: at least one processor; and

A memory connected in communication with the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any one of claims 1 to 5. Rendering method of video content association interface.