CN110300316B - Method and device for implanting push information into video, electronic equipment and storage medium - Google Patents

Method and device for implanting push information into video, electronic equipment and storage medium Download PDF

Info

Publication number
CN110300316B
CN110300316B CN201910704475.1A CN201910704475A CN110300316B CN 110300316 B CN110300316 B CN 110300316B CN 201910704475 A CN201910704475 A CN 201910704475A CN 110300316 B CN110300316 B CN 110300316B
Authority
CN
China
Prior art keywords
image
video
foreground
implanted
frame image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910704475.1A
Other languages
Chinese (zh)
Other versions
CN110300316A (en
Inventor
生辉
陈戈
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201910704475.1A priority Critical patent/CN110300316B/en
Publication of CN110300316A publication Critical patent/CN110300316A/en
Application granted granted Critical
Publication of CN110300316B publication Critical patent/CN110300316B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2668—Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/812—Monomedia components thereof involving advertisement data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Abstract

The embodiment of the invention provides a method, a device, electronic equipment and a storage medium for implanting push information into a video; the method comprises the following steps: carrying out entity identification on a frame image of a video to be implanted, and determining an entity area corresponding to an entity in the frame image; acquiring an image to be implanted carrying push information of a corresponding entity area; implanting an image to be implanted into the solid area of the frame image by taking the solid area of the frame image as a background and the corresponding image to be implanted as a foreground to obtain a target frame image; adjusting image parameters of the foreground in the target frame image to enable the image parameters of the foreground to be matched with the image parameters of the background; and carrying out video synthesis based on the adjusted target frame image to obtain a target video. Therefore, the harmony degree of the push information and the video content in the target video can be improved.

Description

Method and device for implanting push information into video, electronic equipment and storage medium
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a method and a device for implanting push information into a video, electronic equipment and a storage medium.
Background
In a video playing scene, besides the content of the video itself, there is a need to display push information, and one of the display modes of the push information is to embed the push information into the video so as to display the embedded push information in the video playing process.
In the related art, for the implantation of the push information in the video, a professional is required to manually complete, and the implanted push information is manually adjusted, so as to avoid a visual obtrusiveness caused by the incoordination between the implanted push information and the video content, however, the implantation method has low implantation efficiency and high labor cost.
Disclosure of Invention
The embodiment of the invention provides a method, a device, electronic equipment and a storage medium for implanting push information into a video, which can automatically realize the implantation of the push information into the video and the harmony of the push information and the video content after the implantation of the push information.
The embodiment of the invention provides a method for implanting push information in a video, which comprises the following steps:
carrying out entity identification on a frame image of a video to be implanted, and determining an entity area corresponding to the entity in the frame image;
acquiring an image to be implanted which corresponds to the entity area and carries push information;
implanting the image to be implanted into the solid area of the frame image by taking the solid area of the frame image as a background and the corresponding image to be implanted as a foreground to obtain a target frame image;
adjusting image parameters of the foreground in the target frame image to enable the image parameters of the foreground to be matched with the image parameters of the background;
and carrying out video synthesis based on the adjusted target frame image to obtain a target video.
The embodiment of the invention also provides a device for implanting push information in the video, which comprises:
the identification unit is used for carrying out entity identification on a frame image of a video to be implanted and determining an entity area corresponding to the entity in the frame image;
the acquisition unit is used for acquiring the image to be implanted which corresponds to the entity area and carries push information;
an implantation unit, configured to implant the image to be implanted in the solid area of the frame image to obtain a target frame image, with the solid area of the frame image as a background and the corresponding image to be implanted as a foreground;
the adjusting unit is used for adjusting the image parameters of the foreground in the target frame image so that the image parameters of the foreground are matched with the image parameters of the background;
and the synthesizing unit is used for carrying out video synthesis on the basis of the adjusted target frame image to obtain a target video.
An embodiment of the present invention further provides an electronic device, including:
a memory for storing executable instructions;
and the processor is used for implementing the method for implanting the push information into the video provided by the embodiment of the invention when the processor executes the executable instructions stored in the memory.
The embodiment of the invention also provides a storage medium, which stores executable instructions and is used for causing a processor to execute the executable instructions so as to realize the method for implanting the push information into the video provided by the embodiment of the invention.
The application of the embodiment of the invention has the following beneficial effects:
1) implanting the image to be implanted into the solid area of the frame image of the video to be implanted by taking the solid area of the frame image as a background and the corresponding image to be implanted as a foreground to obtain a target frame image, thereby realizing the automatic implantation of the push information into the video to be implanted and improving the implantation efficiency of the push information;
2) by adjusting the image parameters of the foreground in the target frame image, the image parameters of the foreground are matched with those of the background, so that the automatic harmony processing of the foreground and the background in the video frame image is realized after the push information is implanted, the harmony of the push information and the video content in the target video is realized, and the receiving degree of the push information by a user when watching the video is greatly improved.
Drawings
Fig. 1 is a schematic diagram of a system 100 for embedding push information in video according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a hardware structure of a server according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating a method for embedding push information in a video according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating an alternative data structure of a video according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a frame image of a video before embedding push information according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a frame image of a video after push information is implanted according to an embodiment of the present invention;
fig. 7 is a schematic diagram of screening an image to be implanted corresponding to a desktop area according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a three-dimensional model and a poster for determining a matching presentation angle with a desktop according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of implanting an image to be implanted in a photo frame region according to an embodiment of the present invention;
fig. 10 is a schematic diagram of implanting an image to be implanted in a screen area of a television according to an embodiment of the present invention;
fig. 11 is a schematic diagram illustrating adjustment of image parameters of a foreground in an intercepted image according to an embodiment of the present invention;
FIG. 12 is a schematic diagram of a target frame image after image parameter adjustment according to an embodiment of the present invention;
fig. 13 is a flowchart illustrating a method for embedding push information in a video according to an embodiment of the present invention;
fig. 14 is a flowchart illustrating a method for embedding push information in a video according to an embodiment of the present invention;
FIG. 15 is a display diagram of an advantage of advertisement placement in a video according to an embodiment of the present invention;
FIG. 16 is a schematic diagram illustrating the annual estimated revenue of advertisements embedded in a video according to an embodiment of the present invention;
fig. 17 is a schematic structural diagram of an apparatus for embedding push information in a video according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail with reference to the accompanying drawings, the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.
In the description that follows, references to the terms "first", "second", and the like, are intended only to distinguish similar objects and not to indicate a particular ordering for the objects, it being understood that "first", "second", and the like may be interchanged under certain circumstances or sequences of events to enable embodiments of the invention described herein to be practiced in other than the order illustrated or described herein.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.
Before further detailed description of the embodiments of the present invention, terms and expressions mentioned in the embodiments of the present invention are explained, and the terms and expressions mentioned in the embodiments of the present invention are applied to the following explanations.
1) The entity, as used herein, refers to a main body or a real object to be embedded in a video for bearing push information, such as a desktop, a wall surface, a photo frame, a screen of an electronic device, and the like.
2) A frame image, which is the minimum unit of a video and is a static image; for example, when video information is played, a picture at any time is frozen, i.e., a frame image is obtained.
3) The shot is the basic unit of video data, and in the video shooting process, a section of video continuously shot by a camera is called a shot.
4) Key frame images, which can be used to represent images of the shot content, after the shot structure is cut out, the key frame images are used to represent the underlying features of each shot, so as to perform further video structuring; in a video shot, the number of key frames is much smaller than the number of frame images contained in the shot.
5) In response to the condition or state on which the performed operation depends, one or more of the performed operations may be in real-time or may have a set delay when the dependent condition or state is satisfied; there is no restriction on the order of execution of the operations performed unless otherwise specified.
Fig. 1 is an alternative architecture diagram of a system 100 for embedding push information in video according to an embodiment of the present invention, and referring to fig. 1, to implement supporting an exemplary application, terminals (including a terminal 400-1 and a terminal 400-2) are connected to a server 200 through a network 300, where the network 300 may be a wide area network or a local area network, or a combination of the two, and data transmission is implemented using a wireless or wired link.
A terminal (e.g., terminal 400-1) configured to send an implantation request for pushing information to the server 200, where the implantation request carries a video identifier of a video to be implanted, and in some embodiments, the implantation request may also carry an image to be implanted;
the server 200 is configured to perform entity identification on a frame image of a video to be implanted based on the received implantation request, and determine an entity area corresponding to an entity in the frame image; acquiring an image to be implanted carrying push information of a corresponding entity area; implanting an image to be implanted into the solid area of the frame image by taking the solid area of the frame image as a background and the corresponding image to be implanted as a foreground to obtain a target frame image; adjusting image parameters of the foreground in the target frame image to enable the image parameters of the foreground to be matched with the image parameters of the background; performing video synthesis based on the adjusted target frame image to obtain a target video, and returning the target video to the terminal;
here, in practical applications, the server 200 may be a single server configured to support various services, or may be a server cluster.
The terminal (terminal 400-1 and/or terminal 400-2) is further configured to play the target video embedded with the push information.
In practical applications, the terminal may be various types of user terminals such as a smart phone, a tablet computer, a notebook computer, and the like, and may also be a wearable computing device, a Personal Digital Assistant (PDA), a desktop computer, a cellular phone, a media player, a navigation device, a game console, a television, and the like, or a combination of any two or more of these data processing devices or other data processing devices.
An electronic device implementing the method for embedding push information in video according to the embodiment of the present invention is described below. In some embodiments, the electronic device may be a terminal of various types, such as a smartphone, a tablet computer, a notebook computer, or the like, and may also be a server. The embodiment of the invention takes the electronic equipment as an example of the server, and the hardware structure of the server is explained in detail.
Fig. 2 is a schematic diagram of a hardware structure of a server according to an embodiment of the present invention, and it is understood that fig. 2 only shows an exemplary structure of the server, and not a whole structure, and a part of or the whole structure shown in fig. 2 may be implemented as needed. Referring to fig. 2, a server provided in an embodiment of the present invention includes: at least one processor 201, memory 202, user interface 203, and at least one network interface 204. The various components in the server are coupled together by a bus system 205. It will be appreciated that the bus system 205 is used to enable communications among the components. The bus system 205 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 205 in fig. 2.
The user interface 203 may include, among other things, a display, a keyboard, a mouse, a trackball, a click wheel, a key, a button, a touch pad, or a touch screen.
It will be appreciated that the memory 202 can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory.
The memory 202 in embodiments of the present invention is used to store various types of data to support the operation of the server. Examples of such data include: any executable instructions for operating on a server, such as executable instructions, may be included in the program for implementing the method of an embodiment of the invention.
The method for embedding push information in a video disclosed by the embodiment of the invention can be implemented by the processor 201. The processor 201 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the method for embedding push information in video may be implemented by an integrated logic circuit of hardware in the processor 201 or by instructions in the form of software. The Processor 201 may be a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 201 may implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present invention. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed by the embodiment of the invention can be directly implemented by a hardware decoding processor, or can be implemented by combining hardware and software modules in the decoding processor. The software module may be located in a storage medium located in the memory 202, and the processor 201 reads the information in the memory 202, and completes the steps of the method for embedding push information in video provided by the embodiment of the present invention in combination with hardware thereof.
Next, a method for embedding push information in a video according to an embodiment of the present invention is described, fig. 3 is a flowchart illustrating a method for embedding push information in a video according to an embodiment of the present invention, and in some embodiments, the method for embedding push information in a video may be implemented by a server or a terminal, or implemented by a server and a terminal in a cooperation manner, for example, implemented by the server 200 in fig. 1, and with reference to fig. 1 and 3, the method for embedding push information in a video according to an embodiment of the present invention includes:
step 301: and the server identifies the entity of the frame image of the video to be implanted and determines the entity area corresponding to the entity in the frame image.
In practical applications, the video to be embedded can be either a complete video, such as a complete movie file, or a video segment, such as a segment excerpt of a movie.
In practical application, the entity is a main body or an object to be embedded in a video for bearing push information, such as a desktop, a wall surface, a photo frame, and a screen of an electronic device (e.g., a television screen); in practical implementation, the entity in the frame image may be identified by a Single-polygon Multi-Box Detector (SSD), or may be identified by a mask RCNN, which is an example segmentation algorithm, and the embodiment of the present invention is not limited thereto. In practical application, entities needing to be identified can be preset, such as only identifying a desktop in a frame image, or identifying a wall surface and a photo frame in the frame image.
In some embodiments, entity identification may be performed on a frame image of a video to be implanted, and an entity area corresponding to an entity in the frame image may be determined as follows:
extracting key frames of the video to be implanted, performing entity identification on the key frames of the video to be implanted, and determining entity areas corresponding to entities in each key frame.
In some embodiments, entity identification may be further performed on a frame image of the video to be implanted, and an entity area corresponding to an entity in the frame image is determined by:
segmenting a video to be implanted to obtain a plurality of video segments; and respectively carrying out entity identification on the plurality of video fragments, and determining entity areas corresponding to the entities in the frame images of the video fragments.
In practical application, different entity identifications can be carried out on different video fragments, for example, a video is divided into 10 video fragments, desktop identification is carried out on 1 st to 3 rd video fragments, photo frame identification is carried out on 4 th to 6 th video fragments, and wall identification is carried out on 7 th to 10 th video fragments; of course, the same entity identification can be performed for different video segments, and still taking the example that the video is divided into 10 video segments, the entity identification of the desktop and the photo frame is performed for 10 video segments.
Here, a data structure of a video will be explained. Fig. 4 is a schematic diagram of an alternative data structure of a video according to an embodiment of the present invention, referring to fig. 4, the video data can be divided into four levels of movies, scenes, shots, and frames in structure, and a visually continuous video is formed by continuously showing a still image on a screen or a display, where the still image is a video frame; in the video shooting process, a section of video continuously shot by a camera is called a shot, the shot is a basic unit of video data, a plurality of shots with similar contents form a scene, the shots describe the same event from different angles, and a movie consists of a plurality of scenes and describes a complete story.
Based on the data structure of the video, in practical implementation, the implanted video can be segmented in the following way to obtain a plurality of video segments: carrying out shot switching detection on a video frame of a video to be implanted to obtain a plurality of shots corresponding to the video to be implanted; and segmenting the video to be implanted based on the plurality of lenses to obtain video segments corresponding to the lenses.
Here, the lens switching detection will be described. In practical applications, shot cut detection can use the characteristics of shots when they are cut to find the position where the cut occurs, so as to divide the whole video into individual shots. For example, shot cut detection for a video to be detected can be achieved as follows: calculating the difference degree of pixel points at the same position in adjacent video frames of the video to be detected by adopting an interframe pixel point matching method, determining the number of the pixel points of which the difference degree exceeds a first difference threshold value in the two adjacent video frames, and determining that the shot switching occurs between the two video frames when the preset number threshold value is reached.
Based on the data structure of the video, when the data structure is actually implemented, the video to be implanted can be segmented in the following mode to obtain a plurality of video fragments: carrying out scene switching detection on video frames of a video to be implanted to obtain a plurality of scenes corresponding to the video to be implanted; and segmenting the video to be implanted based on a plurality of scenes to obtain video segments corresponding to the scenes.
Here, in practical applications, the scene change detection of the video to be detected can be implemented as follows: and calculating the histogram difference degree of adjacent video frames of the video to be detected, and determining that scene switching occurs between two video frames of which the histogram difference degree reaches a second difference threshold value.
Step 302: and the server acquires the image to be implanted carrying the push information corresponding to the entity area.
In practical applications, the pushed information may be an advertisement, specifically, an embedded advertisement, also called Video-In, which is a soft advertisement form, and refers to an entity area In a Video frame, such as a desktop, a wall surface, a photo frame, a bar counter, a billboard, and the like, where the pushed information is embedded. Fig. 5 is a schematic diagram of a frame image of a video before push information is implanted, according to an embodiment of the present invention, fig. 6 is a schematic diagram of a frame image of a video after push information is implanted, and it can be known from fig. 5 and fig. 6 that iced black tea serving as push information is implanted in a desktop area in a video frame.
In some embodiments, the push information may include at least one of: pushing entities and characters; wherein, the pushing entity is an entity which displays the advertisement in a tangible material form mode, such as a coffee cup with a specific shape; the push text is a text that is used for displaying an advertisement with specific content in a text form, for example, the text is used for describing the function of a specific electronic device.
In practical implementation, different entity areas may correspond to different pieces of push information, or different entity areas may correspond to the same piece of push information. For example, in the case that the entity area is a desktop and a photo frame, the desktop may carry push information including a three-dimensional model and a poster form, and the photo frame may carry push information including a poster form; here, a poster is one of the presentation forms of visual communication, and displays advertisement information in a specific form by completely combining elements such as pictures, characters, colors, and spaces.
The description will be given by taking the identified entity as the desktop and the corresponding entity area as the desktop area. In some embodiments, the push information corresponding to the desktop area includes a push entity, and for the same push entity, there may be images to be implanted at different presentation angles, and for different presentation angles of the desktop, an image to be implanted that matches the presentation angle of the desktop may be selected.
In some embodiments, the server may obtain the to-be-implanted image carrying the push information of the corresponding entity area by: acquiring a video identifier corresponding to a video to be implanted, determining a pushing entity corresponding to a desktop area of the video to be implanted based on the acquired video identifier, and acquiring a first presentation angle of the pushing entity in an image to be implanted; acquiring a second presentation angle of an entity presented in the entity area; and determining the image to be implanted with the first presentation angle matched with the second presentation angle, wherein the image to be implanted is the image to be implanted corresponding to the entity area.
Here, taking the pushing entity as iced black tea as an example, corresponding to a desktop area, a plurality of to-be-implanted images including iced black tea entities exist, and presentation angles of the iced black tea entities in different to-be-implanted images are different, referring to fig. 7, fig. 7 is a schematic diagram of screening the to-be-implanted images corresponding to the desktop area provided by the embodiment of the present invention, in actual implementation, edge detection is performed on the desktop and the to-be-implanted images respectively, then straight line fitting is performed on the lower edges of the desktop and the pushing entity respectively, a first presentation angle and a second presentation angle are determined respectively based on the straight lines obtained by fitting, and the to-be-implanted image with the smallest difference between the first presentation angle and the second presentation angle corresponding to the pushing entity in the plurality of to-be-implanted images is selected and is used as the to-be-implanted image corresponding to the entity area. When the push information includes a three-dimensional model and a poster, a first presentation angle of the three-dimensional model is obtained in the same manner, and the three-dimensional model and the poster matched with a second presentation angle are determined, referring to fig. 8, and fig. 8 is a schematic diagram of the three-dimensional model and the poster matched with a desktop and determined in the presentation angle provided by the embodiment of the present invention.
The identified entity is a photo frame, a wall surface or an electronic device screen, and the corresponding entity area is a photo frame area, a wall surface area or an electronic device screen area. In some embodiments, the server may obtain the to-be-implanted image carrying the push information of the corresponding entity area by: acquiring a video identifier corresponding to a video to be implanted, and determining that an image to be implanted corresponding to the video to be implanted is at least one of the following images based on the video identifier: the image embedding method comprises the steps of embedding an image to be embedded corresponding to a photo frame region, embedding an image to be embedded corresponding to a wall surface region and embedding an image to be embedded corresponding to an electronic equipment screen region.
Step 303: and the server takes the entity area of the frame image as a background and the corresponding image to be implanted as a foreground, and implants the image to be implanted in the entity area of the frame image to obtain the target frame image.
In practical application, when a frame image is a key frame directly extracted from a video, for each key frame, a server takes an entity area of the key frame image as a background and a corresponding image to be implanted as a foreground, and implants the image to be implanted in the center of the entity area of the key frame image to obtain a target frame image.
In practical application, when a frame image is a key frame of a video fragment, for each video fragment, a server takes an entity area of the key frame image in the video fragment as a background and a corresponding image to be implanted as a foreground, and implants the image to be implanted in the center of the entity area of the key frame image to obtain a target frame image.
Taking the solid area as a photo frame area and a tv screen as an example, referring to fig. 9, fig. 9 is a schematic view of implanting an image to be implanted in the photo frame area according to an embodiment of the present invention, in fig. 9, reference numeral 91 is a photo frame area before the image to be implanted is implanted, and reference numeral 92 is a photo frame area after the image to be implanted is implanted. Referring to fig. 10, fig. 10 is a schematic view of implanting an image to be implanted in a screen area of a television according to an embodiment of the present invention, in fig. 10, reference numeral 11 is a picture frame area before the image to be implanted is implanted, and reference numeral 12 is a picture frame area after the image to be implanted is implanted.
Step 304: and the server adjusts the image parameters of the foreground in the target frame image so that the image parameters of the foreground are matched with the image parameters of the background.
In some embodiments, the server may adjust image parameters of the foreground in the target frame image directly based on the background in the target frame image, so that the image parameters of the foreground and the image parameters of the background match; in some embodiments, the server may perform image interception on the target frame image to obtain an intercepted image containing a foreground, and then adjust image parameters of the foreground in the intercepted image based on a background of the intercepted image (a local background of the target frame image) so that the image parameters of the foreground are matched with the image parameters of the background; the area of the intercepted image is a constant multiple of the area of the foreground, for example, the foreground is taken as the center, and twice the area of the foreground is the size of the intercepted image, so that the target frame image is intercepted.
Taking image interception on the target frame image to obtain an intercepted image containing the foreground as an example, the adjustment of the image parameters of the foreground is explained.
In practical applications, the image parameters include at least one of: brightness, saturation, blur, and shadow length.
In some embodiments, the image parameters include brightness, and the image parameters of the foreground in the target frame image may be adjusted such that the image parameters of the foreground and the image parameters of the background match by:
acquiring the brightness difference between the foreground and the background of the intercepted image; converting the foreground of the intercepted image into a hue-saturation-brightness (HSV) image; and adjusting the brightness of the pixel points of the V layer in the HSV image based on the brightness difference. For example: respectively calculating the average brightness of the foreground and the background of the intercepted image, and calculating the brightness difference diff between the average brightness of the foreground and the average brightness of the backgroundvConverting the foreground of the intercepted image into an HSV image, and then performing complementary diff on each pixel point of a V layer in the HSV imagevAnd (5) completing the harmony of the brightness of the foreground and the background by 0.8 operation.
In some embodiments, the image parameters include saturation, and the image parameters of the foreground in the target frame image may be adjusted such that the image parameters of the foreground and the image parameters of the background match by:
acquiring the saturation difference between the foreground and the background of the intercepted image; converting the foreground of the intercepted image into an HSV image; and adjusting the saturation of the pixel points of the S layer in the HSV image based on the saturation difference. For example: respectively calculating the average saturation of the foreground and the background of the intercepted image, and calculating the brightness difference diff between the average saturation of the foreground and the average saturation of the backgroundsConverting the foreground of the intercepted image into an HSV image, and then performing completion diff on each pixel point of an S layer in the HS V imagesAnd (5) completing the saturation harmony of the foreground and the background by 0.8 operation.
In some embodiments, the image parameters include blurriness, and the image parameters of the foreground in the target frame image may be adjusted such that the image parameters of the foreground and the image parameters of the background match by:
respectively carrying out Laplace transformation on the foreground and the background of the intercepted image, and determining a standard deviation based on a transformation result; and when the standard deviation reaches a standard deviation threshold value, carrying out Gaussian blur processing on the foreground of the intercepted image so as to enable the blur degree of the foreground of the intercepted image to be matched with the blur degree of the background. In practical application, the standard deviation threshold may be set according to actual needs, for example, the standard deviation threshold is set to 1.4, and when the calculated standard deviation exceeds 1.4, it indicates that the ambiguity between the foreground and the background in the captured image is significantly different, which is caused by background blurring when a large aperture and a focal length are matched for shooting, so that the foreground may be directly subjected to gaussian blurring operation to flatten the ambiguity and the background.
In some embodiments, the image parameters include shadow length, and the image parameters of the foreground in the target frame image may be adjusted such that the image parameters of the foreground and the image parameters of the background match by:
respectively carrying out shadow detection on the foreground and the background of the intercepted image, and determining the shadow length proportion of the foreground and the background of the intercepted image; and when the shadow length proportion is not in the preset proportion range, adjusting the shadow length of the foreground of the intercepted image, so that the shadow length proportion of the foreground and the background of the intercepted image is in the preset proportion range. In practical application, the size of the proportional range can be set according to actual needs, for example, the preset proportional range is [0.2,0.8], when the shadow length ratio of the foreground to the background of the intercepted image is 0.1 and is not in the range of [0.2,0.8], the shadow length of the foreground of the intercepted image is adjusted, and if the shadow length ratio of the foreground to the background of the intercepted image is adjusted to 0.5 according to a default value; when the shadow length ratio of the foreground and the background of the intercepted image is 0.6, the shadow length ratio is not adjusted.
In some embodiments, when the image parameters include brightness, saturation, blur, and shadow length, the adjustment sequence of the brightness, saturation, blur, and shadow length of the foreground in the captured image is not limited, fig. 11 is a schematic diagram of adjusting the image parameters of the foreground in the captured image according to the embodiment of the present invention, and referring to fig. 11, the brightness, saturation, blur, and shadow length of the foreground in the captured image are sequentially adjusted.
In practical implementation, after the image parameter map of the foreground in the captured image is adjusted, the adjusted captured image needs to be returned to the original position, so as to obtain an adjusted target frame image. Fig. 12 is a schematic diagram of a target frame image after image parameter adjustment according to an embodiment of the present invention, and referring to fig. 12, a reference numeral 21 is a schematic diagram of an effect of a user manually implanting an image to be implanted carrying push information which is milk tea, and a reference numeral 22 is a schematic diagram of an effect of implanting an image to be implanted carrying push information which is milk tea according to an embodiment of the present invention, so that it is obvious that the harmony degree between a foreground (milk tea image) and a background (video frame image) implanted according to the above-described scheme of the present invention is higher.
Step 305: and the server carries out video synthesis based on the adjusted target frame image to obtain a target video.
In some embodiments, in the case of splitting the video to be implanted, the adjusted target frame image includes an adjusted target frame image corresponding to each video segment, and accordingly, video synthesis may be performed in the following manner to obtain a target video: respectively carrying out video coding on the basis of the adjusted target frame images in the video slices to obtain target video slices corresponding to the video slices; and fusing the target video fragments corresponding to the video fragments to obtain the target video.
In some embodiments, the frame image for entity identification is a key frame image of the video to be implanted, and accordingly, the video synthesis may be performed in the following manner to obtain the target video: positioning an entity area of a non-key frame image of a video to be implanted in a target tracking mode; implanting the foreground in the adjusted target frame image into the entity area of the non-key frame image in an affine transformation mode to obtain a target non-key frame image; and carrying out video coding on the adjusted target frame image and the target non-key frame image to obtain a target video.
In some embodiments, after obtaining the target video, the server further sends the target video to the terminal, so that the terminal plays the target video implanted with the push information.
The method for embedding push information in a video provided by the embodiment of the invention is continuously described. Fig. 13 is a flowchart illustrating a method for embedding push information in a video according to an embodiment of the present invention, and referring to fig. 13, the method for embedding push information in a video according to the embodiment of the present invention includes:
step 401: the server inquires the video identification of the video to be implanted, and obtains the video to be implanted based on the video identification of the video to be implanted.
In practical application, a to-be-implanted video library is set and used for storing a plurality of to-be-implanted videos to be implanted with push information, a server can inquire video identifications of the to-be-implanted videos to be processed based on the sequence of adding the to-be-implanted video library, the video identifications of the to-be-implanted videos can be video IDs of the to-be-implanted videos, and the to-be-implanted videos are obtained based on the video IDs of the to-be-implanted videos, namely video files of the to-be-implanted videos are obtained.
Step 402: and carrying out segmentation processing on the video to be implanted to obtain a plurality of video fragments.
Here, in practical implementation, the segmentation of the video to be embedded can be achieved as follows:
and carrying out shot switching detection on a video frame of the video to be implanted to obtain a plurality of shots corresponding to the video to be implanted, and segmenting the video to be implanted based on the obtained plurality of shots to obtain video segments corresponding to the shots.
Step 403: and respectively carrying out entity identification on each video fragment, and determining an entity area corresponding to an entity in the key frame image of each video fragment.
Here, taking an example that the entity includes a desktop, a wall, and a photo frame, the desktop, the wall, and the photo frame included in the frame image included in each video segment are respectively detected through entity identification, and a corresponding desktop area, a corresponding wall area, and a corresponding photo frame area are determined. In practical implementation, the entity identification for each video segment may be entity identification of a key frame image included in the video segment.
Step 404: and respectively acquiring the images to be implanted which correspond to the entity areas and carry the push information.
Here, in practical application, there is an image library to be implanted corresponding to a video to be implanted, different videos to be implanted may correspond to different image libraries to be implanted, a plurality of images to be implanted are stored in the image library to be implanted, and different entity areas may correspond to different images to be implanted.
Exemplarily, a corresponding image library to be implanted is determined according to a video ID of a video to be implanted, the image library to be implanted stores an image to be implanted carrying push information as an automobile, an image to be implanted carrying push information as milk tea, an image to be implanted carrying push information as pure milk, and the like, the image to be implanted corresponding to a wall area in the video to be implanted is determined as the image to be implanted carrying push information as the automobile, and the image to be implanted corresponding to a desktop area is determined as the image to be implanted carrying push information as pure milk.
In some embodiments, for the case that the entity area is a desktop area, since the presenting angles of the desktop are different, in order to make the harmony between the push entity implanted into the desktop area and the desktop higher, an image to be implanted, in which the presenting angle of the push entity (automobile) matches the presenting angle of the desktop, needs to be selected. For a specific selection manner, reference is made to the foregoing embodiments, which are not described herein again.
Step 405: and implanting the corresponding image to be implanted into the entity area of the key frame image by taking the entity area corresponding to the entity in the key frame image as a background and the image to be implanted corresponding to the entity area as a foreground to obtain the target frame image.
Step 406: and respectively carrying out image interception on the target frame images of the video fragments to obtain intercepted images containing the foreground.
Here, in practical implementation, the image capture may be performed as follows: and taking the minimum rectangular frame containing the image to be implanted as a boundary, and expanding the area of the rectangular frame in an equal proportion to be a constant multiple of the area of the image to be implanted to obtain a rectangular intercepted image containing the foreground. Referring to fig. 11, a minimum rectangular frame 02 including an image 01 to be implanted is used as a boundary, and the area of the rectangular frame is expanded in an equal proportion to 2 times that of the image to be implanted, so that a rectangular truncated image 03 including a foreground (the image 01 to be implanted) is obtained. It should be noted that, the shape of the truncated image is not limited in the embodiment of the present invention, for example, a minimum circular frame of the image to be implanted may be used as a boundary, and the area of the circular frame is proportionally expanded to be a constant multiple of the area of the image to be implanted, so as to obtain a circular truncated image including the foreground.
Step 407: and respectively and sequentially adjusting the brightness, the saturation, the fuzziness and the shadow length of the foreground in each intercepted image so as to enable the foreground of the intercepted image to be matched with the background.
Here, for the adjustment process of the brightness, saturation, blur degree, and shadow length of the foreground in the captured image, reference may be made to the foregoing description, and details are not repeated here.
Step 408: and respectively returning the adjusted intercepted images to corresponding target frame images.
The target frame images implanted frame by frame are fused into video fragments, and audio information in the original video fragments is fused, so that the fusion of the whole video fragments is completed.
Step 409: and carrying out video coding on the basis of the restored target frame image corresponding to each video fragment to obtain a target video fragment corresponding to each video fragment.
Step 410: and fusing the target video fragments corresponding to the video fragments to obtain the target video.
By applying the embodiment of the invention, the entity area of the frame image of the video to be implanted is taken as the background, the corresponding image to be implanted is taken as the foreground, and the image to be implanted is implanted in the entity area of the frame image to obtain the target frame image, so that the automatic implantation of the push information in the video to be implanted is realized, and the implantation efficiency of the push information is improved; by adjusting the image parameters of the foreground in the target frame image, the image parameters of the foreground are matched with those of the background, so that the automatic harmony processing of the foreground and the background in the video frame image after the push information is implanted is realized, the harmony of the push information and the video content in the target video is realized, and the receiving degree of the push information when a user watches the video is greatly improved.
Next, a method for implanting push information into a video provided by an embodiment of the present invention is described, taking push information as an advertisement and taking implanting an advertisement on a desktop of a video as an example. Fig. 14 is a flowchart illustrating a method for embedding push information in a video according to an embodiment of the present invention, and referring to fig. 14, the method for embedding push information in a video according to an embodiment of the present invention includes the following main steps:
step 1: complete video material input
Here, in actual implementation, the server may query the video ID of the video to be embedded, and request a corresponding video source file according to the queried video ID.
Step 2: single-shot video slicing
In practical implementation, shot switching detection is performed on a video to be implanted, and a video source file of the video to be implanted is split into video fragments corresponding to each shot.
And step 3: multi-mode video implantation advertisement space detection algorithm
Video can be viewed as a data stream made up of a series of temporally dependent image frames. Generally, when the content of a video episode changes, a shot switch occurs to shift from one shot content to another. The video contains rich visual, auditory and caption information, the bottom layer characteristics can be color, texture, shape, tone, text and the like, and a multi-mode analysis method is adopted, namely two or more types of modal information are used for processing at the same time to realize video data analysis.
In practical applications, in the first key frame image of a video segment, an area of an ad slot is specified, and this area may be provided as follows: based on a multi-mode video implantation advertisement space detection algorithm, directly using an output advertisement space frame to be selected as a tracking area of a background; alternatively, the tracking area is set empirically.
And 4, step 4: object orientation selection of desktop scene
In the practical implementation, an automatic object orientation screening technology is utilized to automatically screen out the picture with the highest harmony degree between the orientation angle and the background from the massive foreground advertisement object pictures to complete the implantation; for example, the edge detection is performed on the foreground real object and the background desktop, and the foreground real object whose slope difference with the edge line of the background desktop meets the preset condition is selected as the object to be implanted.
And 5: foreground rendering
For foreground pictures (such as objects, three-dimensional props or posters) to be implanted into a desktop area, the saturation, brightness, ambiguity and the like of the foreground pictures are greatly different from the background, the local background of the foreground implanted area is used as a template, the brightness, the saturation, the ambiguity and the length depth of the inverted image (namely the shadow length) of the foreground are adjusted, and the harmony of the visual feelings of the foreground and the background after implantation is realized.
Specifically, the adjustment of image parameters such as saturation, brightness, and blur of the foreground can be realized by the following operations:
1) local background calibration
In actual implementation, a foreground (advertisement picture, namely an image to be implanted) is implanted into a desktop central area in a key frame image of a video fragment, a peripheral frame of the implanted foreground is taken as a boundary, the area of the implanted foreground is expanded to be twice as large as the area of the implanted foreground in an equal proportion, image interception is carried out according to the expanded size, a local background is determined, and the local background is taken as a data parameter calibration acquisition source for subsequent self-adaptive foreground parameter adjustment.
2) Adaptive brightness adjustment
Calculating the average brightness of the implanted foreground and the local background, and calculating the brightness difference diffvConverting the foreground picture into an HSV picture, and performing completion diff on each pixel point of the 3 rd layer V picture layervAnd operation of 0.8, completing the harmony of the brightness of the foreground and the background.
3) Adaptive saturation adjustment
Calculating the average saturation of the implanted foreground and the local background, and calculating the saturation difference diffsAnd converting the foreground picture into an HSV picture, and performing complementary diff on each pixel point of the layer 2S picture layers0.8 operation before and after completionThe saturation of the scene is harmonious.
4) Adaptive ambiguity adjustment
And (3) performing Laplacian transformation on the foreground and the background respectively and calculating a standard deviation, wherein if the standard deviation difference exceeds 1.4, the fuzzy degrees of the foreground and the background are obviously different, which is caused by background blurring when a large aperture and a focal length are matched for shooting, so that Gaussian blur operation is directly performed on the foreground, and the fuzzy degree of the foreground and the background are leveled.
5) Adaptive inverse image length and depth adjustment
And detecting the shadow of the background by using a BDRAR algorithm, and increasing the shadow of the foreground according to a preset shadow ratio of the foreground to the background.
Step 6: desktop ad spot tracking and implantation
And tracking the advertisement position based on a motion tracking technology, and implanting the foreground output in the last step into the video through an affine change technology.
And 7: video slicing fusion
Fusing the video segments after completing the advertisement implantation to form a new complete video.
By applying the embodiment of the invention, the automatic advertisement implantation in the video and the harmony of the advertisement as the foreground after the advertisement implantation and the video content as the background are realized, the labor cost is saved, and compared with the prior condition of manually rendering the foreground, the operation time is reduced from 20 minutes to 30 seconds, and the time cost is saved.
By applying the above embodiment of the present invention, for the advertisers, the advantage of the form of advertisement placement in the video is shown in fig. 15, according to the advertisement marketing trend report in 2018, and "73% of the advertisers will increase and maintain the expenses of soft advertisements such as placement". Compared with the traditional advertisement, the method has the advantages of high reach rate, small cooperation risk, budget conservation and the like. Especially during the period when members (video VIPs) are greatly expanded, the traditional front-and middle-patch advertising traffic is limited in growth. The soft advertising form which is based on the later technology and is not limited by member reach is more meaningful. Meanwhile, there are situations where advertisers gamble plays, compared to traditional commercial-style advertisements (where advertising placement needs to be negotiated prior to the capture of the heddles and television plays). With the advertisement implanting technology, an advertiser can implant advertisements in the middle and later periods of a video according to the popularity of an episode and a comprehensive art, so that the risk of advertisement investment is reduced.
With the above embodiment of the present invention, for the traffic side, fig. 16 is a schematic diagram of the annual estimated income of the Video with advertisement implantation provided by the embodiment of the present invention, and referring to fig. 16, new advertisement traffic is derived In a Video-In (advertisement implantation) manner according to the plan of the home-made drama (integrated art and tv drama) of the Video platform. Based on this traffic and the selling form of contract advertisements, a market size of about 36 billion dollars can be extrapolated. It can be seen that with the market's increasing acceptance of the Video-In advertising format, this advertising format can become an important profit growth point for Video platforms.
The description continues on the device for embedding push information in video provided by the embodiment of the present invention. Fig. 17 is a schematic structural diagram of an apparatus for embedding push information in video according to an embodiment of the present invention, and referring to fig. 17, an apparatus 170 for embedding push information in video according to an embodiment of the present invention includes:
the identification unit 171 is configured to perform entity identification on a frame image of a video to be implanted, and determine an entity area corresponding to the entity in the frame image;
an obtaining unit 172, configured to obtain an image to be implanted that carries push information and corresponds to the entity area;
an implanting unit 173, configured to implant the image to be implanted into the solid area of the frame image by using the solid area of the frame image as a background and using the corresponding image to be implanted as a foreground, so as to obtain a target frame image;
an adjusting unit 174, configured to adjust an image parameter of the foreground in the target frame image so that the image parameter of the foreground matches the image parameter of the background;
and a synthesizing unit 175, configured to perform video synthesis based on the adjusted target frame image to obtain a target video.
In some embodiments, the identification unit is further configured to perform segmentation processing on the video to be implanted to obtain a plurality of video segments;
and respectively carrying out entity identification on the plurality of video fragments, and determining an entity area corresponding to the entity in the frame image of each video fragment.
In some embodiments, the identification unit is further configured to perform shot switching detection on a video frame of the video to be implanted to obtain a plurality of shots corresponding to the video to be implanted;
and segmenting the video to be implanted based on the plurality of lenses to obtain video segments corresponding to the lenses.
In some embodiments, the synthesizing unit is further configured to perform video coding based on the adjusted target frame image in each of the video slices, respectively, to obtain a target video slice corresponding to each of the video slices;
and fusing the target video fragments corresponding to the video fragments to obtain the target video.
In some embodiments, the obtaining unit is further configured to, in response to that a push entity is included in the push information, obtain a first presentation angle of the push entity in an image to be implanted;
acquiring a second presentation angle of the entity presented in the entity area;
and determining the image to be implanted with the first presentation angle matched with the second presentation angle as the image to be implanted corresponding to the entity area.
In some embodiments, the apparatus further comprises:
and the intercepting unit is used for intercepting the target frame image to obtain an intercepted image containing the foreground, and the area of the intercepted image is a constant multiple of the area of the foreground.
In some embodiments, the adjusting unit is further configured to, in response to the image parameter including brightness, obtain a brightness difference between a foreground and a background of the captured image;
converting the foreground of the intercepted image into a hue-saturation-brightness (HSV) image;
and adjusting the brightness of the pixel points of the V layer in the HSV image based on the brightness difference.
In some embodiments, the adjusting unit is further configured to, in response to the image parameter including a saturation, obtain a saturation difference between a foreground and a background of the captured image;
converting the foreground of the intercepted image into an HSV image;
and adjusting the saturation of the pixel points of the S layer in the HSV image based on the saturation difference.
In some embodiments, the adjusting unit is further configured to, in response to the image parameter including a degree of blur, respectively perform laplacian transform on a foreground and a background of the truncated image, and determine a standard deviation based on a transform result;
and when the standard deviation reaches a standard deviation threshold value, carrying out Gaussian blur processing on the foreground of the intercepted image so as to enable the blur degree of the foreground of the intercepted image to be matched with the blur degree of the background.
In some embodiments, the adjusting unit is further configured to, in response to that the image parameter includes a shadow length, perform shadow detection on the foreground and the background of the captured image respectively, and determine a shadow length ratio of the foreground and the background of the captured image;
and when the shadow length proportion is not in a preset proportion range, adjusting the shadow length of the foreground of the intercepted image, so that the shadow length proportion of the foreground and the background of the intercepted image is in the preset proportion range.
In some embodiments, the synthesizing unit is further configured to, in response to the frame image being a key frame image of the video to be implanted, locate an entity region of a non-key frame image of the video to be implanted by means of target tracking;
implanting the foreground in the adjusted target frame image in the entity region of the non-key frame image in an affine transformation mode to obtain a target non-key frame image;
and carrying out video coding on the adjusted target frame image and the target non-key frame image to obtain the target video.
Here, it should be noted that: the above description related to the apparatus is similar to the above description of the method, and for the technical details not disclosed in the apparatus according to the embodiment of the present invention, please refer to the description of the method embodiment of the present invention.
An embodiment of the present invention further provides an electronic device, where the electronic device includes:
a memory for storing an executable program;
and the processor is used for implementing the method for implanting push information into the video provided by the embodiment of the invention when executing the executable program stored in the memory.
Embodiments of the present invention further provide a storage medium storing executable instructions, where the executable instructions are stored, and when executed by a processor, will cause the processor to execute the method for embedding push information in a video provided by the embodiments of the present invention.
All or part of the steps of the embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Random Access Memory (RAM), a Read-Only Memory (ROM), a magnetic disk, and an optical disk.
Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a RAM, a ROM, a magnetic or optical disk, or various other media that can store program code.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

Claims (13)

1. A method for embedding push information in video, the method comprising:
segmenting a video to be implanted to obtain video segments corresponding to all shots or video segments corresponding to all scenes;
performing entity identification on the key frame images in the video fragments obtained by segmentation, and determining entity areas corresponding to entities in the key frame images of the video fragments;
acquiring an image to be implanted which corresponds to the entity area and carries push information;
implanting the image to be implanted into the entity area of the key frame image by taking the entity area of the key frame image as a background and the corresponding image to be implanted as a foreground to obtain a target key frame image;
adjusting image parameters of the foreground in the target key frame image to enable the image parameters of the foreground to be matched with the image parameters of the background;
positioning an entity area of a non-key frame image of the video to be implanted in a target tracking mode;
aiming at each video fragment, implanting the adjusted foreground in the target key frame image in the entity area of the non-key frame image to obtain a target non-key frame image;
and carrying out video coding on the adjusted target key frame image and the target non-key frame image to obtain a target video.
2. The method of claim 1, wherein the segmenting the video to be implanted to obtain video slices corresponding to respective shots or video slices corresponding to respective scenes comprises:
carrying out shot switching detection on a video frame of a video to be implanted to obtain a plurality of shots corresponding to the video to be implanted; segmenting the video to be implanted based on the plurality of lenses to obtain video segments corresponding to the lenses; alternatively, the first and second electrodes may be,
performing scene switching detection on the video frame of the video to be implanted to obtain a plurality of scenes corresponding to the video to be implanted; and segmenting the video to be implanted based on the plurality of scenes to obtain video segments corresponding to the scenes.
3. The method of claim 1, wherein the video coding the adjusted target key frame image and the target non-key frame image to obtain a target video comprises:
performing video coding on the adjusted target key frame image and the target non-key frame image in each video fragment respectively to obtain target video fragments corresponding to each video fragment;
and fusing the target video fragments corresponding to the video fragments to obtain the target video.
4. The method of claim 1, wherein the obtaining of the image to be implanted carrying push information corresponding to the physical area comprises:
responding to the fact that the push information comprises a push entity, and acquiring a first presentation angle of the push entity in the image to be implanted;
acquiring a second presentation angle of the entity presented in the entity area;
and determining the image to be implanted with the first presentation angle matched with the second presentation angle as the image to be implanted corresponding to the entity area.
5. The method of claim 1, wherein the method further comprises:
and carrying out image interception on the target key frame image to obtain an intercepted image containing the foreground, wherein the area of the intercepted image is a constant multiple of the area of the foreground.
6. The method of claim 5, wherein the adjusting the image parameters of the foreground in the target key-frame image such that the image parameters of the foreground and the image parameters of the background match comprises:
acquiring the brightness difference between the foreground and the background of the intercepted image;
converting the foreground of the intercepted image into a hue-saturation-brightness (HSV) image;
and adjusting the brightness of the pixel points of the V layer in the HSV image based on the brightness difference.
7. The method of claim 5, wherein the adjusting the image parameters of the foreground in the target key-frame image such that the image parameters of the foreground and the image parameters of the background match comprises:
acquiring the saturation difference between the foreground and the background of the intercepted image;
converting the foreground of the intercepted image into an HSV image;
and adjusting the saturation of the pixel points of the S layer in the HSV image based on the saturation difference.
8. The method of claim 5, wherein the adjusting the image parameters of the foreground in the target key-frame image such that the image parameters of the foreground and the image parameters of the background match comprises:
respectively carrying out Laplace transformation on the foreground and the background of the intercepted image, and determining a standard deviation based on a transformation result;
and when the standard deviation reaches a standard deviation threshold value, carrying out Gaussian blur processing on the foreground of the intercepted image so as to enable the blur degree of the foreground of the intercepted image to be matched with the blur degree of the background.
9. The method of claim 5, wherein the adjusting the image parameters of the foreground in the target key-frame image such that the image parameters of the foreground and the image parameters of the background match comprises:
respectively carrying out shadow detection on the foreground and the background of the intercepted image, and determining the shadow length proportion of the foreground and the background of the intercepted image;
and when the shadow length proportion is not in a preset proportion range, adjusting the shadow length of the foreground of the intercepted image, so that the shadow length proportion of the foreground and the background of the intercepted image is in the preset proportion range.
10. The method of claim 1, wherein the implanting the adjusted foreground in the target key frame image in the solid region of the non-key frame image to obtain the target non-key frame image comprises:
implanting the foreground in the adjusted target key frame image into the entity area of the non-key frame image in an affine transformation mode to obtain the target non-key frame image.
11. An apparatus for embedding push information in video, the apparatus comprising:
the identification unit is used for segmenting a video frame of a video to be implanted to obtain video segments corresponding to all the lenses or video segments corresponding to all the scenes; performing entity identification on the key frame images in the video fragments obtained by segmentation, and determining entity areas corresponding to entities in the key frame images of the video fragments;
the acquisition unit is used for acquiring the image to be implanted which corresponds to the entity area and carries push information;
an implantation unit, configured to implant the image to be implanted in the entity region of the key frame image to obtain a target key frame image, with the entity region of the key frame image as a background and the corresponding image to be implanted as a foreground;
the adjusting unit is used for adjusting the image parameters of the foreground in the target key frame image so that the image parameters of the foreground are matched with the image parameters of the background;
the synthesizing unit is used for positioning the entity area of the non-key frame image of the video to be implanted in a target tracking mode; aiming at each video fragment, implanting the adjusted foreground in the target key frame image in the entity area of the non-key frame image to obtain a target non-key frame image; and carrying out video coding on the adjusted target key frame image and the target non-key frame image to obtain a target video.
12. An electronic device, characterized in that the electronic device comprises:
a memory for storing executable instructions;
a processor configured to implement the method of embedding push information in video according to any one of claims 1 to 10 when executing the executable instructions stored in the memory.
13. A computer-readable storage medium storing executable instructions for causing a processor to perform the method of embedding push information in video according to any one of claims 1 to 10 when the processor executes the method.
CN201910704475.1A 2019-07-31 2019-07-31 Method and device for implanting push information into video, electronic equipment and storage medium Active CN110300316B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910704475.1A CN110300316B (en) 2019-07-31 2019-07-31 Method and device for implanting push information into video, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910704475.1A CN110300316B (en) 2019-07-31 2019-07-31 Method and device for implanting push information into video, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN110300316A CN110300316A (en) 2019-10-01
CN110300316B true CN110300316B (en) 2022-02-11

Family

ID=68032437

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910704475.1A Active CN110300316B (en) 2019-07-31 2019-07-31 Method and device for implanting push information into video, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN110300316B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110213629B (en) * 2019-06-27 2022-02-11 腾讯科技（深圳）有限公司 Information implantation method, device, server and storage medium
CN111182335B (en) * 2019-10-18 2021-08-20 腾讯科技（深圳）有限公司 Streaming media processing method, device, equipment and computer readable storage medium
CN111246315B (en) * 2020-01-21 2022-06-03 咪咕视讯科技有限公司 Video interaction method, system, edge node, client and storage medium
CN113225587B (en) * 2020-02-06 2023-04-28 阿里巴巴集团控股有限公司 Video processing method, video processing device and electronic equipment
CN111988657A (en) * 2020-08-05 2020-11-24 网宿科技股份有限公司 Advertisement insertion method and device
CN112860944B (en) * 2021-02-05 2023-07-25 北京百度网讯科技有限公司 Video rendering method, apparatus, device, storage medium, and computer program product
CN113489919A (en) * 2021-06-21 2021-10-08 北京德风新征程科技有限公司 Digital video production system based on internet big data
CN113691835B (en) * 2021-10-21 2022-01-21 星河视效科技(北京)有限公司 Video implantation method, device, equipment and computer readable storage medium
CN114760517B (en) * 2022-04-15 2024-02-02 广州华多网络科技有限公司 Image moving embedding method and device, equipment, medium and product thereof

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103607554A (en) * 2013-10-21 2014-02-26 无锡易视腾科技有限公司 Fully-automatic face seamless synthesis-based video synthesis method
CN104170392A (en) * 2013-11-21 2014-11-26 华为技术有限公司 Method, device, system and terminal of inplanting advertisements in files
CN104700354A (en) * 2015-03-31 2015-06-10 北京奇艺世纪科技有限公司 Information embedding method and device
CN104766229A (en) * 2015-04-22 2015-07-08 合一信息技术（北京）有限公司 Implantable advertisement putting method

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009032922A1 (en) * 2007-09-04 2009-03-12 Objectvideo, Inc. Stationary target detection by exploiting changes in background model
CN106991641B (en) * 2017-03-10 2020-12-29 北京小米移动软件有限公司 Picture implantation method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103607554A (en) * 2013-10-21 2014-02-26 无锡易视腾科技有限公司 Fully-automatic face seamless synthesis-based video synthesis method
CN104170392A (en) * 2013-11-21 2014-11-26 华为技术有限公司 Method, device, system and terminal of inplanting advertisements in files
CN104700354A (en) * 2015-03-31 2015-06-10 北京奇艺世纪科技有限公司 Information embedding method and device
CN104766229A (en) * 2015-04-22 2015-07-08 合一信息技术（北京）有限公司 Implantable advertisement putting method

Also Published As

Publication number Publication date
CN110300316A (en) 2019-10-01

Similar Documents

Publication Publication Date Title
CN110300316B (en) Method and device for implanting push information into video, electronic equipment and storage medium
US11861903B2 (en) Methods and apparatus to measure brand exposure in media streams
CN104066003B (en) Method and device for playing advertisement in video
US10672104B2 (en) Method and apparatus for generating an extrapolated image based on object detection
KR20190075177A (en) Context-based augmented ad
CN110324679B (en) Video data processing method and device
CN109982141B (en) Method for analyzing video image area and implanting advertisement by using AI technology
CN110213629B (en) Information implantation method, device, server and storage medium
CN113516696A (en) Video advertisement implanting method and device, electronic equipment and storage medium
CN110099298B (en) Multimedia content processing method and terminal equipment
CN111654747B (en) Bullet screen display method and device
CN116761037B (en) Method, device, equipment and medium for video implantation of multimedia information
CN104093069A (en) Video playing method and player device
CN113132744A (en) Processing method, model, electronic device and computer storage medium of live broadcast barrage
CN112752110A (en) Video presentation method and device, computing equipment and storage medium
CN111698541A (en) Video playing method and device, computer equipment and storage medium
US10785546B2 (en) Optimizing product placement in a media
WO2022236842A1 (en) Advertisement replacement or addition processing method, system and apparatus
WO2022018628A1 (en) Smart overlay : dynamic positioning of the graphics
CN108882022A (en) Recommend method, apparatus, medium and the calculating equipment of film
CN114998957A (en) Automatic exposure data processing method, device, equipment and system
CN117745624A (en) Green screen video image matting method, computer equipment and storage medium
Ploumis Tone mapping operator for high dynamic range video
CA2643532A1 (en) Methods and apparatus to measure brand exposure in media streams and to specify regions of interest in associated video frames

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant