CN114051166A - Method, device, electronic equipment and storage medium for implanting advertisement in video - Google Patents

Method, device, electronic equipment and storage medium for implanting advertisement in video Download PDF

Info

Publication number
CN114051166A
CN114051166A CN202010725796.2A CN202010725796A CN114051166A CN 114051166 A CN114051166 A CN 114051166A CN 202010725796 A CN202010725796 A CN 202010725796A CN 114051166 A CN114051166 A CN 114051166A
Authority
CN
China
Prior art keywords
advertisement
video
implanted
content
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010725796.2A
Other languages
Chinese (zh)
Other versions
CN114051166B (en
Inventor
石峰
郭小燕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Dajia Internet Information Technology Co Ltd
Original Assignee
Beijing Dajia Internet Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Dajia Internet Information Technology Co Ltd filed Critical Beijing Dajia Internet Information Technology Co Ltd
Priority to CN202010725796.2A priority Critical patent/CN114051166B/en
Publication of CN114051166A publication Critical patent/CN114051166A/en
Application granted granted Critical
Publication of CN114051166B publication Critical patent/CN114051166B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44213—Monitoring of end-user related data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0277—Online advertisement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44213—Monitoring of end-user related data
    • H04N21/44218—Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/4508—Management of client data or end-user data
    • H04N21/4532—Management of client data or end-user data involving end-user characteristics, e.g. viewer profile, preferences
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/466—Learning process for intelligent management, e.g. learning user preferences for recommending movies
    • H04N21/4667—Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4728—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Social Psychology (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Strategic Management (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Human Computer Interaction (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Development Economics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Geometry (AREA)
  • General Business, Economics & Management (AREA)
  • Game Theory and Decision Science (AREA)
  • Computer Graphics (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The present disclosure relates to a method, an apparatus, an electronic device, and a storage medium for implanting an advertisement in a video. Wherein, the method comprises the following steps: acquiring a sight line staying area of a video viewer, and taking the sight line staying area as an interested area when the video viewer watches the video; acquiring an advertisement to be implanted; generating a three-dimensional space position for delivering the advertisement to be implanted in the video according to the advertisement to be implanted and the region of interest; and putting the advertisement to be implanted on the three-dimensional space position so as to blend the advertisement to be implanted into the native content of the video. The method and the device can utilize the three-dimensional space position to seamlessly integrate the advertisement into the original content of the video, so that the user is not disturbed to watch the advertisement to the greatest extent, and the advertisement investment return rate is improved in a more intelligent and invisible mode.

Description

Method, device, electronic equipment and storage medium for implanting advertisement in video
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to a method and an apparatus for embedding an advertisement in a video, an electronic device, and a storage medium.
Background
With the development of internet technology, internet video traffic is greatly increased in recent years, and the appearance of various novel UGCs (User Generated Content) such as short videos and live broadcasts prompts internet videos to be richer and richer, and meanwhile, audience groups are also larger and larger. In such a case, merchants tend to place more advertisements in internet videos in pursuit of higher return on investment for advertisements.
In the related art, there are two main video advertisement delivery modes in the internet platform: one is that the advertisement appears in the form of native content of a content platform, and the other is that the advertisement is directly embedded into a video in a two-dimensional plane manner, which is also called a woundplast advertisement.
However, there are problems that: the two advertisement putting modes can damage the content watching experience to a certain extent, and cause the user to feel dislike, thereby reducing the return rate of the advertisements.
Disclosure of Invention
The present disclosure provides a method, an apparatus, an electronic device, and a storage medium for implanting an advertisement in a video, so as to at least solve the problem that an advertisement delivery manner in the related art may damage a video content viewing experience, cause a user's dislike, and further reduce a return rate of the advertisement. The technical scheme of the disclosure is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided a method for placing advertisements in videos, including:
acquiring a sight line staying area of a video viewer, and taking the sight line staying area as an interested area when the video viewer watches a video;
acquiring an advertisement to be implanted;
generating a three-dimensional space position for delivering the advertisement to be implanted in the video according to the advertisement to be implanted and the region of interest; and
and putting the advertisement to be implanted on the three-dimensional space position so as to blend the advertisement to be implanted into the native content of the video.
In some embodiments of the present disclosure, the obtaining of the advertisement to be implanted includes: acquiring the content category of the video and the attribute information of the video viewer; and acquiring the advertisement to be implanted according to the content category of the video and the attribute information of the video viewer.
In some embodiments of the present disclosure, the obtaining of the content category of the video includes: extracting visual features and audio features of the video; and inputting the visual features and the audio features into a video content classification model to obtain the content category of the video.
In some embodiments of the present disclosure, the obtaining of the content category of the video includes: extracting a plurality of key frames of the video; performing content classification on each key frame to obtain a classification result of each key frame; and counting the classification result of each key frame to obtain the content category of the video.
In some embodiments of the present disclosure, the generating, according to the advertisement to be implanted and the region of interest, a three-dimensional spatial location for delivering the advertisement to be implanted in the video includes: acquiring the content type of the advertisement to be implanted; determining whether an implantation object of the advertisement to be implanted exists in the region of interest according to the content type of the advertisement to be implanted; when the implantation object to be implanted with the advertisement exists in the region of interest, acquiring three-dimensional information of the implantation object in the video; and generating a three-dimensional space position for delivering the advertisement to be implanted according to the three-dimensional information of the implanted object in the video.
In some embodiments of the present disclosure, the method further comprises: when the implant object to be implanted with the advertisement does not exist in the region of interest, generating a virtual implant object according to the content type of the advertisement to be implanted; acquiring three-dimensional information of the virtual implanted object in the video; and generating a three-dimensional space position for delivering the advertisement to be implanted in the video according to the three-dimensional information of the virtual implanted object in the video.
In some embodiments of the present disclosure, the determining whether an implantation object to be implanted with the advertisement exists in the region of interest according to the content type of the advertisement to be implanted includes: acquiring at least one object within the region of interest; determining a type of each object; determining whether an object used for bearing the advertisement to be implanted exists in the at least one object according to the content type of the advertisement to be implanted and the type of each object; when the object used for bearing the advertisement to be implanted does not exist in the at least one object, determining that the object to be implanted with the advertisement does not exist in the region of interest; when the object used for bearing the advertisement to be implanted exists in the at least one object, determining the object used for bearing the advertisement to be implanted as the implanted object, and determining that the implanted object with the advertisement to be implanted exists in the region of interest.
In some embodiments of the present disclosure, the method further comprises: and in the process of delivering the advertisement to be implanted on the three-dimensional space position, adjusting the three-dimensional space orientation of the content of the advertisement to be implanted so as to enable the three-dimensional space orientation of the content of the advertisement to be implanted to be consistent with the three-dimensional space orientation of an object in the three-dimensional space position.
In some embodiments of the present disclosure, after placing the advertisement to be implanted on the three-dimensional spatial location, the method further comprises: tracking the area with the advertisement in the video frame by frame in real time in a three-dimensional space; when the three-dimensional information of the area is tracked to change in the current frame, acquiring the three-dimensional information of the area in the current frame; and adjusting the three-dimensional space orientation of the content with the advertisement delivered in real time according to the three-dimensional information of the region in the current frame so as to enable the three-dimensional space orientation of the content with the advertisement delivered to be consistent with the three-dimensional space orientation of the object in the region.
According to a second aspect of the embodiments of the present disclosure, there is provided an apparatus for placing advertisements in videos, including:
the first acquisition module is configured to acquire a sight line staying area of a video viewer and take the sight line staying area as an interested area when the video viewer watches videos;
a second obtaining module configured to obtain an advertisement to be implanted;
a generating module configured to generate a three-dimensional space position for delivering the advertisement to be implanted in the video according to the advertisement to be implanted and the region of interest; and
a placement module configured to place the advertisement to be placed on the three-dimensional spatial location to incorporate the advertisement to be placed into native content of the video.
In some embodiments of the present disclosure, the second obtaining module comprises: a first acquisition unit configured to acquire a content category of the video and attribute information of the video viewer; and the second acquisition unit is configured to acquire the advertisement to be implanted according to the content category of the video and the attribute information of the video viewer.
In some embodiments of the present disclosure, the first obtaining unit is configured to: extracting visual features and audio features of the video; and inputting the visual features and the audio features into a video content classification model to obtain the content category of the video.
In some embodiments of the present disclosure, the first obtaining unit is configured to: extracting a plurality of key frames of the video; performing content classification on each key frame to obtain a classification result of each key frame; and counting the classification result of each key frame to obtain the content category of the video.
In some embodiments of the disclosure, the generation module is configured to: acquiring the content type of the advertisement to be implanted; determining whether an implantation object of the advertisement to be implanted exists in the region of interest according to the content type of the advertisement to be implanted; when the implantation object to be implanted with the advertisement exists in the region of interest, acquiring three-dimensional information of the implantation object in the video; and generating a three-dimensional space position for delivering the advertisement to be implanted according to the three-dimensional information of the implanted object in the video.
In some embodiments of the disclosure, the generation module is further configured to: when the implant object to be implanted with the advertisement does not exist in the region of interest, generating a virtual implant object according to the content type of the advertisement to be implanted; acquiring three-dimensional information of the virtual implanted object in the video; and generating a three-dimensional space position for delivering the advertisement to be implanted in the video according to the three-dimensional information of the virtual implanted object in the video.
In some embodiments of the disclosure, the generation module is configured to: acquiring at least one object within the region of interest; determining a type of each object; determining whether an object used for bearing the advertisement to be implanted exists in the at least one object according to the content type of the advertisement to be implanted and the type of each object; when the object used for bearing the advertisement to be implanted does not exist in the at least one object, determining that the object to be implanted with the advertisement does not exist in the region of interest; when the object used for bearing the advertisement to be implanted exists in the at least one object, determining the object used for bearing the advertisement to be implanted as the implanted object, and determining that the implanted object with the advertisement to be implanted exists in the region of interest.
In some embodiments of the present disclosure, the apparatus further comprises: the first adjusting module is configured to adjust the three-dimensional spatial orientation of the content to be advertised in the process of delivering the content to be advertised in the three-dimensional spatial position, so that the three-dimensional spatial orientation of the content to be advertised is consistent with the three-dimensional spatial orientation of an object in the three-dimensional spatial position.
In some embodiments of the present disclosure, the apparatus further comprises: the tracking module is configured to track the region with the advertisement in the video in a three-dimensional space frame by frame in real time; and the second adjusting module is configured to acquire the three-dimensional information of the region in the current frame when the three-dimensional information of the region is tracked to change in the current frame, and adjust the three-dimensional spatial orientation of the content with the advertisement delivered according to the three-dimensional information of the region in the current frame in real time, so that the three-dimensional spatial orientation of the content with the advertisement delivered is consistent with the three-dimensional spatial orientation of the object in the region.
According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of placing advertisements in videos according to the first aspect of the embodiments of the present disclosure.
According to a fourth aspect of the embodiments of the present disclosure, there is provided a storage medium, wherein instructions, when executed by a processor of an electronic device, enable the electronic device to perform the method for placing advertisements in videos according to the first aspect of the embodiments of the present disclosure.
According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, wherein instructions of the computer program product, when executed by a processor, perform the method for placing advertisements in videos according to the first aspect.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
the method comprises the steps of automatically mining and generating a three-dimensional space position suitable for advertisement putting in an internet video stream through artificial intelligence technologies such as sight tracking, interesting region detection, three-dimensional reconstruction and depth estimation, and further utilizing the three-dimensional space position to seamlessly integrate the advertisement into the original content of the video, so that the user is not disturbed to watch the advertisement to the greatest extent, and the advertisement investment return rate is improved in a more intelligent and invisible mode. In addition, because the implanted advertisements generally have a time concept, the dimension of the advertisement implantation is promoted to a four-dimensional space (i.e., time + three-dimensional space) by the method, so that the video implanted in the advertisement is more real and natural, the implantation effect is closer to the real physical world, and the more vivid implantation effect is realized. According to the advertisement implanting method, the advertisement implanting effect can be synchronously seen in the video shooting process or the video watching process (such as a video watcher watches a live video) without depending on the video post-processing in the advertisement implanting process, and the video implanting efficiency is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is a flow diagram illustrating a method of placing advertisements in a video according to an example embodiment.
FIG. 2 is a flow diagram illustrating another method of placing advertisements in a video in accordance with an example embodiment.
FIG. 3 is a flow chart illustrating yet another method of placing advertisements in a video according to an example embodiment.
Fig. 4 is a diagram illustrating an original content display effect of a short video according to an exemplary embodiment.
Fig. 5 is a diagram illustrating the effect of advertising on the original content of the short video shown in fig. 4.
Fig. 6 is a block diagram illustrating an apparatus for placing advertisements in videos according to an example embodiment.
Fig. 7 is a block diagram illustrating another apparatus for placing advertisements in videos according to an example embodiment.
Fig. 8 is a block diagram illustrating yet another apparatus for placing advertisements in videos according to an example embodiment.
Fig. 9 is a block diagram illustrating yet another apparatus for placing advertisements in videos according to an example embodiment.
Fig. 10 is a block diagram illustrating an electronic device 1000 in accordance with an example embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Fig. 1 is a flowchart illustrating a method for placing advertisements in a video according to an exemplary embodiment, where the method for placing advertisements in a video is used in an electronic device, as shown in fig. 1, and includes the following steps.
In step S11, the line-of-sight dwell region of the video viewer is acquired, and the line-of-sight dwell region is taken as the region of interest when the video viewer views the video.
For example, an application scenario of the method for implanting the advertisement in the video according to the embodiment of the present disclosure may be that the user is recording a video scenario, and the like. Where the video may be a short video or other type of video. In addition, the essence of live broadcasting is to record a video and push the video recorded in real time, and a video viewer can watch the video recorded in real time by a main broadcasting in a stream pulling mode to realize the effect of watching the live broadcasting, so that the application scene of the embodiment of the disclosure can also be that a user watches a video scene (for example, watching a live broadcasting scene), that is, the user watches the live broadcasting.
In some embodiments of the present disclosure, the gaze dwell area of the video viewer may be obtained by a gaze area detection mode. Specifically, the first step: detecting human eye feature points by using an SDM (supervisory drop Method) detection algorithm, finding out an optimal solution of the feature points after multiple iterations, determining six feature points of a left eye and a right eye, and positioning a human eye contour central point by using geometric knowledge; step two: acquiring the central position of the iris by utilizing the image gradient information; step three: fitting the human eye contour by adopting a least square method ellipse fitting algorithm, and then determining the opening and closing state of the human eye according to the aspect ratio of the fitting ellipse; step four: if the aspect ratio is larger than the set opening threshold value, the human eyes belong to an opening state, and the step five is skipped; if the aspect ratio is smaller than the set opening threshold value, the human eyes belong to a closed state, and the output human eyes are closed; step five: calculating the distance between the center point of the human eye contour and the actual pupil, comparing the distance with the given center radius, judging that the human eye sight line area is not in the middle when the distance is larger than the given center radius, and skipping to the sixth step; when the distance is smaller than the given central radius, outputting the human eye sight area as the middle; step six: if the aspect ratio is smaller than or equal to a given critical threshold value, the sight line area is positioned to be 'lower left' and 'lower right', the distance and the relative position of the center point position x0 of the human eye outline and the actual pupil position x are compared, when x is less than x0-0.4 abb, the sight line area of the human eye is output to be 'lower right', and when x is greater than x0+0.4 abb, the sight line area of the human eye is output to be 'lower left'; b is the short semi-axial length of the iris; otherwise, jumping to the seventh step; step seven: and if the aspect ratio is larger than a given critical threshold value, the sight line region is positioned to the upper left and the upper right, the distance and the relative position of the central point position of the human eye outline and the actual pupil position are compared, when x is less than x0-0.4 abb, the sight line region of the human eye is output to be the upper right, and when x is more than x0+0.4 abb, the sight line region of the human eye is output to be the upper left.
In other embodiments of the present disclosure, the gaze dwell area of a video viewer watching a video may be determined by a gaze tracking algorithm. The gaze tracking algorithm may include, but is not limited to: a sight line tracking algorithm based on a 2D model, a binocular eye line tracking algorithm based on an eyeball reconstruction out-of-plane straight line model, a sight line tracking algorithm based on an image processing technology, a sight line tracking algorithm based on pupil-cornea reflection and the like.
Optionally, the obtained sight line dwell region of the video viewer is used as a region of interest when the video viewer watches the video. For example, if the user records a short video with a mobile phone and the line of sight stays in the area above the short video display interface, the area above the short video display interface may be used as the area of interest when the user watches the video. For another example, if the user watches a short video with a mobile phone and the sight line stays in the middle area of the short video display interface, the middle area of the short video display interface may be used as the region of interest when the user watches the video.
In step S12, an advertisement to be placed is acquired.
Optionally, the advertisement to be implanted is obtained from an advertisement database. Wherein the advertisement database may be configured on a server.
In step S13, a three-dimensional spatial location for delivering the advertisement to be implanted is generated in the video according to the advertisement to be implanted and the region of interest.
Optionally, after the advertisement to be implanted is obtained and the region of interest when the video viewer watches the video is determined, a three-dimensional space position suitable for delivering the advertisement to be implanted may be generated in the video according to the region of interest and the advertisement content of the advertisement to be implanted. The purpose of generating the three-dimensional space position in the video is to be able to place the advertisement to be placed in the video, that is, because the video has corresponding scene content, if an advertisement is to be placed on a certain area of the video, the advertisement can be placed on the area only by using three-dimensional information, so that the area where the advertisement is placed displays the advertisement and does not display the blocked content any more, thereby achieving the purpose of placing the advertisement in the video. Therefore, in order to realize that the advertisement is really merged into the native content of the video, a three-dimensional space position suitable for putting the advertisement to be implanted can be generated in the video by utilizing a depth estimation mode according to the content and the region of interest of the advertisement to be implanted, so that the advertisement to be implanted can be put into the video by utilizing the three-dimensional space position later. The purpose of generating the three-dimensional space position in the embodiment of the present disclosure is to be able to use the depth information of the position to place an advertisement to be implanted on the position, so that the advertisement covers the original content on the position, and the advertisement content is displayed on the position.
In step S14, the advertisement to be implanted is placed on the three-dimensional space position to be incorporated into the native content of the video.
Optionally, after generating a three-dimensional space position suitable for placing the advertisement to be implanted in the video, the advertisement to be implanted may be placed on the three-dimensional space position, so as to integrate the advertisement to be implanted into the native content of the video. It can be understood that, in order to implement the placement of the advertisement to be implanted into the video content, a three-dimensional space position suitable for the placement of the advertisement to be implanted needs to be generated in the video, and the advertisement to be implanted is placed on the three-dimensional space position, so that the original content of the video on the position is shielded by the advertisement to be implanted, and the advertisement to be implanted is displayed on the position of the video, thereby implementing the seamless integration of the advertisement content into the original content of the video.
According to the method for implanting the advertisement into the video, the sight line staying area of a video viewer can be obtained, the sight line staying area is used as the region of interest of the video viewer when the video viewer watches the video, the advertisement to be implanted is obtained, the three-dimensional space position suitable for putting the advertisement to be implanted is generated in the video according to the advertisement to be implanted and the region of interest, and the advertisement to be implanted is put on the three-dimensional space position so as to be integrated into the original content of the video. Therefore, the three-dimensional space position suitable for advertisement putting in the internet video stream is automatically mined and generated through the artificial intelligence technologies such as sight line tracking, region-of-interest detection, three-dimensional reconstruction and depth estimation, the advertisement is seamlessly integrated into the original content of the video by utilizing the three-dimensional space position, the watching of a user is not disturbed to the greatest extent, and the advertisement investment return rate is improved in a more intelligent and invisible mode. In addition, because the implanted advertisements generally have a time concept, the dimension of the advertisement implantation is promoted to a four-dimensional space (i.e., time + three-dimensional space) by the method, so that the video implanted in the advertisement is more real and natural, the implantation effect is closer to the real physical world, and the more vivid implantation effect is realized. According to the advertisement implanting method, the advertisement implanting effect can be synchronously seen in the video shooting process or the video watching process (such as a video watcher watches a live video) without depending on the video post-processing in the advertisement implanting process, and the video implanting efficiency is improved.
In order to further improve the user experience, the implanted advertisement is enabled to better meet the user requirements, and personalized recommendation of the advertisement is achieved. The video content and attribute information of the video viewer can be utilized to personalize and recommend advertisements for the video viewer. In particular, fig. 2 is a flow chart illustrating another method of placing advertisements in a video, as shown in fig. 2, according to an example embodiment, which includes the following steps.
In step S21, a line-of-sight stay area of the video viewer is acquired.
In step S22, the line-of-sight stay region is set as a region of interest when the video viewer watches the video.
In step S23, the content category of the video is acquired.
Optionally, the video content is classified using a video content understanding algorithm to obtain a content category of the video. Alternatively, an image recognition algorithm may be used to identify the content category of the key frames in the video and determine the content category of the video by counting the classification results of the sequence of key frames. Two example implementations of these will be given below:
as an example possible implementation, visual and audio features of a video may be extracted and input to a video content classification model to obtain a content category of the video. Therefore, the classification of the video content can be realized through the visual characteristic and the audio semantic characteristic of the video, and the efficiency of video content analysis is improved.
In this disclosure, the video content classification model may be a model obtained by pre-training with training data, where the training data may include visual feature samples, audio feature samples, and video content labels corresponding to the samples, and a classifier is trained based on the training data, so as to obtain the video content classification model. The video content classification model can be used for classifying the video content and determining the content category of the video.
As another example of possible implementation manners, a plurality of key frames of the video may be extracted, each key frame is subjected to content classification to obtain a classification result of each key frame, and the classification result of each key frame is counted to obtain a content category of the video. It can be understood that the video content understanding algorithm can directly classify the video, and the image recognition algorithm needs to firstly frame the video, extract a plurality of key frames in the video, classify the key frame images obtained by frame extraction, and determine the content category of the video by counting the classification result of the image sequence. For example, if the classification results of consecutive frames of key frame images are all cosmetic contents, the video can be determined to be a cosmetic video. Therefore, the key frames in the video are classified, and the classification result of the key frames is counted to determine the content classification of the video, so that the accuracy of video content analysis can be improved.
In step S24, attribute information of the video viewer is acquired.
In the disclosed embodiment, the attribute information may include, but is not limited to, identity information, professional information, hobby information, and the like. The identity information may include, but is not limited to, gender, age, location, etc.
In step S25, the advertisement to be implanted is acquired according to the content category of the video and the attribute information of the video viewer.
Optionally, according to the content category of the video and the attribute information of the video viewer, retrieving is performed in an advertisement database to obtain the advertisement to be implanted, which is matched with the content category of the video and the attribute information of the video viewer, so that personalized advertisement recommendation for the video viewer is realized.
In step S26, a three-dimensional spatial location for delivering the advertisement to be implanted is generated in the video according to the advertisement to be implanted and the region of interest.
In step S27, the advertisement to be implanted is placed on the three-dimensional space position to be incorporated into the native content of the video.
According to the method for implanting the advertisement into the video, the advertisement can be recommended to the video viewer in a personalized mode by utilizing the video content and the attribute information of the video viewer, so that the implanted advertisement can better meet the requirements of the user, the personalized recommendation of the advertisement is realized, and the user experience is further improved.
To enable seamless incorporation of the implanted advertisements into the native content of the video, the spatial implantation dimension of the video advertisements may be expanded from two dimensions to three dimensions. Additionally, to enable placement of advertisements into a video, an implant suitable for carrying the advertisement can be identified from the video for placement of the advertisement onto the implant. Specifically, in some embodiments of the present disclosure, as shown in fig. 3, the specific implementation process of generating a three-dimensional spatial position suitable for delivering the advertisement to be implanted in the video according to the advertisement to be implanted and the region of interest may include the following steps:
in step S31, the content type of the advertisement to be placed is acquired.
Optionally, the advertisement database stores advertisements and content types thereof, and when the advertisement to be implanted is obtained from the advertisement database, the content type corresponding to the advertisement to be implanted may also be obtained from the advertisement database. In other embodiments of the present disclosure, the advertisement to be implanted may be classified by a video content understanding algorithm or an image recognition algorithm, so as to obtain the content type of the advertisement to be implanted.
In step S32, it is determined whether an implant object to be implanted with an advertisement exists in the region of interest according to the content type of the advertisement to be implanted.
Wherein the implanted object may be understood as an object in the region of interest of the video, which may include, but is not limited to, a wall, a surface of an object in a scene, clothing, a table, the ground, etc.
In some embodiments of the present disclosure, at least one object in the region of interest may be acquired, a type of each object may be determined, and whether an object that can be used to carry the advertisement to be implanted exists in the at least one object may be determined according to a content type of the advertisement to be implanted and the type of each object, and when an object that can be used to carry the advertisement to be implanted does not exist in the at least one object, it may be determined that an object that can be used to implant the advertisement does not exist in the region of interest; when an object capable of being used for bearing the advertisement to be implanted exists in at least one object, the object capable of being used for bearing the advertisement to be implanted is determined as an implantation object, and the implantation object capable of being used for implanting the advertisement is judged to exist in the region of interest.
For example, an object detection algorithm may be utilized to analyze at least one object within the region of interest and determine the type of each object, such as whether the object is a wall, or an object surface, or a garment, or a table top or floor, etc. Determining whether an object suitable for bearing the advertisement to be implanted exists in at least one object according to the content type of the advertisement to be implanted and the type of each object, and supposing that the following objects exist in the region of interest: the method comprises the steps that the ground and a table are provided, the content type of the advertisement to be implanted is a milk bottle, namely the advertisement is a milk advertisement, the content of the advertisement is a milk bottle, at the moment, whether an object suitable for bearing the advertisement to be implanted exists in an interested area or not can be determined according to the content type of the advertisement to be implanted and the type of each object, and the milk bottle is placed on the table to be closer to the real life and have a real effect, so that a desktop in the interested area can be used as an implanted object suitable for bearing the advertisement to be implanted, namely, the implanted object capable of being implanted with the advertisement exists in the interested area is judged, and the implanted object is the desktop.
For another example, assuming that there are wall surfaces and clothes in the region of interest and assuming that the content type of the advertisement to be implanted is a two-dimensional LOGO, since the clothes are more eye-catching, the clothes in the region of interest may be determined as an implant object suitable for carrying the advertisement to be implanted, or both the clothes and the wall surfaces in the region of interest may be regarded as implant objects suitable for carrying the advertisement to be implanted.
For another example, if only the wall and the ground are identified in the region of interest, that is, the content of the video is an open indoor scene, if the content type of the advertisement to be implanted is a milk bottle, that is, the advertisement is a milk advertisement, and the content of the advertisement is a milk bottle, since the milk bottle is placed on a table to be closer to the real life, the effect is more real, it can be determined that there is no implanted object in the region of interest for implanting the advertisement.
In step S33, when an implant object to be implanted with an advertisement exists in the region of interest, three-dimensional information of the implant object within the video is acquired.
Optionally, when it is determined that an implant object capable of being used for implanting the advertisement exists in the region of interest, the depth estimation algorithm may be used to obtain three-dimensional information of the implant object in the video, for example, the monocular depth estimation algorithm may be used to obtain depth information of the implant object in the video, so that the three-dimensional information of the implant object in the video may be obtained.
In step S34, a three-dimensional spatial position for placing the advertisement to be implanted is generated according to the three-dimensional information of the implantation object in the video.
Optionally, three-dimensional information of content in the advertisement to be implanted is acquired, at least a partial region of the implanted object is used as a position for delivering the advertisement to be implanted according to the three-dimensional information of the content in the advertisement to be implanted and the three-dimensional information of the implanted object in the video, and in order to deliver the advertisement to be implanted to the position, a three-dimensional space position needs to be generated at the position, so that the three-dimensional space position capable of delivering the advertisement to be implanted is obtained.
It should be noted that the above object detection algorithm may be replaced by an image salient region detection algorithm, a panorama segmentation algorithm, or a semantic segmentation algorithm. The image salient region detection algorithm, the panorama segmentation algorithm and the semantic segmentation algorithm can obtain the position information of the implanted object through the connected domain detection algorithm after salient region detection and segmentation are completed.
In order to enable the advertisement implanting effect to be closer to the physical world and be closer to the reality, in some embodiments of the present disclosure, when an implanting object to be implanted with an advertisement does not exist in the region of interest, a virtual implanting object is generated according to the content type of the advertisement to be implanted; acquiring three-dimensional information of a virtual implanted object in a video; and generating a three-dimensional space position which can be used for putting the advertisement to be implanted in the video according to the three-dimensional information of the virtual implanted object in the video. That is to say, when it is determined that there is no implanted object to be implanted with an advertisement in the region of interest, a virtual implanted object capable of bearing the advertisement to be implanted can be generated according to the content type of the advertisement to be implanted, and then a three-dimensional space position suitable for delivering the advertisement to be implanted is generated in the video according to the three-dimensional information of the virtual implanted object in the video. For example, if only the wall and the ground are identified in the region of interest, that is, the content of the video is an open indoor scene, if the content type of the advertisement to be implanted is a milk bottle, that is, the advertisement is a milk advertisement, the content of the advertisement is a milk bottle, since the milk bottle is placed on a table to be closer to the real life and the effect is more real, a virtual table can be introduced on the ground by introducing a three-dimensional model generation mode, the virtual table is used as a virtual implantation object, and a three-dimensional space position suitable for placing the advertisement to be implanted is generated in the video according to the three-dimensional information of the three-dimensional virtual implantation object model of the table, that is, the milk bottle can be placed on the generated virtual table, the advertisement can be implanted in the synthesized virtual object or scene according to the requirement, and support is provided for richer advertisement originality, therefore, the advertisement implanting effect is closer to the physical world and the reality.
In order to further improve the advertisement implanting effect, in some embodiments of the present disclosure, during the process of placing the advertisement to be implanted on the three-dimensional spatial position, the three-dimensional spatial orientation of the content to be implanted with the advertisement may be adjusted, so that the three-dimensional spatial orientation of the content to be implanted with the advertisement is consistent with the three-dimensional spatial orientation of the object in the three-dimensional spatial position. As an example, the object in the three-dimensional space may be an original object of the video (i.e. the implantation object that can be used to carry the advertisement to be implanted), or may be a virtual object generated to be able to carry the advertisement to be implanted. For example, in the process of delivering the advertisement to be implanted in the three-dimensional space position, the three-dimensional space orientation of the content of the advertisement to be implanted can be adjusted according to the three-dimensional space orientation of the implanted object or the virtual implanted object, so that the three-dimensional space orientation of the advertisement content is consistent with the three-dimensional space orientation of the object in the three-dimensional space position, people can feel that the advertisement content is a part of the video native content, no sense of incongruity exists, and the advertisement implantation effect can be further improved.
In order to further improve the exposure rate of advertisement putting, the implantation effect is closer to the real physical world, and the more vivid implantation effect is realized. In some embodiments of the present disclosure, after the advertisement to be placed is placed in the three-dimensional space, the area of the video where the advertisement has been placed is tracked in real time frame by frame in the three-dimensional space, when the three-dimensional information of the tracked area changes in the current frame, the three-dimensional information of the area in the current frame is obtained, and the three-dimensional spatial orientation of the content where the advertisement has been placed is adjusted in real time according to the three-dimensional information of the area in the current frame, so that the three-dimensional spatial orientation of the content where the advertisement has been placed is consistent with the three-dimensional spatial orientation of the object in the area. For example, after the advertisement to be implanted is placed in the three-dimensional space, the three-dimensional space of the area in which the advertisement has been placed in the video can be tracked frame by frame in real time with six degrees of freedom, and when the three-dimensional information of the area is tracked to change in the current frame, the three-dimensional space orientation of the content in which the advertisement has been placed is adjusted in real time according to the three-dimensional novelty of the area in the current frame, so that the three-dimensional space orientation of the content in which the advertisement has been placed is consistent with the three-dimensional space orientation of the object in the area, thereby enabling the advertisement content to be seamlessly integrated into the original content of the video, enabling the video in which the advertisement has been implanted to be more realistic and natural, enabling the implantation effect to be closer to the real physical world, and further realizing more vivid implantation effect.
For example, assuming that the user is recording a short video, the original content display effect of the short video may be as shown in fig. 4. At this time, the sight line tracking algorithm can be used for determining the sight line dwell area of the user watching the video, so as to serve as the area of interest of the user when watching the video, and the area of interest is assumed to be the area where the person is located. Assuming that the advertisement to be implanted is a panda LOGO, at this time, it can be determined that an implantation object, such as clothes, suitable for implanting the advertisement to be implanted exists in the region of interest according to the content type of the advertisement to be implanted, and at this time, a three-dimensional space position suitable for delivering the advertisement to be implanted can be generated according to three-dimensional information of the clothes in the video. And then, the advertisement of the panda LOGO is released in the three-dimensional space position, in the releasing process, the three-dimensional orientation of the panda LOGO content can be adjusted according to the three-dimensional space orientation of clothes, so that the three-dimensional orientation of the panda LOGO content is consistent with the three-dimensional space orientation of the clothes, the region in which the advertisement is implanted in the video is tracked in real time frame by frame in 6 degrees of freedom in the three-dimensional space, and the three-dimensional orientation of the advertisement content is adjusted in real time according to the tracking result and implanted. For example, as shown in fig. 5, for the effect after the advertisement is implanted in the video, the panda LOGO content is superimposed on the surface of the garment in the video, so that the advertisement content is seamlessly integrated into the original content of the video, the video implanted with the advertisement is more real and natural, the implantation effect is closer to the real physical world, and the more vivid implantation effect is realized.
Fig. 6 is a block diagram illustrating an apparatus for placing advertisements in videos according to an example embodiment. Referring to fig. 6, the apparatus 600 includes: a first acquisition module 610, a second acquisition module 620, a generation module 630, and a delivery module 640.
Specifically, the first acquiring module 610 is configured to acquire a line-of-sight dwell area of the video viewer, and take the line-of-sight dwell area as an area of interest when the video viewer watches the video.
The second retrieval module 620 is configured to retrieve the ad to be implanted.
The generating module 630 is configured to generate a three-dimensional spatial location in the video for placement of the advertisement to be implanted according to the advertisement to be implanted and the region of interest. In some embodiments of the present disclosure, the generation module 630 is configured to: acquiring the content type of the advertisement to be implanted; determining whether an implantation object to be implanted with the advertisement exists in the region of interest according to the content type of the advertisement to be implanted; when an implanted object to be implanted with an advertisement exists in the region of interest, acquiring three-dimensional information of the implanted object in the video; and generating a three-dimensional space position for delivering the advertisement to be implanted according to the three-dimensional information of the implanted object in the video.
In some embodiments of the present disclosure, the generation module 630 is further configured to: when the implant object to be implanted with the advertisement does not exist in the region of interest, generating a virtual implant object according to the content type of the advertisement to be implanted; acquiring three-dimensional information of a virtual implanted object in a video; and generating a three-dimensional space position for delivering the advertisement to be implanted in the video according to the three-dimensional information of the virtual implanted object in the video.
In the embodiment of the present disclosure, the specific implementation process of the generating module 630 determining whether there is an implanted object to be implanted with an advertisement in the region of interest according to the content type of the advertisement to be implanted may be as follows: acquiring at least one object within a region of interest; determining a type of each object; determining whether an object used for bearing the advertisement to be implanted exists in at least one object according to the content type of the advertisement to be implanted and the type of each object; when an object used for bearing the advertisement to be implanted does not exist in at least one object, determining that an implantation object in which the advertisement to be implanted does not exist in the region of interest; when an object used for bearing the advertisement to be implanted exists in at least one object, determining the object used for bearing the advertisement to be implanted as an implanted object, and determining that the implanted object in which the advertisement to be implanted exists in the region of interest.
The placement module 640 is configured to place the ad to be placed on the three-dimensional spatial location to incorporate the ad to be placed into the native content of the video.
In some embodiments of the present disclosure, as shown in fig. 7, the second obtaining module 620 may include: a first acquisition unit 621 and a second acquisition unit 622. Wherein, the first obtaining unit 621 is configured to obtain a content category of the video and attribute information of a video viewer; the second obtaining unit 622 is configured to obtain the advertisement to be implanted according to the content category of the video and the attribute information of the video viewer.
In some embodiments of the present disclosure, the first obtaining unit 621 is configured to: extracting visual features and audio features of the video; and inputting the visual characteristics and the audio characteristics into a video content classification model to obtain the content category of the video.
In some embodiments of the present disclosure, the first obtaining unit 621 is configured to: extracting a plurality of key frames of a video; classifying the content of each key frame to obtain a classification result of each key frame; and counting the classification result of each key frame to obtain the content category of the video.
In some embodiments of the present disclosure, as shown in fig. 8, the apparatus 600 for placing advertisements in videos may further include: a first adjustment module 650. The first adjusting module 650 is configured to adjust the three-dimensional spatial orientation of the content to be advertised during the process of delivering the content to be advertised in the three-dimensional spatial position, so that the three-dimensional spatial orientation of the content to be advertised is consistent with the three-dimensional spatial orientation of the object in the three-dimensional spatial position.
In some embodiments of the present disclosure, as shown in fig. 9, the apparatus 600 for placing advertisements in videos may further include: a tracking module 660 and a second adjustment module 670. Wherein the tracking module 660 is configured to track the region of the video with the advertisement in three-dimensional space frame by frame in real time; the second adjusting module 660 is configured to, when the three-dimensional information of the region is tracked to change in the current frame, obtain the three-dimensional information of the region in the current frame, and adjust the three-dimensional spatial orientation of the advertised content in real time according to the three-dimensional information of the region in the current frame, so that the three-dimensional spatial orientation of the advertised content is consistent with the three-dimensional spatial orientation of the object in the region.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
According to the device for implanting the advertisement into the video, the three-dimensional space position suitable for advertisement putting in the internet video stream can be automatically mined and generated through the artificial intelligent technologies such as sight tracking, region-of-interest detection, three-dimensional reconstruction and depth estimation, the advertisement is seamlessly integrated into the original content of the video by utilizing the three-dimensional space position, the purpose that the user is not disturbed to watch and can be noticed at the same time is achieved to the maximum extent, and the return rate of the advertisement investment is improved in a more intelligent and invisible mode. In addition, because the implanted advertisements generally have a time concept, the dimension of the advertisement implantation is promoted to a four-dimensional space (i.e., time + three-dimensional space) by the method, so that the video implanted in the advertisement is more real and natural, the implantation effect is closer to the real physical world, and the more vivid implantation effect is realized. According to the advertisement implanting method, the advertisement implanting effect can be synchronously seen in the video shooting process or the video watching process (such as a video watcher watches a live video) without depending on the video post-processing in the advertisement implanting process, and the video implanting efficiency is improved. To implement the above embodiments, the present disclosure also provides an electronic device, and fig. 10 is a block diagram of an electronic device 1000 shown according to an exemplary embodiment. For example, the electronic device 1000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 10, electronic device 1000 may include one or more of the following components: processing component 1002, memory 1004, power component 1006, multimedia component 1008, audio component 1010, input/output (I/O) interface 1012, sensor component 1014, and communications component 1016.
The processing component 1002 generally controls overall operation of the electronic device 1000, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 1002 may include one or more processors 1020 to execute instructions to perform all or a portion of the steps of the methods described above. Further, processing component 1002 may include one or more modules that facilitate interaction between processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate interaction between the multimedia component 1008 and the processing component 1002.
The memory 1004 is configured to store various types of data to support operations at the electronic device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 1004 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The power supply component 1006 provides power to the various components of the electronic device 1000. The power components 1006 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 1000.
The multimedia component 1008 includes a touch-sensitive display screen that provides an output interface between the electronic device 1000 and a user. In some embodiments, the touch display screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1008 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 1010 is configured to output and/or input audio signals. For example, the audio component 1010 may include a Microphone (MIC) configured to receive external audio signals when the electronic device 1000 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in the memory 1004 or transmitted via the communication component 1016. In some embodiments, audio component 1010 also includes a speaker for outputting audio signals.
I/O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 1014 includes one or more sensors for providing various aspects of status assessment for the electronic device 1000. For example, the sensor assembly 1014 may detect an open/closed state of the electronic device 1000, the relative positioning of components, such as a display and keypad of the electronic device 1000, the sensor assembly 1014 may also detect a change in position of the electronic device 1000 or a component of the electronic device 1000, the presence or absence of user contact with the electronic device 1000, orientation or acceleration/deceleration of the electronic device 1000, and a change in temperature of the electronic device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 1014 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 1016 is configured to facilitate wired or wireless communication between the electronic device 1000 and other devices. The electronic device 1000 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communications component 1016 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the electronic device 1000 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described method of advertising in video.
In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 1004 comprising instructions, executable by the processor 1020 of the electronic device 1000 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
A non-transitory computer readable storage medium having instructions therein, which when executed by a processor of an electronic device 1000, enable the electronic device 1000 to perform a method of placing advertisements in videos.
A computer program product having instructions which, when executed by a processor of an electronic device 1000, enable the electronic device 1000 to perform a method of placing advertisements in videos.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (10)

1. A method for placing advertisements in a video, comprising:
acquiring a sight line staying area of a video viewer, and taking the sight line staying area as an interested area when the video viewer watches a video;
acquiring an advertisement to be implanted;
generating a three-dimensional space position for delivering the advertisement to be implanted in the video according to the advertisement to be implanted and the region of interest; and
and putting the advertisement to be implanted on the three-dimensional space position so as to blend the advertisement to be implanted into the native content of the video.
2. The method for implanting advertisement in video according to claim 1, wherein the obtaining of the advertisement to be implanted comprises:
acquiring the content category of the video and the attribute information of the video viewer;
and acquiring the advertisement to be implanted according to the content category of the video and the attribute information of the video viewer.
3. The method for advertising in video according to claim 2, wherein the obtaining the content category of the video comprises:
extracting visual features and audio features of the video;
and inputting the visual features and the audio features into a video content classification model to obtain the content category of the video.
4. The method for advertising in video according to claim 2, wherein the obtaining the content category of the video comprises:
extracting a plurality of key frames of the video;
performing content classification on each key frame to obtain a classification result of each key frame;
and counting the classification result of each key frame to obtain the content category of the video.
5. The method according to claim 1, wherein the generating a three-dimensional spatial position for delivering the advertisement to be implanted in the video according to the advertisement to be implanted and the region of interest comprises:
acquiring the content type of the advertisement to be implanted;
determining whether an implantation object of the advertisement to be implanted exists in the region of interest according to the content type of the advertisement to be implanted;
when the implantation object to be implanted with the advertisement exists in the region of interest, acquiring three-dimensional information of the implantation object in the video;
and generating a three-dimensional space position for delivering the advertisement to be implanted according to the three-dimensional information of the implanted object in the video.
6. The method of advertising in a video according to claim 5, further comprising:
when the implant object to be implanted with the advertisement does not exist in the region of interest, generating a virtual implant object according to the content type of the advertisement to be implanted;
acquiring three-dimensional information of the virtual implanted object in the video;
and generating a three-dimensional space position for delivering the advertisement to be implanted in the video according to the three-dimensional information of the virtual implanted object in the video.
7. The method for implanting advertisement in video according to claim 5, wherein the determining whether there is an implantation object of the advertisement to be implanted in the region of interest according to the content type of the advertisement to be implanted comprises:
acquiring at least one object within the region of interest;
determining a type of each object;
determining whether an object used for bearing the advertisement to be implanted exists in the at least one object according to the content type of the advertisement to be implanted and the type of each object;
when the object used for bearing the advertisement to be implanted does not exist in the at least one object, determining that the object to be implanted with the advertisement does not exist in the region of interest;
when the object used for bearing the advertisement to be implanted exists in the at least one object, determining the object used for bearing the advertisement to be implanted as the implanted object, and determining that the implanted object with the advertisement to be implanted exists in the region of interest.
8. An apparatus for placing advertisements in a video, comprising:
the first acquisition module is configured to acquire a sight line staying area of a video viewer and take the sight line staying area as an interested area when the video viewer watches videos;
a second obtaining module configured to obtain an advertisement to be implanted;
a generating module configured to generate a three-dimensional space position for delivering the advertisement to be implanted in the video according to the advertisement to be implanted and the region of interest; and
a placement module configured to place the advertisement to be placed on the three-dimensional spatial location to incorporate the advertisement to be placed into native content of the video.
9. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement a method of placing advertisements in videos as claimed in any one of claims 1 to 7.
10. A storage medium having instructions that, when executed by a processor of an electronic device, enable the electronic device to perform a method of placing advertisements in videos as claimed in any one of claims 1 to 7.
CN202010725796.2A 2020-07-24 2020-07-24 Method, device, electronic equipment and storage medium for implanting advertisement in video Active CN114051166B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010725796.2A CN114051166B (en) 2020-07-24 2020-07-24 Method, device, electronic equipment and storage medium for implanting advertisement in video

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010725796.2A CN114051166B (en) 2020-07-24 2020-07-24 Method, device, electronic equipment and storage medium for implanting advertisement in video

Publications (2)

Publication Number Publication Date
CN114051166A true CN114051166A (en) 2022-02-15
CN114051166B CN114051166B (en) 2024-03-29

Family

ID=80204320

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010725796.2A Active CN114051166B (en) 2020-07-24 2020-07-24 Method, device, electronic equipment and storage medium for implanting advertisement in video

Country Status (1)

Country Link
CN (1) CN114051166B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116308530A (en) * 2023-05-16 2023-06-23 飞狐信息技术（天津）有限公司 Advertisement implantation method, advertisement implantation device, advertisement implantation equipment and readable storage medium
CN117939184A (en) * 2024-03-25 2024-04-26 成都索贝数码科技股份有限公司 Advertisement implantation method, device, equipment and medium for sports rebroadcasting field

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120101878A1 (en) * 2010-10-25 2012-04-26 Hon Hai Precision Industry Co., Ltd. Advertisement display system and method
CN104066003A (en) * 2014-06-16 2014-09-24 百度在线网络技术（北京）有限公司 Method and device for playing advertisement in video
CN107341435A (en) * 2016-08-19 2017-11-10 北京市商汤科技开发有限公司 Processing method, device and the terminal device of video image
CN108076359A (en) * 2017-01-24 2018-05-25 北京市商汤科技开发有限公司 Methods of exhibiting, device and the electronic equipment of business object
CN109670860A (en) * 2018-11-27 2019-04-23 平安科技（深圳）有限公司 Advertisement placement method, device, electronic equipment and computer readable storage medium
CN110458820A (en) * 2019-08-06 2019-11-15 腾讯科技（深圳）有限公司 A kind of multimedia messages method for implantation, device, equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120101878A1 (en) * 2010-10-25 2012-04-26 Hon Hai Precision Industry Co., Ltd. Advertisement display system and method
CN104066003A (en) * 2014-06-16 2014-09-24 百度在线网络技术（北京）有限公司 Method and device for playing advertisement in video
CN107341435A (en) * 2016-08-19 2017-11-10 北京市商汤科技开发有限公司 Processing method, device and the terminal device of video image
CN108076359A (en) * 2017-01-24 2018-05-25 北京市商汤科技开发有限公司 Methods of exhibiting, device and the electronic equipment of business object
CN109670860A (en) * 2018-11-27 2019-04-23 平安科技（深圳）有限公司 Advertisement placement method, device, electronic equipment and computer readable storage medium
CN110458820A (en) * 2019-08-06 2019-11-15 腾讯科技（深圳）有限公司 A kind of multimedia messages method for implantation, device, equipment and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116308530A (en) * 2023-05-16 2023-06-23 飞狐信息技术（天津）有限公司 Advertisement implantation method, advertisement implantation device, advertisement implantation equipment and readable storage medium
CN117939184A (en) * 2024-03-25 2024-04-26 成都索贝数码科技股份有限公司 Advertisement implantation method, device, equipment and medium for sports rebroadcasting field

Also Published As

Publication number Publication date
CN114051166B (en) 2024-03-29

Similar Documents

Publication Publication Date Title
CN110662083B (en) Data processing method and device, electronic equipment and storage medium
CN106792004B (en) Content item pushing method, device and system
CN109637518B (en) Virtual anchor implementation method and device
KR101910346B1 (en) Picture processing method and apparatus
CN110517185B (en) Image processing method, device, electronic equipment and storage medium
CN106776890B (en) Method and device for adjusting video playing progress
CN112153400B (en) Live broadcast interaction method and device, electronic equipment and storage medium
KR102319423B1 (en) Context-Based Augmented Advertising
CN107948667B (en) Method and device for adding display special effect in live video
WO2019037615A1 (en) Video processing method and device, and device for video processing
CN109429078B (en) Video processing method and device for video processing
KR20160012902A (en) Method and device for playing advertisements based on associated information between audiences
CN111506758B (en) Method, device, computer equipment and storage medium for determining article name
US20140223474A1 (en) Interactive media systems
CN111986076A (en) Image processing method and device, interactive display device and electronic equipment
CN111368127B (en) Image processing method, image processing device, computer equipment and storage medium
CN113099297B (en) Method and device for generating click video, electronic equipment and storage medium
CN114051166B (en) Method, device, electronic equipment and storage medium for implanting advertisement in video
CN109509195B (en) Foreground processing method and device, electronic equipment and storage medium
KR20220026470A (en) Method for extracting video clip, apparatus for extracting video clip, and storage medium
CN111753135A (en) Video display method, device, terminal, server, system and storage medium
CN112464031A (en) Interaction method, interaction device, electronic equipment and storage medium
CN111556352A (en) Multimedia resource sharing method and device, electronic equipment and storage medium
CN112000266A (en) Page display method and device, electronic equipment and storage medium
CN113689530A (en) Method and device for driving digital person and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant