WO2018107914A1

WO2018107914A1 - Video analysis platform, matching method, and accurate advertisement push method and system

Info

Publication number: WO2018107914A1
Application number: PCT/CN2017/108384
Authority: WO
Inventors: 张鹏程; 刘勇; 张家明
Original assignee: 中兴通讯股份有限公司
Priority date: 2016-12-16
Filing date: 2017-10-30
Publication date: 2018-06-21
Also published as: CN106686404A; CN106686404B

Abstract

Embodiments of the present application disclose a matching method, an accurate advertisement push method and system, and a video analysis platform. The matching method comprises: obtaining a resource to be matched, and extracting images from the resource to be matched, to obtain a first image set; obtaining a second video resource, and extracting images from the second video resource, to obtain a second image set; comparing the images in the first image set with the images in the second image set, to obtain a similarity level; and if a similarity level between a first image in the first image set and a second image in the second image set meets a first preset condition, associating the resource to be matched to which the first image belongs with the second video resource to which the second image belongs. In the technical solution provided in the embodiments of this application, association is performed automatically according to a similarity level by means of image processing, thereby reducing manual intervention, and achieving high association efficiency.

Description

Video analysis platform, matching method, accurate advertising method and system

Technical field

The invention relates to an image processing technology, in particular to a video analysis platform, a matching method, a precise advertising method and a system.

Background technique

In the process of on-demand video playback, inserting commercial videos and advertising images is a common application system advertising method. When the user views, the advertisements required by the user will be highly recognized by the user, and the more the goal of the advertisement can be achieved. If the advertisement is randomly given to the user, it will not only arouse the user's dislike, but may also cause the user to reject the application system itself.

Advertising when users place video on demand, the currently used precision delivery technologies include:

The first solution: analyze the obtained user data, user behavior, and deliver advertisements that the user may need. The solution needs to store the information of each user, and takes up a lot of space in the case of a large number of users.

The second option: manual intervention, custom insert ads at a fixed point in time. The program requires manual intervention for each video, which is labor intensive and less efficient when the amount of video is large.

Summary of the invention

Embodiments of the present invention provide a matching method, a method and system for accurately placing advertisements, and a video analysis platform, which can implement video matching, reduce space occupation, and reduce manual intervention.

In order to achieve the object of the present invention, an embodiment of the present invention provides a matching method, including:

Obtaining a to-be-matched resource, extracting a picture from the to-be-matched resource, and obtaining a first picture set;

Obtaining a second video resource, and extracting a picture from the second video resource according to one or more of an image, a sound, and a subtitle of the second video resource, to obtain a second picture set;

Comparing the picture in the first picture set with the picture in the second picture set, Get similarity;

When the similarity between the first picture in the first picture set and the second picture in the second picture set satisfies a first preset condition, the to-be-matched resource to which the first picture belongs and the first The second video resource to which the two pictures belong is associated.

Optionally, extracting a picture from the to-be-matched resource or extracting a picture from the second video resource includes:

Taking the to-be-matched video or the second video resource in the to-be-matched resource as an input video, after decoding the input video, using the decoded input video as a to-be-extracted video; or, decoding the decoded video The input video is segmented according to the scenario, the segmented video is obtained, and the divided videos are compared, and the segmented video set is obtained according to the comparison result, and the segmented video in the segmented video set is used as the to-be-extracted video, where the segmented video set is The comparison result of any two divided videos satisfies the second preset condition;

Extracting a picture from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video.

Optionally, the extracting the picture from the to-be-matched resource includes: when the to-be-matched resource includes a to-be-matched picture, extracting the to-be-matched picture to join the first picture set.

Optionally, the dividing the decoded input video according to the scenario includes:

Determining whether the scene is switched according to one or more of background and sound in the input video;

The position where the scene switching occurs in the input video is divided.

Optionally, determining whether the scene is switched according to one or more of a background and a sound in the input video comprises: converting an audio signal in the input video from a time domain to a frequency domain, and acquiring the audio signal. The energy curve obtains the position of the maximum value and the minimum value of the energy curve as the position at which the scene switching occurs, and satisfies: the position of the adjacent two scenes that are acquired according to the energy curve is greater than the distance A distance threshold.

Optionally, the extracting the image from the to-be-extracted video according to one or more of the image, the sound, and the subtitle of the to-be-extracted video includes:

Extracting a video frame in the to-be-processed video as a key frame, which is referred to as a current key frame, and extracting a next key frame by sequentially: sequentially selecting a video frame after the current key frame in the to-be-processed video and the current key Comparing the frame to obtain a second similarity, if the second similarity between the second video frame and the current key frame satisfies a third preset condition and is closest to the current key frame, extracting the second video frame And using the second video frame as the current key frame, extracting the next key frame in the same manner; if there is no video frame that meets the third preset condition with the second similarity between the current key frame, The process of extracting a picture of the video to be processed ends, and the extracted key frame is taken as the extracted picture;

Wherein, initially, the starting frame of the to-be-processed video is taken as the current key frame.

Optionally, the extracting the video frame from the to-be-extracted video according to one or more of the image, the sound, and the subtitle of the to-be-extracted video, as the extracted image, includes:

Converting an audio signal in the to-be-extracted video from a time domain to a frequency domain, acquiring an energy curve of the audio signal, acquiring a maximum value and a minimum value of the energy curve, and extracting the maximum value and The video frame corresponding to the minimum value position.

Searching for a subtitle of the video to be extracted, and if a preset keyword is searched, extracting a video frame corresponding to the subtitle including the keyword; wherein the subtitle of the to-be-extracted video is from the video to be extracted Obtaining or performing voice recognition acquisition on the audio of the video to be extracted.

Optionally, the similarity between the first picture in the first picture set and the second picture in the second picture set that meets the first preset condition is:

The similarity between the first picture and the second picture is greater than a first similarity threshold.

Optionally, the comparing the picture in the first picture set with the picture in the second picture set, and obtaining the similarity includes:

Extracting a first feature value of the picture in the first picture set according to the feature model obtained by the deep learning algorithm, and extracting a second picture in the second picture set according to the feature model The feature value compares the first feature value and the second feature value to obtain a similarity.

The embodiment of the invention further provides a video analysis platform, comprising a data acquisition module and a video picture analysis processing layer, wherein:

The data acquisition module is configured to acquire the to-be-matched resource and acquire the second video resource.

The video picture analysis processing layer includes a video processing module and a picture analysis module, where:

The video processing module is configured to: extract a picture from the to-be-matched resource to obtain a first picture set; and select one or more of the second video resource according to one or more of an image, a sound, and a subtitle of the second video resource. Extracting a picture to obtain a second picture set;

The image analysis module is configured to compare the picture in the first picture set with the picture in the second picture set to obtain a similarity; when the first picture and the first picture in the first picture set When the similarity of the second picture in the second picture set satisfies the first preset condition, the to-be-matched resource to which the first picture belongs and the second video resource to which the second picture belongs are associated.

Optionally, the video analysis platform further includes a video decoding layer, configured to decode the to-be-matched video resource in the to-be-matched resource, and then transmit the video resource as an input video to the video image analysis processing layer, and After the second video resource is decoded, it is transmitted as an input video to the video picture analysis processing layer;

The extracting, by the video processing module, the picture from the to-be-matched resource or the second video resource includes:

The input video is used as the to-be-extracted video; or the input video is segmented according to the scene, the segmented video is obtained, the segmented videos of the scenes are compared, and the segmented video set is obtained according to the comparison result, and the segmented video set is obtained. The split video in the video is to be extracted, wherein the comparison result of any two divided videos in the split video set satisfies a second preset condition;

Optionally, the extracting, by the video processing module, the picture from the to-be-matched resource includes: when the to-be-matched resource includes a to-be-matched picture, extracting the to-be-matched picture to join the first picture set.

Optionally, the video processing module includes a video segmentation unit, configured to determine whether the scene is switched according to one or more of a background and a sound in the input video, where the scene switching occurs in the input video. Segmenting; and comparing the divided videos to obtain the segmented video set.

Optionally, determining, by the video segmentation unit, whether the scenario is switched according to one or more of a background and a sound in the input video comprises: converting an audio signal in the input video from a time domain to a frequency domain, and acquiring The energy curve of the audio signal acquires the position of the maximum value and the minimum value of the energy curve as the position where the scene switching occurs, and satisfies: the position of the adjacent two scenes that are acquired according to the energy curve The distance is greater than the first distance threshold.

Optionally, the video processing module includes a multi-modal extraction key frame unit, and is configured to extract a video frame from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video. picture of.

Optionally, the multi-modal extraction key frame unit extracts a video frame from the to-be-extracted video as the extracted image according to one or more of the image, the sound, and the subtitle of the to-be-extracted video.

Optionally, the picture analysis module includes a deep learning feature model unit, a key frame feature extraction unit, and a feature matching association unit, where:

The deep learning feature module unit is configured to learn the labeled sample by using a deep learning algorithm to obtain a feature model;

The key frame feature extraction unit is configured to extract a first feature value of a picture in the first picture set according to the feature model, and extract a second feature value of the picture in the second picture set according to the feature model;

The feature matching association unit is configured to compare the first feature value and the second feature value to obtain a similarity, in the first picture and the second picture set in the first picture set When the similarity of the second picture meets the first preset condition, the to-be-matched resource to which the first picture belongs and the second video resource to which the second picture belongs are associated.

The embodiment of the invention further provides a method for accurately placing advertisements, including:

Get the creative, extract the image from the creative, and get a collection of ad images;

Obtaining a service video, extracting a picture from the service video according to one or more of an image, a sound, and a subtitle of the service video, to obtain a service picture set;

Comparing the advertisement picture in the advertisement picture set with the service picture in the service picture set to obtain a similarity;

When the similarity between the first advertisement picture in the advertisement picture set and the first service picture in the service picture set satisfies a first preset condition, the first creative material and the location to which the first advertisement picture belongs The first service video to which the first service picture belongs is associated, and the association information is generated, where the association information includes: identifier information of the first service video, and identifier information of the first advertisement material.

Optionally, extracting a picture from the creative or extracting a picture from the business video includes:

Taking the advertisement video or the service video in the creative as an input video, after decoding the input video, using the decoded input video as a to-be-extracted video; or, decoding the input video according to the scene Performing segmentation, obtaining a segmented video, comparing the divided videos, and obtaining a segmented video set according to the comparison result, and using the segmented video in the segmented video set as a to-be-extracted video, where any two segments in the segmented video set are segmented The comparison result of the video satisfies the second preset condition;

Optionally, extracting the image from the creative material includes: when the advertisement image includes an advertisement image, extracting the advertisement image to join the advertisement image collection.

Determining whether the scene is switched according to one or more of the background and the sound in the input video; and dividing the position where the scene switching occurs in the input video.

Optionally, determining whether the scene is switched according to one or more of a background and a sound in the input video comprises: converting an audio signal in the input video from a time domain to a frequency domain, and acquiring the audio. The energy curve of the signal obtains the position of the maximum value and the minimum value of the energy curve as the position where the scene switching occurs, and satisfies: the position of the adjacent two scenes that are acquired according to the energy curve is greater than the distance The first distance threshold.

Optionally, extracting, according to one or more images, sounds, and subtitles of the to-be-extracted video, a video frame from the to-be-extracted video as the extracted image includes:

Converting an audio signal in the to-be-extracted video from a time domain to a frequency domain, acquiring an energy curve of the audio signal, acquiring a maximum value and a minimum value of the energy curve, and extracting the maximum value and The video frame corresponding to the minimum value position is taken as the extracted picture.

Optionally, the similarity between the first advertisement image in the advertisement image set and the first service image in the service image collection meets the first preset condition, that is:

The similarity between the first advertisement picture and the first service picture is greater than a first similarity threshold.

Optionally, comparing the advertisement picture in the advertisement picture set with the service picture in the service picture set, and obtaining the similarity includes:

Extracting a first feature value of the image in the advertisement picture set according to the feature model obtained by the deep learning algorithm, extracting a second feature value of the picture in the service picture set according to the feature model, and using the first feature value and The second feature values are compared to obtain a similarity.

Optionally, the association information further includes: insertion location information that is inserted by the first creative material when the first service video is played, and the insertion location information is in the first according to the first service picture. The location in a business video is determined.

Optionally, the method further includes:

Performing background modeling and target detection on the service video, extracting a target of the motion to obtain a service video target object, determining an appearance time and a disappearance time of the service video target object in a service video to which the service video belongs, and the service video target An object is associated with a picture of the service picture set that includes the service video target object;

The associated information further includes insertion location information that is inserted by the first creative material when the first service video is played, and the insertion location information is based on the first service video target object associated with the first service picture. The appearance time and disappearance time in the first service video are determined.

Optionally, the method further includes:

Performing background modeling and target detection on the advertisement video in the creative, extracting the target of the motion to obtain the creative target object, and including the creative target object and the first image set containing the creative target object The image is associated, and the advertisement image in the creative is used as a target object of the creative and associated with the advertisement image to which it belongs;

Acquiring the first advertisement associated with the first advertisement image when the similarity between the first advertisement image in the advertisement image set and the first service image in the service image collection meets the first preset condition The target object is configured with an advertisement object identifier, and the first service video target object associated with the first service image is configured with the same advertisement object identifier as the first creative target object, thereby indicating the first advertisement image The associated first creative is associated with the first business video to which the first business image belongs, wherein the advertising object identifier indicates an advertising theme for which the first creative target object is targeted.

Optionally, the association information further includes an advertisement object identifier that is configured by the first creative target object.

Optionally, the method further includes:

After receiving the query request sent by the portal, determining a second advertisement image associated with the second creative target object carried in the query request, and searching for the similarity with the second advertisement image in the service image collection to satisfy the first And a fourth set of service picture sub-sets, the service picture sub-set is sent to the portal, so that the portal selects a second service picture from the service picture subset according to a user operation, and the selected The second service video target object is configured with the same advertisement object identifier as the second creative target object, thereby indicating the second creative material to which the second creative target object belongs and the second service The second business video to which the video target object belongs is associated.

The embodiment of the present invention further provides a video analysis platform, including a data acquisition module and a video image analysis processing layer, where the video image analysis processing layer includes a video processing module and a picture analysis module, where:

The data acquisition module is configured to acquire a creative material and obtain a business video;

The video processing module is configured to: extract a picture from the creative material to obtain an advertisement picture set; and extract a picture from the service video according to one or more of an image, a sound, and a subtitle of the service video, Obtain a collection of business images;

The image analysis module is configured to compare the advertisement image in the advertisement image collection with the business image in the business image collection to obtain a similarity; when the advertisement image collection When the similarity between the first advertisement picture and the first service picture in the service picture set meets the first preset condition, the first creative material to which the first advertisement picture belongs and the first service picture belong to The first service video is associated and generates association information, where the association information includes: identifier information of the first service video, and identifier information of the first advertisement material.

Optionally, the video analytics platform further includes a video decoding layer, configured to decode the advertisement video in the creative as an input video to the video image analysis processing layer, and decode the service video. And then transmitted as an input video to the video picture analysis processing layer;

The video processing module extracting a picture from the creative or extracting a picture from the business video includes:

Optionally, the video processing module extracting the image from the creative material includes: when the advertisement material includes an advertisement image, extracting the advertisement image to join the advertisement image collection.

Optionally, the video processing module includes a video segmentation unit, configured to determine whether the scene is switched according to one or more of a background and a sound in the input video, and perform segmentation at a position where the scene switching occurs in the input video. .

Optionally, determining, by the video segmentation unit, whether the scenario is switched according to one or more of the background and the sound in the input video comprises: converting the audio signal in the input video from a time domain to a frequency domain, and acquiring The energy curve of the audio signal is obtained, and the position of the maximum value and the minimum value of the energy curve is obtained as a position where the scene switching occurs, and the position of the adjacent two scenes that are acquired according to the energy curve is satisfied. The distance is greater than the first distance threshold.

Searching for a subtitle of the to-be-extracted video, if a preset keyword is searched, extracting a video frame corresponding to the subtitle containing the keyword as the extracted image; wherein the subtitle of the to-be-extracted video is from the Obtained in the subtitle file that is to be extracted from the video or Take the audio of the video for speech recognition.

The key frame feature extraction unit is configured to extract a first feature value of the image in the advertisement image set according to the feature model, and extract a second feature value of the image in the service image set according to the feature model;

The feature matching association unit is configured to compare the first feature value and the second feature value to obtain a similarity; when the first advertisement picture and the service picture set in the advertisement picture set When the similarity of the first service picture satisfies the first preset condition, the first creative material to which the first advertisement picture belongs and the first service video to which the first service picture belongs are associated, and the associated information is generated. The association information includes: identifier information of the first service video, and identifier information of the first advertisement material.

Optionally, the video processing module further includes: a background modeling unit and a target detection unit, where:

The background modeling unit is configured to perform background modeling on the service video;

The target detecting unit is configured to perform target detection on the service video, extract a target of the motion to obtain a service video target object, and determine that the service video target object belongs to the business An appearance time and a disappearing time in the service video, the service video target object is associated with a picture of the service picture set that includes the service video target object;

Optionally, the background modeling unit is further configured to perform background modeling on the advertisement video in the creative;

The target detecting unit is further configured to perform target detection on the advertisement video in the creative, extract a target of the motion to obtain a creative target object, and include the target object in the creative and the first image collection The image of the creative target object is associated, and the advertisement image in the creative is used as a target object of the creative and associated with the advertisement image to which it belongs;

The image analysis module is further configured to: when the similarity between the first advertisement image in the advertisement image set and the first service image in the service image collection meets the first preset condition, acquire the An advertisement object identifier configured by the first creative target object associated with an advertisement image, and an advertisement object identifier identical to the first creative target object configured for the first service video target object associated with the first service image And indicating that the first creative material to which the first advertisement image belongs is associated with the first service video to which the first service image belongs, wherein the advertisement object identifier indicates that the first creative target object is targeted Advertising theme.

Optionally, the image analysis module is further configured to:

After receiving the query request sent by the portal, determining a second advertisement image associated with the second creative target object carried in the query request, and searching for the similarity with the second advertisement image in the service image collection to satisfy the first Four preset business picture sub-sets, the business graph Sending a set of slices to the portal, so that the portal selects a second service picture from the subset of service pictures according to a user operation, and configures a second service video target object associated with the selected second service picture with the The second creative target object has the same advertisement object identifier, thereby indicating that the second creative object to which the second creative target object belongs is associated with the second business video to which the second business video target object belongs.

The embodiment of the invention further provides a video analysis platform, comprising: a memory and a processor, wherein:

The memory is configured to store a program;

The processor is configured to: obtain a resource to be matched, extract a picture from the to-be-matched resource, obtain a first picture set, and acquire a second video resource according to the image of the second video resource And extracting a picture from the second video resource to obtain a second picture set; comparing the picture in the first picture set with the picture in the second picture set, and obtaining Similarity; when the similarity of the first picture in the first picture set and the second picture in the second picture set meets the first preset condition, the to-be-matched resource to which the first picture belongs The second video resource to which the second picture belongs is associated.

Optionally, the extracting a picture from the to-be-matched resource or extracting a picture from the second video resource includes:

The position where the scene switching occurs in the input video is divided.

Searching for a subtitle of the video to be extracted, and if a preset keyword is searched, extracting a video frame corresponding to the subtitle including the keyword; wherein the subtitle of the video to be extracted is from Obtaining or performing voice recognition acquisition on the audio of the to-be-extracted video.

Extracting a first feature value of the picture in the first picture set according to the feature model obtained by the deep learning algorithm, and extracting a second feature value of the picture in the second picture set according to the feature model, and using the first feature The value is compared with the second eigenvalue to obtain a similarity.

The memory is configured to store a program;

The processor executing the program to: obtain a creative, extract a picture from the creative, obtain an advertisement picture set; obtain a service video, according to one of an image, a sound, and a subtitle of the service video Or extracting a picture from the service video to obtain a service picture set; comparing the advertisement picture in the advertisement picture set with the service picture in the service picture set to obtain a similarity; when the advertisement picture set When the similarity between the first advertisement picture and the first service picture in the service picture set meets the first preset condition, the first creative material to which the first advertisement picture belongs and the first service picture belong to The first service video is associated and generates association information, where the association information includes: identifier information of the first service video, and identifier information of the first advertisement material.

Taking the advertisement video or the service video in the creative as an input video, after decoding the input video, using the decoded input video as a to-be-extracted video; or, decoding the input video according to the scene Divide, get the split video, and enter each split video Performing a row comparison, obtaining a segmented video set according to the comparison result, and using the segmented video in the segmented video set as a to-be-extracted video, wherein a comparison result of any two divided videos in the segmented video set satisfies a second preset condition;

Converting the audio signal in the to-be-extracted video from the time domain to the frequency domain, and acquiring the audio signal The energy curve of the number obtains the position of the maximum value and the minimum value of the energy curve, and extracts the video frame corresponding to the position of the maximum value and the minimum value as the extracted picture.

Optionally, the similarity between the first advertisement image in the advertisement image set and the first service image in the service image collection meets the first preset condition, where:

Optionally, the processor executes the program, and is further configured to:

The association information further includes that the first creative material plays on the first service video The interpolated location information is performed, and the interpolated location information is determined according to an appearance time and a disappearing time of the first service video target object associated with the first service picture in the first service video.

Optionally, the processor executes the program, and is further configured to:

The embodiment of the invention further provides an accurate advertising system, including a video analysis platform and a number Database, portal, data synchronization interface module and associated information synchronization interface module, wherein

The video analysis platform is configured to: synchronize the creative synchronization information from the advertisement platform by using the data synchronization interface module, obtain the creative material according to the creative synchronization information, extract a picture from the creative material, obtain a set of advertisement images; The data synchronization interface module synchronizes the service video synchronization information from the service platform, acquires the service video according to the service video synchronization information, and extracts the service video according to one or more of the image, sound and subtitle of the service video. a picture, obtaining a business picture set; comparing the advertisement picture in the advertisement picture set with the service picture in the business picture set to obtain a similarity; when the first advertisement picture and the service in the advertisement picture set Correlating the first creative material to which the first advertisement image belongs and the first service video to which the first service image belongs, and generating the similarity of the first service image in the image collection Correlation information, where the association information includes: an identification letter of the first service video Identification information of the first advertisement creatives, the related information by the related information is synchronized to the synchronous interface module of the service platform, and advertising platform;

The database is configured to store related information of the creative and the business video;

The portal is configured to implement interaction between the user and the database and the video analysis platform;

The data synchronization interface module synchronizes creative synchronization information from the advertisement platform to the video analysis platform; and is configured to synchronize service video synchronization information from the service platform to the video analysis platform;

The associated information synchronization interface module is configured to synchronize the associated information from the video analytics platform to the advertising platform and the service platform.

Optionally, the video analysis platform is further configured to perform background modeling and target detection on the advertisement video in the creative, extract a target of the motion to obtain a target object of the creative, and target the target object of the creative. The image in the first image collection that includes the target object of the creative is associated, and the advertisement image in the creative is used as a target object of the creative and associated with the advertisement image to which the creative belongs, and the target object of the creative is stored in the database. The creative target object table in , setting its status to unreviewed; and, performing the business video Background modeling and target detection, extracting a target of the motion to obtain a service video target object, determining an appearance time and a disappearance time of the service video target object in a service video to which the service video object belongs, and the service video target object and the service image The set includes the picture of the service video target object to be associated, and the service video target object is stored in the service video target object table in the database, and the status is set to unaudited;

The portal is configured to implement the interaction between the user and the database and the video analytics platform, including: querying the database for an unreviewed creative target object according to a user operation, and configuring an advertisement for the unreviewed creative target object according to a user operation. The object identifies and modifies its status in the creative target object table as reviewed and not associated;

The video analytics platform compares the advertisement image in the advertisement image set with the service image in the service image collection, and the obtaining similarity includes:

The video analytics platform queries the database for the reviewed unassociated creative target object, receives the reviewed unassociated creative target object returned by the database, and obtains the associated unrecognized creative target object associated with the database. Advertising the image, comparing it with the business image in the business image collection, and obtaining similarity;

The video analysis platform is further configured to: when the similarity between the second service image in the service image set and the advertisement image associated with the reviewed unrelated creative target object satisfies the first preset condition Obtaining an advertisement object identifier configured by the reviewed unassociated creative target object, and configuring, for the second service video target object associated with the second service image, the creative target object that is not associated with the audited The same advertisement object identifier, the status of the reviewed unassociated creative target object in the database is modified, and the status of the second business video target object in the modified database is associated.

Optionally, the portal is further configured to: query the database for the audited unassociated creative target object, obtain the reviewed unassociated creative target object, and initiate a query request to the video analytics platform, Receiving a service picture subset returned by the video analysis platform, and selecting a third service picture from the service picture subset according to a user operation, and configuring a third service video target object associated with the selected third service picture The reviewed is not associated The creative target object has the same advertisement object identifier, and the status of the third business video target object in the database is modified to be associated, and the status of the reviewed unassociated creative target object is modified to be associated;

The video analysis platform is further configured to: after receiving the query request sent by the portal, determine a third advertisement image associated with the creative target object carried in the query request, and find and describe in the service image set The service picture subset of the third advertisement picture satisfies the fourth preset condition, and the service picture subset is sent to the portal.

According to still another embodiment of the present invention, there is also provided a storage medium comprising a stored program, wherein the program is executed to perform the method of any of the above.

According to still another embodiment of the present invention, there is also provided a processor for running a program, wherein the program is executed to perform the method of any of the above.

Compared with the prior art, in the embodiment of the present invention, the service platform and the advertisement platform obtain the service by cutting the business video and the advertisement video, extracting the key frame, extracting the feature value of the image, and associating the business video and the creative material with high similarity. The video and its associated creatives, the business platform plays the creative video (advertising video and / or advertising image) while playing the business video, in order to achieve the effect of accurately advertising the business video.

Other features and advantages of the invention will be set forth in the description which follows, The objectives and other advantages of the invention may be realized and obtained by means of the structure particularly pointed in the appended claims.

DRAWINGS

The drawings are used to provide a further understanding of the technical solutions of the present invention, and constitute a part of the specification, which together with the embodiments of the present application are used to explain the technical solutions of the present invention, and do not constitute a limitation of the technical solutions of the present invention.

1 is a block diagram of an accurate advertising system according to an embodiment of the present invention;

2 is a schematic diagram of a data structure of a video analysis platform according to an embodiment of the present invention;

3 is a flowchart of a method for accurately placing advertisements according to an embodiment of the present invention;

4 is a block diagram of a precise advertisement system implemented by a stand-alone mode according to an embodiment of the present invention;

FIG. 5 is a flow chart of the service of the accurate advertisement advertisement system realized by the single machine mode according to the embodiment of the present invention; FIG.

6 is a block diagram of a precise delivery advertising system implemented in a distributed manner according to an embodiment of the present invention;

FIG. 7 is a flowchart of a service of a precision delivery advertisement system implemented in a distributed manner according to an embodiment of the present invention;

8 is a structural diagram of an IPTV accurate advertisement advertising system according to an embodiment of the present invention;

FIG. 9 is a flowchart of an IPTV accurate advertisement advertising service according to an embodiment of the present invention;

10 is a structural diagram of a video playback software accurate advertising system according to an embodiment of the present invention;

FIG. 11 is a flowchart of accurately placing an advertisement service of a video playing software according to an embodiment of the present invention.

detailed description

The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that, in the case of no conflict, the features in the embodiments and the embodiments in the present application may be arbitrarily combined with each other.

The steps illustrated in the flowchart of the figures may be executed in a computer system such as a set of computer executable instructions. Also, although logical sequences are shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than the ones described herein.

In the embodiment of the present invention, the video and the image (or video and/or image) are processed by the video analysis and the image processing technology to obtain the similarity information, and the business video and the creative are associated according to the similarity to realize automatic association.

An embodiment of the present invention provides a matching method, including:

Obtaining a to-be-matched resource, and extracting a picture from the to-be-matched resource to obtain a first picture set;

Obtaining a second video resource, and extracting a picture from the second video resource to form a second picture set;

Comparing the picture in the first picture set with the picture in the second picture set to obtain a similarity;

The to-be-matched resource may include multiple to-be-matched video resources and/or to-be-matched pictures, and the second video resource may include multiple video resources. Extracting the first picture set from the to-be-matched resource includes: if the to-be-matched resource includes a to-be-matched video resource, extracting a video frame from the to-be-matched video resource to join the first picture set, if the to-be-matched resource Including the picture to be matched, the picture to be matched is directly added to the first picture set.

In an embodiment of the present invention, the to-be-matched resource may only include the to-be-matched picture, and when the picture is to be extracted from the to-be-matched resource, the first picture set is obtained by directly waiting for the matching picture.

In an embodiment of the present invention, extracting a picture from the to-be-matched resource or extracting a picture from the second video resource includes:

After the input video is decoded, the to-be-matched resource or the second video resource is used as an input video, and the image is directly extracted from the decoded input video; or the decoded input video is segmented according to the scene. Obtaining a segmented video, comparing the segmented videos of the scenes to obtain a segmented video set, and extracting a picture from the segmented video in the segmented video set, wherein a comparison result of any two divided videos in the segmented video set satisfies a second Preset conditions.

That is, in the embodiment of the present invention, the video frame may be directly extracted from the to-be-matched video resource in the to-be-matched resource, or the video resource to be matched may be segmented according to the scene to obtain the segmented video, and the segmented videos of each scene are compared. Obtaining a segmented video set according to the comparison result, and extracting a video frame from the segmented video in the segmented video set. The video frame may be directly extracted from the second video resource, or the second video resource may be segmented according to the scene to obtain a segmented video, and then the divided videos are compared, and the segmented video set is obtained according to the comparison result, and the segmentation is performed from the segmented video set. The video is extracted from the video frame.

In an embodiment of the present invention, when the to-be-matched resource includes a to-be-matched picture, extracting the picture from the to-be-matched resource includes: extracting the to-be-matched picture to join the first picture set Hehe.

Here's how to split the video according to the scene. A video usually contains multiple scenes, and the background or sound can be used to determine whether the scene changes. For example, when switching from indoor to outdoor, the scene is switched. For example, when the background sound is converted from street noise to factory noise, scene switching occurs, and from car noise to venue noise, scene switching occurs. In an embodiment of the invention, the dividing the decoded input video according to the scene includes:

Determining whether the scene is switched according to one or more of the background and the sound in the input video, and dividing the position where the scene switching occurs in the input video.

That is, it is possible to judge whether the scene is changed only by the background, or to determine whether the scene is changed by sound, or to determine whether the scene is changed by the background and the sound.

In an embodiment of the present invention, determining whether the scene is switched according to the background includes: performing background estimation on the input video, and performing background update processing to compare whether the adjacent background pictures determine whether the scene is the same scene. When extracting the background estimation, the background estimation may be performed according to several consecutive frames of the input video. For example, three consecutive video frames may be used for background estimation, and when the background image is compared, the global feature of the background image may be extracted to obtain adjacent The Hamming distance of the global features of the two background images is used to determine whether the adjacent background image is the same scene. The background estimation, the background image update, and the like may be implemented by using a mixed Gaussian model (GMM) algorithm. This method is only an example. You can use other methods to extract the background and determine whether the background changes. For example, when comparing background images, you can use other comparison algorithms to determine whether the background image is the same scene. This application does not limit this.

In an embodiment of the present invention, determining whether the scene is switched according to the sound in the input video comprises: converting an audio signal in the input video from a time domain to a frequency domain, and acquiring an energy curve of the audio signal. Obtaining a position of the maximum value and the minimum value of the energy curve as a position where the scene switching occurs, and satisfying, where the distance between two adjacent scenes acquired according to the energy curve is greater than the first distance threshold . Adjacent maxima or minima may have a small distance. When performing scene segmentation, only the maximum and minimum values whose distance is greater than the first distance threshold are extracted, and the scene segmentation is performed. The distance is less than the pole of the first distance threshold. Large and small values are ignored. Its In the process of converting the audio signal from the time domain to the frequency domain, the process of suppressing the signal of the frequency band in which the voice is located is taken. Certainly, the implementation manner of determining whether the scene is switched according to the sound is only an example. The audio may be processed in other manners to determine whether the scene is switched. For example, the background sound in the audio is extracted, and the background sound is used to determine whether the scene changes.

In an embodiment of the present invention, an implementation manner of comparing and dividing each divided video to obtain a divided video set is: extracting a background image from the divided video (the background image may be subjected to background estimation and background update according to the previous segmentation according to the scene) Obtaining in the process, comparing the background images of the divided videos to obtain a degree of similarity. If the similarity degree of the background images of the two divided videos satisfies the second preset condition, it is determined that the scenes of the two divided videos are not similar, if the two divided videos are If the degree of similarity of the background image does not satisfy the second preset condition, it is determined that the two divided video scenes are similar, and one of the two divided videos with similar scenes may be deleted, and all the divided videos are compared and only two similar segments are reserved. One of the videos, the retained split video also needs to be compared with each other, and finally the split video set is obtained, so that the split video in the split video set satisfies the second preset condition. The scheme can merge similar scenes, reduce the number of divided videos, thereby reducing the number of video frames finally extracted, improving system processing speed, and reducing storage requirements. Of course, in other embodiments of the present invention, the background pictures may be used instead of the divided pictures, and the divided videos may be compared in other ways to determine whether the divided videos are similar. In addition, after the video frames are subsequently extracted, the video frames extracted in the divided videos may be compared to determine whether the divided videos are similar. The second preset condition may be set as needed, and the setting of the second preset condition is related to the number of divided videos. If it is desired to divide the video less, the condition for judging whether the divided video is similar may be wider, so that the number of divided videos judged to be similar is increased, and the number of divided videos extracted into the divided video set is reduced. If it is desired to divide the video more, the condition for judging whether the divided video is similar may be stricter, so that the number of divided videos judged to be similar is reduced, and the number of divided videos extracted into the divided video set is increased. Specifically, the second preset condition can be set as needed.

Of course, in other embodiments of the present invention, the video frame may be directly extracted from the to-be-matched video resource and the second video resource for subsequent processing without performing segmentation. The merging process is performed after the video frames are extracted to reduce the number of extracted video frames.

When a video frame is extracted, a representative video frame may be extracted according to an image, a sound, or a subtitle, and the extracted video frame is referred to as a key frame. In an embodiment of the invention, the extracting a picture directly from the decoded input video or extracting a picture from the split video in the split video set includes:

Taking the decoded input video or the divided video in the divided video set as a to-be-extracted video, and extracting a video from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video. The frame is taken as the extracted picture.

In an embodiment of the present invention, extracting a video frame from the to-be-extracted video as the extracted image according to one or more of an image, a sound, and a subtitle of the to-be-extracted video includes:

Extracting a video frame in the to-be-processed video as a key frame, which is referred to as a current key frame, and extracting a next key frame by sequentially: sequentially selecting a video frame after the current key frame in the to-be-processed video and the current key Comparing the frame to obtain a second similarity, if the second similarity between the second video frame and the current key frame satisfies a third preset condition and is closest to the current key frame, extracting the second video frame And using the second video frame as the current key frame, extracting the next key frame in the same manner; if there is no video frame that meets the third preset condition with the second similarity between the current key frame, The process of extracting a picture of the video to be processed ends, and the extracted key frame is taken as the extracted picture; wherein, initially, the starting frame of the to-be-processed video is taken as the current key frame.

In an embodiment of the present invention, the extracting a video frame from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video as the extracted image includes:

Searching for a subtitle of the video to be extracted, and if a preset keyword is searched, extracting a video frame corresponding to the subtitle; wherein the subtitle of the to-be-extracted video is from the to-be-extracted video Obtaining or performing voice recognition acquisition on the audio of the to-be-extracted video. Among them, the keywords are preset. For example, when you need to advertise basketball, you can set basketball, three-pointers, dunks, etc. as keywords. This method can be used as an aid.

In an embodiment of the present invention, the similarity of the first picture in the first picture set and the second picture in the second picture set satisfying the first preset condition is:

The similarity between the first picture and the second picture is greater than a first similarity threshold. Of course, the first preset condition can also be set to other conditions as needed.

In an embodiment of the present invention, comparing the picture in the first picture set with the picture in the second picture set, and obtaining the similarity includes:

Extracting a first feature value of the picture in the first picture set according to the feature model obtained by the deep learning algorithm, and extracting a second feature value of the picture in the second picture set according to the feature model, and using the first feature The value is compared with the second eigenvalue to obtain a similarity. Comparing images to get similarity can use a lot of algorithms, here is just an example, you can use other methods to get similarity as needed.

In an embodiment of the present invention, a background of the to-be-matched video in the to-be-matched resource may be modeled, a target of the motion is detected, a location of the moving target is stored, and a picture including the moving target is associated; and, The second video resource performs background modeling, detects the target of the motion, stores the location of the moving target, and the appearance time and disappearance time of the target in the second video resource, and associates the image containing the moving target, and extracts the image according to the feature model. When the feature value is used, feature extraction is performed using the corresponding feature extractor in the target region and the background region of the image respectively. For example, the target of motion usually includes a human face, so a face detector can be used in the target area for feature extraction. If the moving target detection is not performed, all feature extractors are used for feature extraction in all areas of the picture. If moving target detection has been performed, feature extraction can be performed in the stationary background area using the feature extractor commonly used for the background area. The target area of the motion is extracted using a feature extractor that is usually used for moving targets, thereby reducing the amount of computation in feature extraction.

One application of the above matching method is an application in an advertisement insertion. Can find the ad video Or the advertisement image is similar to the content of the location in the business video, and the association is performed to insert the advertisement video or the advertisement image in the corresponding position in the business video. When the advertisement video needs to be inserted, the advertisement is inserted into the service video related to the advertisement video, and the user is more acceptable than the random insertion in a certain service video, so that the advertisement effect is better. For example, the product advertisement of a celebrity endorsement is inserted in the TV or movie starring the star. For example, if the advertisement of a product is used in the main character of the TV or the movie, the user's acceptance and advertising effect are good. To arbitrarily insert ads into a TV movie. In addition, the matching method can also be used in image search. For example, if a user wants to find a movie clip but does not know the specific movie name and only knows the approximate content, he can find a picture similar to the movie clip and match it in the movie library to find the corresponding movie. For example, if the user wants to find all movies with Christmas scenes, they can also use a Christmas video or Christmas picture to match in the movie library to find all movies with Christmas scenes. The above application is only an example, and the present application can be applied to various scenarios in which a video needs to be matched.

In an embodiment of the invention, the video analysis platform further includes a video decoding layer, configured to Decoding the to-be-matched video resource in the to-be-matched resource, and transmitting the video resource to the video image analysis processing layer as an input video, and decoding the second video resource as an input video to the video image analysis processing layer ;

The input video is used as the to-be-extracted video; or the input video is segmented according to the scene, the segmented video is obtained, the segmented videos of the scenes are compared, and the segmented video set is obtained according to the comparison result, and the segmented video set is obtained. The divided video is used as the to-be-extracted video, wherein the comparison result of any two divided videos in the divided video set satisfies the second preset condition; the second preset condition may be set as needed.

In an embodiment of the present invention, the video processing module extracting a picture from the to-be-matched resource includes: when the to-be-matched resource includes a to-be-matched picture, extracting the to-be-matched picture to join the first picture set.

In an embodiment of the present invention, the video processing module includes a video segmentation unit configured to determine whether a scene is switched according to one or more of a background and a sound in the input video, where the input video occurs Splitting the position of the scene switching; and comparing the divided videos to obtain the divided video set.

In an embodiment of the present invention, determining, by the video segmentation unit, whether the scenario is switched according to the sound in the input video comprises: converting the audio signal in the input video from a time domain to a frequency domain, and acquiring the audio signal. The energy curve obtains the position of the maximum value and the minimum value of the energy curve as the position where the scene switching occurs, and satisfies the position of the adjacent two scenes that are acquired according to the energy curve, and the distance is greater than the distance A distance threshold.

In an embodiment of the present invention, the video processing module includes a multi-modal extraction key frame unit, and is configured to use the split video in the input video or the split video set as a to-be-extracted video, according to the to-be-extracted One or more of images, sounds, and subtitles of the video from The video frame is extracted from the to-be-extracted video as the extracted picture.

In an embodiment of the present invention, the multi-modal extraction key frame unit extracts a video frame according to the image of the to-be-extracted video, including:

Extracting a video frame in the to-be-processed video as a current key frame, and extracting a next video frame by sequentially performing video frames subsequent to the current key frame in the to-be-processed video and the current key frame Comparing the second similarity, if the second similarity between the second video frame and the current key frame satisfies the third preset condition and is closest to the current key frame, extracting the second video frame, The second video frame is used as the current key frame, and the next video frame is extracted in the same manner; if there is no video frame that meets the third preset condition with the second similarity between the current key frame, then the to-be-processed The process of extracting pictures by video ends;

In an embodiment of the present invention, the multi-modal extraction key frame unit extracts a video frame according to the sound of the to-be-extracted video, including:

In an embodiment of the present invention, the multi-modal extraction key frame unit extracts a video frame according to the subtitle of the to-be-extracted video, including:

And searching for a subtitle of the video to be extracted, and if the preset keyword is searched, extracting a video frame corresponding to the subtitle; wherein the subtitle of the to-be-extracted video is from a subtitle file that is included in the to-be-extracted video. Obtaining or performing voice recognition acquisition on the audio of the to-be-extracted video.

In an embodiment of the invention, the picture analysis module includes a deep learning feature model unit, a key frame feature extraction unit, and a feature matching association unit, where:

Applying the above matching method to the advertisement insertion, the embodiment of the invention provides a method for accurately advertising, including:

In an embodiment of the invention, extracting a picture from the creative or extracting a picture from the business video includes:

Taking the advertisement video or the service video in the creative as an input video, after decoding the input video, using the decoded input video as a to-be-extracted video; or, decoding the input video according to the scene Performing segmentation, acquiring a segmented video, comparing each segmented video, obtaining a segmented video set according to the comparison result, and dividing the segmentation in the segmented video set a video as a to-be-extracted video, wherein a comparison result of any two divided videos in the divided video set satisfies a second preset condition;

In an embodiment of the present invention, extracting a picture from the creative material includes: when the advertisement material includes an advertisement picture, extracting the advertisement picture to join the advertisement picture set.

In an embodiment of the invention, the dividing the decoded input video according to the scene includes:

Wherein, according to the background, whether the scene is switched or not depends on the related description in the foregoing matching method for determining whether the scene is switched according to the background.

In an embodiment of the present invention, determining whether the scene is switched according to one or more of a background and a sound in the input video comprises: converting an audio signal in the input video from a time domain to a frequency domain Obtaining an energy curve of the audio signal, acquiring a position of a maximum value and a minimum value of the energy curve, as a position where a scene switching occurs, and satisfying: switching between two adjacent scenes acquired according to the energy curve The location is greater than the first distance threshold.

In an embodiment of the present invention, the similarity between the first advertisement picture in the advertisement picture set and the first service picture in the service picture set satisfies the first preset condition, that is:

In an embodiment of the present invention, comparing the advertisement picture in the advertisement picture set with the service picture in the service picture set, and obtaining the similarity includes:

In an embodiment of the present invention, the association information further includes insertion location information that is inserted by the first creative material when the first service video is played, and the insertion location information is according to the first service. The location of the picture in the first service video is determined.

In an embodiment of the invention, the method further includes:

Perform background modeling and target detection on the business video, and extract the target of the motion to obtain the service a video target object, determining an appearance time and a disappearing time of the service video target object in a service video to which the service video object belongs, and associating the service video target object with a picture of the service picture set that includes the service video target object;

The method for performing target detection may be: scanning a video frame using a sliding window to determine a location of a foreground target of the motion.

In an embodiment of the invention, the method further includes:

In an embodiment of the invention, the association information further includes an advertisement object identifier in which the first creative target object is configured.

In an embodiment of the invention, the method further includes:

After receiving the query request sent by the portal, determining a second advertisement image associated with the second creative target object carried in the query request, and searching for the similarity with the second advertisement image in the service image collection to satisfy the first Four preset business picture sub-sets, the business graph Sending a set of slices to the portal, so that the portal selects a second service picture from the subset of service pictures according to a user operation, and configures a service second service video target object associated with the selected second service picture. The second creative target object has the same advertisement object identifier, thereby indicating that the second creative object to which the second creative target object belongs is associated with the second service video to which the second service video target object belongs.

The embodiment of the present invention further provides a video analysis platform, as shown in FIG. 1 , including a data acquisition module and a video image analysis processing layer, where the video image analysis processing layer includes a video processing module and a picture analysis module, where:

The image analysis module is configured to compare the advertisement image in the advertisement image collection with the business image in the business image collection to obtain a similarity; when the first advertisement image in the advertisement image collection and the Correlating the first creative material to which the first advertisement image belongs and the first service video to which the first service image belongs when the similarity of the first service image in the service image set meets the first preset condition, and The association information is generated, and the association information includes: identifier information of the first service video, and identifier information of the first advertisement material.

The video analysis platform further includes a video decoding layer, configured to decode the advertisement video in the creative as an input video to the video image analysis processing layer, and decode the service video as an input video. Transmitting to the video picture analysis processing layer;

The input video is used as the to-be-extracted video; or the input video is segmented according to the scene, the segmented video is obtained, the segmented videos of the scenes are compared, and the segmented video set is obtained according to the comparison result, and the segmented video set is obtained. The divided video in the video is to be extracted, wherein the comparison result of any two divided videos in the divided video set satisfies the second preset condition;

In an embodiment of the present invention, the video processing module extracting a picture from the creative material includes: when the advertisement material includes an advertisement picture, extracting the advertisement picture to join the advertisement picture set.

In an embodiment of the invention, the video processing module includes a video segmentation unit and a multi-modal extraction key frame unit, wherein:

The dividing unit is configured to determine whether the scene is switched according to one or more of the background and the sound in the input video, and perform segmentation at a position where the scene switching occurs in the input video.

The multi-modal extraction key frame unit is configured to extract a video frame from the to-be-extracted video as the extracted picture according to one or more of an image, a sound, and a subtitle of the to-be-extracted video.

In an embodiment of the invention, the picture analysis module further includes feature index generation / merging unit, the feature index generating/merging unit is configured to perform index generation and index merging, and the index generating comprises: constructing a tree index according to a preset rule for the extracted feature set, and accelerating the target retrieval process; the index merging includes: The index files built in parallel for different feature sets are merged for index incremental update. The index file includes: an original feature file extracted by the feature, and a tree structure description file generated according to the original feature file. The index generates a tree structure description file according to the original feature file extracted by the feature. The merge is based on the original feature file, updating the index is the tree structure description file, which is equivalent to regenerating a new index. It can be updated regularly, the index is updated according to the original feature file set, and the feature index can be updated in real time.

In an embodiment of the invention, the video processing module further includes: a background modeling unit and a target detecting unit, wherein:

The background modeling unit is configured to perform background modeling on the business video; and perform background modeling on the advertisement video in the creative;

The target detecting unit is configured to perform target detection on the service video, extract a target of the motion to obtain a service video target object, and determine an appearance time and a disappearance time of the service video target object in a service video to which the service video object belongs. The business video target object is associated with the image of the business image collection object that includes the business video target object; and, the target video of the advertisement video in the creative is detected, and the target of the motion is extracted to obtain the creative target object, and The creative target object is associated with the image of the first image set that includes the creative target object, and the advertisement image in the creative is used as a target object of the creative and associated with the advertisement image to which it belongs;

The image analysis module is further configured to: when the similarity between the first advertisement image in the advertisement image set and the first service image in the service image collection meets the first preset condition, acquire the An advertisement object identifier configured by the first creative target object associated with an advertisement image, and an advertisement object identifier identical to the first creative target object configured for the first service video target object associated with the first service image Instructing the first creative material to which the first advertisement picture belongs to be associated with the first service video to which the first service picture belongs. The advertisement object identifier indicates an advertisement theme for which the first creative target object is directed.

In an embodiment of the invention, the picture analysis module is further configured to:

After receiving the query request sent by the portal, determining a second advertisement image associated with the second creative target object carried in the query request, and searching for the similarity with the second advertisement image in the service image collection to satisfy the first And a fourth set of service picture sub-sets, the service picture sub-set is sent to the portal, so that the portal selects a second service picture from the service picture subset according to a user operation, and the selected The second service video target object associated with the second business image is configured with the same advertisement object identifier as the second creative target object, thereby indicating the second creative material to which the second creative target object belongs and the second service video. The second business video to which the target object belongs is associated. The fourth preset condition may be that the similarity is between the second similarity threshold and the first similarity threshold, and the second similarity threshold is smaller than the first similarity threshold.

In an embodiment of the invention, the video analytics platform further includes a storage layer, and the storage layer is configured to store related information of the service video and the advertisement video. It is also possible to store part of the information in the database.

Optionally, the storage layer includes: a structured storage module and an unstructured storage module, where the unstructured storage module is configured to store the material library related entity file, and the structured storage module is configured to save and manage the corresponding table structure information. .

The specific stored information refers to FIG. 2, including: 1. a business video directory; 2. a creative directory (video and/or picture); 3. a third-party image; 4. an advertising object; 5. a creative target object (POI) 6 , business video POI. The above information can also be stored in a database independent of the video analytics platform.

For details of the processing of each module in the above video analysis platform, refer to the related description in the method for accurately placing advertisements, and details are not described herein again.

The embodiment of the invention further provides a method for accurately placing advertisements. Referring to FIG. 3, the method includes:

Step 301, the video analytics platform uses the service video/advertisement video data synchronization interface from the advertisement The platform obtains advertisement video synchronization information, where the advertisement video synchronization information carries information such as an operation type, an advertisement video number, and an address of the advertisement video, and may also carry an advertisement video name information and type information (indicating whether the advertisement video or the service video is currently present) ), parsing the advertisement video synchronization information, including:

Get the operation type. If it is a new operation, add an advertisement video entry in the local advertisement video directory information, and download the advertisement video according to the address provided by the relevant field in the advertisement video synchronization information; if it is a deletion operation, then Deleting the corresponding advertisement video entry in the local advertisement video directory information, and deleting the record related to the advertisement video in the creative POI table and the key frame picture table;

Each advertisement video entry in the locally stored advertisement video directory information corresponds to one advertisement video, and each advertisement video entry includes the following information: an advertisement video number of the advertisement video corresponding to the advertisement video entry, an advertisement video name, and may also include The advertisement video download address, the creation time of the advertisement video entry, and the like. Other information can also be stored as needed.

One implementation of the business video/advertising video data synchronization interface is as follows:

Synchronization direction: business platform, advertising platform -> video analysis platform;

Synchronization format: can be set as needed, such as file mode;

File type: can be set as needed, such as dat type;

Data file description: For business video, advertising video entity data information, incremental data can be generated periodically (for example, generated every day). The first full amount. Only sync successful content.

Synchronization mode: Incremental synchronization (first full amount) every preset time (preset time), uploaded by the service platform and advertising platform to the designated server (can be an FTP server), and supports manual triggering immediate synchronization. For example, the preset time may be one day, which is not limited in this application.

File format: The fields can be separated by preset separators, or other conventions; the default separator is for example "|#".

Step 302: The video analytics platform obtains advertisement image synchronization information from the advertisement platform by using an advertisement image data synchronization interface, where the advertisement image synchronization information carries an operation type and an advertisement. The image number and the advertisement image download address information are used to parse the advertisement image synchronization information, including:

Get the operation type. If it is a new operation, add an advertisement image entry in the local advertisement image directory information, and download the advertisement image according to the address provided by the relevant field in the advertisement image synchronization information; if it is a deletion operation, then The corresponding advertisement image entry is deleted from the local advertisement image directory information, and the record related to the advertisement image in the creative POI table is deleted;

Each of the advertisement image entries stored in the local advertisement image directory information corresponds to one advertisement image, and each advertisement image entry includes the following information: an advertisement image number of the advertisement image corresponding to the advertisement image entry, and an advertisement image name. It can also include other information, such as the download address of the advertisement image.

Among them, an implementation of the advertisement picture data synchronization interface is as follows:

Synchronization direction: advertising platform -> video analysis platform;

Synchronization format: can be set as needed, such as file mode;

File type: can be set as needed, such as dat type;

Data file description: For the advertisement picture entity data information, incremental data is generated periodically every day. Full manual generation. Only sync successful content.

Synchronization mode: Incremental synchronization (first full amount) every preset time (preset time), uploaded by the advertising platform to the designated server (can be an FTP server), supports manual triggering immediate synchronization. The video analysis platform goes to the designated server to download the advertisement picture synchronization information.

File format: The fields are separated by default, or other conventions; the default separator is “|#”.

Step 303: The video analysis platform analyzes the advertisement video, including: decoding the advertisement video, outputting by frame, segmenting the advertisement video, extracting key frames, performing target detection, recording target location information, extracting feature values of key frames, and generating features. index;

Step 304: The video analysis platform writes the advertisement video related information into the key frame picture table and the creative POI table, and the default state of the data in the creative POI table is “unaudited”;

Optional, including:

The video analysis platform writes related information of the extracted key frames into the key frame picture table, including: establishing a key frame picture identifier (imageid) for each key frame, and recording the key frame source (here, from the advertisement video); Record the save path of the key frame, and the access address of the key frame; it may also include the advertisement video number to which the key frame belongs.

The video analytics platform writes information about the detected target to the Creative Target Object (POI) table, including: targeting the target as a new creative target object or a new creative POI, which is created for the target in the creative POI table. An entry that records the following information: a unique identifier for the target, a target object identifier, a keyframe image identifier for the keyframe associated with the target, a status of the target (here unreviewed), and an ad video to which the target belongs Ad video number, ad video name, the target object creation time, modification time; location information of the target in the associated key frame picture.

Step 305: The video analysis platform analyzes the advertisement image, including: performing target detection, recording target location information; extracting feature values of the advertisement image, and generating a feature index;

Step 306, the video analysis platform writes related information of the target of the detected advertisement image into the creative POI table;

Writing information about the target to the creative POI table specifically includes:

For each target detected, an entry is created for the target in the creative POI table, and the following information is recorded: a unique identifier is created for the target, the target object identifier, the image identifier of the advertisement image in which the target is located, and the target identifier Status (here default value: "Unreviewed"), the ad image number of the ad image to which the goal belongs, the ad image name, the target creation time, the modification time, and the location information of the target image in the ad. The advertisement picture number and the advertisement picture name may be consistent or inconsistent.

The video analysis platform writes related information of the advertisement image into the key frame picture table, including: creating a key frame image identifier (imageid) for each advertisement picture, and recording the key frame source (here, from the advertisement picture); recording the advertisement The path to save the image, as well as the access address of the ad image.

Step 307, after receiving the query request of the user, the portal initiates a query request to the database, queries the unreviewed creative POI in the creative POI table, and receives the unreviewed creative POI returned by the database;

Step 308: The portal configures advertisement object information for the unreviewed creative POI according to the operation of the user, and adds an advertisement object data to the advertisement object table.

One of the purposes of the object identifier is to identify the advertising theme for which the creative is targeted, for example, to set an objectid for the creative POI that advertises for a brand of car, and another for the creative POI that advertises for a certain food. An objectid, another function is used as an association tag to compare the business video picture with the advertisement picture, and when the association condition is met, the business object POI associated with the service video picture is configured with the same objectid as the creative POI associated with the advertisement picture. Thereby achieving the purpose of the two associations. When the associated information is subsequently generated, the advertisement object identifier is carried in addition to the business video number of the associated business video and the creative number of the creative. A business video may be associated with multiple creatives. In the subsequent business video playback, the creative may be inserted through the objectid. For example, when an advertisement for a brand car is required, only the advertisement target object is identified as the objectid of the brand automobile. Creatives, without inserting other creatives associated with the business video. In other embodiments of the present invention, an identification information may also be added as an association mark, and the service video picture and the advertisement picture are compared by using the objectid as the association mark, and when the comparison result satisfies the association condition, it is related to the service video picture. The business video POI and the advertisement image related to the advertisement POI are configured with the same associated identification information, indicating that the two are associated, and the objectid only identifies the advertisement for which the advertisement is used, and is no longer used to mark the association between the business video and the creative.

Optionally, a new advertisement object data is created in the advertisement object table for the creative POI, and the advertisement object data includes the following information: creating a unique advertisement object identifier (objectid) corresponding to the creative POI, the advertisement object name, setting the advertisement The object type, the image storage path corresponding to the advertisement object, the creation time of the advertisement object, and the modification time of the advertisement object; and may also include information such as an advertisement target brand and an advertisement object manufacturer.

Step 309, the portal modifies the status of the creative POI in the creative POI table as “reviewed unassociated”, and writes the object identifier of the newly created advertisement object into the information of the creative POI, for example, the creative POI table. The objectid field value of the creative POI is refreshed to the objectid field value of the newly added advertisement object data, and the modified information of the creative POI is sent to the database;

Step 310: The video analytics platform obtains the service video synchronization information from the service platform by using the service video/advertisement video data synchronization interface, where the service video synchronization information carries information such as the operation type, the service video number, and the address of the service video, and may also carry the service. The information such as the video name is used to parse the service video synchronization information, including:

Obtain the operation type. If it is a new operation, add a service video entry to the local service video directory information, and download the service video according to the address provided by the related field in the service video synchronization information; if it is a deletion operation, Deleting the corresponding service video entry in the local service video directory information, and deleting the record related to the service video in the service video POI table and the key frame picture table;

Each service video entry stored in the local service video directory information corresponds to a service video, and the service video entry includes the following information: a service video number of the service video corresponding to the service video entry, and a service video name, which may also include Other information, such as the business video download address, the creation time of the business video entry, and the like.

Step 311: The video analysis platform analyzes the service video, including: decoding the service video, outputting the video according to the frame, dividing the service video, extracting the key frame, performing target detection, recording the target location information, extracting the feature value of the key frame image, and generating the feature. For a specific implementation process, refer to the description of the video analysis platform in the device embodiment, and details are not described herein again.

Step 312: The video analysis platform writes related information into the key frame picture table and the service video POI table, and the default state of the data in the service video POI table is “unrelated”;

Optional, including:

The video analysis platform writes related information of the extracted key frames of the service video into the key frame picture table, including: establishing a key frame picture identifier (imageid) for each key frame, and recording The source of the key frame, here is the service video; the storage path of the key frame, and the access address of the key frame; and the service video number to which the key frame belongs. Additional information can also be recorded as needed.

The video analysis platform writes the related information of the detected target into the service video target object (POI) table, including: using the target as a new service video POI, and establishing an entry for the target in the service video POI table, and recording the following information: Create a unique identifier for the target, the target object identifier, the key frame image identifier of the key frame associated with the target, the status of the target (here is unreviewed), the service video number of the service video to which the target belongs, and the service video. Name, the target object creation time, modification time; the location information of the target in the associated key frame picture.

For an implementation of the key frame picture table, see step 304;

Step 313, the video analysis platform queries the creative POI table of the database, and receives the "audited unassociated" creative POI returned by the database;

Step 314: The video analytics platform performs the following operations on each of the reviewed unrelated creative POIs that are received:

For each creative POI, query the local feature index to find the ad image associated with the creative POI (possibly the key frame extracted from the ad video, or the ad image). The similarity is greater than the first similarity threshold. All the business video key frames are automatically associated, and the advertisement object field of the business video POI corresponding to the found service video key frame is set as the advertisement object field value of the creative POI, and the service video POI status is modified to "Associated" and the status of the creative POI in the creative POI table is changed to "Associated";

Step 315: The video analysis platform interacts with the database, and modifies the value and status of the advertisement object field of the service video POI related to the service video POI table, and modifies the state of the creative POI related to the creative POI table;

Step 316: The portal initiates a query operation to the database according to the operation of the user, queries the “POD that has not been audited”, and receives the audited unrelated creative POI information returned by the database;

Step 317 to step 321 are performed on each of the received creative POIs;

Step 317: The portal selects a creative POI according to the operation of the user, and sends a request message for querying the associated image to the video analysis platform.

Step 318: The video analytics platform queries the local feature index to find all service video key frame images whose similarity of the advertisement image associated with the creative POI is located between the second similarity threshold and the first similarity threshold. The second similarity threshold is smaller than the first similarity threshold;

Step 319, the video analysis platform returns the key frame image of the found service video to the portal;

Step 320: The portal displays the returned business video key frame image to the user, and associates the business video key frame image with the advertisement image according to the user operation;

Step 321: The portal refreshes the objectid field of the corresponding service video POI in the service video POI table in the database as the object field value of the associated creative POI according to the operation of the user, and updates the status of the associated service video POI to “has been "Association", the status of the associated advertising video POI is updated to "Associated";

Step 322, the video analysis platform generates an incremental associated information synchronization file:

The associated information synchronization file initially generated by the video analytics platform is full, that is, the associated information synchronization file includes all associated business video and creative material association information, and then the service video whose relationship relationship changes is generated on the basis of the subsequent analysis. The associated information with the creative, such as disassociation, or modification of the association, etc., or the associated information of the newly associated business video and creative.

The video analytics platform generates an associated information synchronization file as follows:

The business video POI and the creative POI having the same objectid in the business video POI table and the creative POI table are searched, and the business video POI and the creative POI having the same objectid are respectively referred to as a first business video POI and a first creative POI, Obtaining the identification information of the first service video to which the first service video POI belongs, obtaining the identification information of the first creative material to which the first creative material POI belongs, and acquiring the appearance of the first service video POI in the first service video Time as the first creative in the first commercial video in the insertion position, get the first The objectid of the creative POI generates the associated information synchronization file. The insertion location may be a disappearance time of the first service video POI in the first service video, or a position between the appearance time and the disappearance time of the first service video POI in the first service video, or Other locations, such as a location after the disappearance time of the first service video POI in the first service video, or a location before the appearance time of the first service video POI in the first service video, and the like.

In another embodiment of the present invention, the target location may not be detected, the creative POI and the service video POI may not be extracted, and the advertisement video key frame, the advertisement picture, the service video key frame, and the advertisement video key frame and/or may be directly used. The advertisement image configures an advertisement object identifier, and searches for a business video key frame whose similarity with the advertisement video key frame and/or the advertisement image is greater than the first similarity threshold, and configures the advertisement video key frame and/or advertisement for the found service video key frame. The same advertisement object identifier as the image, that is, the business video to which the business video key frame having the same advertisement object identifier belongs is associated with the advertisement video key frame and/or the creative material to which the advertisement image belongs. The placement of the creative in the business video is determined by the location of the business video keyframe.

In another embodiment of the present invention, the target location may also be detected, but the creative POI and the business video POI are not extracted, and the advertising video key frame, the advertisement image, the business video key frame, the advertising video key frame and/or are directly used. The advertisement image configures an advertisement object identifier, and searches for a business video key frame whose similarity with the advertisement video key frame and/or the advertisement image is greater than the first similarity threshold, and configures the advertisement video key frame and/or advertisement for the found service video key frame. The same advertisement object identifier as the image, that is, the business video to which the business video key frame having the same advertisement object identifier belongs is associated with the advertisement video key frame and/or the creative material to which the advertisement image belongs. The placement location of the creative in the business video is determined by the location of the business video keyframe, or by the target occurrence time and target disappearance time associated with the business video keyframe.

The associated information synchronization file includes service video identification information (such as a business video number), creative identification information associated with the business video (such as an advertisement video number, an advertisement picture number, or an advertisement picture name), and the creative is in the business video. Insert location.

The associated information synchronization file can be generated periodically or when triggered.

Step 323: The video analytics platform sends the associated information synchronization file to the advertisement platform through the associated information synchronization interface.

Step 324: The video analytics platform sends the associated information synchronization file to the service platform by using the associated information synchronization interface.

An implementation manner of the associated information synchronization interface is provided below. It should be noted that the implementation manner is only an example, and other implementation manners may be used as needed. In an embodiment of the present invention, an implementation manner of the associated information synchronization interface is as follows:

Synchronization direction: video analysis platform -> business platform, advertising platform;

Synchronization format: can be set as needed, such as file mode;

File type: can be set as needed, such as txt type;

Data file description: For the associated information of business videos and creatives, incremental data can be generated on a daily basis. For the first time, it is full.

Synchronization frequency: Incremental synchronization (first full amount) every preset time (preset time), downloaded from the service platform, advertising platform to the designated server (the specified server can be an FTP server), and supports manual triggering immediate download. For example, the preset time may be one day, which is not limited in this application.

File format: The fields can be separated by preset separators, or other conventions; where the default separator is, for example, "|#";

The associated information synchronization file may include the following information: operation type (new association information, modification association information or deletion association information), service video number, associated creative material number, insertion position of the creative in the business video, and the advertisement object. The identifier can also include an ad object type and an ad object name.

The invention is further illustrated by the following specific examples.

Embodiment 1 Single-machine deployment accurate advertising system

Embodiment 1 provides an implementation scheme for a stand-alone deployment accurate advertising system. The program can Save on hardware costs. As shown in FIG. 4, the single-machine deployment precision advertising system includes a video analysis platform, a database connected to the video analysis platform, a data synchronization interface, an associated information synchronization interface, and a portal connected to the database.

FIG. 5 is a flow chart of a method for implementing a precise placement advertisement system based on the single machine shown in FIG. 4. As shown in Figure 5, it includes:

Step 501: The video analysis platform synchronizes the creative synchronization information through the data synchronization interface, and performs parsing;

For specific analysis methods, see steps 301-302.

Step 502: The video analysis platform analyzes the creative, decodes the video, outputs the video according to the frame, extracts the video key frame, performs target detection, records the target location information, extracts the feature value, and generates a feature index.

Referring to step 303, the analysis process in step 305;

Step 503: The video analysis platform writes related information into the video key frame table and the creative POI table, and the default state of the corresponding data in the creative POI table is “unaudited”;

Referring to step 304, step 306;

Step 504, after receiving the query request of the user, the portal initiates a query request to the database, queries the unreviewed creative POI in the creative POI table, and receives the unreviewed creative POI returned by the database;

Step 505: The portal configures advertisement object information for the unaudited creative POI according to the operation of the user, and adds an advertisement object data to the advertisement object table.

Step 506, the portal modifies the status of the creative POI in the creative POI table as “reviewed unassociated”, and writes the object identifier of the newly created advertisement object into the information of the creative POI, for example, the creative POI table. The objectid field value of the creative POI is refreshed to the objectid field value of the newly added advertisement object data, and the modified information of the creative POI is sent to the database;

Step 507: The video analysis platform acquires service video synchronization information through a data synchronization interface. Analyze. The parsing process refers to step 310;

Step 508: The video analysis platform analyzes the service video, including: decoding the video, outputting according to the frame, extracting the video key frame, performing target detection, recording the target location information, extracting the feature value of the key frame image, and generating an index;

Step 509, the video analysis platform writes the related information into the video key frame table and the service video POI table, and the default state of the data in the service video POI table is “unrelated”;

Step 510, the video analysis platform queries the creative POI table of the database, and receives the "audited unassociated" creative POI returned by the database;

Step 511: The video analytics platform performs the following operations on each of the reviewed unrelated creative POIs that are received:

Step 512: The video analysis platform interacts with the database, and modifies an advertisement object field value and a state of the service video POI related to the service video POI table, and modifies a state of the creative POI related to the creative POI table;

Step 513: The portal initiates a query operation to the database according to the operation of the user, queries the “PO that has not been audited”, and receives the audited unrelated creative POI information returned by the database;

Step 914 to step 918 are performed on each of the received creative POIs;

Step 514: The portal selects a creative POI according to the operation of the user, and sends a request message for querying the associated image to the video analysis platform.

Step 515: The video analytics platform queries the local feature index to find all service video key frame images whose similarity of the advertisement image associated with the creative POI is located between the second similarity threshold and the first similarity threshold. The second similarity threshold is smaller than the first similarity threshold;

Step 516: The video analysis platform returns the key frame image of the found service video to the portal.

Step 517: The portal displays the returned service video key frame image to the user, and associates the service video key frame image with the advertisement image according to the user operation;

Step 518: The portal refreshes the objectid field of the corresponding service video POI in the service video POI table in the database as the object field value of the associated creative POI according to the operation of the user, and updates the status of the associated service video POI to “has been "Association", the status of the associated advertising video POI is updated to "Associated";

Step 519, the video analysis platform generates an incremental associated information synchronization file, which is the full amount for the first time;

Step 520: The video analysis platform synchronizes the associated information synchronization file to the associated information synchronization interface.

Embodiment 2 Distributed deployment accurate advertising system

The second embodiment is an implementation scheme of a distributed advertisement system for distributed deployment. When there is a large number of large video analysis, in order to improve the efficiency of video analysis processing, a distributed deployment mode can be used, which is highly efficient in processing video. Can be used for real-time video processing analysis. Refer to Figure 6 for its networking structure. As shown in FIG. 6, the distributed deployment accurate advertising system includes a video analysis platform composed of multiple video analysis servers, a data synchronization interface, an associated information synchronization interface, a database, and a portal.

FIG. 7 is a flowchart of a method for accurately placing an advertisement based on the architecture shown in FIG. 6. As shown in FIG. 7, the method for accurately advertising according to the second embodiment includes:

Step 701: The video analysis platform synchronizes the creative synchronization information through the data synchronization interface, and performs parsing;

For specific analysis methods, see steps 301-302.

Step 702: The video analysis platform cuts the advertisement video into a plurality of sub-advertising videos, and each video analysis server analyzes and processes one sub-advertising video, specifically: decoding the sub-advertising video, outputting by frame, extracting video key frames, and performing target detection. Recording target location information, extracting feature values, and generating a feature index; referring to step 303, the processing in step 305;

Step 703: The video analysis platform writes related information into the video key frame table and the creative POI table, and the default state of the corresponding data in the creative POI table is “unaudited”;

Referring to step 304, step 306;

Step 704, after receiving the query request of the user, the portal initiates a query request to the database, queries the unreviewed creative POI in the creative POI table, and receives the unreviewed creative POI returned by the database;

Step 705: The portal configures advertisement object information for the unreviewed creative POI according to the operation of the user, and adds an advertisement object data to the advertisement object table.

Step 706: The portal modifies the status of the creative POI in the creative POI table as “reviewed unassociated”, and writes the object identifier of the newly created advertisement object into the information of the creative POI, for example, the creative POI table. The objectid field value of the creative POI is refreshed to the objectid field value of the newly added advertisement object data, and the modified information of the creative POI is sent to the database;

Step 707: The video analysis platform acquires the service video synchronization information through the data synchronization interface, and performs parsing. The parsing process refers to step 310;

Step 708: The video analysis platform analyzes the service video, including: cutting the service video into multiple sub-service videos, and each video analysis server processes the sub-service video as follows: decoding, outputting by frame, extracting video key frames, and performing Target detection, recording target location information, extracting feature values of key frame images, and generating an index;

Step 709, the video analysis platform writes the related information into the video key frame table and the service video POI table, and the default state of the data in the service video POI table is “unrelated”;

Step 710, the video analysis platform queries the creative POI table of the database, and receives the "audited unassociated" creative POI returned by the database;

Step 711: The video analytics platform performs the following operations on each of the reviewed unrelated creative POIs that are received:

Step 712: The video analysis platform interacts with the database, and modifies an advertisement object field value and a state of the service video POI related to the service video POI table, and modifies a state of the creative POI related to the creative POI table;

Step 713: The portal initiates a query operation to the database according to the operation of the user, queries the “PO that has not been audited”, and receives the audited unrelated creative POI information returned by the database;

Step 1114 to step 1118 are performed on each of the received creative POIs;

Step 714: The portal selects a creative POI according to the operation of the user, and sends a request message for querying the associated image to the video analysis platform.

Step 715: The video analytics platform queries the local feature index to find all service video key frame images whose similarity of the advertisement image associated with the creative POI is located between the second similarity threshold and the first similarity threshold. The second similarity threshold is smaller than the first similarity threshold;

Step 716: The video analysis platform returns the key frame image of the found service video to the portal.

Step 717: The portal displays the returned service video key frame picture to the user, and associates the service video key frame picture with the advertisement picture according to the user operation;

Step 718: The portal refreshes the objectid field of the corresponding service video POI in the service video POI table in the database as the object field value of the associated creative POI according to the operation of the user, and updates the status of the associated service video POI to “has been "Association", the status of the associated advertising video POI is updated to "Associated";

Step 719, the video analysis platform generates an incremental associated information synchronization file, for the first time is full amount;

Step 720: The video analytics platform synchronizes the associated information synchronization file to the associated information synchronization interface.

Embodiment 3 IPTV accurate advertising system

The third embodiment is an embodiment of a precise application system and an IPTV application. The system architecture diagram is shown in FIG. 8. As shown in FIG. 8, the IPTV precision advertisement advertisement system provided by the embodiment of the present invention includes a precision advertisement advertisement system, a plurality of set top boxes, an IPTV service platform, and an advertisement platform.

FIG. 9 is a flowchart of a method for accurately placing advertisements based on the architecture shown in FIG. 8. As shown in FIG. 9, the method for accurately advertising according to the third embodiment includes:

Step 901: The accurate delivery advertisement system obtains an advertisement video from the advertisement platform;

The obtaining method refers to step 301;

Step 902, the precision delivery advertisement system obtains an advertisement picture from the advertisement platform;

The obtaining method refers to step 302;

The execution order of the steps 901, 902 is not limited, and the step 901 may be performed first, and then the step 902 may be performed, or the step 902 may be performed first, or the step 901 may be performed, or both may be performed simultaneously;

Step 903, the precision delivery advertisement system analyzes the advertisement video;

The analysis process refers to steps 303-304;

Step 904, the precision delivery advertisement system analyzes the advertisement image;

The analysis process refers to steps 305-306;

The execution order of steps 903 and 904 is not limited, and step 903 may be performed first, and then executed. In step 904, step 904 may be performed first, then step 903 may be performed, or both may be performed simultaneously;

Step 905, the precision delivery advertisement system acquires a service video from the IPTV service platform;

The obtaining method refers to step 310;

Step 906: The precision delivery advertisement system analyzes the service video, and associates the service video with the advertisement video and the advertisement image;

The analysis method refers to steps 311-312, and the association process refers to steps 313-321;

Step 907, the precision delivery advertisement system periodically generates an incremental association information synchronization file;

Step 908: The precision advertisement advertisement system synchronizes the incremental association information synchronization file to the IPTV service platform by using an associated information synchronization interface;

Step 909, the precision delivery advertisement system synchronizes the incremental association information synchronization file to the advertisement platform by using an associated information synchronization interface;

There is no pre-order relationship between the steps 908 and 909. Step 909 may be performed first, and then step 908 may be performed, or both of them may be performed at the same time;

Step 910: The set top box receives a service video on demand request initiated by the user through the television terminal.

Step 911, the set top box sends a service video on demand request to the IPTV service platform;

Step 912: After receiving the service video on demand request, the IPTV service platform acquires, according to the association information, an advertisement material associated with the service video that is requested by the advertisement platform; the advertisement material includes an advertisement video and/or an advertisement. image;

Step 913, the IPTV service platform pushes the associated creative material to the set top box;

Step 914: The set top box inserts the associated creative material while playing the service video requested by the user.

Embodiment 4 Video playback software accurately delivers an advertising system

The fourth embodiment is an embodiment of the application of the accurate advertising system and the video playing software. The system architecture diagram of the fourth embodiment refers to FIG. As shown in FIG. 10, the video playback software accurate advertisement advertising system provided in this embodiment includes multiple video playback software clients, an on-demand video service platform, a precision advertisement advertisement system, and an advertisement platform.

FIG. 11 is a method for accurately placing advertisements based on the system architecture shown in FIG. As shown in FIG. 11, the accurate advertising method provided in the fourth embodiment includes:

Step 1101, the accurate advertising system obtains an advertisement video from the advertisement platform;

The obtaining method refers to step 301;

Step 1102, the precision delivery advertisement system obtains an advertisement picture from the advertisement platform;

The obtaining method refers to step 302;

Step 1103, the precision delivery advertisement system analyzes the advertisement video;

The analysis process refers to steps 303-304;

Step 1104, the precision delivery advertisement system analyzes the advertisement image;

The analysis process refers to steps 305-306;

Step 1105: The precision delivery advertisement system acquires a service video from the on-demand video service platform.

Step 1106: The precision delivery advertisement system analyzes the service video, and associates the service video with the advertisement video and the advertisement image;

Step 1107, the precision delivery advertisement system periodically generates an incremental associated information synchronization file;

Step 1108, the precision delivery advertisement system synchronizes the incremental association information synchronization file to the on-demand video service platform by using an associated information synchronization interface;

Step 1109, the precision delivery advertisement system synchronizes the incremental association information synchronization file to the advertisement platform by using an associated information synchronization interface;

Step 1110, the on-demand video service platform receives the video transmission software client sends Business video on demand request;

Step 1111: After receiving the service video on demand request, the on-demand video service platform acquires, according to the association information, the creative associated with the service video that is requested on the advertisement platform; the creative material includes an advertisement video and/or Ad Image;

Step 1112, the on-demand video service platform pushes the creative material to the on-demand video software client;

Step 1113, the on-demand video software client inserts the creative material when playing the on-demand business video; the creative material includes an advertisement video and/or an advertisement image.

In the embodiment of the present invention, the business video and the creative material are processed and analyzed, and the business video and the creative material are automatically associated by comparing the business image and the advertisement image, thereby reducing manual intervention, playing the associated creative material when playing the business video, and reaching the advertisement. A closely related effect to the user’s viewing of the content, enabling accurate advertising. For example, when there is Andy Lau in the business video, the advertisement of Andy Lau’s endorsement is inserted; when the Mercedes-Benz car appears in the business video, the advertisement of Mercedes-Benz is inserted. The embodiment of the invention can automatically complete video analysis processing and association work, and reduce manual intervention.

The embodiment of the invention further provides a computer readable storage medium storing computer executable instructions, which are implemented when the computer executable instructions are executed.

The memory is configured to store a program;

The processor is configured to: obtain a resource to be matched, extract a picture from the to-be-matched resource, obtain a first picture set, and acquire a second video resource according to the image of the second video resource And extracting a picture from the second video resource to obtain a second picture set; comparing the picture in the first picture set with the picture in the second picture set, and obtaining Similarity; when the similarity of the first picture in the first picture set and the second picture in the second picture set meets the first preset condition, the to-be-matched resource to which the first picture belongs Performing the second video resource to which the second picture belongs Association.

The memory is configured to store a program;

The embodiment of the present invention further provides a matching system, including a video analysis platform, a database, a portal, a data synchronization interface module, and an associated information synchronization interface module, where the video analysis platform includes a memory and a processor, where

The memory is configured to store a program;

The processor is configured to execute the program, so that the data synchronization interface module obtains the to-be-matched resource synchronization information from the first platform, and obtains the to-be-matched resource according to the to-be-matched resource synchronization information. The matching resource includes a to-be-matched video resource and/or a to-be-matched picture, and extracts a picture from the to-be-matched resource to obtain a first picture set; and the second video resource synchronization information is obtained from the second platform by using the data synchronization interface module, according to the The second video synchronization information acquires a second video resource, extracts a picture from the second video resource, obtains a second picture set, and performs a picture in the first picture set and a picture in the second picture set. Comparing, obtaining a similarity; when the similarity of the first picture in the first picture set and the second picture in the second picture set meets a first preset condition, Correlating the matching resource with the second video resource to which the second picture belongs to generate association information; and, Synchronizing the association information to the first platform and the second platform by using the associated information synchronization interface module;

The database is configured to store related information of the to-be-matched resource and the second video resource;

The data synchronization interface module synchronizes the to-be-matched resource synchronization information from the first platform to the video analysis platform; and synchronizes the second video resource synchronization information from the second platform to the video analysis platform;

The associated information synchronization interface module is configured to synchronize the association information from the video analysis platform to the first platform and the second platform.

The first platform may be an advertisement platform, and the second platform may be a service platform.

The embodiment of the invention further provides an accurate advertising system, including a video analysis platform, a database, a portal, a data synchronization interface module and an associated information synchronization interface module, wherein

In an embodiment of the present invention, the video analysis platform is further configured to perform background modeling and target detection on the advertisement video in the creative, extract a target of the motion to obtain a target object of the advertisement, and The material target object is associated with the image in the first image collection that includes the creative target object, and the advertisement image in the creative material is used as a target object of the creative object and associated with the advertisement image to which the creative image belongs. The target object is stored in the creative target object table in the database, and its state is set to unaudited; and the background modeling and target detection of the business video are performed, the target of the motion is extracted, the business video target object is obtained, and the service is determined. Correlating the appearance and disappearance time of the video target object in the service video to which it belongs, associating the service video target object with a picture of the service picture set that includes the service video target object, and storing the service video target object Enter the business video target object table in the database and set its status to Nuclear;

The video analysis platform is further configured to: when the second service picture in the service picture set is When the similarity of the advertisement image associated with the reviewed unassociated creative target object satisfies the first preset condition, the obtained advertisement target identifier of the reviewed unassociated creative target object is obtained as The second service video target object configuration associated with the second service picture is identified by the same advertisement object as the reviewed unassociated creative target object, and the status of the reviewed unassociated creative target object in the database is modified. For the association, the state of the second service video target object in the database is modified to be associated.

In an embodiment of the present invention, the portal is further configured to: query the database for the unrecognized creative target object, obtain the reviewed unassociated creative target object, and analyze the video The platform initiates a query request, receives a service picture subset returned by the video analysis platform, and selects a third service picture from the service picture subset according to a user operation, and associates the selected third service picture with the third service. The video target object configures the same advertisement object identifier as the reviewed unassociated creative target object, and the status of the third business video target object in the modified database is associated, and the reviewed unassociated creative target is modified. The state of the object is already associated;

Embodiments of the present invention also provide a storage medium including a stored program, wherein the program described above executes the method of any of the above.

Optionally, in the embodiment, the foregoing storage medium may include, but is not limited to, a USB flash drive, a Read-Only Memory (ROM), and a Random Access Memory (RAM). A variety of media that can store program code, such as a hard disk, a disk, or an optical disk.

Embodiments of the present invention also provide a processor for running a program, wherein the program is executed to perform the steps of any of the above methods.

Although the embodiments disclosed in the present invention are as above, the content described is only for facilitating understanding of the present invention. The embodiments employed are not intended to limit the invention. Any modification and variation in the form and details of the embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention. The scope defined by the appended claims shall prevail.

Industrial applicability

As described above, a video analysis platform, a matching method, and a precise advertisement advertising method and system provided by the embodiments of the present invention have the following beneficial effects: by cutting a business video, an advertisement video, extracting a key frame, and extracting a feature value of the image, associating Business videos and creatives with similar similarities, business platforms and advertising platforms get business videos and their associated creatives, and the business platform plays creatives (advertising videos and/or advertising images) while playing business videos to achieve precision The performance of advertising related to business videos.

Claims

A matching method, including:

Obtaining a to-be-matched resource, extracting a picture from the to-be-matched resource, and obtaining a first picture set;

Obtaining a second video resource, and extracting a picture from the second video resource according to one or more of an image, a sound, and a subtitle of the second video resource, to obtain a second picture set;

Comparing the picture in the first picture set with the picture in the second picture set to obtain a similarity;

When the similarity between the first picture in the first picture set and the second picture in the second picture set satisfies a first preset condition, the to-be-matched resource to which the first picture belongs and the first The second video resource to which the two pictures belong is associated.
The method of claim 1, wherein extracting a picture from the to-be-matched resource or extracting a picture from the second video resource comprises:

Taking the to-be-matched video or the second video resource in the to-be-matched resource as an input video, after decoding the input video, using the decoded input video as a to-be-extracted video; or, decoding the decoded video The input video is segmented according to the scenario, the segmented video is obtained, and the divided videos are compared, and the segmented video set is obtained according to the comparison result, and the segmented video in the segmented video set is used as the to-be-extracted video, where the segmented video set is The comparison result of any two divided videos satisfies the second preset condition;

Extracting a picture from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video.
The method of claim 1, wherein the extracting the picture from the to-be-matched resource comprises: when the to-be-matched resource includes a to-be-matched picture, extracting the to-be-matched picture to join the first picture set .
The method of claim 2, wherein the segmenting the decoded input video according to a scene comprises:

Determining whether the scene is switched according to one or more of background and sound in the input video;

The position where the scene switching occurs in the input video is divided.
The method of claim 4, wherein the determining whether the scene is switched according to one or more of a background and a sound in the input video comprises: converting an audio signal in the input video from a time domain to a frequency a domain, acquiring an energy curve of the audio signal, acquiring a position of a maximum value and a minimum value of the energy curve, as a position where a scene switching occurs, and satisfying: two adjacent occurrence scenes acquired according to the energy curve The location of the switch is greater than the first distance threshold.
The method of claim 2, wherein the extracting a picture from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video comprises:

Extracting a video frame in the to-be-processed video as a key frame, which is referred to as a current key frame, and extracting a next key frame by sequentially: sequentially selecting a video frame after the current key frame in the to-be-processed video and the current key Comparing the frame to obtain a second similarity, if the second similarity between the second video frame and the current key frame satisfies a third preset condition and is closest to the current key frame, extracting the second video frame And using the second video frame as the current key frame, extracting the next key frame in the same manner; if there is no video frame that meets the third preset condition with the second similarity between the current key frame, The process of extracting a picture of the video to be processed ends, and the extracted key frame is taken as the extracted picture;

Wherein, initially, the starting frame of the to-be-processed video is taken as the current key frame.
The method of claim 2, wherein said according to said to-be-extracted video Extracting a video frame from the image to be extracted as one of the images, sounds, and subtitles as the extracted image includes:

Converting an audio signal in the to-be-extracted video from a time domain to a frequency domain, acquiring an energy curve of the audio signal, acquiring a maximum value and a minimum value of the energy curve, and extracting the maximum value and The video frame corresponding to the minimum value position.
The method of claim 2, wherein the extracting a video frame from the image to be extracted according to one or more of an image, a sound, and a subtitle of the to-be-extracted video as the extracted image comprises:

Searching for a subtitle of the video to be extracted, and if a preset keyword is searched, extracting a video frame corresponding to the subtitle including the keyword; wherein the subtitle of the to-be-extracted video is from the video to be extracted Obtaining or performing voice recognition acquisition on the audio of the video to be extracted.
The method of claim 1, wherein the similarity of the first picture in the first picture set and the second picture in the second picture set satisfying the first preset condition means:

The similarity between the first picture and the second picture is greater than a first similarity threshold.
The method of claim 1, wherein the comparing the pictures in the first picture set with the pictures in the second picture set, and obtaining the similarity comprises:

Extracting a first feature value of the picture in the first picture set according to the feature model obtained by the deep learning algorithm, and extracting a second feature value of the picture in the second picture set according to the feature model, and using the first feature The value is compared with the second eigenvalue to obtain a similarity.
A video analysis platform includes a data acquisition module and a video picture analysis processing layer, wherein:

The data acquisition module is configured to acquire the to-be-matched resource and acquire the second video resource.

The video picture analysis processing layer includes a video processing module and a picture analysis module, where:

The video processing module is configured to: extract a picture from the to-be-matched resource to obtain a first picture set; and select one or more of the second video resource according to one or more of an image, a sound, and a subtitle of the second video resource. Extracting a picture to obtain a second picture set;

The image analysis module is configured to compare the picture in the first picture set with the picture in the second picture set to obtain a similarity; when the first picture and the first picture in the first picture set When the similarity of the second picture in the second picture set satisfies the first preset condition, the to-be-matched resource to which the first picture belongs and the second video resource to which the second picture belongs are associated.
The video analytics platform of claim 11 wherein

The video analysis platform further includes a video decoding layer, configured to decode the to-be-matched video resource in the to-be-matched resource, transmit it to the video image analysis processing layer as an input video, and decode the second video resource. And then transmitted as an input video to the video picture analysis processing layer;

The extracting, by the video processing module, the picture from the to-be-matched resource or the second video resource includes:

The input video is used as the to-be-extracted video; or the input video is segmented according to the scene, the segmented video is obtained, the segmented videos of the scenes are compared, and the segmented video set is obtained according to the comparison result, and the segmented video set is obtained. The split video in the video is to be extracted, wherein the comparison result of any two divided videos in the split video set satisfies a second preset condition;

Extracting a picture from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video.
The video analytics platform of claim 11, wherein the extracting the picture from the to-be-matched resource by the video processing module comprises: when the to-be-matched resource includes a picture to be matched, extracting the picture to be matched The first picture collection.
The video analytics platform of claim 12, wherein the video processing module comprises a video segmentation unit configured to determine whether the scene is switched according to one or more of a background and a sound in the input video, Segmentation of the location where the scene switching occurs in the input video; and comparing the divided videos to obtain the divided video set.
The video analysis platform according to claim 14, wherein the video segmentation unit determines whether the scene is switched according to one or more of a background and a sound in the input video, comprising: an audio signal in the input video Converting from the time domain to the frequency domain, acquiring an energy curve of the audio signal, acquiring a position of the maximum value and the minimum value of the energy curve as a position at which scene switching occurs, and satisfying: obtaining according to the energy curve The position where two adjacent scene switches occur is greater than the first distance threshold.
The video analytics platform according to claim 12, wherein said video processing module comprises a multimodal extraction key frame unit, configured to select one or more of said image, sound and subtitle according to said image to be extracted from said The video frame is extracted from the to-be-extracted video as the extracted picture.
The video analysis platform according to claim 16, wherein the multimodal extraction key frame unit extracts a video frame from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video. The extracted images include:

Extracting a video frame in the to-be-processed video as a key frame, which is referred to as a current key frame, and extracting a next key frame by sequentially: sequentially selecting a video frame after the current key frame in the to-be-processed video and the current key Frames are compared to obtain a second similarity, if any The second similarity between the second video frame and the current key frame satisfies a third preset condition and is closest to the current key frame, and the second video frame is extracted, and the second video frame is used as the current key frame. Extracting the next key frame in the same manner; if there is no video frame that satisfies the third preset condition with the second similarity between the current key frame, the process of extracting the picture from the to-be-processed video ends, and the extraction will be performed Key frame as the extracted picture;

Wherein, initially, the starting frame of the to-be-processed video is taken as the current key frame.
The video analysis platform according to claim 16, wherein the multimodal extraction key frame unit extracts a video frame from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video. The extracted images include:

Converting an audio signal in the to-be-extracted video from a time domain to a frequency domain, acquiring an energy curve of the audio signal, acquiring a maximum value and a minimum value of the energy curve, and extracting the maximum value and The video frame corresponding to the minimum value position.
The video analysis platform according to claim 16, wherein the multimodal extraction key frame unit extracts a video frame from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video. The extracted images include:

Searching for a subtitle of the video to be extracted, and if a preset keyword is searched, extracting a video frame corresponding to the subtitle including the keyword; wherein the subtitle of the to-be-extracted video is from the video to be extracted Obtaining or performing voice recognition acquisition on the audio of the video to be extracted.
The video analytics platform of claim 11, wherein the similarity of the first picture in the first picture set and the second picture in the second picture set satisfying the first preset condition means:

The similarity between the first picture and the second picture is greater than a first similarity threshold.
The video analytics platform of claim 11 wherein said image analysis The module includes a deep learning feature model unit, a key frame feature extraction unit, and a feature matching association unit, wherein:

The deep learning feature module unit is configured to learn the labeled sample by using a deep learning algorithm to obtain a feature model;

The key frame feature extraction unit is configured to extract a first feature value of a picture in the first picture set according to the feature model, and extract a second feature value of the picture in the second picture set according to the feature model;

The feature matching association unit is configured to compare the first feature value and the second feature value to obtain a similarity, in the first picture and the second picture set in the first picture set When the similarity of the second picture meets the first preset condition, the to-be-matched resource to which the first picture belongs and the second video resource to which the second picture belongs are associated.
A precise advertising method, including:

Get the creative, extract the image from the creative, and get a collection of ad images;

Obtaining a service video, extracting a picture from the service video according to one or more of an image, a sound, and a subtitle of the service video, to obtain a service picture set;

Comparing the advertisement picture in the advertisement picture set with the service picture in the service picture set to obtain a similarity;

When the similarity between the first advertisement picture in the advertisement picture set and the first service picture in the service picture set satisfies a first preset condition, the first creative material and the location to which the first advertisement picture belongs The first service video to which the first service picture belongs is associated, and the association information is generated, where the association information includes: identifier information of the first service video, and identifier information of the first advertisement material.
The method of claim 22, wherein extracting a picture from the creative or extracting a picture from the business video comprises:

Taking the advertisement video or the service video in the creative as an input video, after decoding the input video, using the decoded input video as a to-be-extracted video; or, decoding the input video according to the scene Performing segmentation, obtaining a segmented video, comparing the divided videos, and obtaining a segmented video set according to the comparison result, and using the segmented video in the segmented video set as a to-be-extracted video, where any two segments in the segmented video set are segmented The comparison result of the video satisfies the second preset condition;

Extracting a picture from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video.
The method of claim 22, wherein extracting a picture from the creative comprises extracting the advertisement picture into the set of advertisement pictures when the advertisement picture includes an advertisement picture.
The method of claim 23, wherein the segmenting the decoded input video according to a scene comprises:

Determining whether the scene is switched according to one or more of the background and the sound in the input video; and dividing the position where the scene switching occurs in the input video.
The method of claim 25, wherein the determining whether the scene is switched according to one or more of a background and a sound in the input video comprises: converting an audio signal in the input video from a time domain to a time domain In the frequency domain, acquiring an energy curve of the audio signal, obtaining a position of a maximum value and a minimum value of the energy curve, as a position where a scene switching occurs, and satisfying: two adjacent occurrences according to the energy curve The position of the scene switching is greater than the first distance threshold.
The method of claim 23, wherein said according to said to-be-extracted view Extracting a video frame from the to-be-extracted video as one or more of the frequency images, sounds, and subtitles as the extracted image includes:

Extracting a video frame in the to-be-processed video as a key frame, which is referred to as a current key frame, and extracting a next key frame by sequentially: sequentially selecting a video frame after the current key frame in the to-be-processed video and the current key Comparing the frame to obtain a second similarity, if the second similarity between the second video frame and the current key frame satisfies a third preset condition and is closest to the current key frame, extracting the second video frame And using the second video frame as the current key frame, extracting the next key frame in the same manner; if there is no video frame that meets the third preset condition with the second similarity between the current key frame, The process of extracting a picture of the video to be processed ends, and the extracted key frame is taken as the extracted picture;

Wherein, initially, the starting frame of the to-be-processed video is taken as the current key frame.
The method of claim 23, wherein extracting a video frame from the image to be extracted as the extracted picture according to one or more of an image, a sound and a subtitle of the to-be-extracted video comprises:

Converting an audio signal in the to-be-extracted video from a time domain to a frequency domain, acquiring an energy curve of the audio signal, acquiring a maximum value and a minimum value of the energy curve, and extracting the maximum value and The video frame corresponding to the minimum value position is taken as the extracted picture.
The method of claim 23, wherein the extracting a video frame from the image to be extracted according to one or more of an image, a sound, and a subtitle of the to-be-extracted video as the extracted image comprises:

Searching for a subtitle of the video to be extracted, and if a preset keyword is searched, extracting a video frame corresponding to the subtitle including the keyword; wherein the subtitle of the to-be-extracted video is from the video to be extracted Obtaining or performing voice recognition acquisition on the audio of the video to be extracted.
The method according to claim 22, wherein the similarity of the first advertisement picture in the advertisement picture set and the first service picture in the service picture set satisfies the first preset condition means:

The similarity between the first advertisement picture and the first service picture is greater than a first similarity threshold.
The method of claim 22, wherein comparing the advertisement picture in the advertisement picture set with the service picture in the service picture set, and obtaining the similarity comprises:

Extracting a first feature value of the image in the advertisement picture set according to the feature model obtained by the deep learning algorithm, extracting a second feature value of the picture in the service picture set according to the feature model, and using the first feature value and The second feature values are compared to obtain a similarity.
The method of claim 22, wherein the association information further includes insertion location information that is inserted by the first creative material when the first service video is played, the insertion location information according to the first The location of a service picture in the first service video is determined.
The method of claim 22, wherein the method further comprises:

Performing background modeling and target detection on the service video, extracting a target of the motion to obtain a service video target object, determining an appearance time and a disappearance time of the service video target object in a service video to which the service video belongs, and the service video target An object is associated with a picture of the service picture set that includes the service video target object;

The associated information further includes insertion location information that is inserted by the first creative material when the first service video is played, and the insertion location information is based on the first service video target object associated with the first service picture. Appearance in the first business video Time and disappearance time are determined.
The method of claim 33, wherein the method further comprises:

Performing background modeling and target detection on the advertisement video in the creative, extracting the target of the motion to obtain the creative target object, and including the creative target object and the first image set containing the creative target object The image is associated, and the advertisement image in the creative is used as a target object of the creative and associated with the advertisement image to which it belongs;

Acquiring the first advertisement associated with the first advertisement image when the similarity between the first advertisement image in the advertisement image set and the first service image in the service image collection meets the first preset condition The target object is configured with an advertisement object identifier, and the first service video target object associated with the first service image is configured with the same advertisement object identifier as the first creative target object, thereby indicating the first advertisement image The associated first creative is associated with the first business video to which the first business image belongs, wherein the advertising object identifier indicates an advertising theme for which the first creative target object is targeted.
The method of claim 34, wherein the association information further comprises an advertisement object identifier in which the first creative target object is configured.
The method of claim 34, wherein the method further comprises:

After receiving the query request sent by the portal, determining a second advertisement image associated with the second creative target object carried in the query request, and searching for the similarity with the second advertisement image in the service image collection to satisfy the first And a fourth set of service picture sub-sets, the service picture sub-set is sent to the portal, so that the portal selects a second service picture from the service picture subset according to a user operation, and the selected The second business video target object is configured with the same advertisement object identifier as the second creative target object, thereby indicating that the second creative target object belongs to The second creative is associated with the second service video to which the second service video target object belongs.
A video analysis platform includes a data acquisition module and a video image analysis processing layer, where the video image analysis processing layer includes a video processing module and a picture analysis module, wherein:

The data acquisition module is configured to acquire a creative material and obtain a business video;

The video processing module is configured to: extract a picture from the creative material to obtain an advertisement picture set; and extract a picture from the service video according to one or more of an image, a sound, and a subtitle of the service video, Obtain a collection of business images;

The image analysis module is configured to compare the advertisement image in the advertisement image collection with the business image in the business image collection to obtain a similarity; when the first advertisement image in the advertisement image collection and the Correlating the first creative material to which the first advertisement image belongs and the first service video to which the first service image belongs when the similarity of the first service image in the service image set meets the first preset condition, and The association information is generated, and the association information includes: identifier information of the first service video, and identifier information of the first advertisement material.
A video analysis platform according to claim 37, wherein

The video analysis platform further includes a video decoding layer, configured to decode the advertisement video in the creative as an input video to the video image analysis processing layer, and decode the service video as an input video. Transmitting to the video picture analysis processing layer;

The video processing module extracting a picture from the creative or extracting a picture from the business video includes:

Using the input video as a video to be extracted; or, the input video is based on The scene is segmented, the segmented video is obtained, the segmented video of each scene is compared, and the segmented video set is obtained according to the comparison result, and the segmented video in the segmented video set is used as the to-be-extracted video, where any of the segmented video sets is The comparison result of the two divided videos satisfies the second preset condition;

Extracting a picture from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video.
The video analytics platform of claim 37, wherein the video processing module extracting a picture from the creative material comprises: when the advertisement material includes an advertisement picture, extracting the advertisement picture to join the advertisement picture set .
The video analytics platform of claim 38, wherein the video processing module comprises a video segmentation unit configured to determine whether the scene is switched according to one or more of a background and a sound in the input video, at the input The position where the scene switching occurs in the video is split.
The video analytics platform of claim 40, wherein the video segmentation unit determines whether the scene is switched according to one or more of a background and a sound in the input video, comprising: locating an audio signal in the input video from Converting the time domain to the frequency domain, acquiring an energy curve of the audio signal, obtaining a position of the maximum value and the minimum value of the energy curve, as a position where the scene switching occurs, and satisfying: a phase acquired according to the energy curve The position where two adjacent scene switches occur is greater than the first distance threshold.
A video analytics platform according to claim 38, wherein said video processing module comprises a multimodal extraction key frame unit, arranged to be one or more of said image, sound and subtitle based on said image to be extracted from said The video frame is extracted from the to-be-extracted video as the extracted picture.
The video analysis platform according to claim 42, wherein said multimodal extraction key frame unit is one or more of an image, a sound and a subtitle according to said to-be-extracted video Extracting a video frame from the to-be-extracted video as the extracted picture includes:

Extracting a video frame in the to-be-processed video as a key frame, which is referred to as a current key frame, and extracting a next key frame by sequentially: sequentially selecting a video frame after the current key frame in the to-be-processed video and the current key Comparing the frame to obtain a second similarity, if the second similarity between the second video frame and the current key frame satisfies a third preset condition and is closest to the current key frame, extracting the second video frame And using the second video frame as the current key frame, extracting the next key frame in the same manner; if there is no video frame that meets the third preset condition with the second similarity between the current key frame, The process of extracting a picture of the video to be processed ends, and the extracted key frame is taken as the extracted picture;

Wherein, initially, the starting frame of the to-be-processed video is taken as the current key frame.
The video analysis platform according to claim 42, wherein the multimodal extraction key frame unit extracts a video frame from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video. The extracted images include:

Converting an audio signal in the to-be-extracted video from a time domain to a frequency domain, acquiring an energy curve of the audio signal, acquiring a maximum value and a minimum value of the energy curve, and extracting the maximum value and The video frame corresponding to the minimum value position is taken as the extracted picture.
The video analysis platform according to claim 42, wherein the multimodal extraction key frame unit extracts a video frame from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video. The extracted images include:

Searching for a subtitle of the to-be-extracted video, if a preset keyword is searched, extracting a video frame corresponding to the subtitle containing the keyword as the extracted image; wherein the subtitle of the to-be-extracted video is from the Acquiring in the subtitle file that is to be extracted from the video or performing speech recognition on the audio of the to-be-extracted video.
A video analytics platform according to claim 37, wherein said advertisement image The similarity between the first advertisement picture in the set and the first service picture in the service picture set satisfies the first preset condition:

The similarity between the first advertisement picture and the first service picture is greater than a first similarity threshold.
The video analytics platform of claim 37, wherein the picture analysis module comprises a deep learning feature model unit, a key frame feature extraction unit, and a feature matching association unit, wherein:

The deep learning feature module unit is configured to learn the labeled sample by using a deep learning algorithm to obtain a feature model;

The key frame feature extraction unit is configured to extract a first feature value of the image in the advertisement image set according to the feature model, and extract a second feature value of the image in the service image set according to the feature model;

The feature matching association unit is configured to compare the first feature value and the second feature value to obtain a similarity; when the first advertisement picture and the service picture set in the advertisement picture set When the similarity of the first service picture satisfies the first preset condition, the first creative material to which the first advertisement picture belongs and the first service video to which the first service picture belongs are associated, and the associated information is generated. The association information includes: identifier information of the first service video, and identifier information of the first advertisement material.
The video analytics platform of claim 37, wherein the associated information further comprises: insertion location information of the first creative material that is inserted during the playing of the first service video, the insertion location information according to the location Determining the location of the first service picture in the first service video.
The video analytics platform of claim 37, wherein the video processing module further comprises: a background modeling unit and a target detection unit, wherein:

The background modeling unit is configured to perform background modeling on the service video;

The target detecting unit is configured to perform target detection on the service video, extract a target of the motion to obtain a service video target object, and determine an appearance time and a disappearance time of the service video target object in a service video to which the service video object belongs. The service video target object is associated with a picture of the service picture set that includes the service video target object;

The associated information further includes insertion location information that is inserted by the first creative material when the first service video is played, and the insertion location information is based on the first service video target object associated with the first service picture. The appearance time and disappearance time in the first service video are determined.
A video analysis platform according to claim 49, wherein

The background modeling unit is further configured to perform background modeling on the advertisement video in the creative;

The target detecting unit is further configured to perform target detection on the advertisement video in the creative, extract a target of the motion to obtain a creative target object, and include the target object in the creative and the first image collection The image of the creative target object is associated, and the advertisement image in the creative is used as a target object of the creative and associated with the advertisement image to which it belongs;

The image analysis module is further configured to: when the similarity between the first advertisement image in the advertisement image set and the first service image in the service image collection meets the first preset condition, acquire the An advertisement object identifier configured by the first creative target object associated with an advertisement image, and an advertisement object identifier identical to the first creative target object configured for the first service video target object associated with the first service image And indicating that the first creative material to which the first advertisement image belongs is associated with the first service video to which the first service image belongs, wherein the advertisement object identifier indicates that the first creative target object is targeted Advertising theme.
The video analytics platform of claim 50, wherein the associated information further comprises an advertising object identifier in which the first creative target object is configured.
The video analytics platform of claim 50, wherein the picture analysis module is further configured to:

After receiving the query request sent by the portal, determining a second advertisement image associated with the second creative target object carried in the query request, and searching for the similarity with the second advertisement image in the service image collection to satisfy the first And a fourth set of service picture sub-sets, the service picture sub-set is sent to the portal, so that the portal selects a second service picture from the service picture subset according to a user operation, and the selected The second service video target object associated with the second business image is configured with the same advertisement object identifier as the second creative target object, thereby indicating the second creative material to which the second creative target object belongs and the second service video. The second business video to which the target object belongs is associated.
A video analysis platform includes: a memory and a processor, wherein:

The memory is configured to store a program;

The processor is configured to: obtain a resource to be matched, extract a picture from the to-be-matched resource, obtain a first picture set, and acquire a second video resource according to the image of the second video resource And extracting a picture from the second video resource to obtain a second picture set; comparing the picture in the first picture set with the picture in the second picture set, and obtaining Similarity; when the similarity of the first picture in the first picture set and the second picture in the second picture set meets the first preset condition, the to-be-matched resource to which the first picture belongs The second video resource to which the second picture belongs is associated.
The video analytics platform of claim 53, wherein the extracting a picture from the to-be-matched resource or extracting a picture from the second video resource comprises:

Taking the to-be-matched video or the second video resource in the to-be-matched resource as an input video, after decoding the input video, using the decoded input video as a to-be-extracted video; or, decoding the decoded video The input video is segmented according to the scenario, the segmented video is obtained, and the divided videos are compared, and the segmented video set is obtained according to the comparison result, and the segmented video in the segmented video set is used as the to-be-extracted video, where the segmented video set is The comparison result of any two divided videos satisfies the second preset condition;

Extracting a picture from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video.
The video analytics platform of claim 54 wherein the segmenting the decoded input video according to the scene comprises:

Determining whether the scene is switched according to one or more of background and sound in the input video;

The position where the scene switching occurs in the input video is divided.
The video analytics platform of claim 55, wherein the determining whether the scene is switched according to one or more of a background and a sound in the input video comprises: converting an audio signal in the input video from a time domain Go to the frequency domain, obtain an energy curve of the audio signal, obtain a position of the maximum value and the minimum value of the energy curve, as a position where the scene switching occurs, and satisfy: two adjacent two obtained according to the energy curve The location where the scene switch occurs is greater than the first distance threshold.
The video analytics platform of claim 54, wherein the extracting a picture from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video comprises:

Extracting a video frame in the to-be-processed video as a key frame, which is called a current key frame, and extracting a next key frame by sequentially: sequentially, the current key frame in the to-be-processed video Subsequent video frames are compared with the current key frame to obtain a second similarity, if there is a second similarity between the second video frame and the current key frame that satisfies a third preset condition and is related to the current key frame The second video frame is extracted, the second video frame is taken as the current key frame, and the next key frame is extracted in the same manner; if the second similarity between the current key frame and the current key frame does not exist, the third pre-preparation is satisfied. Setting the video frame, the process of extracting the picture from the to-be-processed video ends, and extracting the key frame as the extracted picture;

Wherein, initially, the starting frame of the to-be-processed video is taken as the current key frame.
The video analytics platform of claim 54, wherein the extracting the video frame from the image to be extracted according to one or more of the image, the sound and the subtitle of the to-be-extracted video as the extracted image comprises:

Converting an audio signal in the to-be-extracted video from a time domain to a frequency domain, acquiring an energy curve of the audio signal, acquiring a maximum value and a minimum value of the energy curve, and extracting the maximum value and The video frame corresponding to the minimum value position.
The video analytics platform of claim 54, wherein the extracting the video frame from the image to be extracted according to one or more of the image, the sound and the subtitle of the to-be-extracted video as the extracted image comprises:

Searching for a subtitle of the video to be extracted, and if a preset keyword is searched, extracting a video frame corresponding to the subtitle including the keyword; wherein the subtitle of the to-be-extracted video is from the video to be extracted Obtaining or performing voice recognition acquisition on the audio of the video to be extracted.
The video analytics platform of claim 53, wherein the similarity of the first picture in the first picture set and the second picture in the second picture set satisfying the first preset condition means:

The similarity between the first picture and the second picture is greater than a first similarity threshold.
The video analytics platform of claim 53, wherein the comparing the pictures in the first picture set with the pictures in the second picture set to obtain the similarity comprises:

Extracting a first feature value of the picture in the first picture set according to the feature model obtained by the deep learning algorithm, and extracting a second feature value of the picture in the second picture set according to the feature model, and using the first feature The value is compared with the second eigenvalue to obtain a similarity.
A video analysis platform includes: a memory and a processor, wherein:

The memory is configured to store a program;

The processor executing the program to: obtain a creative, extract a picture from the creative, obtain an advertisement picture set; obtain a service video, according to one of an image, a sound, and a subtitle of the service video Or extracting a picture from the service video to obtain a service picture set; comparing the advertisement picture in the advertisement picture set with the service picture in the service picture set to obtain a similarity; when the advertisement picture set When the similarity between the first advertisement picture and the first service picture in the service picture set meets the first preset condition, the first creative material to which the first advertisement picture belongs and the first service picture belong to The first service video is associated and generates association information, where the association information includes: identifier information of the first service video, and identifier information of the first advertisement material.
The video analytics platform of claim 62, wherein extracting a picture from the creative or extracting a picture from the business video comprises:

Taking the advertisement video or the service video in the creative as an input video, after decoding the input video, using the decoded input video as a to-be-extracted video; or, decoding the input video according to the scene Perform segmentation, obtain a segmented video, compare each segmented video, and obtain a segmented video set according to the comparison result, and divide the segment Splitting the video in the video set as the to-be-extracted video, where the comparison result of any two of the divided video sets meets the second preset condition;

Extracting a picture from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video.
The video analytics platform of claim 63, wherein the segmenting the decoded input video according to the scene comprises:

Determining whether the scene is switched according to one or more of the background and the sound in the input video; and dividing the position where the scene switching occurs in the input video.
The video analytics platform of claim 64, wherein the determining whether the scene is switched according to one or more of a background and a sound in the input video comprises: locating an audio signal in the input video from a time domain Converting to the frequency domain, acquiring an energy curve of the audio signal, obtaining a position of a maximum value and a minimum value of the energy curve, as a position at which scene switching occurs, and satisfying: two adjacent ones obtained according to the energy curve The location where the scene switching occurs is greater than the first distance threshold.
The video analytics platform of claim 63, wherein the extracting a video frame from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video comprises:

Extracting a video frame in the to-be-processed video as a key frame, which is referred to as a current key frame, and extracting a next key frame by sequentially: sequentially selecting a video frame after the current key frame in the to-be-processed video and the current key Comparing the frame to obtain a second similarity, if the second similarity between the second video frame and the current key frame satisfies a third preset condition and is closest to the current key frame, extracting the second video frame And using the second video frame as the current key frame, extracting the next key frame in the same manner; if there is no video frame that meets the third preset condition with the second similarity between the current key frame, The process of extracting a picture of the video to be processed ends, and the extracted key frame is taken as the extracted picture;

Wherein, initially, the starting frame of the to-be-processed video is taken as the current key frame.
The video analytics platform of claim 63, wherein extracting a video frame from the to-be-extracted video as the extracted picture according to one or more of an image, a sound and a subtitle of the to-be-extracted video comprises:

Converting an audio signal in the to-be-extracted video from a time domain to a frequency domain, acquiring an energy curve of the audio signal, acquiring a maximum value and a minimum value of the energy curve, and extracting the maximum value and The video frame corresponding to the minimum value position is taken as the extracted picture.
The video analytics platform of claim 63, wherein the extracting a video frame from the to-be-extracted video according to one or more of an image, a sound, and a subtitle of the to-be-extracted video comprises:

Searching for a subtitle of the video to be extracted, and if a preset keyword is searched, extracting a video frame corresponding to the subtitle including the keyword; wherein the subtitle of the to-be-extracted video is from the video to be extracted Obtaining or performing voice recognition acquisition on the audio of the video to be extracted.
The video analytics platform of claim 63, wherein the similarity of the first advertisement picture in the advertisement picture set and the first service picture in the service picture set satisfying the first preset condition means:

The similarity between the first advertisement picture and the first service picture is greater than a first similarity threshold.
The video analytics platform of claim 62, wherein comparing the advertisement image in the advertisement picture set with the service picture in the service picture set, and obtaining the similarity comprises:

Extracting a first feature value of the image in the advertisement picture set according to the feature model obtained by the deep learning algorithm, and extracting a picture in the service picture set according to the feature model The second feature value of the slice compares the first feature value and the second feature value to obtain a similarity.
The video analytics platform of claim 62, wherein the associated information further comprises: insertion location information of the first creative material that is inserted during the playing of the first service video, the insertion location information according to the location Determining the location of the first service picture in the first service video.
The video analytics platform of claim 62 wherein said processor executes said program, further configured to:

Performing background modeling and target detection on the service video, extracting a target of the motion to obtain a service video target object, determining an appearance time and a disappearance time of the service video target object in a service video to which the service video belongs, and the service video target An object is associated with a picture of the service picture set that includes the service video target object;

The associated information further includes insertion location information that is inserted by the first creative material when the first service video is played, and the insertion location information is based on the first service video target object associated with the first service picture. The appearance time and disappearance time in the first service video are determined.
The video analytics platform of claim 72 wherein said processor executes said program, further configured to:

Performing background modeling and target detection on the advertisement video in the creative, extracting the target of the motion to obtain the creative target object, and including the creative target object and the first image set containing the creative target object The image is associated, and the advertisement image in the creative is used as a target object of the creative and associated with the advertisement image to which it belongs;

When in the first advertisement picture and the business picture set in the advertisement picture set Obtaining an advertisement object identifier of the first creative target object associated with the first advertisement image, which is associated with the first service image, when the similarity of the first service image satisfies the first preset condition The first service video target object is configured with the same advertisement object identifier as the first creative target object, thereby indicating that the first creative material to which the first advertisement image belongs and the first service video to which the first service image belongs Correlation, wherein the advertisement object identifier indicates an advertisement theme for which the first creative target object is directed.
The video analytics platform of claim 73, wherein the association information further comprises an advertisement object identifier in which the first creative target object is configured.
The video analytics platform of claim 73 wherein said processor executes said program, further configured to:

After receiving the query request sent by the portal, determining a second advertisement image associated with the second creative target object carried in the query request, and searching for the similarity with the second advertisement image in the service image collection to satisfy the first And a fourth set of service picture sub-sets, the service picture sub-set is sent to the portal, so that the portal selects a second service picture from the service picture subset according to a user operation, and the selected The second service video target object is configured with the same advertisement object identifier as the second creative target object, thereby indicating the second creative material to which the second creative target object belongs and the second service The second business video to which the video target object belongs is associated.
An accurate advertising system, including a video analysis platform, a database, a portal, a data synchronization interface module, and an associated information synchronization interface module, wherein

The video analysis platform is configured to: synchronize the creative synchronization information from the advertisement platform by using the data synchronization interface module, obtain the creative material according to the creative synchronization information, extract a picture from the creative material, obtain a set of advertisement images; The data synchronization interface module synchronizes service video synchronization information from the service platform, and synchronizes according to the service video. Obtaining a service video, extracting a picture from the service video according to one or more of an image, a sound, and a subtitle of the service video, obtaining a service picture set; and displaying an advertisement picture and the service in the advertisement picture set Comparing the service pictures in the picture set to obtain the similarity; when the similarity between the first advertisement picture in the advertisement picture set and the first service picture in the service picture set satisfies the first preset condition, The first creative material to which the first advertisement picture belongs is associated with the first service video to which the first service picture belongs, and the association information is generated, where the association information includes: the identification information of the first service video, and the Decoding the first creative material, and synchronizing the association information to the advertisement platform and the service platform by using the associated information synchronization interface module;

The database is configured to store related information of the creative and the business video;

The portal is configured to implement interaction between the user and the database and the video analysis platform;

The data synchronization interface module synchronizes creative synchronization information from the advertisement platform to the video analysis platform; and is configured to synchronize service video synchronization information from the service platform to the video analysis platform;

The associated information synchronization interface module is configured to synchronize the associated information from the video analytics platform to the advertising platform and the service platform.
The precision advertising system according to claim 76, wherein

The video analysis platform is further configured to: perform background modeling and target detection on the advertisement video in the creative, extract a target of the motion to obtain a creative target object, and use the creative target object and the first image The image of the creative target object in the collection is associated, the advertisement image in the creative is used as the target object of the creative and associated with the advertisement image to which it belongs, and the creative target object is stored in the creative in the database a target object table, setting its state to unaudited; and performing background modeling and target detection on the service video, extracting a target of the motion to obtain a service video target object, and determining a service video to which the service video target object belongs Out of The current video and the disappearance time, the service video target object is associated with the picture of the service picture set containing the service video target object, and the service video target object is stored in the service video target object table in the database, Its status is set to unaudited;

The portal is configured to implement the interaction between the user and the database and the video analytics platform, including: querying the database for an unreviewed creative target object according to a user operation, and configuring an advertisement for the unreviewed creative target object according to a user operation. The object identifies and modifies its status in the creative target object table as reviewed and not associated;

The video analytics platform compares the advertisement image in the advertisement image set with the service image in the service image collection, and the obtaining similarity includes:

The video analytics platform queries the database for the reviewed unassociated creative target object, receives the reviewed unassociated creative target object returned by the database, and obtains the associated unrecognized creative target object associated with the database. Advertising the image, comparing it with the business image in the business image collection, and obtaining similarity;

The video analysis platform is further configured to: when the similarity between the second service image in the service image set and the advertisement image associated with the reviewed unrelated creative target object satisfies the first preset condition Obtaining an advertisement object identifier configured by the reviewed unassociated creative target object, and configuring, for the second service video target object associated with the second service image, the creative target object that is not associated with the audited The same advertisement object identifier, the status of the reviewed unassociated creative target object in the database is modified, and the status of the second business video target object in the modified database is associated.
The precision advertising system according to claim 77, wherein

The portal is further configured to: query the database for the reviewed unassociated creative target object, obtain the reviewed unassociated creative target object, initiate a query request to the video analytics platform, and receive the video. Analysis of the business image subset returned by the platform And selecting, according to a user operation, selecting a third service picture from the service picture subset, and configuring the selected third service picture associated third service video target object with the reviewed unassociated creative target object The same advertisement object identifier, modifying the status of the third service video target object in the database to be associated, and modifying the status of the reviewed unassociated creative target object to be associated;

The video analysis platform is further configured to: after receiving the query request sent by the portal, determine a third advertisement image associated with the creative target object carried in the query request, and find and describe in the service image set The service picture subset of the third advertisement picture satisfies the fourth preset condition, and the service picture subset is sent to the portal.
A storage medium, the storage medium comprising a stored program, wherein the program is executed to perform the method of any one of claims 1-10, 22-36.