CN112580696A - Advertisement label classification method, system and equipment based on video understanding - Google Patents

Advertisement label classification method, system and equipment based on video understanding Download PDF

Info

Publication number
CN112580696A
CN112580696A CN202011393760.5A CN202011393760A CN112580696A CN 112580696 A CN112580696 A CN 112580696A CN 202011393760 A CN202011393760 A CN 202011393760A CN 112580696 A CN112580696 A CN 112580696A
Authority
CN
China
Prior art keywords
video
classification
advertisement
data set
tag
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011393760.5A
Other languages
Chinese (zh)
Inventor
冯希宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Star Media Ltd
Original Assignee
Star Media Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Star Media Ltd filed Critical Star Media Ltd
Priority to CN202011393760.5A priority Critical patent/CN112580696A/en
Publication of CN112580696A publication Critical patent/CN112580696A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241Advertisements
    • G06Q30/0251Targeted advertisements
    • G06Q30/0269Targeted advertisements based on user profile or attribute
    • G06Q30/0271Personalized advertisement
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/466Learning process for intelligent management, e.g. learning user preferences for recommending movies
    • H04N21/4662Learning process for intelligent management, e.g. learning user preferences for recommending movies characterized by learning algorithms
    • H04N21/4666Learning process for intelligent management, e.g. learning user preferences for recommending movies characterized by learning algorithms using neural networks, e.g. processing the feedback provided by the user
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/466Learning process for intelligent management, e.g. learning user preferences for recommending movies
    • H04N21/4668Learning process for intelligent management, e.g. learning user preferences for recommending movies for recommending content, e.g. movies
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81Monomedia components thereof
    • H04N21/812Monomedia components thereof involving advertisement data
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84Generation or processing of descriptive data, e.g. content descriptors
    • H04N21/8405Generation or processing of descriptive data, e.g. content descriptors represented by keywords

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Business, Economics & Management (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Development Economics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Accounting & Taxation (AREA)
  • General Engineering & Computer Science (AREA)
  • Marketing (AREA)
  • Finance (AREA)
  • Strategic Management (AREA)
  • Game Theory and Decision Science (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • General Business, Economics & Management (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention provides a method, a system and equipment for classifying advertisement labels based on video understanding, which comprises the following steps: labeling the advertisement video to finish the preparation of a data set; adopting Resnet-50 as a backbone network, inserting the time sequence conversion model into the Resnet-50 network to construct a preview frequency classification model, and then training the preview frequency classification model by using a data set to generate a video classification model; and carrying out classification prediction on the advertisement videos to be subjected to label classification by using a video classification model to obtain content classification results of the advertisement videos, and carrying out label classification. The invention can analyze the advertisement video content in multiple dimensions, understand the video semantics and automatically classify and tag, thereby greatly saving the manual examination efficiency and saving the cost.

Description

Advertisement label classification method, system and equipment based on video understanding
Technical Field
The invention relates to the technical field of image classification and identification, in particular to a method, a system and equipment for classifying advertisement labels based on video understanding.
Background
With the rapid development of network technology and multimedia technology, video advertisements are also rapidly developed, and how to accurately push an advertisement to a user interested in the advertisement is an urgent problem to be solved.
The key for realizing accurate advertisement pushing is to accurately classify the advertisements and label the advertisements by using labels, the existing advertisement label classification method on the market generally adopts manual examination and labeling, wastes time and labor, and the manual examination and labeling can not ensure real time along with the explosive growth of advertisement video streams; in addition, the subjectivity of manual review is too strong, and for some advertisement video streams with multi-element characteristics, a manually labeled tag system is deficient, so that accurate advertisement recommendation cannot be performed.
Disclosure of Invention
In view of the above problems, an object of the present invention is to provide a method, a system, and a device for classifying advertisement tags based on video understanding, which can analyze advertisement video content in multiple dimensions, understand video semantics, automatically classify and tag, greatly save manual review efficiency, and save cost.
In order to achieve the purpose, the invention is realized by the following technical scheme: a video understanding-based advertisement label classification method comprises the following steps:
s1: labeling the advertisement video to finish the preparation of a data set;
s2: adopting Resnet-50 as a backbone network, inserting the time sequence conversion model into the Resnet-50 network to construct a preview frequency classification model, and then training the preview frequency classification model by using a data set to generate a video classification model;
s3: and carrying out classification prediction on the advertisement videos to be subjected to label classification by using a video classification model to obtain content classification results of the advertisement videos, and carrying out label classification.
Further, the tag labeling categories include: an action class tag, a scene class tag, and an object class tag.
Further, the data set includes: clipped datasets and non-clipped datasets; label category labeling is directly carried out on the clipped data set; the non-clipped dataset is tagged with tag categories by time period.
Further, before S2, the step further includes:
and performing data enhancement on the data set, wherein the data enhancement specifically comprises geometric transformation enhancement and color transformation enhancement.
Further, the geometric transformation enhancement comprises: flipping, rotating, cropping, distorting, and scaling each frame of the ad video.
Further, the color transform enhancement comprises: noise transformation, blur transformation, and color transformation for each frame of the advertisement video.
Correspondingly, the invention also discloses an advertisement label classification system based on video understanding, which comprises the following steps:
the data set preparation unit is used for labeling the advertisement video to complete the preparation of the data set;
the model training unit is used for adopting Resnet-50 as a backbone network, inserting the time sequence conversion model into the Resnet-50 network to construct a pre-video classification model, and then training the pre-video classification model by using a data set to generate a video classification model;
and the model reasoning unit is used for carrying out classification prediction on the advertisement video to be subjected to label classification by using the video classification model to obtain a content classification result of the advertisement video and carry out label classification.
Further, still include: and the data set enhancement unit is used for carrying out geometric transformation enhancement and color transformation enhancement on the data set.
Correspondingly, the invention also discloses advertisement label classification equipment based on video understanding, which comprises the following components:
a memory for storing a computer program;
a processor for implementing the video understanding-based advertisement tag classification method steps as described in any one of the above when the computer program is executed.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the invention, the self-defined label labeling can be carried out on the advertisement video according to the actual application condition, and the data set for classifying the advertisement video is generated.
2. The invention classifies the advertisement videos by using a Temporal Shift Module (TSM) and a 2D neural network, thereby not only ensuring the accuracy, but also ensuring the speed by adding the Temporal Shift Module and realizing the intelligent classification of the advertisement videos.
3. The invention adopts the video understanding technology based on the content, can analyze the video content in multiple dimensions, understand the video semantics and automatically classify and tag, greatly saves the manual review efficiency and the cost, and has important guiding significance for intelligent advertisement recommendation.
Therefore, compared with the prior art, the invention has prominent substantive features and remarkable progress, and the beneficial effects of the implementation are also obvious.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a system block diagram of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made with reference to the accompanying drawings.
A video understanding-based advertisement tag classification method as shown in fig. 1 includes the following steps:
s101: and labeling the advertisement video to finish the preparation of the data set.
Wherein, the label labeling category comprises: action class tags (e.g., tennis), scene class tags (e.g., beach), and object class tags (e.g., car). Various custom tags can also be labeled according to actual applications. The data set includes: clipped datasets and non-clipped datasets; the clipped data set may be directly labeled categories, but for non-clipped data sets it may be necessary to label different label categories for time periods.
S102: data enhancement is performed on the data set.
Including specifically geometric transformation enhancement and color transformation enhancement. Wherein the geometric transformation enhancement comprises: flipping, rotating, cropping, distorting, and scaling each frame of the ad video. The color transform enhancement includes: noise transformation, blur transformation, and color transformation for each frame of the advertisement video.
S103: and adopting Resnet-50 as a backbone network, inserting the time conversion model into the Resnet-50 network to construct a preview frequency classification model, and then training the preview frequency classification model by using a data set to generate a video classification model.
S104: and carrying out classification prediction on the advertisement videos to be subjected to label classification by using a video classification model to obtain content classification results of the advertisement videos, and carrying out label classification.
Correspondingly, as shown in fig. 2, the present invention also discloses an advertisement tag classification system based on video understanding, which includes:
and the data set preparation unit is used for labeling the advertisement video to complete the preparation of the data set.
And the data set enhancement unit is used for carrying out geometric transformation enhancement and color transformation enhancement on the data set.
And the model training unit is used for adopting Resnet-50 as a backbone network, inserting the time sequence conversion model into the Resnet-50 network to construct a pre-video classification model, and then training the pre-video classification model by using a data set to generate a video classification model.
And the model reasoning unit is used for carrying out classification prediction on the advertisement video to be subjected to label classification by using the video classification model to obtain a content classification result of the advertisement video and carry out label classification.
Correspondingly, the invention also discloses advertisement label classification equipment based on video understanding, which comprises the following components:
a memory for storing a computer program;
a processor for implementing the video understanding-based advertisement tag classification method steps as described in any one of the above when the computer program is executed.
Those skilled in the art will readily appreciate that the techniques of the embodiments of the present invention may be implemented as software plus a required general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be embodied in the form of a software product, where the computer software product is stored in a storage medium, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like, and the storage medium can store program codes, and includes instructions for enabling a computer terminal (which may be a personal computer, a server, or a second terminal, a network terminal, and the like) to perform all or part of the steps of the method in the embodiments of the present invention. The same and similar parts in the various embodiments in this specification may be referred to each other. Especially, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to the description in the method embodiment.
In the embodiments provided by the present invention, it should be understood that the disclosed system, system and method can be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, systems or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each module may exist alone physically, or two or more modules are integrated into one unit.
Similarly, each processing unit in the embodiments of the present invention may be integrated into one functional module, or each processing unit may exist physically, or two or more processing units are integrated into one functional module.
The invention is further described with reference to the accompanying drawings and specific embodiments. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Further, it should be understood that various changes or modifications of the present invention may be made by those skilled in the art after reading the teaching of the present invention, and these equivalents also fall within the scope of the present application.

Claims (9)

1. A video understanding-based advertisement label classification method is characterized by comprising the following steps:
s1: labeling the advertisement video to finish the preparation of a data set;
s2: adopting Resnet-50 as a backbone network, inserting a time conversion model into the Resnet-50 network to construct a preview frequency classification model, and then training the preview frequency classification model by using a data set to generate a video classification model;
s3: and carrying out classification prediction on the advertisement videos to be subjected to label classification by using a video classification model to obtain content classification results of the advertisement videos, and carrying out label classification.
2. The video understanding-based advertisement tag classification method according to claim 1, wherein the tag labeling category comprises: an action class tag, a scene class tag, and an object class tag.
3. The video understanding-based advertisement tag classification method according to claim 1, wherein the data set includes: clipped datasets and non-clipped datasets; label category labeling is directly carried out on the clipped data set; the non-clipped dataset is tagged with tag categories by time period.
4. The method for classifying advertisement tags according to claim 1, wherein said step of S2 is preceded by the step of:
and performing data enhancement on the data set, wherein the data enhancement specifically comprises geometric transformation enhancement and color transformation enhancement.
5. The video understanding-based advertisement tag classification method according to claim 4, wherein the geometric transformation enhancement comprises: flipping, rotating, cropping, distorting, and scaling each frame of the ad video.
6. The video understanding-based advertisement tag classification method according to claim 4, wherein the color transformation enhancement comprises: noise transformation, blur transformation, and color transformation for each frame of the advertisement video.
7. An advertisement tag classification system based on video understanding, comprising:
the data set preparation unit is used for labeling the advertisement video to complete the preparation of the data set;
the model training unit is used for adopting Resnet-50 as a backbone network, inserting the time sequence conversion model into the Resnet-50 network to construct a pre-video classification model, and then training the pre-video classification model by using a data set to generate a video classification model;
and the model reasoning unit is used for carrying out classification prediction on the advertisement video to be subjected to label classification by using the video classification model to obtain a content classification result of the advertisement video and carry out label classification.
8. The video understanding-based advertisement tag classification system of claim 7, further comprising:
and the data set enhancement unit is used for carrying out geometric transformation enhancement and color transformation enhancement on the data set.
9. An advertisement tag classification apparatus based on video understanding, comprising:
a memory for storing a computer program;
a processor for implementing the video understanding-based advertisement tag classification method steps of any one of claims 1 to 6 when executing said computer program.
CN202011393760.5A 2020-12-03 2020-12-03 Advertisement label classification method, system and equipment based on video understanding Pending CN112580696A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011393760.5A CN112580696A (en) 2020-12-03 2020-12-03 Advertisement label classification method, system and equipment based on video understanding

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011393760.5A CN112580696A (en) 2020-12-03 2020-12-03 Advertisement label classification method, system and equipment based on video understanding

Publications (1)

Publication Number Publication Date
CN112580696A true CN112580696A (en) 2021-03-30

Family

ID=75126987

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011393760.5A Pending CN112580696A (en) 2020-12-03 2020-12-03 Advertisement label classification method, system and equipment based on video understanding

Country Status (1)

Country Link
CN (1) CN112580696A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115098725A (en) * 2022-05-27 2022-09-23 北京达佳互联信息技术有限公司 Task processing model determining method, video category determining method and device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111008280A (en) * 2019-12-04 2020-04-14 北京百度网讯科技有限公司 Video classification method, device, equipment and storage medium
CN111369299A (en) * 2020-03-11 2020-07-03 腾讯科技(深圳)有限公司 Method, device and equipment for identification and computer readable storage medium
CN111507349A (en) * 2020-04-15 2020-08-07 深源恒际科技有限公司 Dynamic data enhancement method in OCR (optical character recognition) model training
CN111523566A (en) * 2020-03-31 2020-08-11 易视腾科技股份有限公司 Target video clip positioning method and device
CN111666911A (en) * 2020-06-13 2020-09-15 天津大学 Micro-expression data expansion method and device
CN111859023A (en) * 2020-06-11 2020-10-30 中国科学院深圳先进技术研究院 Video classification method, device, equipment and computer readable storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111008280A (en) * 2019-12-04 2020-04-14 北京百度网讯科技有限公司 Video classification method, device, equipment and storage medium
CN111369299A (en) * 2020-03-11 2020-07-03 腾讯科技(深圳)有限公司 Method, device and equipment for identification and computer readable storage medium
CN111523566A (en) * 2020-03-31 2020-08-11 易视腾科技股份有限公司 Target video clip positioning method and device
CN111507349A (en) * 2020-04-15 2020-08-07 深源恒际科技有限公司 Dynamic data enhancement method in OCR (optical character recognition) model training
CN111859023A (en) * 2020-06-11 2020-10-30 中国科学院深圳先进技术研究院 Video classification method, device, equipment and computer readable storage medium
CN111666911A (en) * 2020-06-13 2020-09-15 天津大学 Micro-expression data expansion method and device

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115098725A (en) * 2022-05-27 2022-09-23 北京达佳互联信息技术有限公司 Task processing model determining method, video category determining method and device
CN115098725B (en) * 2022-05-27 2024-06-14 北京达佳互联信息技术有限公司 Task processing model determining method, video category determining method and device

Similar Documents

Publication Publication Date Title
US11483621B2 (en) Big data acquisition and analysis system using intelligent image recognition, and application method thereof
WO2020259510A1 (en) Method and apparatus for detecting information embedding region, electronic device, and storage medium
WO2018157746A1 (en) Recommendation method and apparatus for video data
CN109117777A (en) The method and apparatus for generating information
CN111523566A (en) Target video clip positioning method and device
Saba et al. Analysis of vision based systems to detect real time goal events in soccer videos
CN106611015B (en) Label processing method and device
CN113761253A (en) Video tag determination method, device, equipment and storage medium
EP3249610A1 (en) A method, an apparatus and a computer program product for video object segmentation
CN112364204A (en) Video searching method and device, computer equipment and storage medium
CN112925905B (en) Method, device, electronic equipment and storage medium for extracting video subtitles
CN114429566A (en) Image semantic understanding method, device, equipment and storage medium
Wong et al. Learning to extract and summarize hot item features from multiple auction web sites
CN112580696A (en) Advertisement label classification method, system and equipment based on video understanding
CN114064968A (en) News subtitle abstract generating method and system
CN111914850B (en) Picture feature extraction method, device, server and medium
CN116993978A (en) Small sample segmentation method, system, readable storage medium and computer device
Arif et al. Video representation by dense trajectories motion map applied to human activity recognition
CN114492313B (en) Encoder training method, resource recommendation method and device
CN117009577A (en) Video data processing method, device, equipment and readable storage medium
CN112015936B (en) Method, device, electronic equipment and medium for generating article display diagram
CN113409074A (en) Data processing method and device, electronic equipment and storage medium
CN111818364A (en) Video fusion method, system, device and medium
CN112560408A (en) Text labeling method, text labeling device, text labeling terminal and storage medium
Nimmagadda et al. Perceptual video summarization using keyframes extraction technique

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210330