Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q30/00—Commerce
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
  • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
  • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
  • G06F16/7837—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F18/00—Pattern recognition
  • G06F18/20—Analysing
  • G06F18/24—Classification techniques
  • G06F18/243—Classification techniques relating to the number of classes
  • G06F18/2431—Multiple classes
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
  • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects

Definitions

• the present invention relates to the field of image processing, and in particular, to a personalized advertisement pushing method and system based on user interest learning. Background technique
• Peripheral related ads As shown in Figure 1 (b), when a video is played, a predefined advertisement is displayed on the periphery of the video player (such as a web page, a player border).
• the above three methods of advertising push have been widely used.
• the effect of these advertisements is not ideal.
• the first method when the advertisement is played, the user often browses other webpages and reduces the advertisement effect; the second method has less interference, but the advertisement is often used as the webpage background. Neglect;
• the third method affects the user's normal viewing experience to a certain extent.
• the main problem is that the above-mentioned push advertisements are generally associated with content, and cannot satisfy the personalized interest needs of each user, so the effect achieved by the advertisement is poor. Summary of the invention
• the embodiment of the invention provides a personalized advertisement pushing method and system based on user interest learning, which is used to solve the problem that the existing push advertising has low relevance to the content, and cannot satisfy each user's personalized promotion. interesting needs.
• the embodiment of the invention provides a personalized advertisement pushing method based on user interest learning, which comprises:
• Extracting an object of interest in the video according to the user interest model Extracting an object of interest in the video according to the user interest model
• a plurality of visual features of the object of interest are extracted, and relevant advertising information is retrieved in the advertisement library based on the visual features.
• the embodiment of the invention further provides a personalized advertisement pushing system based on user interest learning, which comprises:
• a interest model learning module configured to learn a plurality of user interest models by multi-task sorting
• an interest object extracting module configured to extract an object of interest in the video according to the user interest model
• an advertisement retrieval module configured to extract the interest A plurality of visual features of the object, the relevant advertising information being retrieved in the advertisement library based on the visual features.
• the embodiment of the present invention obtains the user interest model by using the multi-task sorting learning algorithm, and on this basis, automatically extracts the interest regions in the video for different users, and then uses the interest region to associate the advertisement information.
• the advertisements provided in this way are not only closely related to the video content, but also satisfy the individualization requirements of the users to a certain extent, and realize personalized advertisement push. Close
• FIG. 1 is a schematic diagram of an existing advertisement pushing method
• FIG. 2 is a schematic flowchart of a personalized advertisement pushing method based on user interest learning according to an embodiment of the present invention
• FIG. 3 is a schematic flowchart of a user interest model learning process according to an embodiment of the present invention.
• FIG. 4 is a schematic diagram of a process for extracting a video interest object according to an embodiment of the present invention.
• FIG. 5 is a schematic diagram of a key frame acquired in a video object extraction process according to an embodiment of the present invention. Schematic diagram of the distribution of interest;
• FIG. 6 is a schematic diagram of a personalized advertisement push system based on user interest learning according to an embodiment of the present invention.
• FIG. 7 is a schematic diagram of data flow based on the system shown in FIG. 6 according to an embodiment of the present invention
• FIG. 8 is a schematic diagram of an advertisement push effect generated by the method and system according to an embodiment of the present invention.
• the system firstly uses the multi-task sorting learning algorithm to train the user interest model according to different pre-collected scene collections and user feedback on the scenes, and simultaneously obtains the scene classification and User classification. It includes the underlying visual feature extraction of the scene, the initial random classification of the scene and the user, and the calculation of the interest model parameters. Then, the system detects the key frame while playing the video, classifies the scene corresponding to the key frame according to the scene category obtained in the model learning process, and then calculates the interest degree map by the interest model of each user.
• an area with high interest is generated from the interest degree map as the object of interest, and relevant advertisements are retrieved in the advertisement information base according to various characteristics thereof, and finally the video stream with personalized advertisement is output.
• the feature of the object of interest reflects the visual characteristics of different levels of the object at different levels, including but not limited to color, structure, contour, texture feature.
• the HSV color histogram, Gabor histogram, and the Gabor histogram of the object of interest are extracted in the embodiment of the present invention. SIFT histogram, visual pattern features.
• the simultaneous retrieval method is a fast matching algorithm, and different matching methods are adopted for different features.
• FIG. 2 is a schematic flowchart of a personalized advertisement pushing method based on user interest learning according to an embodiment of the present invention. As shown in FIG. 2, this embodiment may include the following steps:
• Step 201 Interest model learning: Multiple user interest models are obtained through a multi-task sorting learning algorithm.
• the step further includes:
• Step 2011 Get various scenarios and user feedback on each scene.
• the scene may include topics in various aspects, such as advertisements, news, cartoons, movies, and the like. Users can mark objects of interest in these scenes with a simple interaction. Since the points of interest of different users are different even in the same scene, the field can be characterized as follows Scene collections, user collections, and the connections between them:
• Step 2012 Extracting the underlying visual features of each macroblock in each scene from both local and global aspects. Specifically, each scene is divided into a set of macroblocks, and the underlying visual features of each macroblock are calculated. The underlying visual features of the entire scene can be obtained by combining the underlying visual features of the macroblock. In one embodiment, the scene can be divided into 16 by 16 size macroblocks and local contrast features in the multi-scale multi-vision channel are extracted as the underlying visual features of each macroblock.
• Xk2, ... Xk represents a collection of feature vectors for all of its macroblocks.
• the feature vector V k of the entire scene and the feature vector of the object of interest can be obtained by some combination transformation.
• v k is defined as the expected and standard deviation of the feature vectors of each macroblock of the scene, ⁇ Defined as the expected and standard deviation of the feature vectors for each macroblock within the object of interest.
• the preliminary classification can be performed by randomly dividing a scene and user clustering.
• Another intuitive method is to classify the scene and the user separately according to the similarity of the scene content and the similarity of the user's interest feedback.
• similar to the content of the scene by scene feature ho v k in step 302 to obtain the calculated affinity and the user selected object of interest characterized by a ⁇ is calculated for each user.
• Step 2014 Initialize the user interest model based on the results of the two classifications.
• initializing the user interest model requires first constructing an objective function to be optimized.
• the objective function is divided into two parts. In this embodiment, it is formalized as:
• W is the interest model parameter set
• / is the scene classification set
• J is the user classification set
• L( , ⁇ , ⁇ ) is the empirical loss
• ⁇ ( ⁇ , ⁇ ) it is divided into four types of penalty items based on prior knowledge set.
• ⁇ ( , ⁇ , ⁇ ) ⁇ 8 ⁇ s + e u Q u + s d Q d + ⁇ 0 ⁇ c
• Q s refers to the scene clustering penalty, which mainly calculates the difference between the feature vector Vk between the scene and the scene. When the two scenes have the same content but are in different scene classifications, the penalty value is too large.
• the scene clustering penalty as:
• A ⁇ ( - ) 2 [cos(v )] + where cos(vv ti ) represents the scene feature vector ⁇ .
• the cosine distance between and the v is ⁇ , [x]+ is not max(0, x).
• Q u refers to the user clustering penalty, which mainly calculates the difference of the feature ⁇ of the object of interest selected by the user. When the users of the same preference are classified into different categories, the penalty value becomes larger.
• the user clustering penalty as:
• T s is a predefined threshold
• Mi is a constant used to ⁇ ⁇ . folk ⁇ 3—is in the range [0,1].
• Q d refers to the model difference penalty, which mainly calculates the prediction loss of different models under different conditions, and encourages different categories of user models to give different predictions. This is because for the same category of user models, the predictions are different under different scenario categories.
• we define the user clustering penalty as:
• model complexity penalty which is obtained by calculating the sum of the model parameter moduli.
• the model complexity penalty is defined as: In the model update process, we can use this penalty to control the number of users and scene classifications, which can avoid overly complex models.
• Step 2015 Update the scene classification and user classification in turn based on the obtained user interest model.
• Step 2016, training again to get a new user interest model based on the new scenario and the user's classification results.
• Step 2017, determine whether the number of predefined iterations is reached or the objective function is small enough to be a certain value? If yes, proceed to step 2018; if no, return to step 2015.
• Step 2018 The user interest model obtained in the last iteration, and the category of the scene and the user are used as the final user interest model, and the scene and user classification.
• the basis for calculating the interest model in the beginning of 2014 is to minimize the loss of experience.
• the update scenario and user classification in step 2015 are performed based on the obtained user interest model.
• the scene cluster update may predict the error according to the reduced model and improve the content similarity between the scenes
• the user cluster update may be based on Know the interest model and improve the similarity of preferences between users.
• the new user interest model is again calculated from the newly obtained two categories, and the iterative update steps are repeated until the defined conditions are met (the defined number of times or the objective function value is small to a certain extent).
• Step 2018 After that, the obtained scene and user classification and user interest model are used as the basis for extracting the object of interest in future multitasking.
• Step 202 Interest object extraction: According to the user interest model, the object of interest is extracted in the video. Wherein, as shown in FIG. 4, the step further includes:
• Step 2011 A representative key frame in the video stream is detected as a key scenario. Specifically, by calculating the similarity between all frames in a video shot, and looking for the frame with the highest similarity to other frames, it is used as a representative key frame.
• Step 2022 Extract bottom layer visual features of each macro block of the current scene and calculate an overall bottom visual feature of the scene. Specifically, using the same underlying visual features as the interest model learning process, the underlying visual features of each macroblock of the current scene are extracted first, and then the overall underlying visual features of the scene are calculated. In this embodiment, the expected and standard variance of each macroblock feature is taken as the overall feature of the scene.
• Step 2023 classifying the scene according to the overall underlying visual feature. Specifically, the overall underlying visual feature obtained in step 2022 is used as a basis for classifying the scene, and the closest one of the known scene classes is selected. Preferably, a support vector machine can be trained to perform this sorting work.
• the degree of interest of each macroblock in the scene can be sorted by using the known user interest model.
• Step 2024 Sort interest levels of each macro block of the scene according to a user interest model.
• Step 2025 Map the ranking result to the degree of interest of each candidate block to obtain a scene interest degree distribution.
• the results of the sorting in step 2024 are mapped to be converted into a range of values that facilitates the representation of the interest graph, such as a decimal between [0, 1].
• a fourth power function with a range of [0, 1 ] is taken (3 ⁇ 4 ⁇ to perform this mapping work.
• Step 526 Select a candidate block with the highest global interest degree.
• Step 2027 use a region growing algorithm to generate a region with a high degree of interest. It should be noted that a region with a high degree of interest is generated in this step.
• the algorithm is not limited to the region growing algorithm, but may be other algorithms.
• Step 2028 Obtain an object of interest from the region of interest.
• objects of interest to the user are extracted from the video.
• the personalized advertisement pushing method based on user interest learning obtained by the embodiment of the invention obtains the user interest model by using the multi-task sorting learning algorithm, and automatically extracts the interest area in the video for different users, and then uses the interest area to advertise Information association.
• the advertisements provided in this way are not only closely related to the video content, but also satisfy the user's preferences to a certain extent, and realize personalized advertisement push.
• the embodiment of the present invention further provides a personalized advertisement pushing system based on user interest learning, including: an interest model learning module 61, an interest object extraction module 62, and an advertisement retrieval module 63.
• the interest model learning module 61 is configured to learn a plurality of user interest models by multi-task sorting; the interest object extraction module 62 is configured to extract an interest object in the video according to the user interest model; the advertisement retrieval module 63 is configured to extract the Describe various visual features of the object of interest, and retrieve relevant advertising information in the advertisement library according to the visual feature.
• the interest model learning module 61 may further include the following sub-modules:
• a feature extraction sub-module 611 configured to acquire various scenarios in the training data, and extract an underlying visual feature of each macroblock in each scenario;
• the initialization sub-module 612 is configured to randomly combine the user and the scene into multiple categories according to the underlying visual feature, and initialize an interest model for each type of user on each type of scene;
• the optimization sub-module 613 is configured to establish a loss function on the training set by using the initialized interest model as an optimization target, minimize the loss function by an optimization algorithm, and then update each interest model parameter value to optimize the user and the scene. Clustering division;
• the result acquisition sub-module 614 is configured to obtain a final user and scene cluster and a plurality of user interest models.
• the object of interest extraction module 62 may further include the following sub-modules:
• a key frame detection sub-module 621 configured to receive an input video stream, and detect a representative key frame in the video stream
• a feature calculation sub-module 622 configured to calculate, according to an underlying visual feature of the macroblock, an overall visual feature of the scene corresponding to the key frame for each key frame;
• a scene categorization sub-module 623 configured to classify, according to the overall visual feature, a scene corresponding to the key frame into one of a scene category divided in a process of constructing a user interest model;
• the interest degree calculation sub-module 624 is configured to calculate an interest degree distribution map of the scene where the key frame is located according to the user interest model;
• the region growing sub-module 625 is configured to obtain an area of highest interest on the interest degree distribution map by using a region growing algorithm as an object of interest.
• FIG. 7 is a schematic diagram of the flow of data between modules in a personalized advertisement push system based on user interest learning according to an embodiment of the present invention, to further illustrate a personalized advertisement push system based on user interest learning provided by an embodiment of the present invention.
• the connection relationship of each module As shown in Figure 7:
• the predefined scene set and the user interest feedback data stream first enter the feature extraction sub-module 611, and the feature extraction sub-module 611 inputs the extracted underlying visual feature and the user interest feedback into the initialization sub-module. 612.
• the initialization sub-module 612 randomly classifies the user and the scene, and initializes the initial user interest model according to the classification result, obtains a preliminary random scene and a user classification and interest model, and then sends the results to the optimization sub-module 613 through an iterative algorithm.
• the result obtaining sub-module 614 obtains the final user classification result and the scene classification result from the last iteration result, and corresponding The user interest model; in the personalized advertisement push process, the key frame detection sub-module 621 receives the input video stream, detects a key frame whose content is representative, and outputs the key frame to the feature calculation sub-module 622 for the key frame corresponding scene.
• the calculated information data stream accompanying the information provided by the result obtaining sub-module 614 flows through the scene classification sub-module 623 and the interest degree calculation sub-module 624, respectively, to generate an interest degree distribution map of the scene in which the key frame is located, and then, the region grower
• the module 625 outputs the object of interest to the advertisement retrieval module 63 according to the interest degree profile to extract a plurality of visual features and retrieve the advertisement information library, and finally output the video stream with the personalized advertisement.
• the personalized advertisement pushing system based on user interest learning obtained by the embodiment of the present invention obtains the user interest model by using the multi-task sorting learning algorithm, and automatically extracts the interest area in the video for different users, and then uses the interest area for advertising. Information association.
• the advertisements provided in this way are not only closely related to the video content, but also satisfy the user's preferences to a certain extent, and realize personalized advertisement push.
• the advertisement push result generated by the method and system provided by the embodiment of the present invention is as shown in FIG. 8.
• a person skilled in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by using hardware related to program instructions, and the foregoing program may be stored in a computer readable storage medium, when executed, The steps of the foregoing method embodiments are performed; and the foregoing storage medium includes: various media that can store program codes, such as ROM, RAM, disk or optical disk.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Physics & Mathematics (AREA)
• Multimedia (AREA)
• Data Mining & Analysis (AREA)
• Evolutionary Computation (AREA)
• General Engineering & Computer Science (AREA)
• Databases & Information Systems (AREA)
• Business, Economics & Management (AREA)
• Library & Information Science (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Artificial Intelligence (AREA)
• General Health & Medical Sciences (AREA)
• Medical Informatics (AREA)
• Accounting & Taxation (AREA)
• Development Economics (AREA)
• Economics (AREA)
• Finance (AREA)
• Marketing (AREA)
• Strategic Management (AREA)
• General Business, Economics & Management (AREA)
• Software Systems (AREA)
• Life Sciences & Earth Sciences (AREA)
• Bioinformatics & Cheminformatics (AREA)
• Bioinformatics & Computational Biology (AREA)
• Evolutionary Biology (AREA)
• Health & Medical Sciences (AREA)
• Computing Systems (AREA)
• Image Analysis (AREA)
• Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Priority Applications (4)

Applications Claiming Priority (1)

Related Child Applications (1)

Publications (1)

Family

ID=45484999

Family Applications (1)

Country Status (4)

Cited By (5)

Families Citing this family (41)

Citations (3)

Family Cites Families (10)

• 2010
  • 2010-11-29 WO PCT/CN2010/079245 patent/WO2012071696A1/zh not_active Ceased
  • 2010-11-29 CN CN2010800065025A patent/CN102334118B/zh not_active Expired - Fee Related
  • 2010-11-29 EP EP10860233.5A patent/EP2568429A4/en not_active Withdrawn
• 2012
  • 2012-12-10 US US13/709,795 patent/US8750602B2/en active Active

Patent Citations (3)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events