JP5367872B2 - How to provide users with selected content items - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method of providing selected content items to a user. <P>SOLUTION: The selection of content items is based on metadata pre-assigned to content items, typically authored content metadata, and on metadata generated and associated afterwards, called derived content metadata; additionally, the selection of content items can be based also on context metadata, particularly derived context metadata. Derived metadata are automatically generated on the basis of derivation rules corresponding to algorithms to be applied to e.g. the content of content items, authored content metadata and context metadata. User profiles can be used for improving the selection quality; a method is also disclosed for building and maintaining user profiles based on machine learning techniques. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to a method of providing a selected content item to a user. The present invention is used for modern content-based services provided via telecommunications means such as a data network (eg Internet) or a telephone network (eg UMTS network).

The present invention relates to the fields of content filtering, information retrieval, service personalization and user profiling.
The present invention is particularly effective when applied to a rich multimedia content item, that is, a content item that includes various media content components (text, image, audio, video,...).

Every day, a lot of information is released all over the world, and it is informed to people through various information media such as newspapers, TV and the Internet. The amount of information is increasing rapidly.
Unfortunately, the vast amount of information provided by currently available sources can often be overwhelming for an individual, making it impossible to look up information about items that interest him or It may be lost. Therefore, what is needed is a service or ability to provide the user only with information that may be of interest to the user.

  For a long time it has been common to filter content items based on "keywords". Keywords are given to the software application by the user. The software application may be local to the user's computer or may be executed on a remote computer connected to the user's computer, for example via the Internet. The software application returns to the user all available content items associated with the keyword specified by the user.

  Keywords are a very common type of metadata. Conventionally, “metadata” has been defined as “information about information”. For example, “title” and “summary” of “article” are metadata because they provide information about the content of the article, which is information itself.

  More sophisticated methods have been developed for filtering content information. These methods are metadata for indexing various content items, especially “authored metadata” (ie, metadata associated with a content item by the author of the content item or by another person). Based on using.

  Effective filtering of content items often requires knowledge about the user, such as knowledge of user habits and / or user preferences.

  An interesting overview of filtering documents using metadata and user profiles is the paper by Erika Savia et al., “Metadata Based Matching of Documents and User Profiles”, Proc. 8th Finnish Artificial Intelligence Conference, Human and Artificial Information Processing, pages 61-69, 1998.

  A method for classifying and transmitting multimedia data is known from the international patent application WO 02/41579. Multimedia data is analyzed for their content, corresponding metadata is extracted by a metadata extraction module, and a user profile is created. Prior to receiving multimedia data from the central device, the user sets and / or modifies at least a portion of the user data of the user profile via the communication device. Multimedia data is selected by the metadata based on the user profile, from which the content-oriented multimedia data optimized for the user is generated by the repackaging module. The content-oriented multimedia data optimized for the user is stored in the content module database of the central device and provided to the user.

According to this international patent application (page 8, lines 6-14), content-based indexing techniques (eg those described in US Pat. Nos. 5,210,868 and 5,414,644) Metadata is searched based on.
International patent application WO02 / 41579 US Pat. No. 5,210,868 US Pat. No. 5,414,644 A paper by Erika Savia et al., “Metadata Based Matching of Documents and User Profiles”, Proc. 8th Finnish Artificial Intelligence Conference, Human and Artificial Information Processing, pages 61-69, 1998

  The present invention relates to a method for providing a content item to a user, and more particularly to providing a content item selected by considering user preferences.

  The basic idea behind the present invention is to automatically generate metadata and use these generated metadata to select content items to be provided.

  Applicants believe that manual generation of metadata, generally generated metadata (hereinafter also simply referred to as “creation metadata”), is a very time-consuming task (an increasing amount of publication) I can't keep up with the information), and I found it easy to make mistakes and not generally service oriented. Therefore, it is practically very difficult to reach a level of accuracy sufficient to filter a vast amount of public information.

Applicants have realized that it is important to have accurate user profiles, be able to build them and keep them updated automatically.
In addition, Applicants have found that the interaction context when providing content can be advantageously used to build or update a user profile.

  The method of the present invention can be provided by a service provider that provides a service for delivering personalized content to users. The above considerations also apply when content items are provided in both PULL and PUSH modes.

According to the present invention, content and preferably context metadata is automatically generated. Derived metadata (hereinafter also simply referred to as “derived metadata”) is automatically generated based on derivation rules corresponding to algorithms applied to the content items, content metadata created, and raw context metadata, for example. Generated automatically.
The above features provide flexibility and power to the selection method of the present invention.

  In addition to explicit and / or implicit user feedback, derived metadata can also be used advantageously to build and maintain user profiles. In this way, the user profile is constructed accurately and can be maintained accurately for a long time. Preferably, the construction and maintenance (ie, updating) of the user profile is performed using machine learning techniques.

  The present invention includes a first aspect related to processing a content item. Content processing basically corresponds to the appropriate selection of content items. In a preferred embodiment, the present invention also relates to a second aspect related to the processing of user profiles, which basically corresponds to the proper construction and maintenance of user profiles.

  Accurate content selection can be performed based on user profile matching, i.e., because the content items are matched against the user profile, the two aspects are related to each other. User feedback (explicit and / or implicit) about the selected content is advantageously used to build and maintain the user profile.

According to the first aspect of the present invention, when a request for content is received from a user, the following steps are performed.
-Generating a query starting from the above request;
-Identifying a first set of content items based on this query;
-Identifying pre-assigned content metadata (generally created metadata) that can be associated with each content item of this first set;
Preferably identifying raw context metadata representing context information associated with the request; and- pre-assigned content metadata applied to the content item and preferably associated with the content item (if any). Automatically generating derived metadata for each content item based on derivation rules corresponding to the algorithm applied to More preferably, the algorithm is also applied to raw context metadata.

  Preferably, after the derived metadata is generated, the first set of content items is stored in the content item repository. More preferably, derived metadata generated after the user's request and associated with the content item is also stored in the content item repository for future use to avoid repeating the metadata derivation process.

Once the derived metadata associated with each content item in the first set has been generated, the second set of selected content items is provided to the user. The second set is included in the first set of content items. According to a preferred aspect of the invention, the second set of content items is provided by performing the following steps.
-Identifying content metadata (derived and pre-assigned) for each content item;
-Preferably identifying context metadata (derived and raw) for each content item;
-Identifying the user profile of the user who generated the request;
-Matching said first set of content items against a user profile based on at least some of the derived metadata and creating a ranking for this set of content items;
-Providing the user with a second set of content items based on the ranking, wherein the second set corresponds to the first set ordered based on the ranking, or (preferably ) Corresponding to a subset of content items containing the highest ranked content item,
-Preferably collecting user feedback (explicit and / or implicit) about the provided content item.

Each content item belonging to the second set with associated user feedback corresponds to an interaction event, which is preferably stored as a record in the interaction history repository. Thus, according to a preferred embodiment of the present invention, the following steps are performed to update the user's profile.
-Searching multiple interaction history records for the user making the request. Here, each record includes a content item and at least user feedback (explicit and / or implicit) associated with the content item and generally indicated by a user vote. Preferably, the record also includes user requests, raw context metadata, and content metadata associated with the content item (derived and pre-assigned);
-Selecting a machine learning algorithm to build a prediction model for this user;
-Encoding each record (ie, each stored interaction event) as a feature vector. This feature vector is a formal representation adapted for use with a selected machine learning algorithm. The feature vector includes a plurality of elements corresponding to metadata derived in connection with a particular content item and (possibly) pre-assigned metadata, and includes user feedback. If the derived metadata is not present in the record, the derived metadata can be retrieved from a content item repository that stores all of the first set of content items and generally the content item selected after the query;
Building a predictive model (user model) by applying selected machine learning algorithms to the feature vectors (each vector corresponding to an interaction event);
-(More preferably) checking the constructed prediction model; and-updating the user's profile by replacing the old prediction model with the new prediction model.

  Thus, according to a preferred aspect of the present invention, a machine learning method is provided for derived metadata and preferably pre-assigned metadata by considering user feedback to establish a ranking within the first set of content items. Apply to provide a second set of content items. More preferably, the context metadata (raw and derived) is considered as an independent feature in the feature vector to which the machine learning method is applied. If derivation context metadata is not present in the interaction history repository record, derivation of metadata from raw context metadata is generally not computationally heavy, so derivation context metadata can be quickly derived from raw context metadata. Can be derived.

  The invention will become more apparent from the following description when read in conjunction with the accompanying drawings.

FIG. 2 is a block diagram of a system that performs an aspect of the method of the present invention 6 is a flowchart of the main steps when processing a content item according to the present invention. 6 is a flowchart of main steps in processing a user profile according to the present invention; 1 schematically illustrates a data structure that is adapted to store content items and associated metadata and that can be used in the present invention.

DETAILED DESCRIPTION OF THE INVENTION Before proceeding to a detailed description of the present invention, some terms are defined and explained below.
Content-based service As used herein, a content-based service is a software application that uses a set of existing content items to build information content that is valuable to subscribers of the service. The method of collecting the selected content and giving it to the user is determined by the service application logic.

  Users can interact with content-based services in two modes (PULL or PUSH) via the service front end. A content-based service may provide one or both modes. In PULL mode, the user directly accesses the service front end and in some cases initiates the interaction by making certain inputs and immediately obtains the desired content. In the PUSH mode, when a user subscribes to a content-based service (possibly afterwards), the user can make input that can later generate service content. Based on these inputs, when this content is generated, the user is notified to access the service front end to obtain the content.

Content Item In this technical field, a content item is the basic unit of user content interaction for content-based services. In general, content-based services provide content items in response to requests from service subscribers, ie users. The content item is recognized by the user as one entity distributed by the service. However, a content item can also include one or more content components. For example, if the content item is a video of a soccer game, the content item can include the first half of the game (two content components) as a content component.

Examples of content items are as follows.
-Movies or TV programs from, for example, an on-demand media distribution environment;
− News articles from online news readers, for example;
-One web URL result, eg from a web search engine;
-Songs from content sharing network environments;
-Photos from eg online media catalogs;
-Web pages from internet navigation, for example;
-Product pages from e-commerce catalogs, for example.

In general, a content item is a structured object that includes one or more content components.
Each content component is a multimedia element and can be, for example, text, image, audio, video, 3D model, vector graphics, graphical layout.

  Examples of text components include online newspaper article text, text sections of news articles, text contained in web pages, and text descriptions for products in an e-commerce catalog. Examples of image components include photos and diagrams included in web pages, photos included in news pages, and photos included in online media catalogs. Examples of audio components include audio files that contain songs in an on-demand media distribution environment, audio files that contain songs in a content sharing environment, audio tracks in movies, and audio tracks in news articles. . Examples of video components include files containing movies or TV programs in an on-demand media distribution environment, files containing video in a content sharing environment, and video portions of news pages. An example of a three-dimensional model component is a 3D model that represents a piece of furniture on an online e-commerce catalogue. Examples of vector graphics components include flash animations in web pages, SVG [Scalable Vector Graphics] documents. An example of a graphical layout is a graphical layout of a web page.

  Examples of multi-component content items include, for example, a text component (ie, simple text describing a news article) for an news article, an audio-video component (ie, an audio-video sequence that describes and shows the news article), and A news item consisting of an audio component (ie an audio sequence describing a news article).

Content metadata Traditionally, “metadata” was simply defined as “information about information”.
Specifically (according to the W3 [“www”] consortium), content metadata generally consists of a data structure that describes a given content item and can be automatically processed by a machine (eg, a computer).

The content metadata can describe each content component belonging to the content item or the entire content item.
Content items made available by content providers often comprise metadata such as an identification index (eg, subject, field,...). The metadata associated with a content item when the content item is available for content-based services is referred to as pre-assigned metadata. The pre-assigned metadata is typically “authored metadata” (also referred to simply as authored metadata), which is typically content by the author of the content item or another person within the content provider organization. A type of content metadata associated with an item. In general, creation metadata is manually assigned to each content item by an annotation process.

  There are various types of metadata, such as text metadata, keyword metadata, category metadata (category labels with one value from a limited set of values), numeric metadata, and so on. The metadata can have a more complex structure, e.g. derived from the previous kind of configuration (structured metadata) or a semantic network (semantic network metadata) according to e.g. RDF [Resource Description Framework] It can have a more complex structure corresponding to

  Examples of text metadata include a text description associated with a picture in a web page (multiple creation metadata may be provided for the content of each photo on the page), a text summary of the web page content (the creation metadata as a whole) As well as the lyrics of songs related to song items. Examples of keyword metadata include a list of keywords describing topics covered by a news item, a list of keywords related to movie item characteristics (similar to MDB [Internet Movie Database]), and drawn in a photo item A list of keywords that describe the main features of the scene and subject. Examples of category metadata include a “category” label (within a predetermined set of news categories) indicating the news category of the news item, and a “genre” label (determining the music genre of the song item song) A “color” label indicating whether the movie item is “black and white” or “color”. Examples of numeric metadata include an integer corresponding to the movie item's production year, an integer corresponding to the movie item's required time (eg in minutes), and the number of currencies corresponding to the purchase price of the product item in the e-commerce catalog. It is done.

  Structured metadata can also be applied to movie items, for example. In fact, a movie has a cast that can be represented as a list of actor names including age, role in the movie, gender, and the like. A representative example of structured metadata representation is given by the MPEG-7 description standard.

Context Metadata Interaction context (simply called “context”) is an important element of user content interaction in content-based services. In fact, every user content interaction takes place within a context, which often affects user preferences, making an item interesting in a given context or uninteresting in another context .

Interaction context information can also be associated with one or more metadata (referred to as context metadata).
The interaction context is formed from various aspects. In general, the most important aspects are “date and time” (when the interaction takes place), “user location” (where the interaction takes place), “interaction device” (used by the user for the interaction) ), “Content channel” (through which interaction takes place), “environmental state” (in interaction), “physical world state” (in interaction), “user state (In the interaction).

  "User location" can be provided in several ways in several forms (eg spatial coordinates obtained from GPS systems or cellular networks, eg logical coordinates given by short-range wireless beacon systems) , Can send a metadata description of where the user is located.

  The characteristics of the “interaction device” include, for example, its mobility (ie, movable or fixed device), graphic capabilities (eg, size, resolution, number of display colors), sound capabilities (eg, number of audio channels). ), Its brand and type.

  In certain environments, such as on-demand media distribution environments, each interaction involves selecting a “content channel” such as a TV channel or movie provider.

  The “environmental state” can also be derived, for example, from setting environmental options in the interaction device. For example, the mobile phone can be set to “conference”, “work” or “home” mode, or its operation can be set to “ringing” or “silent” mode.

Information about the “physical world state” can be provided by sensors that detect temperature, lighting conditions, humidity, pressure, wind speed, for example.
Information about "user status" includes, for example, detecting acceleration of the user's body (to determine whether the user is standing, walking, running or moving his hand) or heart rate It can be provided by a sensor (to determine the user's stress / relaxation status) that detects some of the user's physiological parameters, such as blood pressure, skin conductivity.

  Context metadata directly derived from one or more physical devices is hereinafter referred to as “raw context metadata”. The physical device can be, for example, a timer, sensor, switch (hardware or software). Often, these devices are integrated into a terminal device (eg, mobile phone, personal computer, etc.) including a user interface.

Metadata Representation It is advantageous to use the same format and an easily extensible format to easily access and handle all types of metadata.

  The MPEG-7 description standard has been found to be particularly suitable for the present invention. In the aspects described later in this specification, this standard was used as a format for all metadata (both created and derived). In particular, content item metadata is organized as a list of zero or more “related materials” blocks (see FIG. 4). Each related document describes the same type of information block, which can refer to an actual content element or an XML [Extensible Markup Language] block that represents metadata. The associated attachment block is always referenced, and this associated attachment block holds the actual content (or metadata) or provides a URL [Uniform Resource Locator] where the content (or metadata) can be found. This configuration can accommodate different storage strategies while providing a centralized access point for both content components and metadata. In addition to the information contained in each related material, content metadata also includes general information groups (such as creation date) owned by any published content, or other information required by the service application logic (content Can also hold a surviving status flag that indicates whether or not it can be used.

  The following is "related material" encoded according to the MPEG-7 standard, showing some common features of the components.

  The above “related materials” refer to the following “related attachments” encoded according to the MPEG-7 standard for actual media raw data, specifically images.

  The same approach can be used to introduce derived metadata. That is, a new “related material” is added to the list, and if necessary, a new “related attachment” can be referenced to accommodate a metadata XML block that does not conform to this “related attachment” schema.

It should be noted that although the metadata representation described herein is preferred, the present invention is not so limited.
Derived Metadata Metadata other than pre-allocated metadata and raw context metadata is provided by the present invention and is referred to as “derived metadata”. This is because this metadata is derived from content information and / or context information (ie, raw context metadata) in or as an interaction event.

  In particular, the derived content metadata is derived directly from the content item (ie, from the content item content) or indirectly from the content item (eg, from content metadata created and associated with the content item). obtain. If the content item is published and later generated by a software program, the derived content metadata is not directly available.

  Similarly, derived context metadata is derived directly from the context item (ie, from the context of the interaction event) or indirectly from the context item (ie, from the raw context metadata associated with the context item). Can be derived. If the context of the interaction event is detected and later generated by a software program, the derived context metadata is not directly available.

Derived metadata can provide more complete and useful information about content and context.
Derived metadata is particularly suitable for being automatically processed by a software program.

In the following, some examples of metadata derivation are described.
An example of metadata derived from the content of a text item (ie, the text itself) is a list of words that appear in the text along with the number of occurrences of each word (text “bag-of-words”). Called expression). This metadata provides information about the overall lexical structure of the text.

  Other examples of metadata derived from the content of the text item (ie the text itself) include text metrics, ie numerical parameters calculated on the text, eg the total length of the text, the sentence or paragraph belonging to the text Average length, average nesting depth of syntactic structure, Gunning's Fog index (for example, for English text), and Gulpease index (for example, for Italian text) Etc.

Examples of metadata derived from the content of an image item (that is, the image itself) include the following.
-Luminance histogram. This is the luminous intensity distribution for the entire pixel of the digital image, giving information about the brightness and contrast of the image.
-Color histogram. This is the distribution of the primary color components (red, green, blue) for the entire pixel of the digital image and gives information about the color composition of the image.
-Spatial frequency components of the image. This is calculated, for example, by a two-dimensional Fourier transform and gives information about the presence of patterns and textures in the image.
-Geometric category metadata. This is generated, for example, by a geometric hash technique and gives information about the presence of shapes such as straight lines, arcs, ellipses, polygons in the image.
-Pattern category metadata. This is generated, for example, by a pattern recognition algorithm and gives information about the presence of unique features such as human faces, animals, plants, landscapes, buildings, flags, technical drawings, pictures, manga, etc. in the image.
-Text metadata. This is generated, for example, by optical character recognition technology and gives information about the characters, numbers and words that appear in the image.
Examples of metadata derived from the content of the sound item (that is, from the sound itself) include the following.
– Audio frequency spectral components. This is calculated, for example, by a fast Fourier transform and gives information about the nature and composition of the sound.
− Speech waveform. This gives information about sound dynamics.
-Pattern category metadata. This is generated, for example, by a pattern recognition algorithm and provides information about the presence of specific features (eg music, speech, applause, specific parts of an explosion) in the audio track.
-Text metadata. This is generated, for example, by a voice recognition technique, and the uttered word or sentence is extracted from the voice track.

The metadata can be derived from the content of the video item (ie from the video itself) by using specific analysis and algorithms.
Information about the temporal structure of the video can be obtained by the scene division analysis technique. For example, this type of analysis shows that a movie is made up of a predetermined number of scenes, the proportion being characterized by intense motion behavior and another proportion being characterized by the presence of loud music.

  A moving object recognition algorithm is used to identify a specific object that is characterized by a specific movement in the video (walking person, talking or singing person, running car, falling object, open door, etc.) ) Can give information about the presence of.

  If the video is decomposed as a series of still images, some metadata extraction techniques used for still images can be applied to the video if there is a way to average the metadata obtained for the series of images. Even applicable.

Examples of metadata derived from the 3D model include the following.
-Total area, total volume, convexity, fractal dimension;
-Pattern category metadata. This is generated, for example, by a pattern recognition algorithm and gives information about the presence of a specific 3D shape (box shape, tube shape, wheel shape, wire shape, human shape, object shape, etc.).

As described above, in general, metadata can be derived from any other metadata.
For example, starting with numeric metadata, symbolic ranges can be generated by truncation techniques that can classify numeric values, which is a way to give a more compact and semantic numeric representation.

  Metadata can also be derived using ontology. Ontology is a formalization of conceptualization using machine-readable expressions. Ontology can be used to combine taxonomy and the relationship between metadata. This allows user preference models to be built on higher order semantic categories and concepts.

  Information on ontology can be found in, for example, the W3 Consortium website (current address is “http://www.w3.org”) and the book “WordNet: An Electronic Lexical Database”, edited by Christian Felbaum, MIT Press, 1998. Can be found in May.

Two examples of metadata derived by ontology are given below.
A simple ontology of time is to classify time values into “daytime” and “nighttime”. A simple ontology for dates is to classify date values into “working days” and “weekends”. Time value metadata and date value metadata are not important in building user preference models. “Day / night” and “working day / weekend” metadata may be more effective. For example, a user preference model may indicate that a user enjoys interaction with content items related to a particular category on weekends at night and does not like it during work day or day.

  Ontology is particularly suitable for generating category metadata. For example, consider three text items A, B, and C, each containing a news article. Article A describes a computer model of the human lung and includes (among other things) the metadata words “computer” and “asthma”. Article B describes robot-assisted surgery and includes the metadata words “software” and “surgeon”. Article C describes the Internet and includes the metadata word “website”. Combining metadata words with vocabulary ontology can augment articles A and B with, for example, a summary category “medicine” that can be added to the metadata of both articles. Similarly, the metadata for all three articles A, B and C can be augmented by the summary categories “Computer Science” and “Technology”. Thus, user interest in these articles can be associated with a summary category rather than a single word.

Derivation of metadata An important aspect of the present invention is the generation (ie, derivation) of metadata. This is done based on derivation rules.

  Derivation rules correspond to algorithms applied to content items and / or context information and / or metadata, and the data to which the derivation rules are applied is commonly referred to as a source.

The derivation rule defines the algorithm by referring to the plug-in module that executes the applied algorithm and the source being processed.
In particular, the following types of sources can be provided.
・ Content components;
-Created content metadata;
Raw context metadata;
Derived metadata (ie derived from other derivation rules);
• Extended analysis.

  Derivation rules may be described, for example, by an XML [Extensible Markup Language] document that identifies the above elements (ie, the modules that execute the algorithm used as well as the required parameters and the input source used).

Some examples of derivation rules are shown below.
The first set of examples relates to news browser applications. In this application, the content item is a news article. Each content item includes two content components: a news article title and body (both in text format). The content of each content item is associated with the date, category, source, and author name of the article as created metadata. Specifically, this metadata is included in the content item.

  The first metadata derivation rule is defined as follows.

  This rule executes the algorithm given by the plug-in module named “BagOfWords” to obtain the text data that makes up the body of the news article as an input (Source), and the “Bag of Words” "(That is, a list of words appearing in the text together with the number of occurrences of each word) is generated as an output (Destination).

  The second metadata derivation rule is defined as follows.

  This rule executes the algorithm given by the plug-in module named “WordOntology” to get the text data that makes up the body of the news article as input and generate the related set of concept words as output I will tell you.

This rule performs the following steps:
The step of generating “bag of words” from the source text. The step of matching each word of the obtained “bag of words” against the “lexical ontology 1” vocabulary ontology. For each matching word, extract the following:

  -The broader word (hypernyms = 2) that is two levels above the word. Vocabulary ontology such as LexicalOntology1 is organized as a tree. Moving from the leaf of the tree toward the root means moving from a word of a specific meaning toward a word representing a more abstract concept. For a given word, the word above the word in the tree is referred to as a “broadword” (eg, “doctor”-“person”-“biology”; “doctor”-“expert”-“worker”) ). In fact, there can be more than just a tree in ontology.

The highest semantic category (topsemanticlevel = “true”) associated with the word (eg the word “doctor” belongs to the highest semantic category “medicine”).
• Counting the occurrences of extracted concepts (broadwords and highest semantic categories) and keeping only those concepts that appear more than 3 times (minoccur = 3). This stage is done to limit the number of concepts obtained and keep only the most relevant ones.

The overall result is a semantic representation of the text body of the news article, taking the form of a set of concept words from a vocabulary ontology, each having its number of occurrences.
The following third and fourth derivation rules are similar to the above first and second derivation rules, respectively, but their source is the title of the news item instead of the body. However, note that the metadata generated from the title forms a different set of metadata than the metadata generated from the body, as declared in the “destination tag”.

  The following rule generates numerical metadata (represented as the total number of words in the body) corresponding to the length of the body of the news by executing the algorithm provided by the plug-in module “TextMetrics”.

  The following rule generates numeric metadata corresponding to the average sentence length in the body of a news article.

  The following rule generates numeric metadata representing the readability estimated for the news article body.

  The next three rules generate three numeric metadata corresponding to the date, number (number, percentage, price) and number of occurrences of a person name in the news body text.

  The second set of examples pertains to a music catalog browser application. In this application, the item is a music work. Each item contains one content component, an audio file that encodes a musical work (eg, in mp3 format). Each item also includes the title, date, genre, performer name and music author name as creation metadata.

  A second set of four metadata derivation rules are defined as follows.

  The above rules generate four numerical metadata, each of which is "beat speed" (ie a quantitative measure of the rhythmic characteristics of a musical work) and "vocal impact" (ie (Weight of voice component), “sound brightness” (ie, a quantitative measure of sound brightness) and “volume dynamics” (ie, a quantitative measure of sound volume over time). This can be done by executing an algorithm provided by a plug-in module named “Advanced Spectral Analysis” based on a spectral analysis technique applied to the audio signal encoded in the audio file.

  The following rule generates text label metadata that represents the decade of a musical work, starting from the year of the work. This is done by a truncation technique provided by a plug-in module named “NumeralDiscretizer” (ie by decomposing the range of numeric metadata (in this case that year) into a composite description (ten years)).

  The following rules generate numeric metadata that represents a rough estimate of the popularity of the central performer of music. This multiplies the name of the central performer to the web search engine provided by the plug-in module named “SearchEngineQuery” and consequently gets an approximate number of hits (the web page containing that name) Is obtained.

Derivation rules using extended analysis sources Extended analysis provides a special kind of source that can be used by derivation rules. In order to analyze the overall structure of the domain, this special type of source specifies the application of the analysis procedure to the entire subset of items (even all items) contained in the repository. In other words, derivation rules using extended analysis are not limited to extracting metadata using only the information contained in an item, but take into account the overall structure of the item's own domain. New metadata can be generated for each item. A domain is a field of application of derivation techniques, i.e. a field that provides personalized and selected content to a user.

  The algorithm that actually performs the analysis specified by the extended analysis can be executed by a dedicated software module or a generic metadata generation module. If a dedicated module is provided, it can be a “plug-in” module that is called by the generic metadata generation module when needed.

  The following are derivation rules based on the extended analysis that affect the music catalog application described above.

  In the analysis specified as the analysis extended in the above derivation rule, the four derived numerical metadata obtained by applying the derivation rule set in the previous example are taken as input. These four numbers represent the four related audio features of each musical work.

  The expanded analysis indicates that a “cluster analysis” is performed on all items (ie, music works) included in the content repository. Cluster analysis is a well-known statistical technique used to identify groups of items (ie, clusters) characterized by similar values when the group represents an interesting regularity in the item's domain. For this music domain, the resulting cluster may categorize music works that have similar audio features, such as proximity to the same music genre.

The algorithm that actually performs the analysis is provided by a plug-in module named “Numerical Cluster Analysis”.
In particular, the expanded analysis in this example shows the cluster analysis method applied (Ward's method), the range of items to which the analysis applies (all items in the repository), and the maximum number of clusters to extract (10 Number of clusters).

  This analysis allows the derivation rules to generate metadata for a given item (musical work) that indicates which cluster is closest to that item. This new metadata identifies the position of the musical work in the “music landscape” represented by the cluster.

Machine Learning Method The machine learning method allows a computer system to perform automatic (ie, by software program) learning from a set of factual data belonging to a particular application field (ie, domain). Given such a data set, machine learning methods can extract patterns and relationships from the data itself.

  The learned patterns and relationships are encoded into a formal and quantitative model by machine learning methods. This model can take various forms depending on the machine learning technique used. Examples of model formats include logical rules, mathematical formulas, and mathematical graphs.

The purpose of the machine learning method is to better understand and quantify the patterns in the data and the relationships between the data in order to obtain a model as a representation of the data.
Most machine learning methods use feature vector representation. When these methods are applied to build a user-preferred predictive model related to the content provided, each feature vector is associated with a content item, including:
-Independent features. Each feature corresponds to derived or pre-assigned metadata associated with the content item, and preferably also corresponds to raw or derived context metadata.
-Target feature (s). This is represented by a score given by the user as feedback (explicit or implicit) regarding the content provided. For example, the feedback is represented by a numerical value of 1 to 10, where a large value corresponds to a positive feedback.

  Each instance of the data set is then represented as a feature vector. For one target feature, the vector representing the instance is “n + 1” dimensional and takes the form:

<Feature 1, Feature 2, ..., Feature n, Target Feature>
A feature vector model is a formal representation of domain data and is suitable for most machine learning methods.

Many discussions about machine learning methods and their applications are described in Tom Mitchell, “Machine Learning”, McGraw-Hill, 1997.
Preferably, the data set (processed by machine learning methods to build a predictive model of the user profile) is composed of content metadata (created and derived) and preferably context metadata (raw And derived ones) as independent features. User feedback is a target feature. The purpose of the machine learning method is to find a model that predicts user preferences (referred to as a user model or predictive model), that is, a machine learning model that represents the relationship between metadata and user feedback. The prediction model thus obtained can then be used to estimate the user's rating for the new content item (providing new metadata) when the new content item becomes available.

An instance of a data set represented by a feature vector corresponds to one interaction event that indicates a user's preference for a content item and takes the following form:
<Content metadata 1,..., Content metadata m, context metadata 1,..., Context metadata p, user vote>
Here, m + p = n.

  If the user indicates a preference for multiple content items, multiple feature vectors are generated, which are formally represented by a matrix (n + 1) × q, where q is the number of interaction events. . For example, if the user indicates a preference for 10 content items, a matrix (n + 1) × 10 is generated. The selected machine learning algorithm is then applied to the matrix.

  Several well-known machine learning methods are useful for this purpose, including decision trees, association rules, neural networks and Bayesian methods, as well as methods designed specifically for building user preference models.

An example of the use of a machine learning method in building a user profile is described below with respect to the music catalog application described above.
In this simple example, a content item (music work) is represented by the following two metadata.
-MusicGenre: Music genre (given as created metadata),
MusicBeatSpeed: Beats per minute of the music work (given as derived metadata by application of the “MusicBeatSpeed” derivation rule).

The interaction context is represented by the following (derived) context metadata.
“Time”: can have either of the two values “daytime” and “nighttime”, indicating whether the user interaction with the content item took place in the daytime or at night (simple temporary (Provided as derived metadata by applying derivation rules based on general ontology).

User preferences are given by the following features.
“UserVote”: can have one of the two values “like” and “dislike”, indicating whether the user gave a positive score or a negative score for the music work.

Thus, for a normal machine learning expression of the domain, a single event of a user indicating a preference for a music work takes the following vector format.
<MusicGenre, MusicBeatSpeed, Time, UserVote>
As an example, a user / item interaction data set is given as follows.

When the decision tree machine learning method is applied to the data set, a user preference model including the following rules is generated.
If Time = “Daytime” and “MusicBeatSpeed> = 125, UserVote =“ Like ”
If Time = “Night” and “MusicBeatSpeed <= 75, UserVote =“ Like ”
Note that the user preference rules used to generate the prediction score are not derivation rules used to generate the derived metadata.

  The above simple predictive model represented by the user preference rule indicates that this particular user prefers very fast music (MusicBeatSpeed> = 125) during the day and quieter music (MusicBeatSpeed <= 75) at night. Is shown.

Note that this model is valid for the majority of the dataset cases (12 out of 13), but not for all.
Detailed Description of Embodiments Preferred embodiments of the present invention will now be described in detail with particular reference to the block diagram (service application) of FIG. In FIG. 1, two types of symbols are used, a square represents a software module, and a cylindrical shape represents a reservoir.

This embodiment illustrates a service provider that provides users with content-based services. This service may be PULL type or PUSH type or both.
Content-based services provide selected content items to users. For example, by providing an address (eg, an Internet address) where the content item is located or accessible to the content item, the content item can be delivered directly or indirectly to the user by the service provider. In general, content items are directly provided by a content provider via a communication network such as a packet data network (for example, the Internet) or a mobile phone network (for example, a UMTS network).

In content-based services, user profiles are built and maintained to provide a better selection of content items.
The content-based service can be divided into the following two tasks.
-Processing of content items,
-User profile processing.

Content Item Processing The processing of content items consists of the following operations.
− Receive a request from the user,
− Select content items based on user requests,
-Preferably, based on the user's presentation profile, the selected content item is formatted and presented to the user (ie, personalization of the content presentation based on service application logic);
− Provide the user with the selected content item.

  In PULL mode, as soon as a user request is received, the service provider identifies the set of content items based on the request and then performs the above steps on this set of content items. This usually means that a particular selected content item is provided in response to the user shortly after the request from the user.

  In PUSH mode, a service provider receives a request from a user and stores the request without an immediate response and generally without immediate processing, after which there are two possibilities. According to a first possibility, after the service provider periodically identifies all newly published content items, the above steps are performed on all newly published content items. According to a second possibility, as soon as one content item is published, the service provider identifies that content item and then performs the above steps on the newly published content item. In the PUSH mode, the content item can be provided in the following two stages. That is, first as soon as the service provider simply notifies the user that certain content items of interest to the user are available and then indicates that the user wants to receive those content items, the service provider Send these content items (directly or indirectly). However, the user can also indicate that he wants to receive only some of these content items.

  Any input by the user is received and processed by a service front-end module (SFEM), for example a PC or mobile terminal. User requests are sent to a service application logic module (SALM) that incorporates logic specific to the content-based service provided. In addition, module SFEM sends raw context metadata (eg, date and time, user location, etc.) associated with the user request to module SALM.

  When the module SALM receives the user request from the module SFEM, the module SALM generates a corresponding content query (step 201 in FIG. 2). Based on this content query (derived from the user request) and its service logic, the module SALM identifies a first set of content items in the content item repository CIR (step 202 in FIG. 2). For example, following the content query, the service application logic portion identifies a first set of content items in the repository CIR related to movies and TV series.

  The repository CIR stores content items as well as content metadata (generally created metadata) pre-assigned to stored content items. As will be described in detail later, the repository CIR can also store derived content metadata associated with the content item (eg, obtained from previous metadata generation caused by the previous content query).

  Even if there is user feedback applied to the first set of content items, the service application may request the user, such as for example explicit preference ("I like" or "I don't like"). Since the amount of input is generally limited, concerns about usefulness can arise. Also, in many cases, such accurate input formatting cannot be performed.

  For the ultimate goal of avoiding poor filtering and maintaining accurate content for the user, the module SALM is designed to create a ranking of the first set of content items based on the user profile. (Step 203 in FIG. 2). The ranking received by module SALM from module MMM can be used by module SALM to select the highest content item and reorder the maintained content items, except for low score content items. This can be done by known methods.

  Accordingly, the module SALM filters the first set of content items with the assistance of the match maker module MMM (step 204 in FIG. 2), and the second in the identified first set of content items. Select a set of content items. As will be explained later, the module MMM is an important element of the filtering operation and is responsible for considering the user's profile (ie the user profile). The second set of content items obtained by the MMM filtering operation is preferably a subset of the first set of content items, but the second set of content items includes all of the first set of content items. Is not excluded. However, it is ordered based on ranking preferences so that the user can view the items based on the rank associated with the content item.

  Module SALM collects, transforms, and formats the content items to present to the user a subset of content items (step 205 of FIG. 2). Instead of presenting the selected content items, they can simply be notified to the user. Presentation and / or notification is performed by the module SFEM.

In accordance with a preferred embodiment of the present invention, the architecture of FIG.
-User Profile Repository (UPR),
-Metadata generation module (MGM).

  Module MGM provides a set of derivation rules to generate derivation metadata (content and / or context). The derivation rules are based on derivation algorithms. In the embodiment of FIG. 1, these algorithms are provided by a derived algorithm module (DAM) that is external to the module MGM and implemented by “plug-in” technology. This allows the algorithm called or called by the module MGM to be stored locally as well as remotely.

Module MMM retrieves:
-Retrieves the user profile of the current user from repository UPR; and-for each content item in said first set of content items, content metadata associated with it, pre-assigned (generally created Retrieve) and possibly derived (as a result of previous interaction events) from the repository CIR.

  In addition, the module MMM receives context metadata associated with the current context from other modules. Specifically, raw context metadata is received from module SFEM via module SALM, and derived context metadata is received from module MGM.

  Module MMM sends raw context metadata to module MGM, requests to generate derived metadata (at least starting with raw context metadata), and receives the generated derived context metadata. In this way, at least some context metadata may be derived on the fly, i.e. during user interaction.

  Module MMM then applies the user profile to the content metadata (pre-assigned and derived) associated with each content item in the identified first set of content items, preferably The user profile is also applied to the context metadata (raw and derived) associated with the current context. Thus, the module MMM matches the first set of content items against the user profile. In this embodiment, the user profile includes at least a predictive model (preferably generated by a machine learning method). A predictive model is applied to each content item of the first set to generate a predicted vote for each content item. The set of predicted votes associated with the first set of content items is used by module MMM to generate a ranking of the first set of content items. The ranking is provided to the module SALM, where the module SALM has a second set of content items formed by the ranked set of content items, or a first set of content items selected as a result of the ranking. Define a second set of content items formed by the subset (eg, including only the highest ranked content items of the first set).

Preferably, in this embodiment,
A user interaction record module DRM, and an interaction history repository IHR,
Is further provided.

  The interaction history can take the form of a series of records, each record containing some information regarding, for example, user requests (or corresponding queries), system responses, context, metadata, and user feedback. Preferably, a composite format is used (eg a link or index instead of a physical item). In general, each record of interaction history corresponds to a different interaction event.

  The module IRM has a task to update the interaction history (step 206 in FIG. 2). In this regard, the module IRM records the user request (received from the module SFEM) directly in the repository IHR. In addition, the module SALM records its response to the user request (for the content item) in the repository IHR by means of the module IRM. The module SALM can also record all or part of the expected voting and / or metadata (content and / or context) used to respond to the user in the repository IHR by the module IRM.

  Advantageously, to save storage space in the repository IHR, only one type of metadata, ie raw context metadata, is stored in the repository IHR (other metadata is always retrieved from the repository CIR. Or can be generated by the module MGM). This may be performed by a module IRM that receives this metadata directly from the module SFEM.

  The service application may require the user to provide user feedback regarding content items that are provided in response to the user's request. The module SALM can also use the module SFEM for this purpose. Representative feedback from users is represented by voting (which can be directly compared to predicted voting). In this case, the module SALM can store this explicit feedback in the repository IHR by means of the module IRM. Advantageously, the service application logic part is designed to leave the user the freedom to provide explicit feedback.

  Alternatively, if the service application logic portion does not provide explicit feedback from the user, the user's behavior can be monitored to obtain implicit feedback (this can be done, for example, by the module SFEM). For example, a vote may be associated with the time a user spends reading a news item provided by a news service. In this case, the module IRM can record this implicit feedback in the repository IHR.

The processing and recording of explicit and / or implicit user feedback (step 207 of FIG. 2) may be performed after each response or at the end of the service interaction session.
If the content item includes multiple content components, the feedback may relate to the entire content item, advantageously, alternatively or in addition, the feedback may relate to each component of the content item Please note that. For example, the user can represent a vote for the entire movie, or separate votes for the video and audio elements. In this case, for example, separate votes are recorded as the interaction history.

Processing User Profiles The process of processing user profiles consists of creating (building) and maintaining (eg updating) user profiles. In the architecture of FIG. 1, the user profile is stored in the repository UPR, and the user profile construction module PBM performs the work of processing the user profile.

This task can be advantageously performed "offline" at night, for example, when the number of user interactions is sufficiently low.
According to this embodiment, the module PBM performs the following steps:
Retrieving the user interaction history (complete interaction history or partial interaction history corresponding to the time frame from the last update of the user profile to the current time) from the repository IHR (FIG. 3). Stage 301). The interaction history includes at least one event, typically a set of events. In general, an event includes a content query (corresponding to a user request), raw context metadata, and a set of selected content items provided in response to the query and preferably user feedback.
Selecting an appropriate machine learning algorithm adapted to build a user-preferred prediction model based on information contained in the interaction history (step 302 in FIG. 3).
-For each interaction event E _{i in the} interaction history, the stage where the PBM module generates an “n + 1” -dimensional feature vector, generally one vector V _i (steps 303, 304 and 305 in FIG. 3). Where n is the sum of the number of features related to content metadata (pre-assigned and derived) and the number of features related to context metadata (raw and derived) It is.
-To build a new predictive model to be incorporated into the user profile, the selected machine learning algorithm is associated with the feature vectors (events E _i , E _j , E _k. (Step 306) applied to V _i , V _j , V _k . A machine learning algorithm can process only one feature vector at a time, or somehow handle a set of feature vectors at a time (the model is processed by processing 13 vectors corresponding to 13 interaction events). As in the example above generated).
-(Preferably) verifying the performance of the newly built prediction model against a given acceptance criterion (generally a “better than before” type criterion) as a condition for updating the user profile (Step 307). For example, a known effective verification technique is “10x cross-validation”, which is based on dividing an event into 10 different divisions (eg 90% of the events are for model building) , 10% of the events are for model validation). Depending on the specific implementation, verification can be integrated into machine learning methods.
-Updating the user profile by replacing the previous model with the new model in the repository UPR (step 308).

Generation of the feature vector V _i related to the interaction event E _j may be performed according to the following steps.
-Searching raw content metadata;
Send the raw context metadata (recorded in the interaction history) to the module MGM and request to generate derived context metadata from the raw context metadata; raw context metadata and derived context metadata ( (Obtained from module MGM) to produce a context feature vector V _ix of dimension p (the two steps are shown as one step 303 in the flowchart of FIG. 3);
-Retrieving content metadata (created and derived) from the repository CIR;
The stage of encoding content metadata to produce a content feature vector V _ic of dimension m (where m + p = n) (the two stages are shown as one stage 304 in FIG. 3);
_-The content feature vector V _ic is combined with the context feature vector V _ix generated in the previous stage, and the user vote is added as the target feature t, so that one feature vector V _i = <V _ix , V _ic , creating t> (step 305 in FIG. 3);
Identifying a machine learning method algorithm _; and applying the machine learning method algorithm to the feature vector Vi to obtain a user-preferred prediction model.

  It should be noted that the above listed steps can be used even in the absence of a previously existing prediction model, i.e. not only for updating user profiles, but also for building new user profiles. In that case, for example, a fictitious user model is used in which positive feedback is assumed for arbitrary content.

  If the user is providing feedback for each component in a multi-component content item, the module PBM should consider this more detailed feedback.

  In the above description, it is assumed that the user has only one user profile. However, the present invention can also be extended if the user has more than one user profile and can switch between them. This is advantageous, for example, when the context metadata is insufficient to accurately describe the interaction context. For example, it is difficult for a terminal (i.e., an interaction device) to automatically determine if the user is at home or at the office (if the user does not set environmental options on the interaction device).

User profile selection can be made at the start of an interaction session and typically includes multiple requests and corresponding responses and implicit or explicit feedback.
Alternatively, user profile selection can be done on the fly at the moment of the feedback operation.

  For example, suppose a user finds a horror movie that the user likes in a movie browsing application. The user gives the item a highly rated first vote and specifies that the first vote is referenced in the “personal” profile. Since horror movies are not good for the user's children, the user also gives a second vote with a very low rating to the same movie, specifying that the second vote is referenced in the “family” profile. Alternatively, the user can set one of its profiles as the current profile, and the votes given by the user are referenced in the set profile. When a user seeks a ranking or recommendation for a movie, the user needs to specify the profile from which the recommendation is given.

  In embodiments where multiple user profiles are provided, the module of FIG. 1 needs to take this multiplicity into account. The module IRM needs to record information about the user profile in the repository IHR. Module PBM needs to select the correct user profile to update. The module MMM needs to select and use the correct user profile to create the ranking of content items.

Claims (14)

A) identifying a first set of content items stored in a repository of content items based on a request by a user , wherein the first set of content items includes a first content Metadata pre-assigned, stage,
B) a computer identifying first context metadata representing context information, wherein the context information is associated with the request by the user;
C) the computer, based on at least one derivation rules corresponding to algorithms to be applied to content items of at least said first set, the first for the set content items of the second content metadata and the second Automatically generating context metadata for
D ) the computer associating the second content metadata and the second context metadata with the first set of content items ;
E ) A computer stored in the repository of the first content metadata , the second content metadata , the first context metadata, the second context metadata, and a user profile wherein based on the user's user profile, said step of providing content items of the second set to the user selected from the first set, and
F) A computer for the first content metadata, the second content metadata, the first context metadata, the second context metadata, and the second set of content items the method based on the feedback from the user, the comprising <br/> of updating the user profile, to provide the selected content items to the user. The algorithm also applies to at least some of said first content metadata, The method of claim 1. Wherein the second content metadata derived from the algorithm applied to a plurality of content items, the method according to any one of claims 1-2. The method according to claim 1 , wherein the user profile includes a prediction model. In step E ), the ranking is based on the second content metadata and the prediction model to determine the second set of content items based on the ranking for each content item of the first set. It provides a method according to claim 4. The ranking for each content item in the first set is provided based on the second content metadata, the second context metadata, and the user profile, and the second set based on the ranking 6. The method of claim 5 , further comprising selecting a content item. How the computer, further comprising the step of associating the user feedback to at least one content item of the second set, according to claim 4. Computer, of the at least one content item of the second set is recorded in a reservoir of the interaction history, interaction history the feedback from the user who is related to the recorded at least one content item see further including the step of recording said reservoir, stage computer, to be recorded in the reservoir of the interaction history at least a portion of said second meta-data used to select the at least one content item The method of claim 7 , further comprising: Computer, see contains the step of recording the request by the user to the reservoir of the interaction history, computer, at least the recorded content item and the predictive model of the user based on the feedback from the user 9. The method of claim 8 , further comprising the step of constructing or updating . Computer, the steps of recording at least a portion of the second metadata to the reservoir of the interaction history, building or updating the predictive model of the user based on the second metadata least recorded comprising the method of claim 9. The method of claim 10 , wherein the predictive model is constructed or updated by at least one machine learning algorithm applied to at least the second content metadata and applied to at least some context metadata . The method according to any one of claims 1 to 11 for providing a content item of the second set via the telecommunication network. Computer program for causing execution of the method according to the computer in any one of claims 1 to 12. Computer, further including the step of collecting the content request from the user, the method according to any one of claims 1 to 12.

Images