WO2008073595A2 - A recommendation system and method of operation therefor - Google Patents

A recommendation system and method of operation therefor Download PDF

Info

Publication number
WO2008073595A2
WO2008073595A2 PCT/US2007/082812 US2007082812W WO2008073595A2 WO 2008073595 A2 WO2008073595 A2 WO 2008073595A2 US 2007082812 W US2007082812 W US 2007082812W WO 2008073595 A2 WO2008073595 A2 WO 2008073595A2
Authority
WO
WIPO (PCT)
Prior art keywords
recommendation
user
engine
engines
response
Prior art date
Application number
PCT/US2007/082812
Other languages
French (fr)
Other versions
WO2008073595A3 (en
Inventor
Nicolas Lhuillier
David Bonnefoy
Makram Bouzid
Jean Millerat
Jerome Picault
Myriam Ribiere
Original Assignee
Motorola, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Motorola, Inc. filed Critical Motorola, Inc.
Publication of WO2008073595A2 publication Critical patent/WO2008073595A2/en
Publication of WO2008073595A3 publication Critical patent/WO2008073595A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25866Management of end-user data
    • H04N21/25891Management of end-user data being end-user preferences
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/4508Management of client data or end-user data
    • H04N21/4532Management of client data or end-user data involving end-user characteristics, e.g. viewer profile, preferences
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/466Learning process for intelligent management, e.g. learning user preferences for recommending movies
    • H04N21/4667Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/475End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
    • H04N21/4755End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data for defining user preferences, e.g. favourite actors or genre
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/482End-user interface for program selection
    • H04N21/4826End-user interface for program selection using recommendation lists, e.g. of programs or channels sorted out according to their score
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/16Analogue secrecy systems; Analogue subscription systems
    • H04N7/173Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • H04N7/17309Transmission or handling of upstream communications
    • H04N7/17318Direct or substantially direct transmission and handling of requests

Definitions

  • the invention relates to a recommendation system and method of operation therefor and in particular, but not exclusively, to a recommendation system for recommending content items such as television programmes, music etc.
  • DVRs Digital Video Recorders
  • PVRs Personal Video Recorders
  • Such devices can comprise functionality for monitoring the viewing/recording preferences of a user. These preferences can be stored in a user profile which subsequently can be used to autonomously select and recommend suitable television programs for viewing or recording.
  • a DVR may automatically record programs which are then recommended to the user, for example by inclusion of the automatically recorded programs in a listing of all the programs recorded by the DVR.
  • a recommendation consists in predicting how much a user may like a particular content item and recommending it if it is considered of sufficient interest.
  • the process of generating recommendations requires that user preferences have been captured so that they can be used as input data by the prediction algorithm.
  • Personalisation technology usually involves a user profile which collects user preferences and a prediction algorithm which uses the preferences to infer suitable personalisation actions.
  • personalisation methods are used by different devices for different purposes, these do not provide a consistent and harmonised personalised user experience.
  • a generic recommendation engine cannot generate pertinent and targeted recommendations for all use-cases (e.g. recommendation of content, personalisation of service, selection of modality) .
  • a single user profile does not efficiently collect information about the various preferences and contexts for a given user.
  • not all user devices (such as a mobile phone) are sufficiently powerful to run a recommendation engine.
  • the Invention seeks to preferably mitigate, alleviate or eliminate one or more of the above mentioned disadvantages singly or in any combination.
  • a recommendation system comprising: a plurality of recommendation engines; a data communication coupling between the recommendation engines; wherein each of the recommendation engines comprises: a user profile for a first user, prediction means for generating recommendations for the first user in response to the user profile, interface means for exchanging recommendation event messages with other recommendation engines on the data communication coupling, and recommendation means for generating a recommendation in response to a received recommendation event message and a recommendation indication from the prediction means; and wherein at least one of the user profile and prediction means of at least some recommendation engines are different .
  • the invention may allow an improved recommendation system comprising a plurality of different recommendation engines interacting via an exchange of recommendation event messages.
  • the invention may for example allow specific and targeted recommendation engines optimised for a specific purpose to interact with each other to provide improved recommendations.
  • very different and incompatible recommendation engines may interact to provide improved recommendation.
  • a content recommendation may be generated for a given content item by a recommendation engine not suitable for such types of content items by the recommendation engine interacting with another recommendation engine optimised therefore .
  • the invention may allow advantages associated with a dedicated and specialised recommendation engine while providing a large application scope.
  • a facilitated design and development may be achieved as individual recommendation engines may be designed and optimised for specific purposes.
  • the invention may allow improved usability and applicability of existing recommendation engines .
  • the invention may allow a harmonised user experience for a plurality of recommendation engines e.g. implemented in different user devices. Furthermore, an improved learning may be achieved resulting in more accurate user profiles and thus improved recommendations.
  • the user profiles of different recommendation engines may be different and may use different data formats.
  • the user profiles of different recommendation engines may e.g. be incompatible such that one user profile cannot be used with the prediction means of another recommendation engine.
  • the user profile and/or the prediction means of different recommendation engines may for example relate to different types of content. Thus, for a certain type of content only some of the recommendation engines may be able to generate a recommendation.
  • the user profile and/or the prediction means of different recommendation engines may be independent. For example, the operation of one recommendation engine may be completely independent of the operation or characteristics of another recommendation engine except for the consideration of the recommendation event messages exchanged on the data communication coupling.
  • the recommendation indication generated by the recommendation means of one recommendation engine may be an indication that no suitable recommendation can be generated.
  • the recommendation indication may for example be a quality indication for a recommendation generated by the prediction means and this quality indication may be used to select between one or more recommendations of the recommendation engine and one or more recommendations received from another recommendation engine via the data communication coupling.
  • the recommendation event messages may use a common data format or protocol with no other coordination or common restrictions being imposed on the individual recommendation engines.
  • the recommendation event messages may comprise or consist in a recommendation request message and a recommendation provision message where the recommendation request message is a request for a recommendation from another recommendation engine and the recommendation provision message is a recommendation provided to another recommendation engine.
  • At least some recommendation engines comprise means for determining whether to respond to a received recommendation request message in response to a determination of whether at least one characteristic of a recommendation request of the received recommendation request message meets a criterion.
  • the recommendation engines may be loosely coupled whereby a given recommendation engine may request a recommendation and the other recommendation engines may individually decide whether to provide such a recommendation of not.
  • a recommendation engine may only respond by generating and sending a recommendation if it has available capacity for generating the recommendation and/or can generate a recommendation with sufficient accuracy.
  • a first recommendation engine comprises: means for receiving a user recommendation request for the first user, means for generating a recommendation request message requesting a recommendation for the first user and for providing the recommendation request message to the interface means of the first recommendation engine for transmission on the data communication coupling in response to the user recommendation request, means for receiving a recommendation provision message comprising a first recommendation for the first user from a second recommendation engine via the interface means of the first recommendation engine, and the recommendation means being arranged to generate the recommendation in response to the first recommendation; and the second recommendation engine comprises: means for receiving the recommendation request message from the interface means of the second recommendation engine, and means for providing the recommendation provision message to the interface means of the second recommendation engine for transmission to the first recommendation engine on the data communication coupling; and wherein the prediction means of the second recommendation engine is arranged to generate the first recommendation in response to user preference data of only the user profile of the second recommendation engine.
  • This may provide an efficient, flexible and/or improved recommendation system.
  • the second recommendation engine is arranged to receive user feedback for the first recommendation engine and to transmit a user feedback indication for the first recommendation to the first recommendation engine; and the first recommendation engine is arranged to update the user profile of the first recommendation engine in response to the user feedback indication. This may provide improved recommendations and in particular may allow improved adaptation of individual user profiles.
  • At least a first recommendation engine comprises: means for receiving a user preference input; means for updating the user profile of the second recommendation engine in response to the user preference input.
  • the recommendation system may be arranged to update a user profile of another recommendation engine in response to the user preference input and/or the modifications to the user profile of the first recommendation engine thereby resulting in improved recommendations and/or improved adaptation of individual user profiles.
  • a method of operation for a recommendation system including: a plurality of recommendation engines, each recommendation engine comprising: a user profile for a first user, prediction means for generating a recommendation for the first user in response to the user profile, and interface means for exchanging recommendation event messages with other recommendation engines on a data communication coupling; the method comprising at least one recommendation engine performing the step of: generating a recommendation in response to a received recommendation event message from a different recommendation engine and a recommendation indication from the prediction means of the at least one recommendation engine; and wherein at least one of the user profile and prediction means of at least some recommendation engines are different.
  • FIG. 1 illustrates a recommendation system in accordance with some embodiments of the invention
  • FIG. 2 illustrates a recommendation engine in accordance with some embodiments of the invention
  • FIG. 3 illustrates an example of a message exchange when generating a recommendation in accordance with some embodiments of the invention.
  • FIG. 4 illustrates a method of operation for a recommendation system in accordance with some embodiments of the invention.
  • FIG. 1 illustrates a recommendation system in accordance with some embodiments of the invention.
  • recommendation engines 101-107 are coupled together by a data communication coupling which in the specific example is an asynchronous event data network 109.
  • the data communication coupling may e.g. be a data network, such as a Local Area Network (LAN) , a cellular communication system, the Internet or any other suitable means for allowing the recommendation engines to exchange messages.
  • LAN Local Area Network
  • the data communication coupling may be a synchronous or asynchronous data bus.
  • a recommendation engine is a recommendation functionality which is capable of providing recommendations independently of any other recommendation function.
  • each of the recommendation engines 101-107 is in itself capable of providing recommendations without relying on information or data received from any other recommendation engine.
  • each of the recommendation engines may operate as stand-alone recommendation engine and may for example provide recommendations even if they are not coupled to the event data network 109.
  • the recommendation engines 101-107 are capable of exchanging recommendation event messages with other recommendation engines 101-107 coupled to the event data network 109.
  • the recommendation engines 101-107 may via the event data network 109 achieve a loose ad-hoc coupling that allows one recommendation engine 101 to request recommendations from other recommendation engines 103-107.
  • the other recommendation engines 103-107 may choose to provide a recommendation or not, and any recommendations received by the first recommendation engine 101 may be used by this to provide a recommendation to the user or application requesting it.
  • Each of the recommendation engines comprises the following main fundamental elements:
  • the user profile which is a repository of information about the user.
  • the user profile may for example comprise a list of user preferences, demographic data for the user etc.
  • a prediction algorithm which generates a recommendation based on the information of the user profile as well as e.g. user context data and input data describing characteristics of the required recommendation, such as specifying a type of content
  • a recommendation function which interfaces with another application or user to initiate and provide the recommendation based on the recommendation generated by the prediction algorithm
  • the recommendation engines 101-107 comprise optional learning functionality for updating, improving and adapting the user profile in response to the operation of the recommendation engines 101-107.
  • the learning functionality can abstract raw user information into higher-level preferences to be stored in the user profile.
  • a recommendation engine In a given recommendation engine, there is a tight coupling between the constituent components.
  • the prediction algorithm requires input data and user profile data to be in a particular format and the learning algorithm must also produce user profile data in the right format. Therefore a recommendation engine is generally dedicated to a specific purpose and domain (e.g. personalisation of a multi-modal user interface, generation of recommendations for television programmes described in a particular format, RSS-feeds recommendations, etc.).
  • the ability to design and develop a recommendation engine for a specific purpose and with limited scope allows an efficient and optimised solution for that purpose and scope. However, it may limit the applicability and may be inconvenient to a user requiring a plurality of recommendation engines for different recommendations (e.g. for different content item types).
  • each of the recommendation engines 101-107 are located in a different user device.
  • one recommendation engine may be located in a car, another in a mobile phone, another in a PVR, another in a television etc.
  • all the recommendation engines 101-107 are customised for the same user and thus the user profiles for all the recommendation engines 101-107 relate to the same user.
  • the user profiles and/or the prediction functionality for the different recommendation engines 101-107 are different.
  • one user profile may be updated differently than another user profile such that the data is different between these.
  • the user profiles may be independent of each other such that any changes or updates to one user profile may be made by the individual recommendation engine without consideration of whether any other user profile is updated.
  • the data protocol or format of the different user profiles may be different such that the user profile of one recommendation engine is incompatible with another recommendation engine and cannot be used with this.
  • the prediction functionality may be different in the different recommendation engines.
  • the prediction algorithms may be different and use completely different evaluations .
  • the recommendation engines 101-107 are independent of each other except for sharing a common communication protocol for communication of recommendation event messages over the event data network 109. Specifically, the recommendation engines 101-107 may be able to provide recommendations for different content types .
  • the first recommendation engine 101 may be part of a mobile phone capable of supporting cellular based television, and the first recommendation engine 101 is accordingly arranged to recommend cellular television programmes.
  • the second recommendation engine 103 is part of an audio player and the second recommendation engine 103 is arranged to provide recommendations of music clips.
  • the third recommendation engine 105 is in the example capable of recommending satellite television programmes and the fourth recommendation engine 107 can provide text document recommendations (e.g. for newspaper articles).
  • the event data network 109 provides for a communication and exchange of recommendation event messages between the recommendation engines 101-107.
  • the recommendation event messages are encoded and communicated using a common format.
  • the recommendation event messages relate to events which are associated with the generation of a recommendation.
  • the recommendation events associated with the recommendation event messages occur because a recommendation is being generated.
  • the event data network 109 may also exchange other messages or data between the recommendation engines 101- 107.
  • the event data network 109 may also support exchange of information that can be used to adapt or update user profiles regardless of whether a specific recommendation is currently being generated.
  • the recommendation event messages may specifically be:
  • a recommendation request message may include data associated with the requested recommendation.
  • the recommendation request message may include data prescribing the type and the format of the input, the expected output and a deadline by which the recommendation is expected.
  • Recommendation Provision Messages which comprise one or more recommendations for the user.
  • a recommendation provision message can include further data identifying or characterising the recommendations.
  • the recommendation provision messages are responses by the recommendation engines 101-107 to recommendation request messages transmitted by other recommendation engines 101-107.
  • Each of the recommendation engines 101-107 comprise functionality for transmitting and receiving the recommendation event messages via the event data network 109.
  • a recommendation engine 101-107 may comprise functionality for receiving any recommendation event message transmitted by other recommendation engines 101-107 specifically addressed to the recommendation engine 101-107.
  • the recommendation engines 101-107 may comprise functionality for receiving broadcast messages from other recommendation engines 101- 107.
  • a recommendation engine 101-107 can comprise functionality for transmitting recommendation event messages to other recommendation engines 101-107, either by specifically addressing other recommendation engines 101-107 or e.g. by using broadcast messages.
  • each recommendation engine 101-107 may comprise functionality for continuously monitoring messages on the data communication coupling.
  • the recommendation engines 101-107 may comprise functionality for monitoring the traffic on the bus and to intercept and decode all recommendation event messages of interest to the recommendation engine.
  • the first recommendation engine 101 executes the prediction algorithm of the first recommendation engine 101 itself.
  • the prediction functionality returns a recommendation indication which may be a simple binary indication of whether a recommendation can be generated or not.
  • a recommendation indication indicating that no recommendation can be generated by the first recommendation engine 101 is generated if the request is for a music clip.
  • the first recommendation engine 101 of the system of FIG. 1 generates and transmits a recommendation request message on the event data network 109.
  • the recommendation request message comprises an indication that a music clip recommendation is required.
  • the recommendation request message is a broadcast message that may be received by all the other recommendation engines 103-107.
  • the other recommendation engines 103-107 receive the recommendation request message and determine whether they can provide a recommendation.
  • the third and fourth recommendation engines 105-107 cannot provide a recommendation for a music clip and simply ignore the recommendation request message.
  • the second recommendation engine 103 can generate a recommendation for a music clip and proceeds to do so based on the prediction functionality and user profile of the second recommendation engine 103.
  • the generation of the recommendation is performed independently of the first recommendation engine 101 and specifically it is independent of the prediction algorithm and user profile of the first recommendation engine 101.
  • the second recommendation engine 103 generates a recommendation provision message comprising an indication of the recommendation and transmits this back to the first recommendation engine 101.
  • the recommendation provision message may be a dedicated message specifically addressed to the first recommendation engine 101.
  • the first recommendation engine 101 receives the recommendation provision message and proceeds to provide the recommendation to the user or application requesting the recommendation.
  • the described system provides for improved recommendation performance and an improved user experience.
  • the system provides for each recommendation engine becoming a possible recommender for any application or user anywhere in the system.
  • the system may provide a distributed personalisation architecture where personalisation components
  • the recommendation system of FIG. 1 uses an architecture which allows an application that requires some personalisation actions (e.g. content recommendations) to contact the most appropriate recommendation engine to provide the recommendation whether this recommendation engine is located locally on the same device or on another device coupled to the local device via a common shared data bus/network.
  • personalisation actions e.g. content recommendations
  • a distributed personalisation architecture is employed where personalisation components asynchronously exchange requests, replies and feedback for personalisation using a common communication format but with the individual recommendation engines all using their own individual data format/ algorithm. Furthermore, recommendation engines and devices need not exchange other information and in particular need not exchange information related to the individual capabilities of the devices or recommendation engines.
  • each recommendation engine may be implemented as functionality embedded in applications or as a separate component.
  • the personalisation capabilities for different types of content or purpose e.g. selection of modality and music recommendations
  • a recommendation engine if a recommendation engine is not able to provide the requested personalisation action, it can publish a recommendation request by transmitting a recommendation request message on the event data network 109 and can then collect a resulting recommendation by receiving a recommendation provision message from an appropriate recommendation engine.
  • the system may be particularly suited for low capability machines such as mobile devices which need only implement simple recommendation engines aimed at their main purpose while still being able to provide recommendations for other purposes by interacting with other recommendation engines in the system.
  • FIG. 2 illustrates the first recommendation engine 101 in more detail. In the example, all the recommendation engines 101-107 operate similarly to the first recommendation engine 101.
  • FIG. 3 illustrates an example of a message exchange when generating a recommendation.
  • the first recommendation engine 101 comprises a prediction processor 201 which is arranged to execute the prediction algorithm to generate one more personalised recommendations.
  • the prediction processor 201 is coupled to a user profile 203 which stores user preference data used to generate the recommendations.
  • any suitable prediction algorithm can be used without detracting from the invention.
  • many different algorithms will be known to the person skilled in the art. For example a naive Bayes classifier which computes the probability that the user will appreciate a piece of content based on prior content the user liked or disliked can be used.
  • the prediction processor 201 is furthermore coupled to a recommendation processor 205.
  • the recommendation processor 205 is operable to control and instigate the generation of a recommendation.
  • the recommendation processor 205 may interface with a user application and/or a user interface.
  • an application 301 may request a recommendation by submitting a user recommendation request to the recommendation processor 205 of the first recommendation engine 101.
  • the recommendation processor 205 initiates the generation of a recommendation from the prediction processor 201.
  • the prediction processor 201 may initially evaluate whether such a recommendation can be provided. For example, the prediction processor 201 may initially assess whether the recommendation request relates to a content item for which the prediction processor 201 and the user profile 203 are able to provide recommendation. For example, if the recommendation is for a music clip, the prediction processor 201 and user profile 203 are not capable of providing recommendation in the specific example where the first recommendation engine 101 relates to cellular television programmes. Instead a recommendation indication indicating that the recommendation cannot be generated by the first recommendation engine 101 is generated.
  • the data format used to describe the content may be different and may prevent the first recommendation engine 101 to reply. Accordingly, it may request a recommendation from a different device capable of handling this content format.
  • the prediction processor 201 proceeds to generate a recommendation for one or more content items and provide these to the recommendation processor 205.
  • the prediction processor 201 may include a recommendation indication which is indicative of the estimated quality or accuracy of the recommendation. For example, if the request is for a recommendation of a cellular television programme, the prediction processor 201 provides such a recommendation as well as an estimated recommendation quality to the recommendation processor 205. The recommendation processor 205 then evaluates whether a recommendation meeting a quality criterion can be generated by the prediction processed 201. Specifically, if the recommendation indication received from the prediction processed 201 is indicating that no recommendation can be provided, the recommendation processor 205 considers that the quality criterion has not been met.
  • the quality criterion may simply be an assessment of whether a recommendation can be generated by the local recommendation engine.
  • the recommendation processor 205 may compare an indication of the estimated quality of the recommendation and consider the quality criterion to be met only if this exceeds a threshold value.
  • the recommendation processor 205 is furthermore coupled to a recommendation receive processor 207 which is arranged to control an interaction with other recommendation engines 103-107 of the system in order to receive a recommendation from these.
  • the recommendation receive processor 207 is coupled to a data network interface 209 which provides an interface to the event data network 109 and which specifically is arranged to transmit and receive recommendation event messages.
  • the recommendation processor 205 initiates the recommendation receive processor 207 to generate and transmit one or more recommendation request messages.
  • the recommendation request message may for example include a set of recommendation candidates.
  • the first recommendation engine 101 may include a set of content items that can be evaluated by other recommendation engines.
  • the first recommendation engine 101 may for example include a listing of all the music clips stored on the mobile phone hosting the first recommendation engine 101.
  • a recommendation engine optimised for music clips may recommend a music clip while ensuring that this is one which is available at the device initiating the request.
  • the recommendation request message may e.g. comprise the type and format of the recommendation that is sought.
  • the request may also include some additional data which shall be used to compute the request (e.g. the content list to be filtered) .
  • the requesting recommendation engine may for instance send a list of possible options to be personalised and which a responding recommendation engine should return with the associated prediction (s) (for instance as a filtered and sorted list) .
  • the requesting recommendation engine may also ask for a recommendation without providing any list of options. In this case the other recommendation engine can use its own options.
  • the requesting recommendation engine may ask for a template of recommendations which can then be matched against a list of possible options to derive actual recommendations .
  • the recommendation receive processor 207 makes the request available to the other peer recommendation engines 103-107 via the data network interface 209.
  • the recommendation request message is distributed to all other recommendation engines 103-107 in the system without consideration of which recommendation engines are available and what their capabilities are.
  • an individual recommendation request message is transmitted to each of the other recommendation engines 103-107.
  • a single broadcast message may be transmitted which can be received by all other recommendation engines 103-107.
  • the recommendation request message may be addressed to a plurality of recommendation engines such as a selected group of recommendation engines.
  • the recommendation request message (s) are received by the other recommendation engines 103-107.
  • the first recommendation engine 101 comprises a recommendation transmit processor 211 coupled to the data network interface 209 and the prediction processor 201.
  • the recommendation transmit processor 211 is arranged receive recommendation request messages from other recommendation engines. In response, the recommendation transmit processor 211 determines whether to generate a recommendation. If so, the recommendation transmit processor 211 initiates a recommendation action by the prediction processor 201. The resulting recommendation is included in a recommendation provision message which is returned to the requesting recommendation engine.
  • each recommendation engine 103-107 comprises similar or identical functionality to the recommendation transmit processor 211.
  • each recommendation engine 103-107 receives a recommendation request message from the first recommendation engine 101. They then proceed to determine whether to respond to the received recommendation request message. This decision may be determined based only on internal characteristics such as whether the recommendation engine has sufficient available computational resource to generate the recommendation.
  • the decision is alternatively or additionally based on at least one characteristic of the recommendation request.
  • the recommendation request message may indicate that the request is for a specific content type, such as a music clip.
  • the other recommendation engines 103-107 may simply evaluate whether they are able to provide such a recommendation (and/or may evaluate if they can support the format used to describe the music clips) .
  • the second recommendation engine 103 is adapted to provide recommendations for music clips and accordingly it decides to generate a recommendation.
  • the third and fourth recommendation engines 105, 107 are not adapted to provide music clip recommendations and therefore decide not to respond to the recommendation request message.
  • each peer recommendation engine individually and independently decides whether to reply or not based on the type and format of the request and optionally other local constraints (e.g. availability).
  • the recommendation engine If the recommendation engine has decided to provide a recommendation response, it proceeds to generate a recommendation based on the user preference data of only the user profile of the recommendation engine. Thus, a recommendation is generated independently of the prediction algorithms and user profiles stored in other devices.
  • the second recommendation engine 103 then proceed to generate a recommendation provision message comprising an identification of the generated recommendation.
  • the generated recommendation may be an identification of one or more content items, e.g. together with a user preference rating for each content item.
  • the recommendation provision message may comprise a list of content item identifiers, a general description (template) for the content item(s) or one or more actual content items (or e.g. content item addresses).
  • the recommendation provision message is then transmitted back to the first recommendation engine 101 by the event data network 109.
  • the first recommendation engine 101 receives the recommendation provision message and feeds it to the recommendation receive processor 207 which extracts the recommendation data and feeds it to the recommendation processor 205.
  • the recommendation processor 205 then proceeds to generate a recommendation for the application/user.
  • the recommendation processor 205 can simply provide the received recommendation to the application/user.
  • the recommendation processor 205 may further evaluate the received recommendations and may for example compare or combine them with recommendations generated by the local recommendation engine. For example, if a list of recommended content items with associated reference values has been received from another recommendation engine these may be compared to a list of recommended content items and associated preference values generated by the local prediction processor 201.
  • the recommendation processor 205 may then select one or more content items having the highest preference value regardless of whether these were generated locally or received from another device .
  • a recommendation provision message was only received from a single other recommendation engine.
  • responses may be received from a plurality of other recommendation engines.
  • a plurality of recommendation engines can be arranged to generate a recommendation provision message in response to the same recommendation request message.
  • each recommendation engine may individually and independently of other recommendation engines decide whether to respond with a recommendation or not.
  • the first recommendation engine 101 may simply wait for a given time interval in which the other recommendation engines 103 -107 may reply with recommendation provision messages. At the end of the time interval, the first recommendation engine 101 may proceed to evaluate the received recommendations (if any) . Specifically, the first recommendation engine 101 may simply collates the recommendations and feed them to the requesting application/user. As another example, the recommendation processor can select one or more recommendations or may e.g. combine the recommendations. For example, recommendations may be fed to the requesting application/user depending on how many other recommendation engines have recommended each individual content item.
  • the recommendation system may furthermore comprise functionality for adapting the user profiles to the user.
  • the second recommendation engine may be arranged to receive user feedback for a generated recommendation which specifically may be the recommendation generated for the first recommendation engine 101.
  • the second recommendation engine 103 may comprise user input means for a user to manually provide a user preference input.
  • the second recommendation engine 103 may accordingly update its local user profile in response to the user preference input.
  • the second recommendation engine 103 can generate a user feedback indication reflecting the user preference input and send this back to the first recommendation engine 101.
  • the first recommendation engine 101 may then proceed to update its local user profile in response to the user feedback indication .
  • the first recommendation engine 101 comprises a learning processor 213 coupled to the data network interface 209 and the user profile 203.
  • the learning processor 213 is arranged to update the user profile 203 in response to locally received information (e.g. a local user preference input) as well as in response to the feedback indication received from the second recommendation engine 103.
  • the user profile 203 of the first recommendation engine 101 may be updated in response to user feedback provided at another recommendation engine 103-107 of the system.
  • the second recommendation engine 103 may specifically transmit a user input message which comprises an indication of the user preference input. This message may reflect the actual user input and the learning processor 213 may specifically update the user profile 203 based on the user input. Specifically the learning processor 213 may proceed to process the received user input from the second recommendation engine 103 in exactly the same way as if the user input had been provided to the first recommendation engine 101.
  • the second recommendation engine 103 may transmit a user profile update message which comprises an indication of how the user profile of the second recommendation engine 103 has been modified.
  • the user profile update message may not provide information of the actual user input but rather may provide information of how the update affected the user profile of the second recommendation engine 103.
  • the user profile update message may comprise an indication of an increased or reduced preference rating for specific content item category.
  • the learning processor 213 may proceed to update the user profile 203 in response to the received user profile update message. For example, if an indication of an increase in the preference rating for specific content item category in the user profile of the second recommendation engine 103 is received, the corresponding content item category of the user profile 203 may also be increased correspondingly.
  • the approach allows the development and use of very specific (and therefore effective) recommenders .
  • the approach may be based on existing recommendation engines and it does not necessitate a development of generic engines.
  • the approach also alleviates the need to describe the capabilities of each recommender in a common format.
  • FIG. 4 illustrates a method of operation for a recommendation system which includes a plurality of recommendation engines.
  • Each recommendation engine comprises: a user profile for a first user; a prediction unit for generating a recommendation for the first user in response to the user profile; and an interface for exchanging recommendation event messages with other recommendation engines on a data communication coupling. At least one of the user profile and prediction units of at least some recommendation engines are different.
  • the method comprises at least one recommendation engine performing step 401 wherein a recommendation is generated in response to a received recommendation event message from a different recommendation engine and a recommendation indication from the prediction unit of the at least one recommendation engine.
  • an embodiment of the invention may be physically, functionally and logically implemented in any suitable way. Indeed the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the invention may be implemented in a single unit or may be physically and functionally distributed between different units and processors .

Abstract

A recommendation system comprises a plurality of recommendation engines (101-107) which are coupled together by a data communication coupling (109). Each of the recommendation engines (101-107) comprises a user profile (203) for a first user, a prediction processor (201) for generating recommendations for the first user in response to the user profile and an interface (209) for exchanging recommendation event messages, such as a request for a recommendation and a reply thereto, with other recommendation engines on the data communication coupling. At least one of the user profile and prediction algorithm of at least some recommendation engines (101-107) are different. A recommendation processor (205) generates a recommendation in response to a received recommendation event message and a recommendation indication from the prediction means. The invention may provide improved recommendation by interworking of independent recommendation engines (101-107).

Description

A RECOMMENDATION SYSTEM AND METHOD OF OPERATION THEREFOR
Field of the invention
The invention relates to a recommendation system and method of operation therefor and in particular, but not exclusively, to a recommendation system for recommending content items such as television programmes, music etc.
Background of the Invention
In recent years, the availability and provision of multimedia and entertainment content has increased substantially. For example, the number of available television and radio channels has grown considerably and the popularity of the Internet has provided new content distribution means. Consequently, users are increasingly provided with a plethora of different types of content from different sources. In order to identify and select the desired content, the user must typically process large amounts of information which can be very cumbersome and impractical.
Similarly, an increasing number of services and applications with many different options and customisation features are becoming available to the user .
Accordingly, significant resources have been invested in research into techniques and algorithms that may provide an improved user experience and assist a user in identifying and selecting content, personalizing services etc .
For example, Digital Video Recorders (DVRs) or Personal Video Recorders (PVRs) which comprise functionality for providing recommendations of television programs to the user based on user preferences are becoming increasingly popular. More specifically, such devices can comprise functionality for monitoring the viewing/recording preferences of a user. These preferences can be stored in a user profile which subsequently can be used to autonomously select and recommend suitable television programs for viewing or recording. E.g. a DVR may automatically record programs which are then recommended to the user, for example by inclusion of the automatically recorded programs in a listing of all the programs recorded by the DVR.
In order to enhance the user experience, it is advantageous to personalise the recommendations to the individual user as much as is possible. In this context, a recommendation consists in predicting how much a user may like a particular content item and recommending it if it is considered of sufficient interest. The process of generating recommendations requires that user preferences have been captured so that they can be used as input data by the prediction algorithm.
Furthermore, people increasingly use a wide range of electronic devices for different purposes and with different capabilities (e.g. cell phone, PDA, MP3 players, set-top boxes, personal computers, etc.) All these devices are running more and more complex applications with many applications using some element of personalisation to provide a better or simplified user experience .
Personalisation technology usually involves a user profile which collects user preferences and a prediction algorithm which uses the preferences to infer suitable personalisation actions. However, as personalisation methods are used by different devices for different purposes, these do not provide a consistent and harmonised personalised user experience.
One of the main problems is that a generic recommendation engine cannot generate pertinent and targeted recommendations for all use-cases (e.g. recommendation of content, personalisation of service, selection of modality) . Additionally, a single user profile does not efficiently collect information about the various preferences and contexts for a given user. Furthermore, not all user devices (such as a mobile phone) are sufficiently powerful to run a recommendation engine.
In order to provide a harmonised user experience for a plurality of user devices it has been proposed to use a centralised recommendation engine or a single centralised user profile that may be accessed by a plurality of prediction algorithms operating on individual devices. However, such approaches tend to have a number of disadvantages and tend to be complex, cumbersome, inflexible, have poor updating capabilities and to provide sub-optimal recommendations. Hence, an improved recommendation system would be advantageous and in particular a system allowing increased flexibility, an improved and potentially more consistent user experience, suitability for different devices, facilitated operation, facilitated design and development, facilitated or improved learning and/or improved recommendations would be advantageous.
Summary of the Invention
Accordingly, the Invention seeks to preferably mitigate, alleviate or eliminate one or more of the above mentioned disadvantages singly or in any combination.
According to a first aspect of the invention there is provided a recommendation system comprising: a plurality of recommendation engines; a data communication coupling between the recommendation engines; wherein each of the recommendation engines comprises: a user profile for a first user, prediction means for generating recommendations for the first user in response to the user profile, interface means for exchanging recommendation event messages with other recommendation engines on the data communication coupling, and recommendation means for generating a recommendation in response to a received recommendation event message and a recommendation indication from the prediction means; and wherein at least one of the user profile and prediction means of at least some recommendation engines are different . The invention may allow an improved recommendation system comprising a plurality of different recommendation engines interacting via an exchange of recommendation event messages. The invention may for example allow specific and targeted recommendation engines optimised for a specific purpose to interact with each other to provide improved recommendations. For example, very different and incompatible recommendation engines may interact to provide improved recommendation. E.g. a content recommendation may be generated for a given content item by a recommendation engine not suitable for such types of content items by the recommendation engine interacting with another recommendation engine optimised therefore .
The invention may allow advantages associated with a dedicated and specialised recommendation engine while providing a large application scope. A facilitated design and development may be achieved as individual recommendation engines may be designed and optimised for specific purposes. The invention may allow improved usability and applicability of existing recommendation engines .
The invention may allow a harmonised user experience for a plurality of recommendation engines e.g. implemented in different user devices. Furthermore, an improved learning may be achieved resulting in more accurate user profiles and thus improved recommendations.
The user profiles of different recommendation engines may be different and may use different data formats. The user profiles of different recommendation engines may e.g. be incompatible such that one user profile cannot be used with the prediction means of another recommendation engine. The user profile and/or the prediction means of different recommendation engines may for example relate to different types of content. Thus, for a certain type of content only some of the recommendation engines may be able to generate a recommendation. The user profile and/or the prediction means of different recommendation engines may be independent. For example, the operation of one recommendation engine may be completely independent of the operation or characteristics of another recommendation engine except for the consideration of the recommendation event messages exchanged on the data communication coupling.
The recommendation indication generated by the recommendation means of one recommendation engine may be an indication that no suitable recommendation can be generated. The recommendation indication may for example be a quality indication for a recommendation generated by the prediction means and this quality indication may be used to select between one or more recommendations of the recommendation engine and one or more recommendations received from another recommendation engine via the data communication coupling.
The recommendation event messages may use a common data format or protocol with no other coordination or common restrictions being imposed on the individual recommendation engines.
The recommendation event messages may comprise or consist in a recommendation request message and a recommendation provision message where the recommendation request message is a request for a recommendation from another recommendation engine and the recommendation provision message is a recommendation provided to another recommendation engine.
According to an optional feature of the invention, at least some recommendation engines comprise means for determining whether to respond to a received recommendation request message in response to a determination of whether at least one characteristic of a recommendation request of the received recommendation request message meets a criterion.
This may allow a flexible and efficient recommendation system. Specifically, the recommendation engines may be loosely coupled whereby a given recommendation engine may request a recommendation and the other recommendation engines may individually decide whether to provide such a recommendation of not. For example, a recommendation engine may only respond by generating and sending a recommendation if it has available capacity for generating the recommendation and/or can generate a recommendation with sufficient accuracy.
According to an optional feature of the invention, a first recommendation engine comprises: means for receiving a user recommendation request for the first user, means for generating a recommendation request message requesting a recommendation for the first user and for providing the recommendation request message to the interface means of the first recommendation engine for transmission on the data communication coupling in response to the user recommendation request, means for receiving a recommendation provision message comprising a first recommendation for the first user from a second recommendation engine via the interface means of the first recommendation engine, and the recommendation means being arranged to generate the recommendation in response to the first recommendation; and the second recommendation engine comprises: means for receiving the recommendation request message from the interface means of the second recommendation engine, and means for providing the recommendation provision message to the interface means of the second recommendation engine for transmission to the first recommendation engine on the data communication coupling; and wherein the prediction means of the second recommendation engine is arranged to generate the first recommendation in response to user preference data of only the user profile of the second recommendation engine.
This may provide an efficient, flexible and/or improved recommendation system.
According to an optional feature of the invention, the second recommendation engine is arranged to receive user feedback for the first recommendation engine and to transmit a user feedback indication for the first recommendation to the first recommendation engine; and the first recommendation engine is arranged to update the user profile of the first recommendation engine in response to the user feedback indication. This may provide improved recommendations and in particular may allow improved adaptation of individual user profiles.
According to an optional feature of the invention, at least a first recommendation engine comprises: means for receiving a user preference input; means for updating the user profile of the second recommendation engine in response to the user preference input.
This may provide improved recommendations and in particular may allow improved adaptation of individual user profiles. The recommendation system may be arranged to update a user profile of another recommendation engine in response to the user preference input and/or the modifications to the user profile of the first recommendation engine thereby resulting in improved recommendations and/or improved adaptation of individual user profiles.
According to a different aspect of the invention, there is provided a method of operation for a recommendation system including: a plurality of recommendation engines, each recommendation engine comprising: a user profile for a first user, prediction means for generating a recommendation for the first user in response to the user profile, and interface means for exchanging recommendation event messages with other recommendation engines on a data communication coupling; the method comprising at least one recommendation engine performing the step of: generating a recommendation in response to a received recommendation event message from a different recommendation engine and a recommendation indication from the prediction means of the at least one recommendation engine; and wherein at least one of the user profile and prediction means of at least some recommendation engines are different.
These and other aspects, features and advantages of the invention will be apparent from and elucidated with reference to the embodiment (s) described hereinafter.
Brief Description of the Drawings
Embodiments of the invention will be described, by way of example only, with reference to the drawings, in which
FIG. 1 illustrates a recommendation system in accordance with some embodiments of the invention;
FIG. 2 illustrates a recommendation engine in accordance with some embodiments of the invention;
FIG. 3 illustrates an example of a message exchange when generating a recommendation in accordance with some embodiments of the invention; and
FIG. 4 illustrates a method of operation for a recommendation system in accordance with some embodiments of the invention.
Detailed Description of Some Embodiments of the Invention The following description focuses on embodiments of the invention applicable to a recommendation system for recommendation of content items such as music, videos, television programmes, documents etc. However, it will be appreciated that the invention is not limited to this application but may be applied to many other types of recommendation including for example service recommendations or user interface adaptation recommendations .
FIG. 1 illustrates a recommendation system in accordance with some embodiments of the invention.
In the system, recommendation engines 101-107 (of which four are shown) are coupled together by a data communication coupling which in the specific example is an asynchronous event data network 109. It will be appreciated that the data communication coupling may e.g. be a data network, such as a Local Area Network (LAN) , a cellular communication system, the Internet or any other suitable means for allowing the recommendation engines to exchange messages. For example, in some embodiments the data communication coupling may be a synchronous or asynchronous data bus.
A recommendation engine is a recommendation functionality which is capable of providing recommendations independently of any other recommendation function. Thus, each of the recommendation engines 101-107 is in itself capable of providing recommendations without relying on information or data received from any other recommendation engine. Thus, each of the recommendation engines may operate as stand-alone recommendation engine and may for example provide recommendations even if they are not coupled to the event data network 109. However, in addition, the recommendation engines 101-107 are capable of exchanging recommendation event messages with other recommendation engines 101-107 coupled to the event data network 109.
In particular, the recommendation engines 101-107 may via the event data network 109 achieve a loose ad-hoc coupling that allows one recommendation engine 101 to request recommendations from other recommendation engines 103-107. In response the other recommendation engines 103-107 may choose to provide a recommendation or not, and any recommendations received by the first recommendation engine 101 may be used by this to provide a recommendation to the user or application requesting it.
Each of the recommendation engines comprises the following main fundamental elements:
a user profile which is a repository of information about the user. The user profile may for example comprise a list of user preferences, demographic data for the user etc
a prediction algorithm which generates a recommendation based on the information of the user profile as well as e.g. user context data and input data describing characteristics of the required recommendation, such as specifying a type of content
(e.g. music, video etc), a service or a modality. A recommendation function which interfaces with another application or user to initiate and provide the recommendation based on the recommendation generated by the prediction algorithm
In addition, the recommendation engines 101-107 comprise optional learning functionality for updating, improving and adapting the user profile in response to the operation of the recommendation engines 101-107. Specifically, the learning functionality can abstract raw user information into higher-level preferences to be stored in the user profile.
In a given recommendation engine, there is a tight coupling between the constituent components. The prediction algorithm requires input data and user profile data to be in a particular format and the learning algorithm must also produce user profile data in the right format. Therefore a recommendation engine is generally dedicated to a specific purpose and domain (e.g. personalisation of a multi-modal user interface, generation of recommendations for television programmes described in a particular format, RSS-feeds recommendations, etc.). The ability to design and develop a recommendation engine for a specific purpose and with limited scope allows an efficient and optimised solution for that purpose and scope. However, it may limit the applicability and may be inconvenient to a user requiring a plurality of recommendation engines for different recommendations (e.g. for different content item types).
In the system of FIG. 1 each of the recommendation engines 101-107 are located in a different user device. For example, one recommendation engine may be located in a car, another in a mobile phone, another in a PVR, another in a television etc. However, in the system all the recommendation engines 101-107 are customised for the same user and thus the user profiles for all the recommendation engines 101-107 relate to the same user.
However, despite relating to the same user, the user profiles and/or the prediction functionality for the different recommendation engines 101-107 are different.
Specifically, one user profile may be updated differently than another user profile such that the data is different between these. Specifically, the user profiles may be independent of each other such that any changes or updates to one user profile may be made by the individual recommendation engine without consideration of whether any other user profile is updated. The data protocol or format of the different user profiles may be different such that the user profile of one recommendation engine is incompatible with another recommendation engine and cannot be used with this.
Alternatively or additionally, the prediction functionality may be different in the different recommendation engines. For example, the prediction algorithms may be different and use completely different evaluations .
Thus, in the system of FIG. 1, the recommendation engines 101-107 are independent of each other except for sharing a common communication protocol for communication of recommendation event messages over the event data network 109. Specifically, the recommendation engines 101-107 may be able to provide recommendations for different content types .
As a specific example, the first recommendation engine 101 may be part of a mobile phone capable of supporting cellular based television, and the first recommendation engine 101 is accordingly arranged to recommend cellular television programmes. In the example, the second recommendation engine 103 is part of an audio player and the second recommendation engine 103 is arranged to provide recommendations of music clips. The third recommendation engine 105 is in the example capable of recommending satellite television programmes and the fourth recommendation engine 107 can provide text document recommendations (e.g. for newspaper articles).
In the system of FIG. 1, the event data network 109 provides for a communication and exchange of recommendation event messages between the recommendation engines 101-107. The recommendation event messages are encoded and communicated using a common format.
The recommendation event messages relate to events which are associated with the generation of a recommendation. Thus, the recommendation events associated with the recommendation event messages occur because a recommendation is being generated. It will be appreciated that in addition to the recommendation event messages, the event data network 109 may also exchange other messages or data between the recommendation engines 101- 107. For example, the event data network 109 may also support exchange of information that can be used to adapt or update user profiles regardless of whether a specific recommendation is currently being generated.
The recommendation event messages may specifically be:
Recommendation Request Messages which comprise a request for a recommendation for the user transmitted from one recommendation engine 101-107 to one or more of the other recommendation engines 101-107. A recommendation request message may include data associated with the requested recommendation. For example, the recommendation request message may include data prescribing the type and the format of the input, the expected output and a deadline by which the recommendation is expected.
Recommendation Provision Messages which comprise one or more recommendations for the user. A recommendation provision message can include further data identifying or characterising the recommendations. The recommendation provision messages are responses by the recommendation engines 101-107 to recommendation request messages transmitted by other recommendation engines 101-107.
Each of the recommendation engines 101-107 comprise functionality for transmitting and receiving the recommendation event messages via the event data network 109. Specifically, a recommendation engine 101-107 may comprise functionality for receiving any recommendation event message transmitted by other recommendation engines 101-107 specifically addressed to the recommendation engine 101-107. In addition, the recommendation engines 101-107 may comprise functionality for receiving broadcast messages from other recommendation engines 101- 107.
In addition, a recommendation engine 101-107 can comprise functionality for transmitting recommendation event messages to other recommendation engines 101-107, either by specifically addressing other recommendation engines 101-107 or e.g. by using broadcast messages.
Specifically, in some embodiments, each recommendation engine 101-107 may comprise functionality for continuously monitoring messages on the data communication coupling. For example, if the data communication coupling is an asynchronous data bus, the recommendation engines 101-107 may comprise functionality for monitoring the traffic on the bus and to intercept and decode all recommendation event messages of interest to the recommendation engine.
In the system of FIG. 1, when a recommendation is requested from the first recommendation engine 101 by a user or an application, the first recommendation engine 101 executes the prediction algorithm of the first recommendation engine 101 itself. The prediction functionality returns a recommendation indication which may be a simple binary indication of whether a recommendation can be generated or not. E.g. in the specific example where the first recommendation engine 101 relates to television programmes a recommendation indication indicating that no recommendation can be generated by the first recommendation engine 101 is generated if the request is for a music clip.
However, rather than reporting that no recommendation can be generated, the first recommendation engine 101 of the system of FIG. 1 generates and transmits a recommendation request message on the event data network 109. The recommendation request message comprises an indication that a music clip recommendation is required. In the example, the recommendation request message is a broadcast message that may be received by all the other recommendation engines 103-107.
The other recommendation engines 103-107 receive the recommendation request message and determine whether they can provide a recommendation. In the example, the third and fourth recommendation engines 105-107 cannot provide a recommendation for a music clip and simply ignore the recommendation request message. However, the second recommendation engine 103 can generate a recommendation for a music clip and proceeds to do so based on the prediction functionality and user profile of the second recommendation engine 103. Thus, the generation of the recommendation is performed independently of the first recommendation engine 101 and specifically it is independent of the prediction algorithm and user profile of the first recommendation engine 101.
The second recommendation engine 103 generates a recommendation provision message comprising an indication of the recommendation and transmits this back to the first recommendation engine 101. The recommendation provision message may be a dedicated message specifically addressed to the first recommendation engine 101.
The first recommendation engine 101 receives the recommendation provision message and proceeds to provide the recommendation to the user or application requesting the recommendation.
Thus, the described system provides for improved recommendation performance and an improved user experience. The system provides for each recommendation engine becoming a possible recommender for any application or user anywhere in the system. Thus, the system may provide a distributed personalisation architecture where personalisation components
(recommendation engines) that relate to a given user exchange requests and replies for personalisation action across device/network boundaries using a common communication format, while internally using their own particular data formats and algorithms.
Thus, the recommendation system of FIG. 1 uses an architecture which allows an application that requires some personalisation actions (e.g. content recommendations) to contact the most appropriate recommendation engine to provide the recommendation whether this recommendation engine is located locally on the same device or on another device coupled to the local device via a common shared data bus/network.
Specifically, a distributed personalisation architecture is employed where personalisation components asynchronously exchange requests, replies and feedback for personalisation using a common communication format but with the individual recommendation engines all using their own individual data format/ algorithm. Furthermore, recommendation engines and devices need not exchange other information and in particular need not exchange information related to the individual capabilities of the devices or recommendation engines.
In the system, each recommendation engine may be implemented as functionality embedded in applications or as a separate component. Also, the personalisation capabilities for different types of content or purpose (e.g. selection of modality and music recommendations) may be supported in a single recommendation engine or by a collection of recommendation engines. E.g. as described above, if a recommendation engine is not able to provide the requested personalisation action, it can publish a recommendation request by transmitting a recommendation request message on the event data network 109 and can then collect a resulting recommendation by receiving a recommendation provision message from an appropriate recommendation engine.
The system may be particularly suited for low capability machines such as mobile devices which need only implement simple recommendation engines aimed at their main purpose while still being able to provide recommendations for other purposes by interacting with other recommendation engines in the system.
FIG. 2 illustrates the first recommendation engine 101 in more detail. In the example, all the recommendation engines 101-107 operate similarly to the first recommendation engine 101. FIG. 3 illustrates an example of a message exchange when generating a recommendation.
The first recommendation engine 101 comprises a prediction processor 201 which is arranged to execute the prediction algorithm to generate one more personalised recommendations. The prediction processor 201 is coupled to a user profile 203 which stores user preference data used to generate the recommendations.
It will be appreciated that any suitable prediction algorithm can be used without detracting from the invention. Also, many different algorithms will be known to the person skilled in the art. For example a naive Bayes classifier which computes the probability that the user will appreciate a piece of content based on prior content the user liked or disliked can be used.
The prediction processor 201 is furthermore coupled to a recommendation processor 205. The recommendation processor 205 is operable to control and instigate the generation of a recommendation. Specifically, the recommendation processor 205 may interface with a user application and/or a user interface.
Specifically, as illustrated in FIG. 3, an application 301 may request a recommendation by submitting a user recommendation request to the recommendation processor 205 of the first recommendation engine 101. In response, the recommendation processor 205 initiates the generation of a recommendation from the prediction processor 201. The prediction processor 201 may initially evaluate whether such a recommendation can be provided. For example, the prediction processor 201 may initially assess whether the recommendation request relates to a content item for which the prediction processor 201 and the user profile 203 are able to provide recommendation. For example, if the recommendation is for a music clip, the prediction processor 201 and user profile 203 are not capable of providing recommendation in the specific example where the first recommendation engine 101 relates to cellular television programmes. Instead a recommendation indication indicating that the recommendation cannot be generated by the first recommendation engine 101 is generated. As another example, even if the request is for the type of content supported by the prediction processor 201 and the user profile 203, the data format used to describe the content may be different and may prevent the first recommendation engine 101 to reply. Accordingly, it may request a recommendation from a different device capable of handling this content format.
If the request is for content items for which the prediction processor 201 can generate the recommendation, the prediction processor 201 proceeds to generate a recommendation for one or more content items and provide these to the recommendation processor 205. In addition, the prediction processor 201 may include a recommendation indication which is indicative of the estimated quality or accuracy of the recommendation. For example, if the request is for a recommendation of a cellular television programme, the prediction processor 201 provides such a recommendation as well as an estimated recommendation quality to the recommendation processor 205. The recommendation processor 205 then evaluates whether a recommendation meeting a quality criterion can be generated by the prediction processed 201. Specifically, if the recommendation indication received from the prediction processed 201 is indicating that no recommendation can be provided, the recommendation processor 205 considers that the quality criterion has not been met. In other words, the quality criterion may simply be an assessment of whether a recommendation can be generated by the local recommendation engine. As another example, the recommendation processor 205 may compare an indication of the estimated quality of the recommendation and consider the quality criterion to be met only if this exceeds a threshold value.
The recommendation processor 205 is furthermore coupled to a recommendation receive processor 207 which is arranged to control an interaction with other recommendation engines 103-107 of the system in order to receive a recommendation from these. The recommendation receive processor 207 is coupled to a data network interface 209 which provides an interface to the event data network 109 and which specifically is arranged to transmit and receive recommendation event messages.
If the quality criterion is not met, the recommendation processor 205 initiates the recommendation receive processor 207 to generate and transmit one or more recommendation request messages. The recommendation request message may for example include a set of recommendation candidates. Thus, in some embodiments, the first recommendation engine 101 may include a set of content items that can be evaluated by other recommendation engines. In this specific example, the first recommendation engine 101 may for example include a listing of all the music clips stored on the mobile phone hosting the first recommendation engine 101. Thus, a recommendation engine optimised for music clips may recommend a music clip while ensuring that this is one which is available at the device initiating the request.
The recommendation request message may e.g. comprise the type and format of the recommendation that is sought. Optionally the request may also include some additional data which shall be used to compute the request (e.g. the content list to be filtered) .
Thus, the requesting recommendation engine may for instance send a list of possible options to be personalised and which a responding recommendation engine should return with the associated prediction (s) (for instance as a filtered and sorted list) . The requesting recommendation engine may also ask for a recommendation without providing any list of options. In this case the other recommendation engine can use its own options. Also, the requesting recommendation engine may ask for a template of recommendations which can then be matched against a list of possible options to derive actual recommendations .
The recommendation receive processor 207 makes the request available to the other peer recommendation engines 103-107 via the data network interface 209. In the example, the recommendation request message is distributed to all other recommendation engines 103-107 in the system without consideration of which recommendation engines are available and what their capabilities are.
In the example of FIG. 3, an individual recommendation request message is transmitted to each of the other recommendation engines 103-107. However, in other embodiments a single broadcast message may be transmitted which can be received by all other recommendation engines 103-107. Also, in some embodiments the recommendation request message may be addressed to a plurality of recommendation engines such as a selected group of recommendation engines.
The recommendation request message (s) are received by the other recommendation engines 103-107. In the example, the first recommendation engine 101 comprises a recommendation transmit processor 211 coupled to the data network interface 209 and the prediction processor 201. The recommendation transmit processor 211 is arranged receive recommendation request messages from other recommendation engines. In response, the recommendation transmit processor 211 determines whether to generate a recommendation. If so, the recommendation transmit processor 211 initiates a recommendation action by the prediction processor 201. The resulting recommendation is included in a recommendation provision message which is returned to the requesting recommendation engine.
In the example, the other recommendation engines 103-107 comprise similar or identical functionality to the recommendation transmit processor 211. Thus, each recommendation engine 103-107 receives a recommendation request message from the first recommendation engine 101. They then proceed to determine whether to respond to the received recommendation request message. This decision may be determined based only on internal characteristics such as whether the recommendation engine has sufficient available computational resource to generate the recommendation.
However, in other embodiments the decision is alternatively or additionally based on at least one characteristic of the recommendation request. For example, the recommendation request message may indicate that the request is for a specific content type, such as a music clip. The other recommendation engines 103-107 may simply evaluate whether they are able to provide such a recommendation (and/or may evaluate if they can support the format used to describe the music clips) . In the example, the second recommendation engine 103 is adapted to provide recommendations for music clips and accordingly it decides to generate a recommendation. However, the third and fourth recommendation engines 105, 107 are not adapted to provide music clip recommendations and therefore decide not to respond to the recommendation request message.
Thus, in the example, each peer recommendation engine individually and independently decides whether to reply or not based on the type and format of the request and optionally other local constraints (e.g. availability).
If the recommendation engine has decided to provide a recommendation response, it proceeds to generate a recommendation based on the user preference data of only the user profile of the recommendation engine. Thus, a recommendation is generated independently of the prediction algorithms and user profiles stored in other devices.
The second recommendation engine 103 then proceed to generate a recommendation provision message comprising an identification of the generated recommendation. The generated recommendation may be an identification of one or more content items, e.g. together with a user preference rating for each content item. Specifically, the recommendation provision message may comprise a list of content item identifiers, a general description (template) for the content item(s) or one or more actual content items (or e.g. content item addresses).
The recommendation provision message is then transmitted back to the first recommendation engine 101 by the event data network 109.
The first recommendation engine 101 receives the recommendation provision message and feeds it to the recommendation receive processor 207 which extracts the recommendation data and feeds it to the recommendation processor 205.
The recommendation processor 205 then proceeds to generate a recommendation for the application/user. In a simple embodiment, the recommendation processor 205 can simply provide the received recommendation to the application/user. However, in more complex embodiments the recommendation processor 205 may further evaluate the received recommendations and may for example compare or combine them with recommendations generated by the local recommendation engine. For example, if a list of recommended content items with associated reference values has been received from another recommendation engine these may be compared to a list of recommended content items and associated preference values generated by the local prediction processor 201. The recommendation processor 205 may then select one or more content items having the highest preference value regardless of whether these were generated locally or received from another device .
In the example, a recommendation provision message was only received from a single other recommendation engine. However, it will be appreciated that in other embodiments or scenarios, responses may be received from a plurality of other recommendation engines. Thus, a plurality of recommendation engines can be arranged to generate a recommendation provision message in response to the same recommendation request message. In particular, each recommendation engine may individually and independently of other recommendation engines decide whether to respond with a recommendation or not.
In some such embodiments, the first recommendation engine 101 may simply wait for a given time interval in which the other recommendation engines 103 -107 may reply with recommendation provision messages. At the end of the time interval, the first recommendation engine 101 may proceed to evaluate the received recommendations (if any) . Specifically, the first recommendation engine 101 may simply collates the recommendations and feed them to the requesting application/user. As another example, the recommendation processor can select one or more recommendations or may e.g. combine the recommendations. For example, recommendations may be fed to the requesting application/user depending on how many other recommendation engines have recommended each individual content item.
The recommendation system may furthermore comprise functionality for adapting the user profiles to the user. For example, the second recommendation engine may be arranged to receive user feedback for a generated recommendation which specifically may be the recommendation generated for the first recommendation engine 101. For example, the second recommendation engine 103 may comprise user input means for a user to manually provide a user preference input.
The second recommendation engine 103 may accordingly update its local user profile in response to the user preference input. In addition, the second recommendation engine 103 can generate a user feedback indication reflecting the user preference input and send this back to the first recommendation engine 101. The first recommendation engine 101 may then proceed to update its local user profile in response to the user feedback indication .
Specifically, in the example the first recommendation engine 101 comprises a learning processor 213 coupled to the data network interface 209 and the user profile 203. The learning processor 213 is arranged to update the user profile 203 in response to locally received information (e.g. a local user preference input) as well as in response to the feedback indication received from the second recommendation engine 103. Thus, the user profile 203 of the first recommendation engine 101 may be updated in response to user feedback provided at another recommendation engine 103-107 of the system.
The second recommendation engine 103 may specifically transmit a user input message which comprises an indication of the user preference input. This message may reflect the actual user input and the learning processor 213 may specifically update the user profile 203 based on the user input. Specifically the learning processor 213 may proceed to process the received user input from the second recommendation engine 103 in exactly the same way as if the user input had been provided to the first recommendation engine 101.
Alternatively or additionally, the second recommendation engine 103 may transmit a user profile update message which comprises an indication of how the user profile of the second recommendation engine 103 has been modified. Thus, the user profile update message may not provide information of the actual user input but rather may provide information of how the update affected the user profile of the second recommendation engine 103. For example, the user profile update message may comprise an indication of an increased or reduced preference rating for specific content item category. The learning processor 213 may proceed to update the user profile 203 in response to the received user profile update message. For example, if an indication of an increase in the preference rating for specific content item category in the user profile of the second recommendation engine 103 is received, the corresponding content item category of the user profile 203 may also be increased correspondingly.
It will be appreciated that the described recommendation system provides many advantages including one or more of the following advantages.
The approach allows the development and use of very specific (and therefore effective) recommenders . The approach may be based on existing recommendation engines and it does not necessitate a development of generic engines. The approach also alleviates the need to describe the capabilities of each recommender in a common format.
From the user point of view, one advantage is the benefit arising from personalisation being achieved by using the best recommender (the one with the best algorithm and the most accurate user profile) for the specific recommendation regardless of the specific device being used by the user. From the mobility point of view, this means for instance that users can receive on their mobile, alerts for content of interest (video, television programmes, News) which are generated by a recommendation engine located at their homes. FIG. 4 illustrates a method of operation for a recommendation system which includes a plurality of recommendation engines. Each recommendation engine comprises: a user profile for a first user; a prediction unit for generating a recommendation for the first user in response to the user profile; and an interface for exchanging recommendation event messages with other recommendation engines on a data communication coupling. At least one of the user profile and prediction units of at least some recommendation engines are different.
The method comprises at least one recommendation engine performing step 401 wherein a recommendation is generated in response to a received recommendation event message from a different recommendation engine and a recommendation indication from the prediction unit of the at least one recommendation engine.
It will be appreciated that the above description for clarity has described embodiments of the invention with reference to different functional units and processors. However, it will be apparent that any suitable distribution of functionality between different functional units or processors may be used without detracting from the invention. For example, functionality illustrated to be performed by separate processors or controllers may be performed by the same processor or controllers. Hence, references to specific functional units are only to be seen as references to suitable means for providing the described functionality rather than indicative of a strict logical or physical structure or organization. The invention can be implemented in any suitable form including hardware, software, firmware or any combination of these. The invention may optionally be implemented at least partly as computer software running on one or more data processors and/or digital signal processors. The elements and components of an embodiment of the invention may be physically, functionally and logically implemented in any suitable way. Indeed the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the invention may be implemented in a single unit or may be physically and functionally distributed between different units and processors .
Although the present invention has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Rather, the scope of the present invention is limited only by the accompanying claims. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in accordance with the invention. In the claims, the term comprising does not exclude the presence of other elements or steps.
Furthermore, although individually listed, a plurality of means, elements or method steps may be implemented by e.g. a single unit or processor. Additionally, although individual features may be included in different claims, these may possibly be advantageously combined, and the inclusion in different claims does not imply that a combination of features is not feasible and/or advantageous. Also the inclusion of a feature in one category of claims does not imply a limitation to this category but rather indicates that the feature is equally applicable to other claim categories as appropriate. Furthermore, the order of features in the claims does not imply any specific order in which the features must be worked and in particular the order of individual steps in a method claim does not imply that the steps must be performed in this order. Rather, the steps may be performed in any suitable order.

Claims

1. A recommendation system comprising: a plurality of recommendation engines; a data communication coupling between the recommendation engines; wherein each of the recommendation engines comprises: a user profile for a first user, prediction means for generating recommendations for the first user in response to the user profile, interface means for exchanging recommendation event messages with other recommendation engines on the data communication coupling, and recommendation means for generating a recommendation in response to a received recommendation event message and a recommendation indication from the prediction means; and wherein at least one of the user profile and prediction means of at least some recommendation engines are different.
2. The recommendation system of claim 1 wherein the user profiles for the at lest two recommendation engines are independent.
3. The recommendation system of claim 1 wherein the at lest two recommendation engines are arranged to provide recommendations for different types of content items.
4. The recommendation system of claim 1 wherein the recommendation event messages comprise at least one of a recommendation request message and a recommendation provision message, the recommendation request message comprising a request for a recommendation for the first user and the recommendation provision message comprising a recommendation for the first user.
5. The recommendation system of claim 4 wherein at least some recommendation engines comprise means for determining whether to respond to a received recommendation request message in response to a determination of whether at least one characteristic of a recommendation request of the received recommendation request message meets a criterion.
6. The recommendation system of claim 1 wherein a first recommendation engine comprises: means for receiving a user recommendation request for the first user, means for generating a recommendation request message requesting a recommendation for the first user and for providing the recommendation request message to the interface means of the first recommendation engine for transmission on the data communication coupling in response to the user recommendation request, means for receiving a recommendation provision message comprising a first recommendation for the first user from a second recommendation engine via the interface means of the first recommendation engine, and the recommendation means being arranged to generate the recommendation in response to the first recommendation; and the second recommendation engine comprises: means for receiving the recommendation request message from the interface means of the second recommendation engine, and means for providing the recommendation provision message to the interface means of the second recommendation engine for transmission to the first recommendation engine on the data communication coupling; and wherein the prediction means of the second recommendation engine is arranged to generate the first recommendation in response to user preference data of only the user profile of the second recommendation engine.
7. The recommendation system of claim 6 wherein the first recommendation engine is arranged to generate the recommendation request message in response to a detection that a recommendation meeting a quality criterion cannot be generated by the prediction means of the first recommendation engine based on the user profile of the first recommendation engine.
8. The recommendation system of claim 6 wherein a plurality of recommendation engines are arranged to generate a recommendation provision message in response to the same recommendation request message from the first recommendation engine; and the first recommendation engine is arranged to generate the user recommendation in response to the recommendations of the received recommendation provision messages.
9. The recommendation system of claim 6 wherein the recommendation request message is addressed to a plurality of recommendation engines.
10. A method of operation for a recommendation system including : a plurality of recommendation engines, each recommendation engine comprising: a user profile for a first user, prediction means for generating a recommendation for the first user in response to the user profile, and interface means for exchanging recommendation event messages with other recommendation engines on a data communication coupling; the method comprising at least one recommendation engine performing the step of: generating a recommendation in response to a received recommendation event message from a different recommendation engine and a recommendation indication from the prediction means of the at least one recommendation engine; and wherein at least one of the user profile and prediction means of at least some recommendation engines are different.
PCT/US2007/082812 2006-12-09 2007-10-29 A recommendation system and method of operation therefor WO2008073595A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB0624551A GB2444520A (en) 2006-12-09 2006-12-09 A recommendation system
GB0624551.8 2006-12-11

Publications (2)

Publication Number Publication Date
WO2008073595A2 true WO2008073595A2 (en) 2008-06-19
WO2008073595A3 WO2008073595A3 (en) 2008-09-12

Family

ID=37711798

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2007/082812 WO2008073595A2 (en) 2006-12-09 2007-10-29 A recommendation system and method of operation therefor

Country Status (2)

Country Link
GB (1) GB2444520A (en)
WO (1) WO2008073595A2 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11429682B2 (en) * 2019-06-25 2022-08-30 Sap Portals Israel Ltd. Artificial crowd intelligence via networking recommendation engines

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060031460A1 (en) * 2002-11-19 2006-02-09 Shouichi Araki Operation log cooperation utilizing device
US20060173838A1 (en) * 2005-01-31 2006-08-03 France Telecom Content navigation service

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
SE9801181L (en) * 1998-04-03 1999-10-04 Telia Ab Improvements in, or with regard to, telecommunications systems
US20020152175A1 (en) * 2001-04-17 2002-10-17 Armstrong John E. Methods and apparatus for the interoperablility and manipulation of data in a computer network

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060031460A1 (en) * 2002-11-19 2006-02-09 Shouichi Araki Operation log cooperation utilizing device
US20060173838A1 (en) * 2005-01-31 2006-08-03 France Telecom Content navigation service

Also Published As

Publication number Publication date
GB0624551D0 (en) 2007-01-17
GB2444520A (en) 2008-06-11
WO2008073595A3 (en) 2008-09-12

Similar Documents

Publication Publication Date Title
US9047367B2 (en) Socially collaborative filtering
GB2447868A (en) A distributed content item recommendation system
US11025583B2 (en) Recommendation system based on common interests in social networks
US20110184899A1 (en) Method and system for generating recommendations of content items
US20090271417A1 (en) Identifying User Relationships from Situational Analysis of User Comments Made on Media Content
RU2452024C1 (en) Content item recommendation
US20090222430A1 (en) Apparatus and Method for Content Recommendation
US9338249B2 (en) Distributed user profile
US7933408B2 (en) Asynchronous download
US9826060B1 (en) Predictive transmission of digital content
US11689771B2 (en) Customized recommendations of multimedia content streams
KR20070086310A (en) Method and apparatus for alert management
US20140074859A1 (en) System and method for rating audiences of network-based content of multiple content publishers
US10834467B2 (en) Single button selection to facilitate actions in a communications network
US8245267B2 (en) Methods, systems, and products for recording media on a user device
CN102714601A (en) Electronic device and operating method of the same
WO2022047074A1 (en) Systems and methods for peer-to-peer music recommendation processing
JP5527853B2 (en) Information provision system
WO2008073595A2 (en) A recommendation system and method of operation therefor
US20070253675A1 (en) Methods, systems, and products for recording media
Bonnefoy et al. “More Like This” or “Not for Me”: Delivering Personalised Recommendations in Multi-user Environments
CN108900593B (en) Distributed processing method for data of storage cabinet
GB2444519A (en) Recommendation system
Zhu et al. A recommender handoff framework for a mobile device
KR20090095000A (en) Life style management system capable of providing circumstance adaptive service and method thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07863597

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07863597

Country of ref document: EP

Kind code of ref document: A2