EP3840307B1 - Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social - Google Patents

Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social Download PDF

Info

Publication number
EP3840307B1
EP3840307B1 EP21157079.1A EP21157079A EP3840307B1 EP 3840307 B1 EP3840307 B1 EP 3840307B1 EP 21157079 A EP21157079 A EP 21157079A EP 3840307 B1 EP3840307 B1 EP 3840307B1
Authority
EP
European Patent Office
Prior art keywords
media content
media
user
bot
interacting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP21157079.1A
Other languages
German (de)
English (en)
Other versions
EP3840307A1 (fr
Inventor
Brendan O'DRISCOLL
Matt Montag
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Spotify AB
Original Assignee
Spotify AB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Spotify AB filed Critical Spotify AB
Priority to EP23195111.2A priority Critical patent/EP4262172A3/fr
Publication of EP3840307A1 publication Critical patent/EP3840307A1/fr
Application granted granted Critical
Publication of EP3840307B1 publication Critical patent/EP3840307B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9535—Search customisation based on user profiles and personalisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/07—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail characterised by the inclusion of specific contents
    • H04L51/10—Multimedia information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/02—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/52—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]

Definitions

  • Embodiments of the invention are generally related to providing digital media content, and are particularly related to a system and method for enabling use of a media content bot in a social messaging environment that supports group chat.
  • an increasingly common mode of user interaction is for users to communicate within a group chat or discussion.
  • the topic of a user's preference in music may arise within the context of a group chat.
  • users are (physically) present in the same physical room, and are (virtually) present in the same group chat room, there may be no way for users to interact in such a manner as to control which music is playing in the physical room, or for the users to engage at the same time with a bot to control music playback or receive music recommendations.
  • a conversational agent can leverage a user's interaction with the bot, to access a media server, which in turn can determine one or more items of media content appropriate for the user and/or their interaction.
  • the bot can recognize music commands within the context of a particular discussion, and/or utilize music recommendation services provided by a digital media content environment, to determine items of media content appropriate for the group of users and/or their interaction.
  • the system enables the use of media playback controls within the group chat, enabling the group of users to determine how and which media content is to be played.
  • a social messaging environment that supports conversational messaging may allow users to communicate in a group chat or discussion, there may be no way for users to interact in such a manner as to control which music is playing, or for the users to engage at the same time with a bot to control music playback or receive music recommendations.
  • described herein is a system and method for enabling use of a media content bot in a social messaging environment that supports group chat.
  • a conversational agent can leverage a user's interaction with the bot, to access a media server, which in turn can determine one or more items of media content appropriate for the user and/or their interaction.
  • the bot can recognize music commands within the context of a particular discussion, and/or utilize music recommendation services provided by a digital media content environment, to determine items of media content appropriate for the group of users and/or their interaction.
  • the system enables the use of media playback controls within the group chat, enabling the group of users to determine how and which media content is to be played.
  • Figure 1 illustrates an exemplary digital media content environment, in accordance with an embodiment.
  • a media device 102 operating as a client device, can receive and play media content provided by a media server system 142 (media server), or by another system or peer device.
  • the media device can be, for example, a personal computer system, handheld entertainment device, tablet device, smartphone, television, audio speaker, in-car entertainment system, or other type of electronic or media device that is adapted or able to prepare a media content for presentation, control the presentation of media content, and/or play or otherwise present media content.
  • Each of the media device and the media server can include, respectively, one or more physical device or computer hardware resources 104, 144, such as one or more processors (CPU), physical memory, network components, or other types of hardware resources.
  • physical device or computer hardware resources 104, 144 such as one or more processors (CPU), physical memory, network components, or other types of hardware resources.
  • a media server can support the simultaneous use of a plurality of client media devices.
  • a client media device can access media content provided by a plurality of media servers, or switch between different media streams produced by one or more media servers.
  • the media device can optionally include a touch-enabled or other type of display screen having a user interface 106, which is adapted to display media options, for example as an array of media tiles, thumbnails, or other format, and to determine a user interaction or input. Selecting a particular media option, for example a particular media tile or thumbnail, can be used as a command by a user and/or the media device, to the media server, to download, stream or otherwise access a corresponding particular media content item or stream of media content.
  • a touch-enabled or other type of display screen having a user interface 106, which is adapted to display media options, for example as an array of media tiles, thumbnails, or other format, and to determine a user interaction or input. Selecting a particular media option, for example a particular media tile or thumbnail, can be used as a command by a user and/or the media device, to the media server, to download, stream or otherwise access a corresponding particular media content item or stream of media content.
  • the media device can also include a software media application 108, together with an in-memory client-side media content buffer 110, and a client-side data buffering logic or software component 112, which can be used to control the playback of media content received from the media server, for playing either at a requesting media device (i.e., controlling device) or at a controlled media device (i.e., controlled device), in the manner of a remote control.
  • a software media application 108 together with an in-memory client-side media content buffer 110, and a client-side data buffering logic or software component 112, which can be used to control the playback of media content received from the media server, for playing either at a requesting media device (i.e., controlling device) or at a controlled media device (i.e., controlled device), in the manner of a remote control.
  • a connected media environment logic or software component 120 can be provided at the media device, either as part of the media application, or separately, for example as a firmware, to enable the media device to participate within a connected media environment (e.g., a Spotify Connect environment) that enables a user to control the playback of media content at such controlled devices.
  • a connected media environment e.g., a Spotify Connect environment
  • the client-side data buffering logic together with the media content buffer, enables a portion of media content items, or samples thereof, to be pre-buffered at a client media device. For example, while media options are being prepared for display on a user interface, e.g., as media tiles or thumbnails, their related media content can be pre-buffered at the same time, and cached by one or more media devices in their media content buffers, for prompt and efficient playback when required.
  • the media server can include an operating system 146 or other processing environment which supports execution of a media server 150 that can be used, for example, to stream music, video, or other forms of media content to a client media device, or to a controlled device.
  • an operating system 146 or other processing environment which supports execution of a media server 150 that can be used, for example, to stream music, video, or other forms of media content to a client media device, or to a controlled device.
  • the media server can provide a subscription-based media streaming service, for which a client media device or user can have an associated account and credentials, and which enable the user's media device to communicate with and receive content from the media server.
  • a received media-access request from a client media device can include information such as, for example, a network address, which identifies a destination media device to which the media server should stream or otherwise provide media content, in response to processing the media-access request.
  • a user may own several media devices, such as a smartphone and an audio speaker, which can play media content received from a media server.
  • identifying information provided with a media-access request can include an identifier, such as an IP address, MAC address, or device name, which identifies that the media-access request is intended for use with a particular destination device.
  • an identifier such as an IP address, MAC address, or device name
  • the media server can then send the requested media and/or forward the media-access request to the audio speaker, even though the request originated at the user's smartphone.
  • One or more application interface(s) 148 can receive requests from client media devices, or from other systems, to retrieve media content from the media server.
  • a context database 162 can store data associated with the presentation of media content by a client media device, including, for example, a current position within a media stream that is being presented by the media device, or a playlist associated with the media stream, or one or more previously-indicated user playback preferences.
  • the media server can transmit context information associated with a media stream to a media device that is presenting that stream, so that the context information can be used by the device, and/or displayed to the user.
  • the context database can be used to store a media device's current media state at the media server, and synchronize that state between devices, in a cloud-like manner. Alternatively, media state can be shared in a peer-to-peer manner, wherein each device is aware of its own current media state which is then synchronized with other devices as needed.
  • the media server can transmit context information associated with an active media content to the newly-appointed destination device, for use by that device in playing the media content.
  • a media content database 164 can include media content, for example music, songs, videos, movies, or other media content, together with metadata describing that media content.
  • the metadata can be used to enable users and client media devices to search within repositories of media content, to locate particular media content items.
  • a server-side media content buffering logic or software component 180 can be used to retrieve or otherwise access media content items, in response to requests from client media devices or other systems, and to populate a server-side media content buffer 181, at a media delivery component or streaming service 152, with streams 182, 184, 186 of corresponding media content data, which can then be returned to the requesting device or to a controlled device.
  • a plurality of client media devices, media server systems, and/or controlled devices can communicate with one another using a network, for example the Internet 190, a local area network, peer-to-peer connection, wireless or cellular network, or other form of network.
  • a user 192 can interact 194 with the user interface at a client media device, and issue requests to access media content, for example the playing of a selected music or video item at their device, or at a controlled device, or the streaming of a media channel or video stream to their device, or to a controlled device.
  • the user's selection of a particular media option can be communicated 196 to the media server, via the server's application interface.
  • the media server can populate its server-side media content buffer at the server 204, with corresponding media content, 206 including one or more streams of media content data, and can then communicate 208 the selected media content to the user's media device, or to a controlled device as appropriate, where it can be buffered in a client-side media content buffer for playing at the device.
  • the system can include a server-side media gateway or access point 220, or other process or component, which operates as a load balancer in providing access to one or more servers, for use in processing requests at those servers.
  • the system can enable communication between a client media device and a server, via an access point at the server, and optionally the use of one or more routers, to allow requests from the client media device to be processed either at that server and/or at other servers.
  • Spotify clients operating on media devices can connect to various Spotify back-end processes via a Spotify "accesspoint", which forwards client requests to other servers, such as sending one or more metadataproxy requests to one of several metadataproxy machines, on behalf of the client or end user.
  • the system can enable access to digital media content via a social messaging environment, for example through the use of a client-side messaging application, or a software conversational agent or "bot".
  • the media server can recommend or otherwise determine items of media content appropriate to a particular user interaction.
  • the recommended media content can be provided to a media device, in the form of a link or other type of reference by which the user of the device can stream, download, access, or otherwise use the media content.
  • Figure 2 illustrates a system that enables access to media content from a social messaging environment, in accordance with an embodiment.
  • a social messaging environment 240 enables, for each user, a user environment 244 that provides a user experience, and the use of a messaging application 246 that enables interaction by the user with the social messaging environment and with other users.
  • social messaging environments including, for example, social networking or social media environments (e.g., Instagram), which support the sharing of picture-based or other messages between different users, or social messaging environments (e.g., Facebook Messenger) which support the use of a media content bot that operates according to a bot conversation logic to provide access to a media content environment.
  • social networking or social media environments e.g., Instagram
  • social messaging environments e.g., Facebook Messenger
  • the media server can include a recommendation application program interface (API), which is accessible, e.g., via a HTTP URL, and which provides access to a recommendation engine 252 that provides media content recommendations in response to interactions or conversations with the social messaging environment.
  • API recommendation application program interface
  • the messaging application or bot can access the media server, via the recommendation API, and provide a user interaction information 245, to the recommendation engine, for its use in determining a media content recommendation 249, and populating 250 the media delivery component or streaming service with the recommended media content 251.
  • the media server can then return, as part of the conversation or other interaction with the user, a media content recommendation in the form of a link or other type of reference by which the user of the device can stream, download, access, or otherwise use the media content.
  • the recommendation engine can determine a media content recommendation based on criteria such as, for example, a user's prior listening history, or favorite media content; other user characteristics such as demographic information; external-sourced information, such as the current weather at the user's locations; or other types of criteria.
  • the link or other type of reference to access the recommended media content can be processed by the media device, to request that the recommended media content be returned either to the media device or to a controlled device, for playback at that device.
  • link or reference to an item of media content can be provided as: https://open.spotify.com/track/01G6a8O85SXIH05upiDN1N
  • the above link can then be passed or otherwise processed by a media server web service API, to request the associated media content.
  • the messaging application or media content bot can either operate anonymously, or can perform an authentication or account verification at the media server, for example, to indicate the user as a Spotify user, and customize the response accordingly.
  • the system enables the use of a conversational agent or bot which provides a conversation-like interaction with a user, including simulating how a human participant might respond or otherwise behave as a conversational partner in response to input from the user.
  • a bot can be configured to scan for particular keywords within a text received as user input during a conversation with the bot, and respond accordingly to those keywords.
  • the bot can be configured to employ natural language processing (NLP), to extract semantic or other useful information from the conversational text, and provide a more complex response.
  • NLP natural language processing
  • the system can employ a media content bot, which leverages a user's interaction with the bot, to access a media server, which in turn can determine one or more items of media content that are appropriate for the user and/or their interaction.
  • the recommended or otherwise determined media content can then be provided to the user, or to a media device, in the form of a link, playlist, or other type of reference by which the user can stream, download, access, or otherwise use the recommended media content.
  • Figure 3 illustrates the use of a social messaging environment with a digital media content environment, in accordance with an embodiment.
  • a social messaging environment enables a messaging application to be used by a user, to interact with the social messaging environment.
  • the social messaging environment can provide a bot store 250, which operates as a shelf of one or more available bot(s) 252, which can be selected and installed 254 by a user, to operate within their user environment.
  • the store can include a media content bot 260, which includes and operates according to a bot conversation logic 262.
  • the media content bot can be used to provide access to a media content environment, from within a social messaging environment.
  • the media content bot can act in a reactive manner, awaiting an input from a user, or can act proactively, sending messages to the user in the manner of a push-notification.
  • the bot can access the media content environment 270, and provide interaction information for use in determining a media content recommendation; and, if appropriate, populating the media delivery component or streaming service with the appropriate recommended media content, for delivery either to the user's media device (208) or to another device (209), for example a controlled device 272, such as an audio speaker or other output device.
  • a controlled device 272 such as an audio speaker or other output device.
  • Figure 4 further illustrates the use of social messaging environment with a digital media content environment, in accordance with an embodiment.
  • the system can respond at appropriate points during a bot interaction 280 or conversation by the user with the media content bot, for example in response to a user's text input or communication with the system or with other users.
  • the user's interaction information can be communicated to the media server via the recommendation API, as input to the recommendation engine.
  • the media server can then return to the media content bot, as part of the conversation or interaction with the user, a media content recommendation, in the form of a link, playlist, or other type of reference by which the user, or the controlled device, can stream, download, access, or otherwise use the recommended media content.
  • a media content recommendation in the form of a link, playlist, or other type of reference by which the user, or the controlled device, can stream, download, access, or otherwise use the recommended media content.
  • the reference to access the recommended media content can be processed by the user's media device as described above, to request that the recommended media content be returned either to the media device, or to a controlled or other device, for playback at that device.
  • the media server can stream or buffer the recommended media content.
  • the media content bot can be configured to parse a stream of input text within a user environment for, e.g., an artist name, such as the user indicating "My favorite artist is Rihanna".
  • the artist name can be passed from the media content bot to the recommendation API.
  • the resultant information received from the recommendation API can be provided in the manner of a conversation, e.g. "Thanks, do you want to hear some new tracks from Rihanna?", and can return a link, playlist, or other type of reference to that item of media content
  • the media content bot can either operate anonymously, or can perform an authentication or account verification, for example, to indicate the user as a Spotify user.
  • the device can be either directed to a landing page (e.g., in a Web browser, or an app), or can request the user to log in, or can play the songs anonymously without requiring a login.
  • a landing page e.g., in a Web browser, or an app
  • the bot can also prompt the user to determine, for example, whether the recommended media content should be played on the user's media device, or at another (i.e., controlled) device.
  • the system enables use of a media content bot in a social messaging environment that supports group chat.
  • the bot can recognize music commands within the context of a particular discussion, and/or utilize music recommendation services provided by a digital media content environment, to determine items of media content appropriate for the group of users and/or their interaction.
  • the system enables the use of media playback controls within the group chat, enabling the group of users to determine how and which media content is to be played.
  • Figure 5 illustrates the use of a media content bot in a social messaging environment that supports group chat, in accordance with an embodiment.
  • the bot can similarly access the media content environment 295, and provide group interaction information, for use in determining a media content recommendation.
  • the media content bot can operate in the form of an assistant or concierge, to assist the group of users in determining, or example, an appropriate media content to play, or at which device the media content should be played.
  • the media server can then respond, if appropriate, by populating the media delivery component or streaming service with the appropriate recommended media content, for delivery to a device, for example to an audio speaker or other controlled output device, reflecting a group-directed control 296.
  • the system enables multiple users, who may or may not be physically present in a same room, to participate in a group chat in such a manner as to control which music is to be played in that room.
  • the system enables multiple users to engage, at the same time, with a bot as part of a group chat, to control the selection of music playback, or to receive music recommendations.
  • the media content bot can be configured to help guide users within the context of a group chat or discussion, to control the playback of a particular media content, as illustrated in the example interaction below:
  • Media Content Bot Users (Group Chat): How can I help you?
  • Oscar Play Justin Bieber ⁇ /spotify queue Justin Sieber ⁇
  • Media Content Bot Sure thing, Oscar!
  • each user can interact with the media content bot, for example to request that a particular item of media content be played.
  • the bot can respond accordingly, for example: Media Content Bot: Users (Group Chat): How can I help you? (Oscar): Play Justin Bieber ⁇ /spotify queue Justin Bieber ⁇ ⁇ (Media Content Bot): Sure thing, Oscar! ⁇ (Alva): Noooo /spotify skip Justin Bieber
  • the media content bot can be configured to respond by waiting, until the group has generally agreed on a next song to play, and instructed the bot accordingly, at which point the bot can proceed as directed:
  • Media Content Bot Users (Group Chat): How can I help you?
  • Media Content Bot Sure thing, Oscar! ⁇
  • Alva Niice ⁇ /spotify play Billy Joel Play Billy Joel next!
  • the above interaction is provided by way of example, to illustrate an interaction between a group of users and a media content bot within the context of a group chat or discussion.
  • Other types or forms of media content bots and interactions can be supported.
  • a similar approach can also be used to enable the group to control other aspects of their physical or virtual environment, via the media content bot and a group chat, for example by controlling the ambient lighting of their room or the volume at which a media content is played, or by determining to promote a particular artist within a social messaging environment.
  • Figure 6 illustrates a process of enabling use of a media content bot in a social messaging environment that supports group chat, in accordance with an embodiment.
  • a social messaging environment that supports conversational messaging is provided, which enables a messaging application to be used by a user, to interact with a media content bot, and access a media content environment.
  • the bot provides interaction information to the media content environment, and provides interaction information, for use in determining an appropriate media, and populating a media delivery component or streaming service with the appropriate media content, for delivery to at least one of the user's media device or to a controlled device.
  • step 306 additional users can use a messaging application, to interact with the media content bot and original user as part of a group chat, for use in accessing the media content environment as part of the group chat.
  • the media content bot enables the group of users to direct the media server to control the playing of an appropriate media content at a controlled or other destination device.
  • the present invention may be conveniently implemented using one or more conventional general purpose or specialized digital computer, computing device, machine, or microprocessor, including one or more processors, memory and/or computer readable storage media programmed according to the teachings of the present disclosure.
  • Appropriate software coding can readily be prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software art.
  • the present invention includes a computer program product which is a non-transitory storage medium or computer readable medium (media) having instructions stored thereon/in which can be used to program a computer to perform any of the processes of the present invention.
  • the storage medium can include, but is not limited to, any type of disk including floppy disks, optical discs, DVD, CD-ROMs, microdrive, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of media or device suitable for storing instructions and/or data.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Information Transfer Between Computers (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Claims (15)

  1. Procédé exécuté par un bot de contenu multimédia (260) fourni dans un environnement de messagerie de réseau social (240) qui prend en charge la conversation de groupe, le procédé comprenant de :
    interagir (244) avec au moins un utilisateur (192) dans le cadre d'une conversation de groupe (294) ;
    reconnaître une commande musicale dans le contexte d'une discussion particulière dans le conversation de groupe (294) ; et
    en réponse à la commande musicale, accéder (295) à un serveur multimédia (150) pour déterminer un élément recommandé de contenu multimédia (206) comprenant de la musique ou de la vidéo, et pour fournir l'élément de contenu multimédia recommandé à un dispositif multimédia (102 ; 272) en vue de la lecture sur le périphérique multimédia.
  2. Procédé selon la revendication 1, dans lequel le bot (260) fonctionne selon une logique conversationnelle de bot (262).
  3. Procédé selon une quelconque des revendications précédentes, dans lequel le au moins un utilisateur est une pluralité d'utilisateurs (192, 290).
  4. Procédé selon la revendication 3, dans lequel l'interaction (244) comprend l'engagement avec plusieurs utilisateurs de la pluralité d'utilisateurs en même temps.
  5. Procédé selon une quelconque des revendications précédentes, dans lequel l'interaction (244) comprend l'interaction (302) avec un utilisateur d'origine (192) avant l'accès (295), puis l'interaction (306) avec des utilisateurs supplémentaires (290).
  6. Procédé selon la revendication 5, comprenant en outre de :
    tandis que les utilisateurs supplémentaires (290) participent au conversation de groupe, permettre (308) aux utilisateurs (290, 192) de diriger le serveur multimédia (150) pour contrôler la lecture au niveau du dispositif multimédia (102 ; 272).
  7. Procédé selon une quelconque des revendications précédentes, dans lequel l'interaction (244) consiste à aider les utilisateurs à déterminer un contenu multimédia (251) à lire ou sur quel dispositif multimédia (102 ; 272) le contenu multimédia (251) doit être lu ou guider le au moins un utilisateur (192) dans le contexte de la conversation de groupe pour contrôler la lecture de l'élément de contenu multimédia (251).
  8. Procédé selon une quelconque des revendications précédentes, dans lequel l'interaction (244) comprend le fait d'agir de manière réactive, en attendant une entrée de l'utilisateur (192).
  9. Procédé selon la revendication 8, dans lequel l'interaction (244) comprend la réponse en attendant qu'au moins un utilisateur (192, 290) se soit généralement mis d'accord sur une prochaine chanson à jouer.
  10. Procédé selon une quelconque des revendications précédentes, dans lequel l'interaction (244) comprend une action proactive, l'envoi d'un message à l'utilisateur (192) en tant que notification push.
  11. Procédé selon une quelconque des revendications précédentes, dans lequel l'interaction (244) comprend la recherche de mots clés particuliers dans un texte reçu en tant qu'entrée utilisateur, ou l'utilisation de NLP pour extraire des informations sémantiques ou d'autres informations utiles du texte, et répondre en conséquence.
  12. Procédé selon une quelconque des revendications précédentes, dans lequel l'accès (295) comprend la réalisation d'une authentification ou d'une vérification de compte au niveau du serveur multimédia (150).
  13. Procédé selon une quelconque des revendications précédentes, dans lequel l'accès (295) comprend l'accès au serveur multimédia (150) via une API.
  14. Produit de programme informatique (250) comprenant des composants exécutables par ordinateur pour amener un bot de contenu multimédia (260) à exécuter toutes les étapes du procédé selon une quelconque des revendications précédentes lorsque les composants exécutables par ordinateur sont exécutés sur des circuits de traitement compris dans l'environnement de messagerie de réseau social (240).
  15. Bot de contenu multimédia (260) configuré pour être fourni dans un environnement de messagerie de réseau social (240) qui prend en charge la conversation de groupe, le bot de contenu multimédia (260) comprenant :
    des moyens pour interagir (244) avec au moins un utilisateur (192) dans le cadre d'une conversation de groupe (294) ;
    des moyens pour reconnaître une commande musicale dans le contexte d'une discussion particulière dans la conversation de groupe (294) ; et
    des moyens pour, en réponse à la commande musicale, accéder (295) à un serveur multimédia (150) pour déterminer un élément recommandé de contenu multimédia (206) comprenant de la musique ou de la vidéo, et pour fournir l'élément de contenu multimédia recommandé à un dispositif multimédia (102 ; 272) en vue de la lecture sur le périphérique multimédia.
EP21157079.1A 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social Active EP3840307B1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP23195111.2A EP4262172A3 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un robot de contenu multimédia dans un environnement de messagerie sociale

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201662440843P 2016-12-30 2016-12-30
EP17208892.4A EP3343844B1 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social
EP19191410.0A EP3585007B1 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
EP17208892.4A Division EP3343844B1 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social
EP19191410.0A Division EP3585007B1 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social

Related Child Applications (1)

Application Number Title Priority Date Filing Date
EP23195111.2A Division EP4262172A3 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un robot de contenu multimédia dans un environnement de messagerie sociale

Publications (2)

Publication Number Publication Date
EP3840307A1 EP3840307A1 (fr) 2021-06-23
EP3840307B1 true EP3840307B1 (fr) 2023-09-06

Family

ID=60781790

Family Applications (4)

Application Number Title Priority Date Filing Date
EP21157079.1A Active EP3840307B1 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social
EP17208892.4A Active EP3343844B1 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social
EP23195111.2A Pending EP4262172A3 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un robot de contenu multimédia dans un environnement de messagerie sociale
EP19191410.0A Active EP3585007B1 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social

Family Applications After (3)

Application Number Title Priority Date Filing Date
EP17208892.4A Active EP3343844B1 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social
EP23195111.2A Pending EP4262172A3 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un robot de contenu multimédia dans un environnement de messagerie sociale
EP19191410.0A Active EP3585007B1 (fr) 2016-12-30 2017-12-20 Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social

Country Status (2)

Country Link
US (2) US10430483B2 (fr)
EP (4) EP3840307B1 (fr)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9634855B2 (en) 2010-05-13 2017-04-25 Alexander Poltorak Electronic personal interactive device that determines topics of interest using a conversational agent
US10854180B2 (en) 2015-09-29 2020-12-01 Amper Music, Inc. Method of and system for controlling the qualities of musical energy embodied in and expressed by digital music to be automatically composed and generated by an automated music composition and generation engine
US9721551B2 (en) 2015-09-29 2017-08-01 Amper Music, Inc. Machines, systems, processes for automated music composition and generation employing linguistic and/or graphical icon based musical experience descriptions
US10063600B1 (en) 2017-06-19 2018-08-28 Spotify Ab Distributed control of media content item during webcast
US20190260826A1 (en) * 2018-02-21 2019-08-22 Artem Gurtovoy P2p video communication with a third-parties
US10706086B1 (en) * 2018-03-12 2020-07-07 Amazon Technologies, Inc. Collaborative-filtering based user simulation for dialog systems
US10964299B1 (en) 2019-10-15 2021-03-30 Shutterstock, Inc. Method of and system for automatically generating digital performances of music compositions using notes selected from virtual musical instruments based on the music-theoretic states of the music compositions
US11037538B2 (en) 2019-10-15 2021-06-15 Shutterstock, Inc. Method of and system for automated musical arrangement and musical instrument performance style transformation supported within an automated music performance system
US11024275B2 (en) 2019-10-15 2021-06-01 Shutterstock, Inc. Method of digitally performing a music composition using virtual musical instruments having performance logic executing within a virtual musical instrument (VMI) library management system
KR102465254B1 (ko) * 2020-03-19 2022-11-09 주식회사 카카오 컨텐츠와 관련된 추천 컨텐츠를 제공하는 방법, 사용자 단말 및 컨텐츠 제공 서버
US11470025B2 (en) 2020-09-21 2022-10-11 Snap Inc. Chats with micro sound clips
US11729122B2 (en) * 2020-12-23 2023-08-15 Atlassian Pty Ltd. Content suggestion system for real-time communication environments
KR20220105017A (ko) * 2021-01-19 2022-07-26 라인플러스 주식회사 관심 컨텐츠를 표시하는 방법, 시스템, 및 컴퓨터 프로그램
US20220353223A1 (en) * 2021-04-29 2022-11-03 Spotify Ab Text command based group listening session playback control
US20230403244A1 (en) * 2021-06-15 2023-12-14 Meta Platforms, Inc. Methods, mediums, and systems for responding to a user service prompt
US11947775B2 (en) * 2021-06-25 2024-04-02 Atlassian Pty Ltd. Service for extracting message data for display in user interface widgets

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2368225B (en) * 2000-10-17 2003-12-10 Hewlett Packard Co Inviting assistant entity into a network communication session
US9405827B2 (en) * 2006-11-30 2016-08-02 Red Hat, Inc. Playlist generation of content gathered from multiple sources
US8091032B2 (en) * 2006-11-30 2012-01-03 Red Hat, Inc. Automatic generation of content recommendations weighted by social network context
US7881984B2 (en) * 2007-03-30 2011-02-01 Amazon Technologies, Inc. Service for providing item recommendations
US8555177B1 (en) * 2009-04-22 2013-10-08 Google, Inc. Real-time interaction during video viewing
US20130159126A1 (en) * 2011-12-16 2013-06-20 Amr Elkady With-me social interactivity platform
US9247306B2 (en) * 2012-05-21 2016-01-26 Intellectual Ventures Fund 83 Llc Forming a multimedia product using video chat
KR102084507B1 (ko) * 2013-09-05 2020-03-04 삼성전자주식회사 홈 네트워크 시스템에서 홈 디바이스를 원격으로 제어하는 방법 및 장치
US20160255126A1 (en) * 2014-03-01 2016-09-01 William Sarris Application and method for conducting group video conversations and meetings on mobile communication devices
US10223432B2 (en) * 2014-09-29 2019-03-05 International Business Machines Corporation Interactive social media access to data systems
US10116596B2 (en) * 2014-09-29 2018-10-30 International Business Machines Corporation Personalizing data system components and data sources as chatbots in a group chat session
US11089132B2 (en) * 2016-03-29 2021-08-10 Microsoft Technology Licensing, Llc Extensibility for context-aware digital personal assistant
US9876909B1 (en) * 2016-07-01 2018-01-23 At&T Intellectual Property I, L.P. System and method for analytics with automated whisper mode
US9940390B1 (en) * 2016-09-27 2018-04-10 Microsoft Technology Licensing, Llc Control system using scoped search and conversational interface

Also Published As

Publication number Publication date
EP3840307A1 (fr) 2021-06-23
US10430483B2 (en) 2019-10-01
EP3343844A1 (fr) 2018-07-04
US11514117B2 (en) 2022-11-29
US20200272673A1 (en) 2020-08-27
EP4262172A2 (fr) 2023-10-18
EP3343844B1 (fr) 2019-08-28
EP3585007B1 (fr) 2021-02-17
EP4262172A3 (fr) 2024-01-17
US20180189408A1 (en) 2018-07-05
EP3585007A1 (fr) 2019-12-25

Similar Documents

Publication Publication Date Title
EP3840307B1 (fr) Système et procédé d'utilisation d'un contenu multimédia « bot » dans un environnement de messagerie social
US11620972B2 (en) System and method for association of a song, music, or other media content with a user's video content
US11297017B2 (en) System and method for providing digital media content with a conversational messaging environment
US9798514B2 (en) System and method for color beat display in a media content environment
US11783359B2 (en) Audio advertising interaction with voice interactive devices
EP3343484A1 (fr) Système et procédé d'association d'une chanson, du musique ou d'un autre contenu multimédia avec le contenu d'un video utilisateur.
US11435892B2 (en) System and method for enabling interaction with an electronic device
US20200264844A1 (en) Communicating shuffled media content
US11868390B2 (en) Communicating shuffled media content
KR102228375B1 (ko) 다수의 스트리밍 콘텐츠들을 재생하는 방법 및 시스템

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AC Divisional application: reference to earlier application

Ref document number: 3585007

Country of ref document: EP

Kind code of ref document: P

Ref document number: 3343844

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20211221

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602017074044

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04L0012580000

Ipc: H04L0051020000

Ref country code: DE

Ref legal event code: R079

Free format text: PREVIOUS MAIN CLASS: H04L0012580000

Ipc: H04L0051020000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: H04L 51/04 20220101ALN20230310BHEP

Ipc: H04L 51/52 20220101ALI20230310BHEP

Ipc: H04L 51/10 20220101ALI20230310BHEP

Ipc: H04L 51/02 20220101AFI20230310BHEP

INTG Intention to grant announced

Effective date: 20230403

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230513

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AC Divisional application: reference to earlier application

Ref document number: 3343844

Country of ref document: EP

Kind code of ref document: P

Ref document number: 3585007

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602017074044

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20230906

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231207

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231106

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231206

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231207

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20231107

Year of fee payment: 7

Ref country code: DE

Payment date: 20231106

Year of fee payment: 7

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1610045

Country of ref document: AT

Kind code of ref document: T

Effective date: 20230906

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240106

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240106

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240108

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602017074044

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

26N No opposition filed

Effective date: 20240607

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20231231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231220

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231231