US20230359655A1 - Enabling exchange of conversational bot skills during conversational chat - Google Patents
Enabling exchange of conversational bot skills during conversational chat Download PDFInfo
- Publication number
- US20230359655A1 US20230359655A1 US18/350,661 US202318350661A US2023359655A1 US 20230359655 A1 US20230359655 A1 US 20230359655A1 US 202318350661 A US202318350661 A US 202318350661A US 2023359655 A1 US2023359655 A1 US 2023359655A1
- Authority
- US
- United States
- Prior art keywords
- conversational
- skill
- bot
- conversational bot
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 claims abstract description 40
- 230000015654 memory Effects 0.000 claims description 15
- 238000004891 communication Methods 0.000 abstract description 36
- 230000000694 effects Effects 0.000 description 37
- 238000012545 processing Methods 0.000 description 28
- 238000003860 storage Methods 0.000 description 27
- 238000013528 artificial neural network Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 10
- 230000009471 action Effects 0.000 description 9
- 238000003058 natural language processing Methods 0.000 description 8
- 230000002776 aggregation Effects 0.000 description 7
- 238000004220 aggregation Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 238000005201 scrubbing Methods 0.000 description 5
- 230000003993 interaction Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 230000004931 aggregating effect Effects 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 101000685088 Buthus occitanus tunetanus Alpha-toxin Bot1 Proteins 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000003490 calendering Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9032—Query formulation
- G06F16/90332—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9538—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/285—Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/02—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
Definitions
- Entities are increasingly using conversational bots to handle user requests and commands related to their goods and services. Users often input complex queries and commands to conversational bots that cannot adequately process those requests for various reasons. For example, some requests are simply not supported by a corresponding conversational bot, and some requests may not be understood correctly. Additionally, while some requests may be understood correctly by a conversational bot, the conversational bot may nonetheless not have the requisite information needed to execute a skill needed to appropriately respond.
- Non-limiting examples of the present disclosure describe systems, methods, and devices for enabling conversational bot communications. Examples described herein relate to one or more conversational bots, which may execute one or more skills. Each skill may be described in one or more manifests.
- a manifest may describe an object that is capable of sending and receiving conversational bot framework-compatible messages.
- a manifest may allow configuration, registration, and publishing data about a conversational bot and/or skill to be authored, stored, and transmitted between services.
- a manifest may describe how to contact a conversational bot, what capabilities a conversational bot offers, and information about how it should be published.
- a user-bot conversation may be initiated via a messaging construct.
- the messaging construct may comprise a web chat interface, an application chat interface, or a digital assistant interface, for example.
- a user may provide a query or command to the conversational bot.
- the conversational bot may determine whether it has a skill that is required to fulfill the query or command. If the conversational bot has the skill, it may simply perform the skill and in some instances communicate the completion of the query or command to the user.
- the conversational bot may determine whether there is another conversational bot that has the skill.
- the conversational bot that received the query or command (the primary conversational bot) may query a manifest marketplace or other skill repository to determine whether there is another conversational bot that may perform the skill and fulfill the query or command for the user. If a determination is made that there is another conversational bot that can execute the skill (the secondary conversational bot) the primary conversational bot may pass control of the conversation to the secondary conversational bot until the query or command is fulfilled. Passing the conversation may simply mean that the secondary conversational bot perform actions and/or communicate with the primary conversational bot and need not necessarily communicate directly via the user chat interface.
- the secondary conversational bot may send requests for information to the primary conversational bot, the primary conversational bot may obtain that information, and it may be passed back to the secondary conversational bot for skill execution. Once the secondary conversational bot has executed a requisite skill to complete an original request, the primary conversational bot may take control of the conversation back over and respond to the original query or command.
- an identity and a classification of a plurality of skills, manifests, and conversational bots may be stored in a manifest marketplace.
- the manifest marketplace may be searchable by skill type.
- the skills, conversational bots, and/or manifests may be scored based on website ranking factors. For example, websites that have associated conversational bots may have a rank score that may be provided to the manifest marketplace.
- the manifest marketplace may use the website ranking and in some cases one or more additional factors to rank conversational bots and associated skills. Skill requests from conversational bots may be routed to the manifest marketplace to determine whether there is a bot that can execute a particular skill needed to fulfill a request.
- FIG. 1 is a schematic diagram illustrating an example distributed computing environment for enabling conversational bot communications.
- FIG. 2 illustrates a skill manifest and associated exemplary elements included in the skill manifest.
- FIG. 3 illustrates a computing device and an exemplary communication with a conversational bot associated with a restaurant entity.
- FIG. 4 illustrates bot-to-bot communications and the processing of a command from the bot communication of FIG. 3 .
- FIG. 5 illustrates a computing device that displays the result of the processing and bot-to-bot communications performed in relation to FIG. 4 .
- FIG. 6 is a schematic diagram illustrating an example distributed computing environment for identifying relevant skills from a manifest marketplace in relation to natural language input queries and commands.
- FIG. 7 A is an exemplary method for enabling conversational bot communications in relation to a natural language input.
- FIG. 7 B is another exemplary method for enabling conversational bot communications in relation to a natural language input.
- FIG. 8 is an exemplary method for enabling conversational bot communications in relation to a natural language input and a manifest marketplace.
- FIGS. 9 and 10 are simplified diagrams of a mobile computing device with which aspects of the disclosure may be practiced.
- FIG. 11 is a block diagram illustrating example physical components of a computing device with which aspects of the disclosure may be practiced.
- FIG. 12 is a simplified block diagram of a distributed computing system in which aspects of the present disclosure may be practiced.
- a conversational bot may be published by, and/or associated with, a particular entity.
- a first company may host a first conversational bot that provides artificial intelligence-driven conversational capabilities in relation to goods and/or services provided by that company
- a second company may host a second conversational bot that provides artificial intelligence-driven conversational capabilities in relation to goods and/or services provided by that company.
- Other entities that a conversational bot may be published by and/or associated with may include individual users, applications, websites, and/or digital assistants.
- a conversational bot may comprise one or more software objects that interact with input (e.g., natural language input from a user, schematic input, object-oriented input) and/or other bots in a conversational manner.
- a conversational bot may be associated with one or more skills, and each skill may be associated with one or more manifests.
- a skill may comprise one or more operations that are performed by a conversational bot to accomplish a task.
- a manifest may comprise one or more objects that describe how to contact a conversational bot, what capabilities a conversational bot offers, and information about how that information should be published.
- a conversational bot and/or one or more corresponding manifests may be hosted by an entity's website or database.
- a plurality of manifests, conversational bots, and/or skills may be stored in one or more repositories.
- Examples of a conversational bot repository include a conversational bot service and a manifest marketplace.
- Websites and repositories that host bot manifests may be searchable, and therefore utilized, by computing devices seeking to interact with corresponding bots.
- Computing devices and their associated applications and interfaces may interact with hosted bots and manifests utilizing one or more application programming interfaces (APIs).
- APIs application programming interfaces
- a determination may be made by the conversational bot as to whether it has a skill (and corresponding manifest) that can fulfill that request or command. If a determination is made that the conversational bot does have a skill that can fulfill the request or command, and the conversational bot has all the information it needs to fulfill that request itself, it may execute the skill and fulfill the request or command. If a determination is made that the conversational bot does have a skill that can fulfill the request or command but that the conversational bot does not have all the information it needs to fulfill that request itself, it may cause a request for the missing information to be surfaced in a chat dialog with a user that can provide that missing information.
- the conversational bot if it has been granted permission to do so by a user (e.g., via account settings, via explicit input), may automatically obtain missing information needed to complete a request or command from a user account database or other information source associated with a requesting user.
- a conversational bot may identify a secondary conversational bot that is capable of executing that skill and thereby fulfilling the request or command.
- a conversational bot service may access a manifest marketplace to identify one or more conversational bots that have a skill that can fulfill a request or command.
- the manifest marketplace may include a skill library that includes an identity of each of a plurality of conversational bots, an identity of one or more skills that are associated with each of those conversational bots, and one or more manifests associated with those one or more skills.
- the manifest marketplace may be populated with skills, conversational bots, and/or manifests via a registration process.
- the manifest marketplace may require that skills are capable of performing, or necessarily perform, one or more actions.
- the manifest marketplace may require that skills of a specific skill type execute an encryption or personal identifiable information (PII) scrubbing action on data they receive in order to be registered with the manifest marketplace.
- PII personal identifiable information
- the manifest marketplace may only register skills/bots for publishers of those skills/bots that go through an authentication process and/or that meet certain authentication requirements.
- websites that have associated conversational bots may include their manifest in the website directory.
- the manifest marketplace may crawl website directories to identify new skills that may be registered with the manifest marketplace.
- manifest marketplaces there may be a plurality of manifest marketplaces.
- private marketplaces may exist.
- manifest marketplaces may exist behind corporate firewalls or inside a government data center.
- the non-public marketplaces may be searchable upon authentication of user domains and/or user credentials.
- a manifest marketplace may be maintained by an administrator. For example, an administrator for a marketplace may only allow certain bots and/or skills to be added to it. Additionally, an administrator for a marketplace may remove certain bots and/or skills as it sees fit and/or based on violations of a terms of service contract.
- the skills and/or the manifests included in the skill library may be classified, tagged, organized, or otherwise associated with a skill type.
- the skills and/or manifests may be searchable and/or matched to natural language queries and commands, and schema-defined queries and commands, that are received by the conversational bot service.
- a conversational bot may only hand off or accept skill execution operations to or from other conversational bots that have one or more skills and/or require one or more particular skills be performed.
- a conversational bot may only hand off or accept skill execution operations to or from other conversational bots that will execute a PII scrubbing skill on any data associated with skill performance.
- a conversational bot may only hand off or accept skill execution operations to or from other conversational bots that will execute a data encryption skill on any data that it sends or receives.
- the secondary conversational bot may execute the skill that has been requested of it by the primary conversational bot (e.g., the bot in textual or audio communication with the user).
- the secondary conversational bot may request that information from the primary bot or a different source.
- the secondary conversational bot may provide a request for additional information that may be sent to the primary conversational bot and surfaced to the user in a conversational interface.
- the user may provide the necessary information that is required to the primary conversational bot, and that information may be provided by the primary conversational bot to the secondary conversational bot.
- the secondary conversational bot may have been provided with access to obtain missing information directly (e.g., from a user database, from the user directly) rather than having to go through the primary conversational bot.
- the secondary conversational bot may directly obtain the missing information and execute one or more skills so that a request or command received by the primary conversational bot can be completed.
- the secondary conversational bot may pass control over the conversation back to the primary conversational bot. The actions by the secondary conversational bot need never be seen by the interacting user. That is, in some examples, the requesting/commanding user need not receive a direct message from the secondary conversational bot.
- the primary conversational bot may relay any requests from the secondary conversational bot to the user or a database associated with the user, and/or the secondary conversational bot may perform actions on the backend that are never seen by the user aside from their results.
- the user may be provided with a seamless experience in which a single conversational bot accomplishes or facilitates all desired user actions.
- the primary conversational bot may perform one or more fallback operations.
- a first fallback operation may comprise causing an indication (e.g., in a chat dialog) that the request or command cannot be fulfilled by the conversational bot to be surfaced.
- a second fallback operation may comprise causing a web search corresponding to the request or command to be performed and causing one or more results from that web search to be surfaced.
- a third fallback operation may comprise identifying one or more related tasks to the request or command that can be fulfilled via execution of an identified skill to be surfaced.
- the systems, methods, and devices described herein provide technical advantages for performing tasks electronically and interacting with conversational bots. Processing costs (e.g., CPU cycles) associated with identifying services that may assist with task completion are reduced in that conversational bots may automatically identify other conversational bots associated with services that may be useful in executing skills that are needed for task completion. Privacy concerns associated with handing off information to third parties are also addressed via the mechanisms described herein in that when a task (e.g., a user query, a user command) is received by a first conversational bot, that conversational bot may require that secondary conversational bots that may complete portions of the task execute privacy-oriented skills (e.g., PII scrubbing skills, data encryption skills).
- privacy-oriented skills e.g., PII scrubbing skills, data encryption skills
- User experience is also enhanced via the mechanisms described herein in that a user may have multiple conversational bots execute skills on her behalf, but only need to interact with a single conversational bot in the process. That is, because the systems, methods, and devices described herein allow for more interaction between conversational bots behind the scenes, the user need not be involved on the front end with each bot handling a request or command.
- FIG. 1 is a schematic diagram illustrating an example distributed computing environment 100 for enabling conversational bot communications.
- Computing environment 100 includes natural language input sub-environment 102 , network and processing sub-environment 124 , data sources sub-environment 114 , manifest marketplace sub-environment 142 , first conversational bot sub-environment 130 , and second conversational bot sub-environment 148 .
- Natural language input sub-environment 102 includes user 104 , laptop computing device 108 , digital assistant audio device 110 , and mobile computing device 112 .
- User 104 issues natural language input 106 A.
- Natural language input 106 A may comprise a command or a query.
- Natural language input 106 A may be received by one of laptop computing device 108 , digital assistant audio device 110 , or mobile computing device 112 .
- natural language input 106 A may be received by a bot chat interface as illustrated by bot chat element 128 on laptop computing device 108 , and bot chat element 129 on mobile computing device 112 .
- a website or application accessed on a computing device may surface a bot chat interface and a natural language input may be typed or audibly received by the bot chat interface.
- natural language input 106 A may be received by a digital assistant service, such as by a digital assistant service associated with digital assistant audio device 110 .
- a digital assistant service such as by a digital assistant service associated with digital assistant audio device 110 .
- natural language input 106 A may be sent from the digital assistant service to a conversational bot service.
- natural language input 106 A is received directly by a bot interface, it may be automatically and/or directly received from a corresponding computing device (e.g., laptop computing device 108 , mobile computing device 129 ) by a conversational bot service.
- Natural language input 106 A may be sent to the conversational bot service via an API.
- Network and processing sub-environment 124 includes network 126 and server computing device 128 . Any and all of the computing devices described herein may communicate via a network, such as network 126 in network and processing sub-environment 124 .
- Network and processing sub-environment 124 may comprise a conversational bot service. That is, a conversational bot service may be hosted by one or more server computing devices, such as server computing device 128 .
- the conversational bot service may comprise a plurality of conversational bots.
- Each of the conversational bots may be associated with one or more manifests.
- a manifest may comprise an interface definition language (IDL) that includes instructions for sending, receiving, and processing commands associated with skills that a conversational bot may perform.
- a skill may comprise one or more activities that may be performed by a conversational bot.
- IDL interface definition language
- Certain bots associated with the conversational bot service may not be associated with their own manifests. That is, a conversational bot that receives inputs and directs those inputs to one or more other conversational bots, but which does not perform any skills itself, may be associated with one or more conversational APIs, but need not be associated with its own manifest or manifests.
- the conversational bot service may include or otherwise be associated with a manifest marketplace.
- the manifest marketplace is illustrated by manifest marketplace sub-environment 142 .
- Manifest marketplace sub-environment 142 includes skill library 146 .
- Skill library 146 includes an identity of each of a plurality of conversational bots, an identity of one or more skills that are associated with each of those conversational bots, and one or more manifests associated with those one or more skills.
- the skills and/or the manifests included in skill library 146 may be classified, tagged, organized, or otherwise associated with a skill type.
- the skills and/or manifests may be searchable and/or matched to natural language queries and commands, and schema-defined queries and commands, that are received by the conversational bot service.
- natural language input 106 D may be provided to manifest marketplace 142 and one or more keywords or phrases included in natural language input 106 D may be matched to one more skills or skill types in manifest marketplace 142 .
- the conversational bot service and/or the manifest marketplace may communicate with one or more data sources as illustrated by data sources sub-environment 114 .
- Data sources sub-environment 114 includes data store 116 , documents 119 , user information 118 , electronic messages 120 , and calendar information 122 .
- Data store 116 is representative of various data sources.
- Data store 116 may comprise data from the World Wide Web, data associated with conversational bot entities, and/or data associated with user accounts.
- data store 116 may comprise data for a plurality of user accounts associated with one or more computing devices, one or more applications (e.g., productivity applications), and/or one or more application suites (e.g., productivity application suites).
- user 104 may have a user account that is associated with one or more applications and/or services and data that includes settings for that user account, preferences for that user account, documents generated by that user account, documents sent by that user account, and/or documents received by that user account, all of which may be included in data store 116 with user information 118 .
- data store 116 may include user information 118 , documents 119 emails 120 , and calendar information 122 (or references thereto) associated with one or more user accounts.
- natural language input 106 A is sent from one of laptop computing device 108 , digital assistant audio device 110 , or mobile computing device 112 to the conversational bot service in network and processing sub-environment 124 as illustrated by natural language input 106 B.
- Natural language input 106 B may be processed by one or more natural language processing models associated with the conversational bot service.
- the one or more natural language processing models may determine whether natural language input 106 B includes a reference to a specific conversational bot, reference to a specific skill, a reference to a specific skill type, reference to a task to be performed, or any combination of the same.
- a schematized request to perform the skill may be sent from the conversational bot service to an endpoint corresponding to the specific conversational bot and the one or more manifests corresponding to the specific skill.
- processing may comprise identifying one or more conversational bots, or entities associated with one or more conversational bots, that a user account for user 104 is associated with. For example, a user token comprising the identity of user 104 and/or user 104 's user account may be provided to skill library 146 and a determination may be made as to one or more conversational bots and/or entities associated with conversational bots that user 104 has an account for.
- the processing may comprise identifying one or more conversational bots and/or entities that are publicly available for use (e.g., a user does not have to have an account to access).
- the processing may further comprise identifying one or more conversational bots that have one or more manifests for performing a skill corresponding to the identified specific skill and/or skill type.
- those conversational bots may be ranked.
- the ranking may comprise one or more metrics (e.g., popularity of website corresponding to conversational bot, popularity of conversational bot by other users, history of using conversational bot by user 104 ).
- the conversational bot service may send a request for performance of the skill to one or more endpoints for the identified manifests.
- the conversational bot service may send the natural language directly to the conversational bot and the conversational bot may identify one or more manifests for performing the specific skill.
- conversational bot A 132 in first conversational bot sub-environment 130 is identified as being able to perform a specific skill identified in natural language input 106 B. As such, the request for performance of the specific skill is sent to conversational bot A 132 via skill request 107 A.
- a conversational bot may initiate and open a conversation between an origination point and the conversational bot when a skill request is received.
- the origination point may comprise a computing device (e.g., laptop computing device 108 , digital assistant audio device 110 , mobile computing device 112 ), an interface where the skill request originated (e.g., bot chat interface 128 , bot chat interface 129 ), and/or a conversational intermediary between an originating natural language input and the conversational bot (e.g., a digital assistant service, the conversational bot service).
- a conversation is initiated and opened by conversational bot A 132 when it receives skill request 107 A.
- skill request 107 A may comprise a request to perform skill A 134 .
- skill request 107 A may be sent to an endpoint specified in manifest A 136 . Skill A 134 may then be performed by conversational bot A 132 .
- request 107 A may include a request to perform an additional skill that is not a skill that may be performed by conversational bot A 132 . That is, the additional skill may not be skill A 134 or one of the other skills, such as skill N 138 and corresponding manifest N 140 , that conversational bot A 132 is capable of performing.
- conversational bot A 132 may identify another conversational bot that is capable of performing the additional skill. For example, conversational bot A may query the conversational bot service and/or the manifest marketplace (e.g., skill library 146 ) to identify one or more bots and corresponding manifests for performing the additional skill.
- the manifest marketplace e.g., skill library 146
- conversational bot A identifies that the additional skill is skill C 152 , which is performable by conversational bot 150 in second conversational bot sub-environment 148 .
- conversational bot A 132 sends skill request 107 B to the endpoint included in manifest C 154 corresponding to skill C 152 .
- Conversational bot B 150 may then perform skill C 152 and return the result either directly to conversational bot A or to the origination point of the request (e.g., laptop computing device 108 , bot chat interface 128 , digital assistant audio computing device 110 , mobile computing device 112 , bot chat interface 129 ) where the result may be caused to be surfaced.
- the origination point of the request e.g., laptop computing device 108 , bot chat interface 128 , digital assistant audio computing device 110 , mobile computing device 112 , bot chat interface 129
- conversational bot B 150 may send a request for that additional information back to conversational bot A 132 .
- conversational bot B 150 may need location information for user 104 , account credential information for user 104 , and/or payment information for user 104 , and that information may be requested from conversational bot A 132 , which may obtain that information and return it to conversational bot B 150 .
- conversational bot A 132 may identify that information directly from user 104 via part of the conversation with user 104 on one of the devices included in natural language input sub-environment 102 .
- conversational bot A 132 may identify that information from an account associated with user 104 (e.g., from user information 118 in data sources sub-environment 114 ). The retrieval of that information may be made via execution of a specific skill and manifest associated with conversational bot A 132 .
- the first conversational bot may turn over control of the open conversation to the second conversational bot.
- the second conversational bot may turn control of the conversation back over to the first conversational bot.
- FIG. 2 illustrates a skill manifest 202 and associated exemplary elements included in the skill manifest.
- Manifest 202 comprises publishing information element 204 , connection element 206 , language element 208 , activities element 210 , and sent activities element 228 .
- Activities element 210 further comprises message element 212 , events element 214 , end of conversation element 216 , invoke element 218 , add element 220 , update element 222 , delete element 224 , and [custom] element 226 .
- Manifest 202 may comprise more or fewer elements than illustrated in this example.
- Manifest 202 may describe an object that is capable of sending and receiving conversational bot framework-compatible messages.
- Manifest 202 may allow configuration, registration, and publishing data about a conversational bot to be authored, stored, and transmitted between services.
- Manifest 202 may describe how to contact a conversational bot, what capabilities a conversational bot offers, and information about how it should be published.
- Manifest 202 may be a flat list of name/value pairs, some of which are primitive, and some of which are complex (nested).
- Manifest 202 may be expressed in the JSON format but may also be projected into in-memory data structures in other languages (e.g., C #, JavaScript).
- Publishing information element 204 may define an identity of a bot associated with manifest 202 that may execute activities included in manifest 202 ; a publisher of the one or more bots associated with manifest 202 ; copyright information for manifest 202 ; and license information for manifest 202 .
- Publishing information element 204 may contain information about how a conversational bot is published into a registry (e.g., a conversational bot service, a manifest marketplace).
- Connection element 206 may define endpoint information for connecting with manifest 202 from one or more geographic locations (e.g., North America endpoint, European endpoint).
- Connection element 206 may include an authentication connection representing a sign-in service containing parameters describing how to use a conversational bot to request user sign-in and consent.
- Language element 208 may define an identity of one or more languages that are supported by the bot associated with manifest 202 .
- Activities element 210 may define operations that the bot can receive, execute, and/or modify. Activities element 210 may represent conversational behaviors made by humans and automated software within chat applications, email, and other text interaction programs. Each activity object may include a type field and represent a single action (e.g., send text content, send multimedia attachments, execute “like” button, typing indicator). Exemplary activities in activities element 210 are described below.
- Message element 212 may define message types that may be received and attempted to be resolved by the bot.
- Message activities may represent content intended to be shown within a conversational interface.
- Message activities may contain text, speech, interactive cards, and binary or unknown attachments.
- Events element 214 may define a plurality of named event type properties that indicate a task that a skill may execute. Events may be custom developed for each skill type. For example, for a flight booking skill there may be a “BookFlight” event that books a flight when called; for a calendar application there may be a “clock” event that checks an electronic calendar for calendar entries at periodic intervals when called; for an investment skill there may be a “GetStockPrice” event that obtains stock prices when called. Event activities may communicate programmatic information from a client or channel to a conversational bot. The meaning of an event activity may be defined by a “name” field, which may be meaningful within the scope of a channel.
- Event activities may be designed to carry both interactive information (such as button clicks) and non-interactive information (such as notification of a client automatically updating an embedded speech model). Event activities are the asynchronous counterpart to invoke activities. Event activities may be identified by a “type” value of event and specific values of the “name” field.
- End of conversation element 216 may comprise a definition for ending an open conversation between a first conversational bot and the conversational bot associated with manifest 202 and/or ending an open conversation between the conversational bot associated with manifest 202 and a user communication element.
- end of conversation activities may signal the end of a conversation from a recipient's perspective. This may be because the conversation has been affirmatively ended, or because the recipient has been removed from the conversation in a way that is indistinguishable from it ending. End of conversation activities may be identified by a “type” value.
- Invoke element 218 may define a plurality of named invoke type properties that indicate a task that a skill will execute.
- Invoke activities may communicate programmatic information from a client or channel to a conversational bot and have a corresponding return payload for use within a channel.
- Invoke activities are the synchronous counterpart to event activities.
- Event activities may be designed to be extensible. Invoke activities may differ only in their ability to return response payloads back to a channel; because the channel must decide where and how to process these response payloads, invoke may only be useful in cases where explicit support for each invoke name has been added to a channel.
- Invoke activities may be identified by a “type” value of “invoke” and specific values of the “name” field.
- Add element 220 may comprise a definition for adding operations and/or content to an existing activity.
- Update element 222 may comprise a definition for updating operations and/or content for an existing activity.
- Delete element 224 may comprise a definition for deleting operations and/or content from an existing activity.
- [Custom] element 226 may comprise custom definitions for activities that may be created by bot developers for use in custom skills.
- Sent activities element 228 may comprise activities that a conversational bot can send to a skill (e.g., to a different conversational bot).
- activities 210 may comprise activities that a conversational bot corresponding to manifest 202 may perform.
- FIG. 3 illustrates a computing device 302 A/ 302 B and an exemplary communication with a conversational bot associated with a restaurant entity.
- Computing device 302 A in sub-environment 300 A, and computing device 302 B in sub-environment 300 B are the same computing device.
- Web browser 304 A is open on computing device 302 A. Web browser 304 A is currently navigated to www.[restaurantwebsite].com. In this example, a user is interacting with the restaurant entity via a conversational bot for that entity. Specifically, real-time bot chat window 306 A is being utilized to communicate with conversational bot A 308 A.
- Conversational bot A 308 may be hosted by a conversational bot service and may be associated with a plurality of manifests (e.g., a manifest for skill A, a manifest for skill B, a manifest for skill C).
- Conversational bot A 308 A opened the dialog in chat window 306 A with the text “Welcome! How can I help you?” The user then responded with the natural language input “I would like to book a table for four at 7 pm tomorrow”.
- conversational bot A 308 B communicates the text “Your table is booked” in chat window 306 B. That is, conversational bot A 308 A may have executed a “BookTable” skill (e.g., one of skills A, B, or C). The user then responded with the natural language input “Book me a car for my dinner.” Additional details regarding the processing of this natural language input are discussed below in relation to FIG. 4 .
- “BookTable” skill e.g., one of skills A, B, or C.
- FIG. 4 illustrates bot-to-bot communications and the processing of a command from the bot communication of FIG. 3 .
- the command at issue in this example is book car request 426 , which originated from the natural language input “Book me a car for my dinner.” That natural language input was received by conversational bot A 416 , which is associated with the restaurant entity.
- Conversational bot A 416 has three skills that it can perform. Specifically, conversational bot A 416 can perform skill A 418 , skill B 420 , and skill C 422 .
- conversational bot A 416 or the conversational bot service on behalf of conversational bot A 416 , receives the natural language input and processes it with one or more natural language processing models, a determination may be made that none of the skills that conversational bot A can perform will fulfill an identified “book car” command.
- conversational bot A 416 determines that it cannot fulfill the “book car” command itself, it may determine whether one or more other bots that the user that generated the corresponding natural language input has access to may fulfill the “book car” command.
- conversational bot A 416 sends book car request 426 and user token 428 to manifest marketplace sub-environment 432 .
- Book car request 426 may comprise the natural language input originally received by conversational bot A 416 or a schematized version of that natural language input.
- User token 428 may comprise an account identity associated with the user that provided the natural language input and/or security credentials associated with that account. This information may be obtained by conversational bot A 416 from user data store 412 .
- Manifest marketplace sub-environment 432 includes skill library 430 .
- Skill library 430 includes an identity of each of a plurality of conversational bots, an identity of one or more skills that are associated with each of those conversational bots, and one or more manifests associated with those one or more skills.
- the skills and/or manifests included in skill library 430 may be classified, tagged, organized, or otherwise associated with a skill type.
- the skills and/or manifests may be searchable and/or matched to natural language queries and commands, and schema-defined queries and commands, that are received by the conversational bot service and/or the manifest marketplace.
- a plurality of book car skill bots (bot B, bot C, bot D) and corresponding manifests that are capable of performing the desired skill and handling book car request 426 are illustrated in book car skill element 434 .
- Each of those book car skill bots and corresponding manifests are associated with a different rideshare service. This is illustrated by rideshare service sub-environment 402 .
- conversational bot B corresponds to rideshare service B database 406
- conversational bot C corresponds to rideshare service C database 408
- conversational bot D corresponds to rideshare service database D 410 .
- the databases for the rideshare services are illustrated as containing their corresponding bots and manifests, the bots and manifests may be maintained separately by the conversational bot service.
- a determination may be made, utilizing user token 428 , via the manifest marketplace and/or communication with one or more of rideshare service B database 406 , rideshare service C database 408 , and/or rideshare service D 410 , that the user account associated with the natural language input book car request only has an account for one or a subset of the rideshare services and/or that settings associated with that user account indicate a preference for one of the rideshare services.
- a determination has been made that the rideshare service that the user account has an account with and/or a preference for using corresponds to rideshare service C database 408 .
- conversation control 442 is passed from conversational bot A 416 to conversational bot C 436 , which is the conversational bot for that rideshare service.
- Conversation control 442 may comprise backend communications and operations that are never surfaced in a communication with a corresponding user.
- conversational control 442 here may comprise conversational bot C 436 communicating with conversational bot A 416 , one or more additional conversational bots, and/or user data store 412 .
- Conversational bot C 436 may perform three skills (skill E 438 , skill F 440 , and skill G 442 ). At least one of those skills corresponds to a “book car” skill that is needed to complete book car request 426 . However, in some instances it may be the case that a bot that has taken control of a conversation needs additional information to complete to execute a skill and complete a request. In this example, conversational bot C 436 makes a determination that it needs one or both of an address where the user needs to be picked up and/or an address where the user needs to be dropped off. Depending on user privacy settings, conversational bot C 436 may obtain that address information directly from user data store 412 if it is available, or from conversational bot A 416 .
- conversational bot C 436 may request the address information from conversational bot A 416 .
- conversational bot A 416 may surface a request for that information in the bot chat window that it has open on one of the computing devices associated with the user, the user may provide that information to conversational bot A 416 , and address information 440 may then be provided from conversational bot A 416 to conversational bot C 436 .
- conversational bot C 436 may execute and fulfill book car request 426 , as indicated by execute car booking skill element 448 .
- conversational bot C 436 may send an “end of conversation” action indication to conversational bot A 416 , which may then take back control of the conversation.
- FIG. 5 illustrates a computing device 502 that displays the result of the processing and bot-to-bot communications performed in relation to FIG. 4 .
- Conversational bot A 508 has taken control of the conversation back from the rideshare conversational bot (conversational bot C 436 ).
- Computing device 502 displays web browser 504 and bot messaging window 506 .
- the user has previously provided the natural language input “Book me a car for my dinner” in messaging window 506 , conversational bot A 508 processed that request in combination with conversational bot C 436 , and the conversational bot notes this in messaging window 506 with the text “Your car from [rideshare service] C will pick you up.
- FIGS. 3 - 5 illustrate bot-to-bot communications and user communications in relation to a bot messaging window in a website
- conversational bot A 508 may communicate with the user via an audio device, via an intermediary digital assistant, via SMS message, and/or via email.
- FIG. 6 is a schematic diagram illustrating an example distributed computing environment 600 for identifying relevant skills from a manifest marketplace in relation to natural language input queries and commands.
- Computing environment 600 includes manifest marketplace 602 , company database sub-environment 630 , and web service sub-environment 638 .
- Manifest marketplace 602 includes skill library 604 .
- Skill library 604 includes an identity for each of a plurality of conversational bots, an identity of one or more skills that are associated with each of those conversational bots, and one or more manifests associated with those one or more skills.
- the skills and/or manifests included in skill library 604 may be classified, tagged, organized, or otherwise associated with a skill type.
- the skills and/or manifests may be searchable and/or matched to natural language queries and commands, and schema-defined queries and commands, that are received by the conversational bot service.
- manifest marketplace 602 includes skill classifications 618 . That is, there are a plurality of skill types for which corresponding manifests and conversational bots have been classified by skill type.
- Skill classifications 618 includes travel booking skills 620 , with corresponding conversational bots A, B and C; book car skills 622 , with corresponding conversational bots D, E and F; investment skills 624 , with corresponding conversational bots G, H and I; weather skills 626 , with corresponding conversational bots J, K and L; and type N skills 618 (exemplary of any miscellaneous skill type), with corresponding conversational bots M, N and O.
- the skill type divisions that are illustrated may be further divided in sub-classes and sub-classes of sub-classes.
- travel booking skills 620 may be divided into flight booking skills, hotel booking skills, train booking skills.
- Book car skills 622 may be divided into car types and ride share types.
- Investment skills 624 may be divided by transaction type, pay or no pay, and cost per transaction.
- Weather skills 626 may be divided by geographic region and radar types.
- Company database sub-environment 630 illustrates three exemplary investment services that each have a corresponding conversational bot. Although the three exemplary investment services are illustrated as being in the same sub-environment, they each correspond to a different service and therefore are separate and distinct from one another.
- investment service A database 632 is associated with conversational bot G, which may perform one or more investment skills.
- Investment service B database 634 is associated with conversational bot H, which may perform one or more investment skills.
- Investment service C database 636 is associated with conversational bot I, which may perform one or more investment skills.
- the skills performed by each of those bots may be the same skill or a different skill.
- the manifests for those skills may differ by publishing elements only, or by a plurality of elements (e.g., endpoints, languages, activities, sent activities).
- Manifest marketplace 602 further includes skill ranking engine 612 , site ranking engine 614 , and natural language processing skill matching engine 616 .
- Skill ranking engine 612 and/or site ranking engine 614 may receive data from web service sub-environment 638 .
- the data may comprise website rank data or data that may be utilized by skill ranking engine 612 and/or site ranking engine 614 to rank websites and associated skills.
- web service sub-environment 638 includes web data 640 for websites and web search engine 642 from one or more search engines.
- Web data 640 may include an identity of a plurality of websites associated with entities that have conversational bots. Web data 640 may further include use data associated with those websites.
- web data 640 may include a number of users that access a website, a frequency of access for a website, engagement data with a website (e.g., amount of time spent by users, number of pages of website viewed, number of scrolls on each page, number of orders placed), and/or engagement data with a conversational bot associated with a web page.
- Web search engine 642 may provide additional information to web data 640 that may be received by skill ranking engine 612 and/or site ranking engine 614 .
- search engine 642 may collect information related to a number of searches for a particular website and/or conversational bot, a number of clicks on websites and/or conversational bot elements, a frequency of searches, and search term patterns. Any of that data may be provided to skill ranking engine 612 and/or site ranking engine 614 .
- Skill ranking engine 614 may utilize data from web service sub-environment 638 , site ranking engine 614 , and/or skill use data from skill library 604 in calculating scores for skills and ranking them.
- the scores may be based on one or more of popularity of skill, popularity of conversational bot associated with skill, and/or popularity of entity associated with conversational bot.
- One or more machine learning models, such as neural network 610 and unsupervised machine learning model 606 may be utilized in calculating scores for websites and skills.
- Natural language processing skill match engine 616 may perform operations associated with identifying relevant words and phrases from a natural language input provided to a conversational bot and matching those words and/or phrases to one or more highest ranked skills. Natural language processing skill match engine 616 may apply one or more language processing models, such as language processing model 608 in scoring words as being relevant to a skill. Language processing model 608 may comprise one or more word and/or sentence embedding layers (e.g., Word2Vec, BERT, Sent2Vec, Embeddings from Language Models (ELMo), recurrent neural network).
- word2Vec e.g., Word2Vec, BERT, Sent2Vec, Embeddings from Language Models (ELMo), recurrent neural network.
- ELMo Language Models
- Language processing model 608 may further comprise a sentence level information aggregation layer for aggregating embeddings from each word in a natural language input into a distinct embedding for one or more sentences included in a natural language input.
- the distinct sentence aggregation layer may apply a neural network to the embeddings for each word.
- the neural network may comprise a gated recurrent unit (GRU) neural network or bidirectional GRU (bi-GRU) neural network.
- the neural network may comprise a long short-term memory (LSTM) neural network, an attention-based aggregation method, etc.
- Language processing model 608 may further comprise a contextual aggregation layer for aggregating each distinct embedding for each of the sentences into a contextual embedding.
- the contextual aggregation layer may apply a neural network to each distinct embedding for each of the sentences.
- the neural network may comprise a GRU neural network, or bi-GRU neural network.
- the neural network may comprise an LSTM neural network, an attention-based aggregation method, etc.
- Language processing model 608 may further comprise a scoring layer for scoring and ranking each of the sentences in a natural language input based on their relevance to a skill type (e.g., a “book car” skill type, “get stock price” skill type).
- the scoring layer may apply a classifier function to each contextual embedding for each of the plurality of sentences (e.g., the embeddings generated by the contextual aggregation layer).
- the classifier function may comprise a sigmoid function.
- Other activation functions e.g., tanh, softplus may be utilized for scoring each sentence.
- FIG. 7 A is an exemplary method 700 A for enabling conversational bot communications in relation to a natural language input.
- the method 700 A begins at a start operation and flow moves to operation 702 A.
- a natural language input is received by a first conversational bot.
- the natural language input may be received via a chat interface in a website, a chat interface in an application, a real-time messaging application, an SMS message, an email, and/or an intermediary digital assistant.
- the natural language input may be associated with a user account. For example, a user may be signed into an account associated with a web browser application, one or more applications, and/or an application service, and the account information may therefore be made available to the conversational bot.
- a skill needed to fulfill one of: a query included in the natural language input, and a command included in the natural language input is identified.
- the skill may be associated with one or more manifests.
- a manifest may comprise one or more objects that describe how to contact a conversational bot, what capabilities a conversational bot offers, and information about how that information should be published.
- the skill may comprise one or more operations that may be performed by a conversational bot to accomplish a task.
- the skill may be identified via application of one or more natural language processing models to the natural language input.
- operation 706 A a determination is made that the skill is not executable by the first conversational bot.
- the conversational bot and/or a conversational bot service may determine that the first conversational bot does not have a manifest that includes one or more activities needed to perform the skill.
- the determination may be made by the conversational bot, a conversational bot service, and/or a manifest marketplace.
- the plurality of bots may be scored and ranked based on factors including: popularity of conversational bot, popularity of website associated with conversational bot, and user preferences.
- a request for execution of the skill is sent from the first conversational bot to the second conversational bot.
- the request may be sent via an API.
- the request may comprise the natural language input.
- the request may comprise a schematized request for execution of the skill.
- the request may include a token describing the user (or user account) that generated the natural language input.
- a request for information needed for the second conversational bot to execute the skill is received by the first conversational bot from the second conversational bot.
- the information may relate to the user that generated the natural language input.
- the second conversational bot may need payment information, time information, location information, among other information types, to execute the skill.
- the first conversational bot may request that information from the user (e.g., via a chat window, via an audio query).
- the first conversational bot when the first conversational bot receives the request for information, it may determine whether that information is available in a user store associated with a user account for the user.
- operation 714 A the information is sent by the first conversational bot to the second conversational bot.
- the second conversational bot may execute the skill and turn control of the conversation back over to the first conversational bot.
- FIG. 7 B is another exemplary method 700 B for enabling conversational bot communications in relation to a natural language input.
- the method 700 B begins at a start operation and flow moves to operation 702 B.
- a request for execution of a skill is received by a second conversational bot from a first conversational bot.
- the request for execution of the skill may be received in the form of the natural language input or a schematized format.
- the second conversational bot may determine whether the first conversational bot executes a PII scrubbing skill. In examples, the second conversational bot may only execute the skill for the first conversational bot if the first conversational bot executes a PII scrubbing skill.
- operation 704 B a determination is made by the second conversational bot that execution of the skill requires additional information.
- the information may relate to the user that generated the natural language input.
- the second conversational bot may need payment information, time information, location information, among other information types, to execute the skill.
- operation 706 B a request for the additional information is sent from the second conversational bot to the first conversational bot.
- the request may be sent to the second conversational bot via an API.
- FIG. 8 is an exemplary method 800 for enabling conversational bot communications in relation to a natural language input and a manifest marketplace.
- the method 800 begins at a start operation and flow moves to operation 802 .
- a marketplace comprising an identity of a plurality of skills is maintained, each of the plurality of skills associated with a manifest and a conversational bot.
- the manifest marketplace may include a skill library that includes an identity of each of a plurality of conversational bots, an identity of one or more skills that are associated with each of those conversational bots, and one or more manifests associated with those one or more skills.
- the skills and/or the manifests included in the skill library may be classified, tagged, organized, or otherwise associated with a skill type.
- the skills and/or manifests may be searchable and/or matched to natural language queries and commands, and schema-defined queries and commands, that are received by a conversational bot service.
- a request for execution of a specific skill is received.
- the request may be received by the conversational bot service and/or a manifest marketplace.
- the request may be in the form of a natural language input or it may be in a schematized format.
- a plurality of conversational bots is identified, each of the plurality of conversational bots being associated with a manifest for executing the specific skill.
- the identification may be made based on keyword and/or phrase matching between a natural language input comprising the request for the specific skill, and categorized skill names in the skill library.
- the plurality of conversational bots may be ranked based on a calculated value for each of the conversational bots.
- the value may be based on one or more of: a relevance of the bot to the requested skill, a popularity of the bot, a popularity of a website associated with the bot, user preferences, and user history of interaction with the conversational bots.
- FIGS. 9 and 10 illustrate a mobile computing device 900 , for example, a mobile telephone, a smart phone, wearable computer (such as smart eyeglasses), a tablet computer, an e-reader, a laptop computer, or other AR compatible computing device, with which embodiments of the disclosure may be practiced.
- a mobile computing device 900 for implementing the aspects is illustrated.
- the mobile computing device 900 is a handheld computer having both input elements and output elements.
- the mobile computing device 900 typically includes a display 905 and one or more input buttons 910 that allow the user to enter information into the mobile computing device 900 .
- the display 905 of the mobile computing device 900 may also function as an input device (e.g., a touch screen display). If included, an optional side input element 915 allows further user input.
- the side input element 915 may be a rotary switch, a button, or any other type of manual input element.
- mobile computing device 900 may incorporate more or fewer input elements.
- the display 905 may not be a touch screen in some embodiments.
- the mobile computing device 900 is a portable phone system, such as a cellular phone.
- the mobile computing device 900 may also include an optional keypad 935 .
- Optional keypad 935 may be a physical keypad or a “soft” keypad generated on the touch screen display.
- the output elements include the display 905 for showing a graphical user interface (GUI), a visual indicator 920 (e.g., a light emitting diode), and/or an audio transducer 925 (e.g., a speaker).
- GUI graphical user interface
- the mobile computing device 900 incorporates a vibration transducer for providing the user with tactile feedback.
- the mobile computing device 900 incorporates input and/or output ports, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., a HDMI port) for sending signals to or receiving signals from an external device.
- FIG. 10 is a block diagram illustrating the architecture of one aspect of a mobile computing device. That is, the mobile computing device 1000 can incorporate a system (e.g., an architecture) 1002 to implement some aspects.
- the system 1002 is implemented as a “smart phone” capable of running one or more applications (e.g., browser, e-mail, calendaring, contact managers, messaging clients, games, and media clients/players).
- the system 1002 is integrated as a computing device, such as an integrated personal digital assistant (PDA) and wireless phone.
- PDA personal digital assistant
- One or more application programs 1066 may be loaded into the memory 1062 and run on or in association with the operating system 1064 .
- Examples of the application programs include phone dialer programs, e-mail programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so forth.
- the system 1002 also includes a non-volatile storage area 1068 within the memory 1062 .
- the non-volatile storage area 1068 may be used to store persistent information that should not be lost if the system 1002 is powered down.
- the application programs 1066 may use and store information in the non-volatile storage area 1068 , such as e-mail or other messages used by an e-mail application, and the like.
- a synchronization application (not shown) also resides on the system 1002 and is programmed to interact with a corresponding synchronization application resident on a host computer to keep the information stored in the non-volatile storage area 1068 synchronized with corresponding information stored at the host computer.
- other applications may be loaded into the memory 1062 and run on the mobile computing device 1000 , including instructions for providing and operating a conversational bot platform.
- the system 1002 has a power supply 1070 , which may be implemented as one or more batteries.
- the power supply 1070 might further include an external power source, such as an AC adapter or a powered docking cradle that supplements or recharges the batteries.
- the system 1002 may also include a radio interface layer 1072 that performs the function of transmitting and receiving radio frequency communications.
- the radio interface layer 1072 facilitates wireless connectivity between the system 1002 and the “outside world,” via a communications carrier or service provider. Transmissions to and from the radio interface layer 1072 are conducted under control of the operating system 1064 . In other words, communications received by the radio interface layer 1072 may be disseminated to the application programs 1066 via the operating system 1064 , and vice versa.
- the visual indicator 920 may be used to provide visual notifications, and/or an audio interface 1074 may be used for producing audible notifications via the audio transducer 925 .
- the visual indicator 920 is a light emitting diode (LED) and the audio transducer 925 is a speaker.
- LED light emitting diode
- the LED may be programmed to remain on indefinitely until the user takes action to indicate the powered-on status of the device.
- the audio interface 1074 is used to provide audible signals to and receive audible signals from the user.
- the audio interface 1074 may also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation.
- the microphone may also serve as an audio sensor to facilitate control of notifications, as will be described below.
- the system 1002 may further include a video interface 1076 that enables an operation of an on-board camera 930 to record still images, video stream, and the like.
- a mobile computing device 1000 implementing the system 1002 may have additional features or functionality.
- the mobile computing device 1000 may also include additional data storage devices (removable and/or non-removable) such as, magnetic disks, optical disks, or tape.
- additional storage is illustrated in FIG. 10 by the non-volatile storage area 1068 .
- Data/information generated or captured by the mobile computing device 1000 and stored via the system 1002 may be stored locally on the mobile computing device 1000 , as described above, or the data may be stored on any number of storage media that may be accessed by the device via the radio interface layer 1072 or via a wired connection between the mobile computing device 1000 and a separate computing device associated with the mobile computing device 1000 , for example, a server computer in a distributed computing network, such as the Internet.
- a server computer in a distributed computing network such as the Internet.
- data/information may be accessed via the mobile computing device 1000 via the radio interface layer 1072 or via a distributed computing network.
- data/information may be readily transferred between computing devices for storage and use according to well-known data/information transfer and storage means, including electronic mail and collaborative data/information sharing systems.
- FIG. 11 is a block diagram illustrating physical components (e.g., hardware) of a computing device 1100 with which aspects of the disclosure may be practiced.
- the computing device components described below may have computer executable instructions for assisting conversational bot interactions.
- the computing device 1100 may include at least one processing unit 1102 and a system memory 1104 .
- the system memory 1104 may comprise, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories.
- the system memory 1104 may include an operating system 1105 suitable for running one or more conversational bot applications and/or services.
- the operating system 1105 may be suitable for controlling the operation of the computing device 1100 .
- embodiments of the disclosure may be practiced in conjunction with a graphics library, other operating systems, or any other application program and is not limited to any particular application or system.
- This basic configuration is illustrated in FIG. 11 by those components within a dashed line 1108 .
- the computing device 1100 may have additional features or functionality.
- the computing device 1100 may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape.
- additional storage is illustrated in FIG. 11 by a removable storage device 1109 and a non-removable storage device 1110 .
- a number of program modules and data files may be stored in the system memory 1104 .
- the program modules 1106 e.g., conversational bot application 1120
- the program modules 1106 may perform processes including, but not limited to, the aspects, as described herein.
- skill matching engine 1111 may perform one or more operations associated with matching a query and/or command included in a natural language input to one or more skills that may be executed by a conversational bot.
- Language processing engine 1113 may perform one or more operations associated with applying one or more natural language processing models to a natural language input to identify one or more tasks and associated skill requests therein.
- Website ranking engine 1115 may perform one or more operations associated with scoring a plurality of websites associated with conversational bots and ranking those bots based on the scores.
- Skill ranking engine 1117 may perform one or more operations associated with scoring skills associated with conversational bots based on match to natural language input and/or schematized request, popularity of bot, popularity of corresponding website, popularity for specific user, and user preferences, for example, and ranking the bots based on those scores.
- embodiments of the disclosure may be practiced in an electrical circuit comprising discrete electronic elements, packaged or integrated electronic chips containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic elements or microprocessors.
- embodiments of the disclosure may be practiced via a system-on-a-chip (SOC) where each or many of the components illustrated in FIG. 11 may be integrated onto a single integrated circuit.
- SOC system-on-a-chip
- Such an SOC device may include one or more processing units, graphics units, communications units, system virtualization units and various application functionality all of which are integrated (or “burned”) onto the chip substrate as a single integrated circuit.
- the functionality, described herein, with respect to the capability of client to switch protocols may be operated via application-specific logic integrated with other components of the computing device 1100 on the single integrated circuit (chip).
- Embodiments of the disclosure may also be practiced using other technologies capable of performing logical operations such as, for example, AND, OR, and NOT, including but not limited to mechanical, optical, fluidic, and quantum technologies.
- embodiments of the disclosure may be practiced within a general purpose computer or in any other circuits or systems.
- the computing device 1100 may also have one or more input device(s) 1112 such as a keyboard, a mouse, a pen, a sound or voice input device, a touch or swipe input device, etc.
- the output device(s) 1114 such as a display, speakers, a printer, etc. may also be included.
- the aforementioned devices are examples and others may be used.
- the computing device 1100 may include one or more communication connections 1116 allowing communications with other computing devices 1150 . Examples of suitable communication connections 1116 include, but are not limited to, radio frequency (RF) transmitter, receiver, and/or transceiver circuitry; universal serial bus (USB), parallel, and/or serial ports.
- RF radio frequency
- USB universal serial bus
- Computer readable media may include computer storage media.
- Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, or program modules.
- the system memory 1104 , the removable storage device 1109 , and the non-removable storage device 1110 are all computer storage media examples (e.g., memory storage).
- Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article of manufacture which can be used to store information and which can be accessed by the computing device 1100 . Any such computer storage media may be part of the computing device 1100 . Computer storage media does not include a carrier wave or other propagated or modulated data signal.
- Communication media may be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media.
- modulated data signal may describe a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal.
- communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.
- RF radio frequency
- FIG. 12 illustrates one aspect of the architecture of a system for processing data received at a computing system from a remote source, such as a personal/general computer 1204 , tablet computing device 1206 , or mobile computing device 1208 , as described above.
- Content displayed at server device 1202 may be stored in different communication channels or other storage types. For example, various documents may be stored using a directory service 1222 , a web portal 1224 , a mailbox service 1226 , an instant messaging store 1228 , or a social networking site 1230 .
- the program modules 1106 may be employed by a client that communicates with server device 1202 , and/or the program modules 1106 may be employed by server device 1202 .
- the server device 1202 may provide data to and from a client computing device such as a personal/general computer 1204 , a tablet computing device 1206 and/or a mobile computing device 1208 (e.g., a smart phone) through a network 1215 .
- a client computing device such as a personal/general computer 1204 , a tablet computing device 1206 and/or a mobile computing device 1208 (e.g., a smart phone) through a network 1215 .
- the computer systems described herein may be embodied in a personal/general computer 1204 , a tablet computing device 1206 and/or a mobile computing device 1208 (e.g., a smart phone). Any of these embodiments of the computing devices may obtain content from the store 1216 , in addition to receiving graphical data useable to be either pre-processed at a graphic-originating system, or post-processed at a receiving computing system.
Abstract
In non-limiting examples of the present disclosure, systems, methods, and devices for enabling conversational bot communications are presented. A marketplace comprising an identity of a plurality of skills may be maintained, each of the plurality of skills associated with a manifest and a conversational bot. A request for execution of a specific skill may be received. A plurality of conversational bots may be identified, each of the plurality of conversational bots being associated with a manifest for executing the specific skill. The plurality of conversational bots may be ranked. The specific skill may be caused to be executed by the highest ranked conversational bot. In some examples, a plurality of highest ranked conversational bots may be surfaced and a bot that is selected may execute the specific skill.
Description
- This application claims priority to and is a continuation of U.S. patent application Ser. No. 16/780,696, filed Feb. 3, 2020, and entitled “MARKETPLACE CONVERSATIONAL BOT SKILLS,” which claims priority to U.S. Provisional Patent Application No. 62/949,327, filed Dec. 17, 2019, and entitled “MARKETPLACE FOR CONVERSATIONAL BOT SKILLS,” each of which are hereby incorporated by reference in their entireties for all purposes.
- Entities are increasingly using conversational bots to handle user requests and commands related to their goods and services. Users often input complex queries and commands to conversational bots that cannot adequately process those requests for various reasons. For example, some requests are simply not supported by a corresponding conversational bot, and some requests may not be understood correctly. Additionally, while some requests may be understood correctly by a conversational bot, the conversational bot may nonetheless not have the requisite information needed to execute a skill needed to appropriately respond.
- It is with respect to this general technical environment that aspects of the present technology disclosed herein have been contemplated. Furthermore, although a general environment has been discussed, it should be understood that the examples described herein should not be limited to the general environment identified in the background.
- This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description section. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. Additional aspects, features, and/or advantages of examples will be set forth in part in the description which follows and, in part, will be apparent from the description or may be learned by practice of the disclosure.
- Non-limiting examples of the present disclosure describe systems, methods, and devices for enabling conversational bot communications. Examples described herein relate to one or more conversational bots, which may execute one or more skills. Each skill may be described in one or more manifests. A manifest may describe an object that is capable of sending and receiving conversational bot framework-compatible messages. A manifest may allow configuration, registration, and publishing data about a conversational bot and/or skill to be authored, stored, and transmitted between services. A manifest may describe how to contact a conversational bot, what capabilities a conversational bot offers, and information about how it should be published.
- In examples, a user-bot conversation may be initiated via a messaging construct. The messaging construct may comprise a web chat interface, an application chat interface, or a digital assistant interface, for example. A user may provide a query or command to the conversational bot. The conversational bot may determine whether it has a skill that is required to fulfill the query or command. If the conversational bot has the skill, it may simply perform the skill and in some instances communicate the completion of the query or command to the user.
- In examples where the conversational bot does not have the skill required to fulfill the query or command, it may determine whether there is another conversational bot that has the skill. The conversational bot that received the query or command (the primary conversational bot) may query a manifest marketplace or other skill repository to determine whether there is another conversational bot that may perform the skill and fulfill the query or command for the user. If a determination is made that there is another conversational bot that can execute the skill (the secondary conversational bot) the primary conversational bot may pass control of the conversation to the secondary conversational bot until the query or command is fulfilled. Passing the conversation may simply mean that the secondary conversational bot perform actions and/or communicate with the primary conversational bot and need not necessarily communicate directly via the user chat interface. For example, the secondary conversational bot may send requests for information to the primary conversational bot, the primary conversational bot may obtain that information, and it may be passed back to the secondary conversational bot for skill execution. Once the secondary conversational bot has executed a requisite skill to complete an original request, the primary conversational bot may take control of the conversation back over and respond to the original query or command.
- In some examples, an identity and a classification of a plurality of skills, manifests, and conversational bots may be stored in a manifest marketplace. The manifest marketplace may be searchable by skill type. In some examples, the skills, conversational bots, and/or manifests, may be scored based on website ranking factors. For example, websites that have associated conversational bots may have a rank score that may be provided to the manifest marketplace. The manifest marketplace may use the website ranking and in some cases one or more additional factors to rank conversational bots and associated skills. Skill requests from conversational bots may be routed to the manifest marketplace to determine whether there is a bot that can execute a particular skill needed to fulfill a request.
- Non-limiting and non-exhaustive examples are described with reference to the following figures:
-
FIG. 1 is a schematic diagram illustrating an example distributed computing environment for enabling conversational bot communications. -
FIG. 2 illustrates a skill manifest and associated exemplary elements included in the skill manifest. -
FIG. 3 illustrates a computing device and an exemplary communication with a conversational bot associated with a restaurant entity. -
FIG. 4 illustrates bot-to-bot communications and the processing of a command from the bot communication ofFIG. 3 . -
FIG. 5 illustrates a computing device that displays the result of the processing and bot-to-bot communications performed in relation toFIG. 4 . -
FIG. 6 is a schematic diagram illustrating an example distributed computing environment for identifying relevant skills from a manifest marketplace in relation to natural language input queries and commands. -
FIG. 7A is an exemplary method for enabling conversational bot communications in relation to a natural language input. -
FIG. 7B is another exemplary method for enabling conversational bot communications in relation to a natural language input. -
FIG. 8 is an exemplary method for enabling conversational bot communications in relation to a natural language input and a manifest marketplace. -
FIGS. 9 and 10 are simplified diagrams of a mobile computing device with which aspects of the disclosure may be practiced. -
FIG. 11 is a block diagram illustrating example physical components of a computing device with which aspects of the disclosure may be practiced. -
FIG. 12 is a simplified block diagram of a distributed computing system in which aspects of the present disclosure may be practiced. - Various embodiments will be described in detail with reference to the drawings, wherein like reference numerals represent like parts and assemblies throughout the several views. Reference to various embodiments does not limit the scope of the claims attached hereto. Additionally, any examples set forth in this specification are not intended to be limiting and merely set forth some of the many possible embodiments for the appended claims.
- The various embodiments and examples described above are provided by way of illustration only and should not be construed to limit the claims attached hereto. Those skilled in the art will readily recognize various modifications and changes that may be made without following the example embodiments and applications illustrated and described herein, and without departing from the true spirit and scope of the claims.
- Examples of the disclosure provide systems, methods, and devices for enabling conversational bot communications. A conversational bot may be published by, and/or associated with, a particular entity. For example, a first company may host a first conversational bot that provides artificial intelligence-driven conversational capabilities in relation to goods and/or services provided by that company, and a second company may host a second conversational bot that provides artificial intelligence-driven conversational capabilities in relation to goods and/or services provided by that company. Other entities that a conversational bot may be published by and/or associated with may include individual users, applications, websites, and/or digital assistants.
- A conversational bot may comprise one or more software objects that interact with input (e.g., natural language input from a user, schematic input, object-oriented input) and/or other bots in a conversational manner. A conversational bot may be associated with one or more skills, and each skill may be associated with one or more manifests. A skill may comprise one or more operations that are performed by a conversational bot to accomplish a task. A manifest may comprise one or more objects that describe how to contact a conversational bot, what capabilities a conversational bot offers, and information about how that information should be published.
- In some examples, a conversational bot and/or one or more corresponding manifests may be hosted by an entity's website or database. In additional examples, a plurality of manifests, conversational bots, and/or skills may be stored in one or more repositories. Examples of a conversational bot repository include a conversational bot service and a manifest marketplace. Websites and repositories that host bot manifests may be searchable, and therefore utilized, by computing devices seeking to interact with corresponding bots. Computing devices and their associated applications and interfaces may interact with hosted bots and manifests utilizing one or more application programming interfaces (APIs).
- In examples, when a conversational bot receives a request or command, a determination may be made by the conversational bot as to whether it has a skill (and corresponding manifest) that can fulfill that request or command. If a determination is made that the conversational bot does have a skill that can fulfill the request or command, and the conversational bot has all the information it needs to fulfill that request itself, it may execute the skill and fulfill the request or command. If a determination is made that the conversational bot does have a skill that can fulfill the request or command but that the conversational bot does not have all the information it needs to fulfill that request itself, it may cause a request for the missing information to be surfaced in a chat dialog with a user that can provide that missing information. In some examples, the conversational bot, if it has been granted permission to do so by a user (e.g., via account settings, via explicit input), may automatically obtain missing information needed to complete a request or command from a user account database or other information source associated with a requesting user.
- In examples where a conversational bot does not have a skill needed to fulfill a request or command, the conversational bot, or a conversational bot service, may identify a secondary conversational bot that is capable of executing that skill and thereby fulfilling the request or command. In some examples, a conversational bot service may access a manifest marketplace to identify one or more conversational bots that have a skill that can fulfill a request or command. The manifest marketplace may include a skill library that includes an identity of each of a plurality of conversational bots, an identity of one or more skills that are associated with each of those conversational bots, and one or more manifests associated with those one or more skills. In some examples, the manifest marketplace may be populated with skills, conversational bots, and/or manifests via a registration process. In some examples, the manifest marketplace may require that skills are capable of performing, or necessarily perform, one or more actions. For example, the manifest marketplace may require that skills of a specific skill type execute an encryption or personal identifiable information (PII) scrubbing action on data they receive in order to be registered with the manifest marketplace. In additional examples, the manifest marketplace may only register skills/bots for publishers of those skills/bots that go through an authentication process and/or that meet certain authentication requirements. In some examples, websites that have associated conversational bots may include their manifest in the website directory. Thus, the manifest marketplace may crawl website directories to identify new skills that may be registered with the manifest marketplace.
- In examples, rather than having only a single manifest marketplace, there may be a plurality of manifest marketplaces. For example, there may be one or more public marketplaces as well as one or more domain specific marketplaces. Additionally, private marketplaces may exist. For example, manifest marketplaces may exist behind corporate firewalls or inside a government data center. The non-public marketplaces may be searchable upon authentication of user domains and/or user credentials. According to some examples, a manifest marketplace may be maintained by an administrator. For example, an administrator for a marketplace may only allow certain bots and/or skills to be added to it. Additionally, an administrator for a marketplace may remove certain bots and/or skills as it sees fit and/or based on violations of a terms of service contract.
- In examples, the skills and/or the manifests included in the skill library may be classified, tagged, organized, or otherwise associated with a skill type. As such, the skills and/or manifests may be searchable and/or matched to natural language queries and commands, and schema-defined queries and commands, that are received by the conversational bot service. According to some examples, a conversational bot may only hand off or accept skill execution operations to or from other conversational bots that have one or more skills and/or require one or more particular skills be performed. For example, a conversational bot may only hand off or accept skill execution operations to or from other conversational bots that will execute a PII scrubbing skill on any data associated with skill performance. In another example, a conversational bot may only hand off or accept skill execution operations to or from other conversational bots that will execute a data encryption skill on any data that it sends or receives.
- In examples where a secondary conversational bot has been passed control of a conversation and/or execution of a skill, the secondary conversational bot may execute the skill that has been requested of it by the primary conversational bot (e.g., the bot in textual or audio communication with the user). In instances where the secondary conversational bot needs additional information to execute a skill, the secondary conversational bot may request that information from the primary bot or a different source. For example, the secondary conversational bot may provide a request for additional information that may be sent to the primary conversational bot and surfaced to the user in a conversational interface. The user may provide the necessary information that is required to the primary conversational bot, and that information may be provided by the primary conversational bot to the secondary conversational bot. In other examples, the secondary conversational bot may have been provided with access to obtain missing information directly (e.g., from a user database, from the user directly) rather than having to go through the primary conversational bot. In such instances, the secondary conversational bot may directly obtain the missing information and execute one or more skills so that a request or command received by the primary conversational bot can be completed. Once the request or command has been completed via execution of one or more skills by the secondary conversational bot, the secondary conversational bot may pass control over the conversation back to the primary conversational bot. The actions by the secondary conversational bot need never be seen by the interacting user. That is, in some examples, the requesting/commanding user need not receive a direct message from the secondary conversational bot. Rather, the primary conversational bot may relay any requests from the secondary conversational bot to the user or a database associated with the user, and/or the secondary conversational bot may perform actions on the backend that are never seen by the user aside from their results. In this way, the user may be provided with a seamless experience in which a single conversational bot accomplishes or facilitates all desired user actions.
- In examples where a determination is made that a primary conversational bot does not have a skill needed to fulfill a request or command, and a further determination is made that there are no secondary conversational bots that have the skill needed to fulfill the request or command, the primary conversational bot may perform one or more fallback operations. A first fallback operation may comprise causing an indication (e.g., in a chat dialog) that the request or command cannot be fulfilled by the conversational bot to be surfaced. A second fallback operation may comprise causing a web search corresponding to the request or command to be performed and causing one or more results from that web search to be surfaced. A third fallback operation may comprise identifying one or more related tasks to the request or command that can be fulfilled via execution of an identified skill to be surfaced.
- The systems, methods, and devices described herein provide technical advantages for performing tasks electronically and interacting with conversational bots. Processing costs (e.g., CPU cycles) associated with identifying services that may assist with task completion are reduced in that conversational bots may automatically identify other conversational bots associated with services that may be useful in executing skills that are needed for task completion. Privacy concerns associated with handing off information to third parties are also addressed via the mechanisms described herein in that when a task (e.g., a user query, a user command) is received by a first conversational bot, that conversational bot may require that secondary conversational bots that may complete portions of the task execute privacy-oriented skills (e.g., PII scrubbing skills, data encryption skills). User experience is also enhanced via the mechanisms described herein in that a user may have multiple conversational bots execute skills on her behalf, but only need to interact with a single conversational bot in the process. That is, because the systems, methods, and devices described herein allow for more interaction between conversational bots behind the scenes, the user need not be involved on the front end with each bot handling a request or command.
-
FIG. 1 is a schematic diagram illustrating an example distributedcomputing environment 100 for enabling conversational bot communications.Computing environment 100 includes naturallanguage input sub-environment 102, network and processing sub-environment 124, data sources sub-environment 114,manifest marketplace sub-environment 142, firstconversational bot sub-environment 130, and secondconversational bot sub-environment 148. - Natural
language input sub-environment 102 includesuser 104,laptop computing device 108, digitalassistant audio device 110, andmobile computing device 112.User 104 issuesnatural language input 106A.Natural language input 106A may comprise a command or a query.Natural language input 106A may be received by one oflaptop computing device 108, digitalassistant audio device 110, ormobile computing device 112. In examples,natural language input 106A may be received by a bot chat interface as illustrated bybot chat element 128 onlaptop computing device 108, andbot chat element 129 onmobile computing device 112. For example, a website or application accessed on a computing device may surface a bot chat interface and a natural language input may be typed or audibly received by the bot chat interface. In other examples,natural language input 106A may be received by a digital assistant service, such as by a digital assistant service associated with digitalassistant audio device 110. In examples wherenatural language input 106A is received by a digital assistant service, it may be sent from the digital assistant service to a conversational bot service. In examples wherenatural language input 106A is received directly by a bot interface, it may be automatically and/or directly received from a corresponding computing device (e.g.,laptop computing device 108, mobile computing device 129) by a conversational bot service.Natural language input 106A may be sent to the conversational bot service via an API. - Network and
processing sub-environment 124 includesnetwork 126 andserver computing device 128. Any and all of the computing devices described herein may communicate via a network, such asnetwork 126 in network andprocessing sub-environment 124. Network and processing sub-environment 124 may comprise a conversational bot service. That is, a conversational bot service may be hosted by one or more server computing devices, such asserver computing device 128. The conversational bot service may comprise a plurality of conversational bots. Each of the conversational bots may be associated with one or more manifests. A manifest may comprise an interface definition language (IDL) that includes instructions for sending, receiving, and processing commands associated with skills that a conversational bot may perform. A skill may comprise one or more activities that may be performed by a conversational bot. Additional details related to the elements that may be included in a manifest are described below in relation toFIG. 2 . Certain bots associated with the conversational bot service may not be associated with their own manifests. That is, a conversational bot that receives inputs and directs those inputs to one or more other conversational bots, but which does not perform any skills itself, may be associated with one or more conversational APIs, but need not be associated with its own manifest or manifests. - In some examples, the conversational bot service may include or otherwise be associated with a manifest marketplace. The manifest marketplace is illustrated by
manifest marketplace sub-environment 142.Manifest marketplace sub-environment 142 includesskill library 146.Skill library 146 includes an identity of each of a plurality of conversational bots, an identity of one or more skills that are associated with each of those conversational bots, and one or more manifests associated with those one or more skills. In examples, the skills and/or the manifests included inskill library 146 may be classified, tagged, organized, or otherwise associated with a skill type. As such, the skills and/or manifests may be searchable and/or matched to natural language queries and commands, and schema-defined queries and commands, that are received by the conversational bot service. For example,natural language input 106D may be provided to manifestmarketplace 142 and one or more keywords or phrases included innatural language input 106D may be matched to one more skills or skill types inmanifest marketplace 142. - The conversational bot service and/or the manifest marketplace may communicate with one or more data sources as illustrated by data sources sub-environment 114. Data sources sub-environment 114 includes
data store 116,documents 119,user information 118,electronic messages 120, andcalendar information 122.Data store 116 is representative of various data sources.Data store 116 may comprise data from the World Wide Web, data associated with conversational bot entities, and/or data associated with user accounts. Regarding data associated with user accounts,data store 116 may comprise data for a plurality of user accounts associated with one or more computing devices, one or more applications (e.g., productivity applications), and/or one or more application suites (e.g., productivity application suites). For example,user 104 may have a user account that is associated with one or more applications and/or services and data that includes settings for that user account, preferences for that user account, documents generated by that user account, documents sent by that user account, and/or documents received by that user account, all of which may be included indata store 116 withuser information 118. Thus, in this example,data store 116 may includeuser information 118,documents 119emails 120, and calendar information 122 (or references thereto) associated with one or more user accounts. - In this example,
natural language input 106A is sent from one oflaptop computing device 108, digitalassistant audio device 110, ormobile computing device 112 to the conversational bot service in network and processing sub-environment 124 as illustrated by natural language input 106B. Natural language input 106B may be processed by one or more natural language processing models associated with the conversational bot service. In examples, the one or more natural language processing models may determine whether natural language input 106B includes a reference to a specific conversational bot, reference to a specific skill, a reference to a specific skill type, reference to a task to be performed, or any combination of the same. In examples where a specific skill and a specific conversational bot are identified in natural language input 106B, a schematized request to perform the skill may be sent from the conversational bot service to an endpoint corresponding to the specific conversational bot and the one or more manifests corresponding to the specific skill. - In examples where a specific conversational bot is not identified, but a specific skill and/or skill type is identified, further processing may be performed to determine a specific conversational bot and corresponding one or more manifests to send a request to. In some examples, the processing may comprise identifying one or more conversational bots, or entities associated with one or more conversational bots, that a user account for
user 104 is associated with. For example, a user token comprising the identity ofuser 104 and/oruser 104's user account may be provided toskill library 146 and a determination may be made as to one or more conversational bots and/or entities associated with conversational bots thatuser 104 has an account for. In additional examples, the processing may comprise identifying one or more conversational bots and/or entities that are publicly available for use (e.g., a user does not have to have an account to access). The processing may further comprise identifying one or more conversational bots that have one or more manifests for performing a skill corresponding to the identified specific skill and/or skill type. In examples where more than one conversational bot is identified as being capable of performing an identified specific skill from natural language input 106B, those conversational bots may be ranked. The ranking may comprise one or more metrics (e.g., popularity of website corresponding to conversational bot, popularity of conversational bot by other users, history of using conversational bot by user 104). - In some examples, once a conversational bot and one or more manifests associated with the conversational bot have been identified for performing the specific skill, the conversational bot service may send a request for performance of the skill to one or more endpoints for the identified manifests. In other examples, if a conversational bot is identified for performing the specific skill, but a specific manifest has not been identified, the conversational bot service may send the natural language directly to the conversational bot and the conversational bot may identify one or more manifests for performing the specific skill. In this example,
conversational bot A 132 in firstconversational bot sub-environment 130 is identified as being able to perform a specific skill identified in natural language input 106B. As such, the request for performance of the specific skill is sent toconversational bot A 132 viaskill request 107A. - In examples, a conversational bot may initiate and open a conversation between an origination point and the conversational bot when a skill request is received. The origination point may comprise a computing device (e.g.,
laptop computing device 108, digitalassistant audio device 110, mobile computing device 112), an interface where the skill request originated (e.g.,bot chat interface 128, bot chat interface 129), and/or a conversational intermediary between an originating natural language input and the conversational bot (e.g., a digital assistant service, the conversational bot service). Thus, in this example, a conversation is initiated and opened byconversational bot A 132 when it receivesskill request 107A. As an example,skill request 107A may comprise a request to performskill A 134. Thus,skill request 107A may be sent to an endpoint specified inmanifest A 136.Skill A 134 may then be performed byconversational bot A 132. - In addition to
request 107A including a request to performskill A 134,request 107A may include a request to perform an additional skill that is not a skill that may be performed byconversational bot A 132. That is, the additional skill may not beskill A 134 or one of the other skills, such asskill N 138 and correspondingmanifest N 140, thatconversational bot A 132 is capable of performing. In examples,conversational bot A 132 may identify another conversational bot that is capable of performing the additional skill. For example, conversational bot A may query the conversational bot service and/or the manifest marketplace (e.g., skill library 146) to identify one or more bots and corresponding manifests for performing the additional skill. - In this example, conversational bot A identifies that the additional skill is
skill C 152, which is performable byconversational bot 150 in secondconversational bot sub-environment 148. As such,conversational bot A 132 sendsskill request 107B to the endpoint included inmanifest C 154 corresponding toskill C 152.Conversational bot B 150 may then performskill C 152 and return the result either directly to conversational bot A or to the origination point of the request (e.g.,laptop computing device 108,bot chat interface 128, digital assistantaudio computing device 110,mobile computing device 112, bot chat interface 129) where the result may be caused to be surfaced. - According to examples, if
conversational bot B 150 requires additional information not specified inskill request 107B to fulfill the request,conversational bot B 150 may send a request for that additional information back toconversational bot A 132. For example,conversational bot B 150 may need location information foruser 104, account credential information foruser 104, and/or payment information foruser 104, and that information may be requested fromconversational bot A 132, which may obtain that information and return it toconversational bot B 150. In some examples,conversational bot A 132 may identify that information directly fromuser 104 via part of the conversation withuser 104 on one of the devices included in naturallanguage input sub-environment 102. In other examples,conversational bot A 132 may identify that information from an account associated with user 104 (e.g., fromuser information 118 in data sources sub-environment 114). The retrieval of that information may be made via execution of a specific skill and manifest associated withconversational bot A 132. - In examples where a first conversational bot sends a subsequent request to a second conversational bot, the first conversational bot may turn over control of the open conversation to the second conversational bot. Once the second conversational bot has completed the portion of the request corresponding to its skills, the second conversational bot may turn control of the conversation back over to the first conversational bot.
-
FIG. 2 illustrates askill manifest 202 and associated exemplary elements included in the skill manifest. Manifest 202 comprisespublishing information element 204,connection element 206,language element 208,activities element 210, and sentactivities element 228.Activities element 210 further comprisesmessage element 212,events element 214, end ofconversation element 216, invokeelement 218, addelement 220,update element 222, deleteelement 224, and [custom]element 226. Manifest 202 may comprise more or fewer elements than illustrated in this example. Manifest 202 may describe an object that is capable of sending and receiving conversational bot framework-compatible messages. Manifest 202 may allow configuration, registration, and publishing data about a conversational bot to be authored, stored, and transmitted between services. Manifest 202 may describe how to contact a conversational bot, what capabilities a conversational bot offers, and information about how it should be published. Manifest 202 may be a flat list of name/value pairs, some of which are primitive, and some of which are complex (nested). Manifest 202 may be expressed in the JSON format but may also be projected into in-memory data structures in other languages (e.g., C #, JavaScript). -
Publishing information element 204 may define an identity of a bot associated withmanifest 202 that may execute activities included inmanifest 202; a publisher of the one or more bots associated withmanifest 202; copyright information formanifest 202; and license information formanifest 202.Publishing information element 204 may contain information about how a conversational bot is published into a registry (e.g., a conversational bot service, a manifest marketplace). -
Connection element 206 may define endpoint information for connecting withmanifest 202 from one or more geographic locations (e.g., North America endpoint, European endpoint).Connection element 206 may include an authentication connection representing a sign-in service containing parameters describing how to use a conversational bot to request user sign-in and consent. -
Language element 208 may define an identity of one or more languages that are supported by the bot associated withmanifest 202. -
Activities element 210 may define operations that the bot can receive, execute, and/or modify.Activities element 210 may represent conversational behaviors made by humans and automated software within chat applications, email, and other text interaction programs. Each activity object may include a type field and represent a single action (e.g., send text content, send multimedia attachments, execute “like” button, typing indicator). Exemplary activities inactivities element 210 are described below. -
Message element 212 may define message types that may be received and attempted to be resolved by the bot. Message activities may represent content intended to be shown within a conversational interface. Message activities may contain text, speech, interactive cards, and binary or unknown attachments. -
Events element 214 may define a plurality of named event type properties that indicate a task that a skill may execute. Events may be custom developed for each skill type. For example, for a flight booking skill there may be a “BookFlight” event that books a flight when called; for a calendar application there may be a “clock” event that checks an electronic calendar for calendar entries at periodic intervals when called; for an investment skill there may be a “GetStockPrice” event that obtains stock prices when called. Event activities may communicate programmatic information from a client or channel to a conversational bot. The meaning of an event activity may be defined by a “name” field, which may be meaningful within the scope of a channel. Event activities may be designed to carry both interactive information (such as button clicks) and non-interactive information (such as notification of a client automatically updating an embedded speech model). Event activities are the asynchronous counterpart to invoke activities. Event activities may be identified by a “type” value of event and specific values of the “name” field. - End of
conversation element 216 may comprise a definition for ending an open conversation between a first conversational bot and the conversational bot associated withmanifest 202 and/or ending an open conversation between the conversational bot associated withmanifest 202 and a user communication element. In some examples, end of conversation activities may signal the end of a conversation from a recipient's perspective. This may be because the conversation has been affirmatively ended, or because the recipient has been removed from the conversation in a way that is indistinguishable from it ending. End of conversation activities may be identified by a “type” value. - Invoke
element 218 may define a plurality of named invoke type properties that indicate a task that a skill will execute. Invoke activities may communicate programmatic information from a client or channel to a conversational bot and have a corresponding return payload for use within a channel. Invoke activities are the synchronous counterpart to event activities. Event activities may be designed to be extensible. Invoke activities may differ only in their ability to return response payloads back to a channel; because the channel must decide where and how to process these response payloads, invoke may only be useful in cases where explicit support for each invoke name has been added to a channel. Invoke activities may be identified by a “type” value of “invoke” and specific values of the “name” field. - Add
element 220 may comprise a definition for adding operations and/or content to an existing activity. -
Update element 222 may comprise a definition for updating operations and/or content for an existing activity. -
Delete element 224 may comprise a definition for deleting operations and/or content from an existing activity. - [Custom]
element 226 may comprise custom definitions for activities that may be created by bot developers for use in custom skills. -
Sent activities element 228 may comprise activities that a conversational bot can send to a skill (e.g., to a different conversational bot). In contrast,activities 210 may comprise activities that a conversational bot corresponding to manifest 202 may perform. -
FIG. 3 illustrates acomputing device 302A/302B and an exemplary communication with a conversational bot associated with a restaurant entity.Computing device 302A in sub-environment 300A, andcomputing device 302B in sub-environment 300B are the same computing device. -
Web browser 304A is open oncomputing device 302A.Web browser 304A is currently navigated to www.[restaurantwebsite].com. In this example, a user is interacting with the restaurant entity via a conversational bot for that entity. Specifically, real-timebot chat window 306A is being utilized to communicate with conversational bot A 308A. Conversational bot A 308 may be hosted by a conversational bot service and may be associated with a plurality of manifests (e.g., a manifest for skill A, a manifest for skill B, a manifest for skill C). -
Conversational bot A 308A opened the dialog inchat window 306A with the text “Welcome! How can I help you?” The user then responded with the natural language input “I would like to book a table for four at 7 pm tomorrow”. - Moving to sub-environment 300B, in response to the user's natural language input discussed above, conversational bot A 308B communicates the text “Your table is booked” in
chat window 306B. That is, conversational bot A 308A may have executed a “BookTable” skill (e.g., one of skills A, B, or C). The user then responded with the natural language input “Book me a car for my dinner.” Additional details regarding the processing of this natural language input are discussed below in relation toFIG. 4 . -
FIG. 4 illustrates bot-to-bot communications and the processing of a command from the bot communication ofFIG. 3 . The command at issue in this example isbook car request 426, which originated from the natural language input “Book me a car for my dinner.” That natural language input was received byconversational bot A 416, which is associated with the restaurant entity.Conversational bot A 416 has three skills that it can perform. Specifically,conversational bot A 416 can performskill A 418,skill B 420, andskill C 422. However, whenconversational bot A 416, or the conversational bot service on behalf ofconversational bot A 416, receives the natural language input and processes it with one or more natural language processing models, a determination may be made that none of the skills that conversational bot A can perform will fulfill an identified “book car” command. - Because
conversational bot A 416 determines that it cannot fulfill the “book car” command itself, it may determine whether one or more other bots that the user that generated the corresponding natural language input has access to may fulfill the “book car” command. In this example,conversational bot A 416 sendsbook car request 426 anduser token 428 to manifestmarketplace sub-environment 432.Book car request 426 may comprise the natural language input originally received byconversational bot A 416 or a schematized version of that natural language input.User token 428 may comprise an account identity associated with the user that provided the natural language input and/or security credentials associated with that account. This information may be obtained byconversational bot A 416 fromuser data store 412. -
Manifest marketplace sub-environment 432 includesskill library 430.Skill library 430 includes an identity of each of a plurality of conversational bots, an identity of one or more skills that are associated with each of those conversational bots, and one or more manifests associated with those one or more skills. In examples, the skills and/or manifests included inskill library 430 may be classified, tagged, organized, or otherwise associated with a skill type. As such, the skills and/or manifests may be searchable and/or matched to natural language queries and commands, and schema-defined queries and commands, that are received by the conversational bot service and/or the manifest marketplace. - In this example, a determination is made of a skill type associated with
book car request 426 of “book car” skill. Thus, a plurality of book car skill bots (bot B, bot C, bot D) and corresponding manifests that are capable of performing the desired skill and handlingbook car request 426 are illustrated in bookcar skill element 434. Each of those book car skill bots and corresponding manifests are associated with a different rideshare service. This is illustrated byrideshare service sub-environment 402. For example, conversational bot B corresponds to rideshareservice B database 406, conversational bot C corresponds to rideshareservice C database 408, and conversational bot D corresponds to rideshareservice database D 410. Although the databases for the rideshare services are illustrated as containing their corresponding bots and manifests, the bots and manifests may be maintained separately by the conversational bot service. - According to examples, a determination may be made, utilizing
user token 428, via the manifest marketplace and/or communication with one or more of rideshareservice B database 406, rideshareservice C database 408, and/orrideshare service D 410, that the user account associated with the natural language input book car request only has an account for one or a subset of the rideshare services and/or that settings associated with that user account indicate a preference for one of the rideshare services. In this example, a determination has been made that the rideshare service that the user account has an account with and/or a preference for using corresponds to rideshareservice C database 408. As such,conversation control 442 is passed fromconversational bot A 416 toconversational bot C 436, which is the conversational bot for that rideshare service.Conversation control 442 may comprise backend communications and operations that are never surfaced in a communication with a corresponding user. For example,conversational control 442 here may compriseconversational bot C 436 communicating withconversational bot A 416, one or more additional conversational bots, and/oruser data store 412. -
Conversational bot C 436 may perform three skills (skill E 438,skill F 440, and skill G 442). At least one of those skills corresponds to a “book car” skill that is needed to completebook car request 426. However, in some instances it may be the case that a bot that has taken control of a conversation needs additional information to complete to execute a skill and complete a request. In this example,conversational bot C 436 makes a determination that it needs one or both of an address where the user needs to be picked up and/or an address where the user needs to be dropped off. Depending on user privacy settings,conversational bot C 436 may obtain that address information directly fromuser data store 412 if it is available, or fromconversational bot A 416. In examples where that information is not available fromuser data store 412,conversational bot C 436 may request the address information fromconversational bot A 416. As such,conversational bot A 416 may surface a request for that information in the bot chat window that it has open on one of the computing devices associated with the user, the user may provide that information toconversational bot A 416, and addressinformation 440 may then be provided fromconversational bot A 416 toconversational bot C 436. - Once
conversational bot C 436 has all the requisite information needed to execute the “book car” skill it may execute and fulfillbook car request 426, as indicated by execute carbooking skill element 448. Upon executing the “book car” skill,conversational bot C 436 may send an “end of conversation” action indication toconversational bot A 416, which may then take back control of the conversation. -
FIG. 5 illustrates acomputing device 502 that displays the result of the processing and bot-to-bot communications performed in relation toFIG. 4 .Conversational bot A 508 has taken control of the conversation back from the rideshare conversational bot (conversational bot C 436).Computing device 502displays web browser 504 andbot messaging window 506. The user has previously provided the natural language input “Book me a car for my dinner” inmessaging window 506,conversational bot A 508 processed that request in combination withconversational bot C 436, and the conversational bot notes this inmessaging window 506 with the text “Your car from [rideshare service] C will pick you up. - Although
FIGS. 3-5 illustrate bot-to-bot communications and user communications in relation to a bot messaging window in a website, other messaging constructs are within the scope of this disclosure. For example,conversational bot A 508 may communicate with the user via an audio device, via an intermediary digital assistant, via SMS message, and/or via email. -
FIG. 6 is a schematic diagram illustrating an example distributedcomputing environment 600 for identifying relevant skills from a manifest marketplace in relation to natural language input queries and commands.Computing environment 600 includesmanifest marketplace 602,company database sub-environment 630, andweb service sub-environment 638. -
Manifest marketplace 602 includesskill library 604.Skill library 604 includes an identity for each of a plurality of conversational bots, an identity of one or more skills that are associated with each of those conversational bots, and one or more manifests associated with those one or more skills. In examples, the skills and/or manifests included inskill library 604 may be classified, tagged, organized, or otherwise associated with a skill type. As such, the skills and/or manifests may be searchable and/or matched to natural language queries and commands, and schema-defined queries and commands, that are received by the conversational bot service. - In this example,
manifest marketplace 602 includesskill classifications 618. That is, there are a plurality of skill types for which corresponding manifests and conversational bots have been classified by skill type.Skill classifications 618 includestravel booking skills 620, with corresponding conversational bots A, B and C;book car skills 622, with corresponding conversational bots D, E and F;investment skills 624, with corresponding conversational bots G, H and I;weather skills 626, with corresponding conversational bots J, K and L; and type N skills 618 (exemplary of any miscellaneous skill type), with corresponding conversational bots M, N and O. The skill type divisions that are illustrated may be further divided in sub-classes and sub-classes of sub-classes. For example,travel booking skills 620 may be divided into flight booking skills, hotel booking skills, train booking skills.Book car skills 622 may be divided into car types and ride share types.Investment skills 624 may be divided by transaction type, pay or no pay, and cost per transaction.Weather skills 626 may be divided by geographic region and radar types. -
Company database sub-environment 630 illustrates three exemplary investment services that each have a corresponding conversational bot. Although the three exemplary investment services are illustrated as being in the same sub-environment, they each correspond to a different service and therefore are separate and distinct from one another. For example, investmentservice A database 632 is associated with conversational bot G, which may perform one or more investment skills. Investmentservice B database 634 is associated with conversational bot H, which may perform one or more investment skills. Investmentservice C database 636 is associated with conversational bot I, which may perform one or more investment skills. The skills performed by each of those bots may be the same skill or a different skill. The manifests for those skills may differ by publishing elements only, or by a plurality of elements (e.g., endpoints, languages, activities, sent activities). -
Manifest marketplace 602 further includesskill ranking engine 612,site ranking engine 614, and natural language processingskill matching engine 616.Skill ranking engine 612 and/orsite ranking engine 614 may receive data fromweb service sub-environment 638. The data may comprise website rank data or data that may be utilized byskill ranking engine 612 and/orsite ranking engine 614 to rank websites and associated skills. For example,web service sub-environment 638 includesweb data 640 for websites andweb search engine 642 from one or more search engines.Web data 640 may include an identity of a plurality of websites associated with entities that have conversational bots.Web data 640 may further include use data associated with those websites. For example,web data 640 may include a number of users that access a website, a frequency of access for a website, engagement data with a website (e.g., amount of time spent by users, number of pages of website viewed, number of scrolls on each page, number of orders placed), and/or engagement data with a conversational bot associated with a web page.Web search engine 642 may provide additional information toweb data 640 that may be received byskill ranking engine 612 and/orsite ranking engine 614. For example,search engine 642 may collect information related to a number of searches for a particular website and/or conversational bot, a number of clicks on websites and/or conversational bot elements, a frequency of searches, and search term patterns. Any of that data may be provided toskill ranking engine 612 and/orsite ranking engine 614. -
Skill ranking engine 614 may utilize data fromweb service sub-environment 638,site ranking engine 614, and/or skill use data fromskill library 604 in calculating scores for skills and ranking them. The scores may be based on one or more of popularity of skill, popularity of conversational bot associated with skill, and/or popularity of entity associated with conversational bot. One or more machine learning models, such asneural network 610 and unsupervisedmachine learning model 606 may be utilized in calculating scores for websites and skills. - Natural language processing
skill match engine 616 may perform operations associated with identifying relevant words and phrases from a natural language input provided to a conversational bot and matching those words and/or phrases to one or more highest ranked skills. Natural language processingskill match engine 616 may apply one or more language processing models, such aslanguage processing model 608 in scoring words as being relevant to a skill.Language processing model 608 may comprise one or more word and/or sentence embedding layers (e.g., Word2Vec, BERT, Sent2Vec, Embeddings from Language Models (ELMo), recurrent neural network). -
Language processing model 608 may further comprise a sentence level information aggregation layer for aggregating embeddings from each word in a natural language input into a distinct embedding for one or more sentences included in a natural language input. The distinct sentence aggregation layer may apply a neural network to the embeddings for each word. In examples, the neural network may comprise a gated recurrent unit (GRU) neural network or bidirectional GRU (bi-GRU) neural network. In other examples the neural network may comprise a long short-term memory (LSTM) neural network, an attention-based aggregation method, etc. -
Language processing model 608 may further comprise a contextual aggregation layer for aggregating each distinct embedding for each of the sentences into a contextual embedding. In aggregating the distinct embeddings for each sentence, the contextual aggregation layer may apply a neural network to each distinct embedding for each of the sentences. In examples, the neural network may comprise a GRU neural network, or bi-GRU neural network. In other examples, the neural network may comprise an LSTM neural network, an attention-based aggregation method, etc. -
Language processing model 608 may further comprise a scoring layer for scoring and ranking each of the sentences in a natural language input based on their relevance to a skill type (e.g., a “book car” skill type, “get stock price” skill type). In scoring and ranking each sentence, the scoring layer may apply a classifier function to each contextual embedding for each of the plurality of sentences (e.g., the embeddings generated by the contextual aggregation layer). In examples, the classifier function may comprise a sigmoid function. Other activation functions (e.g., tanh, softplus) may be utilized for scoring each sentence. -
FIG. 7A is anexemplary method 700A for enabling conversational bot communications in relation to a natural language input. Themethod 700A begins at a start operation and flow moves tooperation 702A. - At
operation 702A a natural language input is received by a first conversational bot. The natural language input may be received via a chat interface in a website, a chat interface in an application, a real-time messaging application, an SMS message, an email, and/or an intermediary digital assistant. The natural language input may be associated with a user account. For example, a user may be signed into an account associated with a web browser application, one or more applications, and/or an application service, and the account information may therefore be made available to the conversational bot. - From
operation 702A flow continues tooperation 704A where a skill needed to fulfill one of: a query included in the natural language input, and a command included in the natural language input, is identified. The skill may be associated with one or more manifests. A manifest may comprise one or more objects that describe how to contact a conversational bot, what capabilities a conversational bot offers, and information about how that information should be published. The skill may comprise one or more operations that may be performed by a conversational bot to accomplish a task. The skill may be identified via application of one or more natural language processing models to the natural language input. - From
operation 704A flow continues tooperation 706A where a determination is made that the skill is not executable by the first conversational bot. for example, the conversational bot and/or a conversational bot service may determine that the first conversational bot does not have a manifest that includes one or more activities needed to perform the skill. - From
operation 706A flow continues tooperation 708A where a determination is made that the skill is executable by a second conversational bot. The determination may be made by the conversational bot, a conversational bot service, and/or a manifest marketplace. In some examples, there may be more than one conversational bot that is identified as capable of executing the skill. In such examples, the plurality of bots may be scored and ranked based on factors including: popularity of conversational bot, popularity of website associated with conversational bot, and user preferences. - From
operation 708A flow continues tooperation 710A where a request for execution of the skill is sent from the first conversational bot to the second conversational bot. The request may be sent via an API. In some examples, the request may comprise the natural language input. In other examples, the request may comprise a schematized request for execution of the skill. In additional examples, the request may include a token describing the user (or user account) that generated the natural language input. - From
operation 710A flow continues tooperation 712A where a request for information needed for the second conversational bot to execute the skill is received by the first conversational bot from the second conversational bot. The information may relate to the user that generated the natural language input. For example, the second conversational bot may need payment information, time information, location information, among other information types, to execute the skill. In some examples, when the first conversational bot receives the request for information, it may request that information from the user (e.g., via a chat window, via an audio query). In other examples, when the first conversational bot receives the request for information, it may determine whether that information is available in a user store associated with a user account for the user. - From
operation 712A flow continues tooperation 714A where the information is sent by the first conversational bot to the second conversational bot. The second conversational bot may execute the skill and turn control of the conversation back over to the first conversational bot. - From
operation 714A flow moves to an end operation and themethod 700A ends. -
FIG. 7B is anotherexemplary method 700B for enabling conversational bot communications in relation to a natural language input. Themethod 700B begins at a start operation and flow moves tooperation 702B. - At
operation 702B a request for execution of a skill is received by a second conversational bot from a first conversational bot. The request for execution of the skill may be received in the form of the natural language input or a schematized format. According to some examples, the second conversational bot may determine whether the first conversational bot executes a PII scrubbing skill. In examples, the second conversational bot may only execute the skill for the first conversational bot if the first conversational bot executes a PII scrubbing skill. - From
operation 702B flow continues to operation 704B where a determination is made by the second conversational bot that execution of the skill requires additional information. The information may relate to the user that generated the natural language input. For example, the second conversational bot may need payment information, time information, location information, among other information types, to execute the skill. - From operation 704B flow continues to
operation 706B where a request for the additional information is sent from the second conversational bot to the first conversational bot. The request may be sent to the second conversational bot via an API. - From
operation 706B flow continues tooperation 708B where the additional information is received by the second conversational bot. - From
operation 708B flow continues tooperation 710B where the second conversational bot executes the skill. The second conversational bot may then pass control of the conversation back to the first conversational bot. - From
operation 710 B flow moves to an end operation and themethod 700B ends. -
FIG. 8 is anexemplary method 800 for enabling conversational bot communications in relation to a natural language input and a manifest marketplace. Themethod 800 begins at a start operation and flow moves tooperation 802. - At operation 802 a marketplace comprising an identity of a plurality of skills is maintained, each of the plurality of skills associated with a manifest and a conversational bot. The manifest marketplace may include a skill library that includes an identity of each of a plurality of conversational bots, an identity of one or more skills that are associated with each of those conversational bots, and one or more manifests associated with those one or more skills. In examples, the skills and/or the manifests included in the skill library may be classified, tagged, organized, or otherwise associated with a skill type. As such, the skills and/or manifests may be searchable and/or matched to natural language queries and commands, and schema-defined queries and commands, that are received by a conversational bot service.
- From
operation 802 flow continues tooperation 804 where a request for execution of a specific skill is received. The request may be received by the conversational bot service and/or a manifest marketplace. The request may be in the form of a natural language input or it may be in a schematized format. - From
operation 804 flow continues tooperation 806 where a plurality of conversational bots is identified, each of the plurality of conversational bots being associated with a manifest for executing the specific skill. In examples, the identification may be made based on keyword and/or phrase matching between a natural language input comprising the request for the specific skill, and categorized skill names in the skill library. - From
operation 806 flow continues tooperation 808 where the plurality of conversational bots is ranked. The plurality of conversational bots may be ranked based on a calculated value for each of the conversational bots. The value may be based on one or more of: a relevance of the bot to the requested skill, a popularity of the bot, a popularity of a website associated with the bot, user preferences, and user history of interaction with the conversational bots. - From
operation 808 flow continues tooperation 810 where the skill is caused to be executed by the highest-ranked conversational bot. - From
operation 810 flow moves to an end operation and themethod 800 ends. -
FIGS. 9 and 10 illustrate amobile computing device 900, for example, a mobile telephone, a smart phone, wearable computer (such as smart eyeglasses), a tablet computer, an e-reader, a laptop computer, or other AR compatible computing device, with which embodiments of the disclosure may be practiced. With reference toFIG. 9 , one aspect of amobile computing device 900 for implementing the aspects is illustrated. In a basic configuration, themobile computing device 900 is a handheld computer having both input elements and output elements. Themobile computing device 900 typically includes adisplay 905 and one ormore input buttons 910 that allow the user to enter information into themobile computing device 900. Thedisplay 905 of themobile computing device 900 may also function as an input device (e.g., a touch screen display). If included, an optionalside input element 915 allows further user input. Theside input element 915 may be a rotary switch, a button, or any other type of manual input element. In alternative aspects,mobile computing device 900 may incorporate more or fewer input elements. For example, thedisplay 905 may not be a touch screen in some embodiments. In yet another alternative embodiment, themobile computing device 900 is a portable phone system, such as a cellular phone. Themobile computing device 900 may also include anoptional keypad 935.Optional keypad 935 may be a physical keypad or a “soft” keypad generated on the touch screen display. In various embodiments, the output elements include thedisplay 905 for showing a graphical user interface (GUI), a visual indicator 920 (e.g., a light emitting diode), and/or an audio transducer 925 (e.g., a speaker). In some aspects, themobile computing device 900 incorporates a vibration transducer for providing the user with tactile feedback. In yet another aspect, themobile computing device 900 incorporates input and/or output ports, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., a HDMI port) for sending signals to or receiving signals from an external device. -
FIG. 10 is a block diagram illustrating the architecture of one aspect of a mobile computing device. That is, themobile computing device 1000 can incorporate a system (e.g., an architecture) 1002 to implement some aspects. In one embodiment, thesystem 1002 is implemented as a “smart phone” capable of running one or more applications (e.g., browser, e-mail, calendaring, contact managers, messaging clients, games, and media clients/players). In some aspects, thesystem 1002 is integrated as a computing device, such as an integrated personal digital assistant (PDA) and wireless phone. - One or
more application programs 1066 may be loaded into thememory 1062 and run on or in association with theoperating system 1064. Examples of the application programs include phone dialer programs, e-mail programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so forth. Thesystem 1002 also includes anon-volatile storage area 1068 within thememory 1062. Thenon-volatile storage area 1068 may be used to store persistent information that should not be lost if thesystem 1002 is powered down. Theapplication programs 1066 may use and store information in thenon-volatile storage area 1068, such as e-mail or other messages used by an e-mail application, and the like. A synchronization application (not shown) also resides on thesystem 1002 and is programmed to interact with a corresponding synchronization application resident on a host computer to keep the information stored in thenon-volatile storage area 1068 synchronized with corresponding information stored at the host computer. As should be appreciated, other applications may be loaded into thememory 1062 and run on themobile computing device 1000, including instructions for providing and operating a conversational bot platform. - The
system 1002 has apower supply 1070, which may be implemented as one or more batteries. Thepower supply 1070 might further include an external power source, such as an AC adapter or a powered docking cradle that supplements or recharges the batteries. - The
system 1002 may also include aradio interface layer 1072 that performs the function of transmitting and receiving radio frequency communications. Theradio interface layer 1072 facilitates wireless connectivity between thesystem 1002 and the “outside world,” via a communications carrier or service provider. Transmissions to and from theradio interface layer 1072 are conducted under control of theoperating system 1064. In other words, communications received by theradio interface layer 1072 may be disseminated to theapplication programs 1066 via theoperating system 1064, and vice versa. - The
visual indicator 920 may be used to provide visual notifications, and/or anaudio interface 1074 may be used for producing audible notifications via theaudio transducer 925. In the illustrated embodiment, thevisual indicator 920 is a light emitting diode (LED) and theaudio transducer 925 is a speaker. These devices may be directly coupled to thepower supply 1070 so that when activated, they remain on for a duration dictated by the notification mechanism even though theprocessor 1060 and other components might shut down for conserving battery power. The LED may be programmed to remain on indefinitely until the user takes action to indicate the powered-on status of the device. Theaudio interface 1074 is used to provide audible signals to and receive audible signals from the user. For example, in addition to being coupled to theaudio transducer 925, theaudio interface 1074 may also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. In accordance with embodiments of the present disclosure, the microphone may also serve as an audio sensor to facilitate control of notifications, as will be described below. Thesystem 1002 may further include avideo interface 1076 that enables an operation of an on-board camera 930 to record still images, video stream, and the like. - A
mobile computing device 1000 implementing thesystem 1002 may have additional features or functionality. For example, themobile computing device 1000 may also include additional data storage devices (removable and/or non-removable) such as, magnetic disks, optical disks, or tape. Such additional storage is illustrated inFIG. 10 by thenon-volatile storage area 1068. - Data/information generated or captured by the
mobile computing device 1000 and stored via thesystem 1002 may be stored locally on themobile computing device 1000, as described above, or the data may be stored on any number of storage media that may be accessed by the device via theradio interface layer 1072 or via a wired connection between themobile computing device 1000 and a separate computing device associated with themobile computing device 1000, for example, a server computer in a distributed computing network, such as the Internet. As should be appreciated such data/information may be accessed via themobile computing device 1000 via theradio interface layer 1072 or via a distributed computing network. Similarly, such data/information may be readily transferred between computing devices for storage and use according to well-known data/information transfer and storage means, including electronic mail and collaborative data/information sharing systems. -
FIG. 11 is a block diagram illustrating physical components (e.g., hardware) of acomputing device 1100 with which aspects of the disclosure may be practiced. The computing device components described below may have computer executable instructions for assisting conversational bot interactions. In a basic configuration, thecomputing device 1100 may include at least oneprocessing unit 1102 and asystem memory 1104. Depending on the configuration and type of computing device, thesystem memory 1104 may comprise, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. Thesystem memory 1104 may include anoperating system 1105 suitable for running one or more conversational bot applications and/or services. Theoperating system 1105, for example, may be suitable for controlling the operation of thecomputing device 1100. Furthermore, embodiments of the disclosure may be practiced in conjunction with a graphics library, other operating systems, or any other application program and is not limited to any particular application or system. This basic configuration is illustrated inFIG. 11 by those components within a dashedline 1108. Thecomputing device 1100 may have additional features or functionality. For example, thecomputing device 1100 may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated inFIG. 11 by aremovable storage device 1109 and anon-removable storage device 1110. - As stated above, a number of program modules and data files may be stored in the
system memory 1104. While executing on theprocessing unit 1102, the program modules 1106 (e.g., conversational bot application 1120) may perform processes including, but not limited to, the aspects, as described herein. According to examples,skill matching engine 1111 may perform one or more operations associated with matching a query and/or command included in a natural language input to one or more skills that may be executed by a conversational bot.Language processing engine 1113 may perform one or more operations associated with applying one or more natural language processing models to a natural language input to identify one or more tasks and associated skill requests therein.Website ranking engine 1115 may perform one or more operations associated with scoring a plurality of websites associated with conversational bots and ranking those bots based on the scores.Skill ranking engine 1117 may perform one or more operations associated with scoring skills associated with conversational bots based on match to natural language input and/or schematized request, popularity of bot, popularity of corresponding website, popularity for specific user, and user preferences, for example, and ranking the bots based on those scores. - Furthermore, embodiments of the disclosure may be practiced in an electrical circuit comprising discrete electronic elements, packaged or integrated electronic chips containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic elements or microprocessors. For example, embodiments of the disclosure may be practiced via a system-on-a-chip (SOC) where each or many of the components illustrated in
FIG. 11 may be integrated onto a single integrated circuit. Such an SOC device may include one or more processing units, graphics units, communications units, system virtualization units and various application functionality all of which are integrated (or “burned”) onto the chip substrate as a single integrated circuit. When operating via an SOC, the functionality, described herein, with respect to the capability of client to switch protocols may be operated via application-specific logic integrated with other components of thecomputing device 1100 on the single integrated circuit (chip). Embodiments of the disclosure may also be practiced using other technologies capable of performing logical operations such as, for example, AND, OR, and NOT, including but not limited to mechanical, optical, fluidic, and quantum technologies. In addition, embodiments of the disclosure may be practiced within a general purpose computer or in any other circuits or systems. - The
computing device 1100 may also have one or more input device(s) 1112 such as a keyboard, a mouse, a pen, a sound or voice input device, a touch or swipe input device, etc. The output device(s) 1114 such as a display, speakers, a printer, etc. may also be included. The aforementioned devices are examples and others may be used. Thecomputing device 1100 may include one ormore communication connections 1116 allowing communications withother computing devices 1150. Examples ofsuitable communication connections 1116 include, but are not limited to, radio frequency (RF) transmitter, receiver, and/or transceiver circuitry; universal serial bus (USB), parallel, and/or serial ports. - The term computer readable media as used herein may include computer storage media. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, or program modules. The
system memory 1104, theremovable storage device 1109, and thenon-removable storage device 1110 are all computer storage media examples (e.g., memory storage). Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article of manufacture which can be used to store information and which can be accessed by thecomputing device 1100. Any such computer storage media may be part of thecomputing device 1100. Computer storage media does not include a carrier wave or other propagated or modulated data signal. - Communication media may be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. The term “modulated data signal” may describe a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.
-
FIG. 12 illustrates one aspect of the architecture of a system for processing data received at a computing system from a remote source, such as a personal/general computer 1204,tablet computing device 1206, ormobile computing device 1208, as described above. Content displayed atserver device 1202 may be stored in different communication channels or other storage types. For example, various documents may be stored using adirectory service 1222, aweb portal 1224, amailbox service 1226, aninstant messaging store 1228, or asocial networking site 1230. Theprogram modules 1106 may be employed by a client that communicates withserver device 1202, and/or theprogram modules 1106 may be employed byserver device 1202. Theserver device 1202 may provide data to and from a client computing device such as a personal/general computer 1204, atablet computing device 1206 and/or a mobile computing device 1208 (e.g., a smart phone) through anetwork 1215. By way of example, the computer systems described herein may be embodied in a personal/general computer 1204, atablet computing device 1206 and/or a mobile computing device 1208 (e.g., a smart phone). Any of these embodiments of the computing devices may obtain content from thestore 1216, in addition to receiving graphical data useable to be either pre-processed at a graphic-originating system, or post-processed at a receiving computing system. - Aspects of the present disclosure, for example, are described above with reference to block diagrams and/or operational illustrations of methods, systems, and computer program products according to aspects of the disclosure. The functions/acts noted in the blocks may occur out of the order as shown in any flowchart. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved.
- The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the disclosure as claimed in any way. The aspects, examples, and details provided in this application are considered sufficient to convey possession and enable others to make and use the best mode of claimed disclosure. The claimed disclosure should not be construed as being limited to any aspect, example, or detail provided in this application. Regardless of whether shown and described in combination or separately, the various features (both structural and methodological) are intended to be selectively included or omitted to produce an embodiment with a particular set of features. Having been provided with the description and illustration of the present disclosure, one skilled in the art may envision variations, modifications, and alternate aspects falling within the spirit of the broader aspects of the general inventive concept embodied in this application that do not depart from the broader scope of the claimed disclosure.
- The various embodiments described above are provided by way of illustration only and should not be construed to limit the claims attached hereto. Those skilled in the art will readily recognize various modifications and changes that may be made without following the example embodiments and applications illustrated and described herein, and without departing from the true spirit and scope of the following claims.
Claims (20)
1. A computer-implemented method, comprising:
receiving, by a first conversational bot, a natural language input in a conversational chat;
identifying a skill for fulfilling one of a query or a command in the natural language input;
determining that the skill is not executable by the first conversational bot and is executable by a second conversational bot;
transmitting a request for the skill from the first conversational bot to the second conversational bot;
receiving, by the first conversational bot from the second conversational bot, a request for information regarding the request for the skill;
transmitting the information from the first conversational bot to the second conversational bot;
providing, by the first conversational bot, control of the conversational chat to the second conversational bot for execution of the skill; and
receiving, by the first conversation bot from the second conversational bot, the control of the conversational chat after execution of the skill.
2. The computer-implemented method of claim 1 , wherein the natural language input is associated with a user account.
3. The computer-implemented method of claim 1 , wherein the natural language input is received via one of a chat interface in a website, a chat interface in an application, a real-time messaging application, a Short Message Service “SMS” message, an email, and an intermediary digital assistant.
4. The computer-implemented method of claim 1 , wherein the skill is associated with a manifest comprising an object that describes how to contact the second conversational bot.
5. The computer-implemented method of claim 1 , wherein determining that the skill is not executable by the first conversational bot comprises:
determining that the first conversational bot does not include a manifest that includes the skill.
6. The computer-implemented method of claim 1 , further comprising:
identifying a plurality of conversational bots that can execute the skill, the plurality of conversational bots comprising the second conversational bot; and
ranking the plurality of conversational bots for executing the skill.
7. The computer-implemented method of claim 6 , wherein the ranking is based on one or more factors including popularity of the respective conversational bot, popularity of a website associated with the respective conversational bot, and user preferences identified based on a user account associated with the natural language input.
8. The computer-implemented method of claim 1 , wherein the request for the skill comprises the natural language input.
9. The computer-implemented method of claim 1 , wherein the request for the skill comprises a token describing a user account associated with the natural language input.
10. The computer-implemented method of claim 1 , wherein the information comprises one of payment information, time information, location information, or a combination thereof.
11. The computer-implemented method of claim 1 , further comprising:
requesting, by the first conversational bot via the conversational chat, the information.
12. The computer-implemented method of claim 1 , further comprising:
obtaining the information from a user store associated with a user account associated with the natural language input.
13. A system, comprising:
one or more processors; and
a memory having stored thereon instructions that, upon execution by the one or more processors, cause the one or more processors to:
receive a natural language input in a conversational chat;
identify a skill for fulfilling one of a query or a command in the natural language input;
determine that the skill is not executable by a first conversational bot and is executable by a second conversational bot;
transmit a request for the skill to the second conversational bot;
receive, from the second conversational bot, a request for information regarding the request for the skill;
transmit the information to the second conversational bot;
provide control of the conversational chat to the second conversational bot for execution of the skill; and
receive the control of the conversational chat from the second conversational bot after execution of the skill.
14. The system of claim 13 , wherein the instructions to determine that the skill is not executable by the first conversational bot comprises further instructions that, upon execution by the one or more processors, cause the one or more processors to:
determine that the first conversational bot does not include a manifest that includes the skill, wherein a manifest comprises an object that associates a conversational bot with a skill and describes how to contact the conversational bot.
15. The system of claim 13 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
identify a plurality of conversational bots that can execute the skill, the plurality of conversational bots comprising the second conversational bot; and
rank the plurality of conversational bots for executing the skill.
16. The system of claim 15 , wherein the instructions to rank the plurality of conversational bots bases the rank on one or more factors including popularity of the respective conversational bot, popularity of a website associated with the respective conversational bot, and user preferences identified based on a user account associated with the natural language input.
17. The system of claim 13 , wherein the request for the skill comprises one or more of the natural language input and a token describing a user account associated with the natural language input.
18. The system of claim 13 , wherein the information comprises one of payment information, time information, location information, or a combination thereof.
19. The system of claim 13 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to: request, via the conversational chat, the information.
20. The system of claim 13 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
obtain the information from a user store associated with a user account associated with the natural language input.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/350,661 US20230359655A1 (en) | 2019-12-17 | 2023-07-11 | Enabling exchange of conversational bot skills during conversational chat |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201962949327P | 2019-12-17 | 2019-12-17 | |
US16/780,696 US11741140B2 (en) | 2019-12-17 | 2020-02-03 | Marketplace for conversational bot skills |
US18/350,661 US20230359655A1 (en) | 2019-12-17 | 2023-07-11 | Enabling exchange of conversational bot skills during conversational chat |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/780,696 Continuation US11741140B2 (en) | 2019-12-17 | 2020-02-03 | Marketplace for conversational bot skills |
Publications (1)
Publication Number | Publication Date |
---|---|
US20230359655A1 true US20230359655A1 (en) | 2023-11-09 |
Family
ID=76318036
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/780,696 Active 2041-08-25 US11741140B2 (en) | 2019-12-17 | 2020-02-03 | Marketplace for conversational bot skills |
US18/350,661 Pending US20230359655A1 (en) | 2019-12-17 | 2023-07-11 | Enabling exchange of conversational bot skills during conversational chat |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/780,696 Active 2041-08-25 US11741140B2 (en) | 2019-12-17 | 2020-02-03 | Marketplace for conversational bot skills |
Country Status (3)
Country | Link |
---|---|
US (2) | US11741140B2 (en) |
EP (1) | EP4078401A1 (en) |
WO (1) | WO2021126426A1 (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11321532B2 (en) | 2019-12-17 | 2022-05-03 | Microsoft Technology Licensing, Llc | Conversational manifests for enabling complex bot communications |
KR20220046964A (en) * | 2020-10-08 | 2022-04-15 | 삼성전자주식회사 | Electronic apparatus for responding to question using multi chat-bot and control method thereof |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7835997B2 (en) | 2007-04-23 | 2010-11-16 | Sudhir Rajkhowa | System for therapy |
US11138388B2 (en) * | 2016-12-22 | 2021-10-05 | Verizon Media Inc. | Method and system for facilitating a user-machine conversation |
US10581765B2 (en) * | 2017-04-03 | 2020-03-03 | Microsoft Technology Licensing, Llc | Conversation bot discovery and response fusion |
US20180332167A1 (en) * | 2017-05-09 | 2018-11-15 | Microsoft Technology Licensing, Llc | Chat bot search results integration |
US11394667B2 (en) * | 2017-10-03 | 2022-07-19 | Rupert Labs, Inc. | Chatbot skills systems and methods |
US10691764B2 (en) * | 2017-10-23 | 2020-06-23 | International Business Machines Corporation | Search engine optimization techniques |
US10749822B2 (en) | 2018-09-20 | 2020-08-18 | The Toronto-Dominion Bank | Chat bot conversation manager |
US11290574B2 (en) * | 2019-05-20 | 2022-03-29 | Citrix Systems, Inc. | Systems and methods for aggregating skills provided by a plurality of digital assistants |
US11651768B2 (en) | 2019-09-16 | 2023-05-16 | Oracle International Corporation | Stop word data augmentation for natural language processing |
US11645458B2 (en) | 2019-10-28 | 2023-05-09 | Paypal, Inc. | Systems and methods for automatically scrubbing sensitive data |
US11159457B2 (en) * | 2019-11-12 | 2021-10-26 | International Business Machines Corporation | Chatbot orchestration |
US11321532B2 (en) | 2019-12-17 | 2022-05-03 | Microsoft Technology Licensing, Llc | Conversational manifests for enabling complex bot communications |
-
2020
- 2020-02-03 US US16/780,696 patent/US11741140B2/en active Active
- 2020-11-13 WO PCT/US2020/060317 patent/WO2021126426A1/en unknown
- 2020-11-13 EP EP20820677.1A patent/EP4078401A1/en active Pending
-
2023
- 2023-07-11 US US18/350,661 patent/US20230359655A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
WO2021126426A1 (en) | 2021-06-24 |
US20210182341A1 (en) | 2021-06-17 |
US11741140B2 (en) | 2023-08-29 |
EP4078401A1 (en) | 2022-10-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11715289B2 (en) | Generating multi-perspective responses by assistant systems | |
Khan et al. | Build better chatbots | |
US11321532B2 (en) | Conversational manifests for enabling complex bot communications | |
US10360300B2 (en) | Multi-turn cross-domain natural language understanding systems, building platforms, and methods | |
US10552218B2 (en) | Dynamic context of tasks | |
EP3991119A1 (en) | Memory grounded conversational reasoning and question answering for assistant systems | |
EP3465469B1 (en) | Intelligent capture, storage, and retrieval of information for task completion | |
US20230359655A1 (en) | Enabling exchange of conversational bot skills during conversational chat | |
US10911389B2 (en) | Rich preview of bundled content | |
US20180196784A1 (en) | Dynamic content generation | |
US11861319B2 (en) | Chatbot conducting a virtual social dialogue | |
CN111247778A (en) | Conversational/multi-turn problem understanding using WEB intelligence | |
US10474439B2 (en) | Systems and methods for building conversational understanding systems | |
WO2021107951A1 (en) | Interfacing with applications via dynamically updating natural language processing | |
WO2020263459A1 (en) | Electronic concierge for automating event planning and application curation | |
WO2023014454A1 (en) | Context-aware observational entity recognition and capture | |
WO2023003675A1 (en) | Enterprise knowledge base system for community mediation | |
US20210397624A1 (en) | Contextual insight system | |
WO2023278124A1 (en) | Classification and augmentation of unstructured data for autofill | |
US20220398247A1 (en) | Natural language triggering for search answer extensibility |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MULLINS, CHRISTOPHER LEE;GELLOCK, SCOTT JEFF;LAIRD-MCCONNELL, THOMAS MATTHEW;AND OTHERS;SIGNING DATES FROM 20200120 TO 20200131;REEL/FRAME:064319/0045 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |