JP2021176092A - Virtual Assistant Domain Selection Analysis - Google Patents

Abstract

To provide a user interface for a virtual assistant platform to configure a domain enable for a virtual assistant by an application developer.SOLUTION: A test query set may be uploaded, and statistical analysis may be displayed for the number of test query provided by each selected domain and the use cost of each domain. Cost may vary according to a complex price setting model. A user interface provides a table, a cost stack chart, and a display view of a histogram to notify a decision to trade off cost and benefit to virtual assistant user experience. A platform interface shows possible responses expected from various domains relative to each query. A platform provider promotes a particular selected domain.SELECTED DRAWING: Figure 1

Description

Background More and more people are using virtual assistants such as SoundHound's Hound, Google Assistant, Amazon Alexa, Baidu Secret, Apple Siri, Line / Neighbor Clover, Microsoft Cortana, KT Gigajiny, SK Telecom NUGU and Open Source Microft. ing. First-generation virtual assistants were only available on corporate branded computers, smartphones or home speaker devices that offered virtual assistant functionality. They weren't configurable and only supported custom features that the company developed in-house.

The current virtual assistant may be text-enabled. They can understand written natural language texts and make written natural language responses. Some embodiments are voice-aware. They may understand the spoken natural language and respond in the form of text, synthetic speech, images, graphic animations, audio and video clips.

Overview Companies that manufacture and sell applications (apps) or devices such as automobiles, robots, shops and equipment, including virtual assistants (“VA”), do not necessarily want to invest in creating VA “from scratch” on their own. I'm not. These companies want the flexibility to incorporate existing natural language features as part of their custom virtual assistants. These companies find it frustrating to be unable to configure virtual assistants, especially to meet their requirements.

A domain is a unit of natural language function. A domain is characterized by a set of sentences that the domain can interpret, and by its ability to respond and act in interpreting the recognized sentence. Examples of domains include weather domains that answer questions about weather, as well as calendar domains that allow users to coordinate and manage appointments, that is, request actions and ask questions.

Some VA development platforms (“platforms”), such as SoundHound's Houndify, are virtual assistant developers (“VA developers”” by providing a menu of domains available on the development platform. ) Is supported. On such platforms, VA developers choose domains to include in their own designed custom virtual assistants, including an array of domains already known to the platform. Some embodiments provide the use of such a platform to configure a virtual assistant. Some embodiments include such a platform. In some embodiments, the platform provides an interface for providing the domain so that the domain provider is integrated with the virtual assistant. The information provided by the domain provider is sufficient to support the functionality of the domain in the virtual assistant that selects the domain. In some embodiments, some domains are free. In some embodiments, the provided domain charges an associated fee for its use. In various embodiments, payments are made to the domain provider by the platform provider or by other parties such as the VA developer, the user or the VA developer through the platform provider. In some embodiments, the platform provider promotes a particular domain more than any other domain.

Some embodiments provide a VA developer to enter test queries for a virtual assistant to see how the virtual assistant behaves in response to these queries. Some embodiments provide to show and compare the behavior of the virtual assistant under different conditions for different domains selected from the domain menu.

Some embodiments provide a VA developer to enter a large test query set. Some embodiments display a view of the analysis of the virtual assistant's behavior in response to a test query set and domain selection. Some embodiments provide analysis of the costs associated with responding to all queries in the set. Some embodiments provide analysis of some of the test queries in a set that can be interpreted by the virtual assistant.

Some embodiments provide a display view for analyzing the success rate and cost of interpreting queries in a set under the conditions of various selections of domains. Some embodiments provide a tabular view of domain results. Some embodiments provide a pricing model for each domain. Some embodiments provide typical queries that are recognized by each domain. Some embodiments provide a test query set and the total cost of responding to queries from the test query set. Some embodiments provide a cost stack view of the per-domain cost of responding to queries in the test set. Some embodiments provide a histogram view of the queries that are responded per domain. Some embodiments indicate that some domains are free and others charge costs.

Some embodiments provide a graphical user interface with pointer icons. In some such embodiments, the corresponding information is displayed in a pop-up message when the pointer is hovering over a particular portion of the display view.

Some embodiments provide sorting or filtering a menu of domains according to useful criteria. Some embodiments provide display of different domains in a menu in different ways such as background color, text color, font and typeface.

Some embodiments provide parameterized pricing models for domains, such as linear, piecewise linear, or expression-based pricing models. Some such embodiments provide tools to help VA developers make cost-effective domain selection.

Some embodiments are computer readable media. Some embodiments are cloud servers. Some embodiments are mobile devices. Some embodiments are systems of devices that communicate with cloud servers. Some embodiments are self-contained devices that implement a natural language virtual assistant configured by a platform.

The information flow from the domain provider to the user via the platform provider and the VA developer in the embodiment is shown. Shows the test query set. A cloud-based system consisting of a domain, a platform, and a virtual assistant in an embodiment is shown. A single device system consisting of a virtual assistant composed of a domain and a platform in the embodiment is shown. In embodiments, it is shown that the user interacts with a virtual assistant that interprets the query and provides a response according to the domain. In an embodiment, it is shown that a VA developer uses a test query to guide the configuration of a natural language virtual assistant through a platform that promotes a domain. In the embodiment, it shows a complete system in which a user queries, a VA developer configures, and a domain provider provides information to the user. In the embodiment, select which domains to enable, indicate how many test query sets each domain will answer, and per domain it takes to respond to queries in the pricing model and test set. Shows the display of the menu of domains that provide to show the cost. A cost stack chart of a set of enabled domains for a test query set in an embodiment is shown. A cost stack chart of a set of enabled domains for a test query set in an embodiment is shown. Shown is a histogram of queries from the test query set responsive to each of the set of enabled domains in the embodiment. In the embodiment, a histogram of query responses per domain, highlighted under the hovering pointer and showing domain-specific information, is shown. In the embodiment, the display of the menu of the domain in which the promoted domain is prominently shown is shown. Enter a test query to show the domains that can respond to this query and the display to see the filtered menu of those responses. Shown is a linear pricing model. A piecewise linear pricing model is shown. An expression-based pricing model is shown. Indicates a rotary, non-temporary computer-readable medium. Indicates a packaged solid-state non-transitory computer-readable medium. Indicates a server rack. The block diagram of the server is shown. Indicates a packaged system-on-chip. A block diagram of the system on chip is shown.

Detailed Description Various embodiments of the invention that describe the various features are described below. In general, in embodiments, the described features can be used in any combination.

Many companies manufacture or sell devices or apps that can be enhanced by integrating virtual assistants. Many virtual assistants are voice-based. Examples of voice-enabled devices are cars, robots, shops, devices and smartphones. All of them can have an integrated virtual assistant. The user communicates with the virtual assistant by issuing a natural language query, which is usually either a request for information or a request for action. The query may be a spoken query or a written query. Responses from the virtual assistant may include voice, text, graphics, audio or video recording, and action.

Developing intelligent multifunctional virtual assistants is complex and requires expertise and investment for each domain that the virtual assistant can handle, in addition to the general architecture. When developing a virtual assistant, it makes sense for developers to purchase or in-license domain software from the virtual assistant development platform provider. This allows platform providers to hire enough professionals to build a good platform and allows various VA developers to share costs.

In such an ecosystem, the logic or software to implement the virtual assistant is stored on a computer-readable medium within a network-attached server operated by the platform provider or within a device that does not need to be connected to the network. , Device logic or software is configured using the platform. In a typical networked embodiment, a client app or remote device uses an application programming interface (“API”) to access a server over the Internet. Some single-device virtual assistants work autonomously, others only with network access, and the combination of local and remote virtual assistant features creates an intermittent network. Some work with access. Whether connected to a network or not, the virtual assistant receives a natural language query from the user, interprets the query, and provides the user with a response. Zero or more domains may recognize each query. If there is ambiguity, the virtual assistant selects one of the competing domains and determines the response.

A virtual assistant developer is someone (mostly a trained technician) who works for a company that integrates virtual assistants into specific devices and apps. VA developers use the platform to configure virtual assistants to specific requirements. Virtual assistants generally support multiple domains. Some domains provide information such as weather, news, trivia, restaurant searches, simple calculations and Wikipedia searches. Some domains provide services such as thermostat control or lighting, music streaming, SMS text message sending, and calendar appointment management. Some domains may be as simple as telling time. Some domains can be complex, such as providing travel agency services, and others can be very useful to the user, such as the ability to locate a child. Many domains access Web APIs to access specific or dynamic information or to produce the requested behavior. Some domains provide information and services from third-party providers in exchange for payment. For example, some weather domains access the National Weather Service API to obtain weather forecast information.

The platform provides VA developers with a way to select a domain to include in the virtual assistant, among other features. Generally, the domain is provided by the domain provider entity. Some platforms support thousands of third-party domain providers. Some domain providers are small businesses or even individual programmers.

In some embodiments, the query set recognized by the domain is defined by the semantic grammar code. Such grammar code is specifically created for each domain by a domain developer (mostly a trained technician) who creates rules for interpreting a query as a specific information request or a specific action request. Will be done. In such an embodiment, the natural language interpretation is based on the domain grammar code. Domain grammars include syntax rules used by natural language processing systems to parse queries. In domain grammar, syntactic rules are extended by semantic extensions. Extension is a function that constructs a component interpretation from the interpretation of zero or more subcomponents, and the subcomponents are judged by parsing. In some embodiments, a domain grammar is used to recognize and interpret queries against the domain. In some embodiments, at least some of the interpreters include neural networks that are trained using machine learning. In any case, the interpretation of the query assigns values to semantic slots based on the information from the query. In some embodiments, an intent slot that represents the user's overall intent to the query, such as meeting scheduling or flight booking, is encoded.

In an exemplary embodiment, the semantic grammar for requesting a weather forecast may have (1) a component whose syntactic component is named <location>, and the corresponding semantic slot is named "location". , Its value must be considered a place in any example, (2) the syntax component may have another component named <time>, and the corresponding semantic slot is named "time". And its value must be considered as time. When the user asks, "What kind of weather is Timbuktu tomorrow?", This embodiment uses a weather domain syntax with <location> and <time> components and values the intended slot "weather_information". You can fill it with, fill the location slot with the value "Timbukto", and fill the time slot with the value "Tomorrow" or perhaps the corresponding calendar date.

Similarly, when the user says, "Where is the nearest French restaurant with at least 4 stars?", The system can use a restaurant domain grammar with syntax rules for <cooking_type> and <rating> ingredients. The corresponding slots for the dish type and rating will be filled with values that convey "French cuisine" and "at least 4 stars" respectively. In another example, when the user says "text: love mom", the system recognizes the query with the SMS texting domain grammar and fills the intended slot with "send_text" to correspond to "mama". Refer to the contacts in the user's contact list to fill the recipient slot and fill the message slot with "I love you". The interpretation of the query will be formed in these slots, and the completion of the interpretation of the query specified by intent will be to send the text content of the message to the recipient's mobile phone. When the user says, "How many claws does a cat have?", The system recognizes the knowledge domain query, fills the intent slot with "Wikipedia search" (says), and fills the type slot with "cat". Fill, fill the attribute slot with "claw", and fill the question slot with "how many". Other natural language approaches are also possible.

Some queries can be interpreted by multiple domains. It is common to have such duplicate syntax when different domain providers compete to provide the same domain functionality. For example, the platform may provide four competing domains for booking hotels, all of which can interpret the query "Tell me a hotel in Paris." To help VA developers deal with this situation, the platform can configure the virtual assistant to include certain domains in its functionality and not other domains, or other domains. You can configure the virtual assistant to select a specific domain in preference to. Such choices can be made in a "hard" or "soft" way.

Hard choices are made during configuration. Some domains are enabled in the virtual assistant and all other domains are disabled. This configuration step allows the VA developer to control which domains are involved in query interpretation at runtime. During production, query interpretation considers only enabled domains.

Soft choices are made at runtime. Selection between query interpretations of competing domains (all of which are enabled) is generally made by selecting the highest scoring interpretation based on the scoring scheme. Scoring techniques can be used to prioritize domains. That is, domain A can always take precedence over B when the enabled domains A and B conflict and interpret the query. Scoring can also address the issue of ambiguity between unrelated domains, but such semantic conflicts are less frequent. For example, the query "How high is Denver?" Can be interpreted by a geographic information domain that knows the altitude of a city, but also by a weather domain where "height" refers to temperature. be able to. The second interpretation, which may be unlikely in the absence of context, is "What is the weather like in Chicago?" [Answer: 56 degrees] How high is Denver? This is a fairly possible interpretation in the context of weather and temperature in conversations such as. Parsing and interpretation scores may be context sensitive and scoring can take into account all of these and domain priorities.

For the selection of valid configurations, the platform can provide empirical tools intended to help VA developers make wise decisions. In some embodiments, the cost effectiveness of a domain is (1) coverage, i.e. which part of the query is recognized by the domain, (2) accuracy, i.e. which part of the recognized query is recognized by the domain. It can be correctly interpreted and (3) measured against the test set of queries in terms of cost considerations.

In terms of cost, according to some embodiments, the platform is aware of the pricing model associated with each domain and can provide pricing information to the VA developer. Many platforms will offer a specific domain for free, such as a domain to inform time or a domain to do simple calculations. However, many domain providers want to offer high-value domains and be compensated for their use of their data and services. For example, a domain for trading stocks may charge a fee for each stock trading requested by the user. In some virtual assistants, the user pays the VA developer to use the domain, and the VA developer pays the domain provider directly or through the platform. In some cases, the platform may increase the cost of the domain to recover the added value provided by the platform.

In practice, the platform or VA developer may discount or subsidize a particular domain to attract more customers. In pricing models, the price per query is often subject to quantity discounts. All such considerations can be part of the domain's pricing model.

Coverage and accuracy are measured based on the test query set that the platform receives from VA developers. A "hit" is registered when the enabled domain recognizes the query. The test query is "covered" if it hits at least one enabled domain. Coverage is measured (absolutely) by the number of test queries covered, or (relatively) by the corresponding part of the test query. Accuracy is measured (absolutely) by the number of test queries that are processed correctly, or (relatively) by the corresponding part of the test query that is covered. Accuracy can be measured if the test set contains query interpretation data or if the test set contains query data.

In some embodiments, testing accuracy requires the interpretation of the test query that would be made to the user query during production, but it is necessary to execute (execute) the test query. do not. Accuracy is based on the equivalence (or matching integrity) of the interpretation (which is an internal representation of the meaning of the query). In some embodiments, testing accuracy requires both interpretation and execution of the test query. And the accuracy is based on the equivalence (or matching consistency) of the response.

In some embodiments, the VA developer can enter the test query as if it were received from the user, and the platform can: (1) which domain can recognize the query. One or more of (2) interpretation of the query, (3) information required to perform the interpretation and respond to the query, and (4) cost of providing the response can be shown to the VA developer. ..

In some embodiments, the VA developer enters a large test query set and the platform provides statistical information about the domains that this query set will hit and the cumulative cost of the test set. In some embodiments, the query may hit a large number of domains. In other embodiments, the system ensures that there are no more than one hit per query. One type of statistic is a histogram of hits per domain across the query set. Another type of statistic is the domain name, whether each domain is enabled for the virtual assistant, the pricing model for accessing each domain (such as the cost per simple query), and for each domain. The number of queries in the test set to hit, the costs associated with enabling the domain in the virtual assistant, and some of the queries in the test set that will hit at least one of the enabled domains. , And the total cost of processing all queries in the test set using the enabled domain, a table with one or more of them.

It is possible for some queries to require information from more than one domain. For example, the query "What's the weather at home for the team that won the last Super Bowl match?" Is a virtual assistant using the sports information domain to see which team won the last Super Bowl match and that. It will be necessary to locate the home and then use the city information to look up the weather information using the weather domain. Therefore, the number of domains hit by the virtual assistant responding to the test query set will be greater than the number of queries, and the cost of the query will be higher than if each test query hits only one domain.

Some queries are not recognized by any domain. In some embodiments, such non-functional queries are responded with default information such as web search results. As a result, the number of hits will also be less than the number of queries.

In general, the more domains a virtual assistant has, the more queries the virtual assistant can respond to with satisfaction. The more satisfied the users are with the virtual assistant, the more users will use the virtual assistant. The more users who use the virtual assistant, the more incentives domain providers have to create and improve their domains. This creates a virtuous circle of ever-increasing domains and virtual assistants that benefits users and benefits all participating companies.

The following is a description of certain typical embodiments shown in the drawings.
Participants Figure 1 shows a diagram of the virtual assistant ecosystem and its operation. Various domain providers provide information and services to Platform 12. Specifically, the weather information domain provider provides the weather domain 11a to platform 12, the map information domain provider provides the map domain 11b to platform 12, and the text messaging domain provider provides the text messaging domain 11c to platform 12. To provide. In addition to the domain grammar, the information provided to the platform may include pricing models and test data.

The VA developer uses the platform 12 to create virtual assistants for devices 13a, mobile phones 13b, automobiles 13c, etc., all of which deliver VA functionality to multiple users 14. Some users use more than one type of virtual assistant. Information and services generally flow from the domain, through the platform, through the VA, from left to right, and reach the user.

Generally, payments flow in the opposite direction. In various embodiments, the user pays directly to the VA developer, platform provider or domain provider. In various embodiments, the VA developer pays directly to the platform or domain provider. In some embodiments, the platform provider pays the domain provider. In some embodiments, the platform provider offers virtual assistant developers discounts or access to some domains that are losing money in order to win the business or increase total usage. offer. In some embodiments, the domain provider pays or offers a discount to the platform provider in exchange for the platform provider to promote the domain. The techniques disclosed herein do not limit the business relationships that can be made using this framework.

In some embodiments, the enterprise is both a domain provider and a virtual assistant developer. For example, a car virtual assistant needs its own local domain to answer state queries about the car, such as fuel level, or to request actions such as turning on the headlights. .. In addition, in some cases, an external domain such as a domain having weather information is included. In another example, a video player set-top box sold by a particular video content provider needs to be able to access the domain for retrieving that video content as part of its custom virtual assistant. This domain may be local or external. In some embodiments, some domains are dedicated to some virtual assistants and not available to other virtual assistants.

FIG. 2 shows an example of a query set specific to a general-purpose virtual assistant. Queries for weather are the most frequent, but other common queries are local business searches, map directions and traffic conditions, queries for news or sports, queries for trivia information, and various other types. Is a query. Since each query may appear multiple times in the test set, the alternative expression known as a bag is to associate each unique query with the number, regardless of the order. For example, there are three queries in this set that say "how is the weather?" Taking this a step further, each query can be given a real-valued weight (rather than just a frequency) that indicates to the experimenter its importance to other queries. Such weights can be normalized so that the sum is 1.

Clouds and Devices FIG. 3 shows a user 31 communicating with a first app 32 and a second app 33. These apps communicate with the platform 35 over the network 34. The app developer 36 has previously used platform 35 to configure a first virtual assistant for the first app 32 and a second virtual assistant for the second app 33. In some embodiments, different developers configure each app.

On platform 35, multiple domains are available for configuration selection. These plurality of available domains may be in the hundreds, thousands, or even more. The domain is provided by the domain provider. They are made available on the platform via domain registration or ingestion process. During registration, the information provided about the domain includes a unique ID, simple name, description, examples of recognized queries, and a domain in an executable or interpretable form (source code, object code, neural network or semantic grammar, etc.) ) May be included. In the present disclosure, the term "grammar" or "semantic grammar" generally refers to a description of code such as a source, object, neural network or arbitrary data structure instance that can be executed to parse or recognize a query.

The virtual assistant configuration created by the developer 36 includes a selection of the first domain 37 and the second domain 38. When the user 31 queries any of the apps, the user sends the query to the platform 35, which interprets the query and issues a web API from the first domain 37 or the second domain 38. Use to get the right information.

FIG. 4 shows a user 41 communicating with an app 42 integrated within a device 44 that operates independently of the platform 45. This figure is not drawn according to a certain ratio. The app developer 46 uses platform 45 in advance to configure a virtual assistant for the app. An app integrator is a company that designs, manufactures, or sells products and services that integrate virtual assistant apps internally. The platform provided the software needed by the app integrator to integrate the configured virtual assistant into the app 42.

When the app 42 receives a query from the user 41, it interprets them using an integrated interpreter and requests information from the local information domain 47 accordingly, or requests a service action from the local service domain 48. .. For example, in automotive devices, the information domain 47 has a grammar and provides information for queries such as "how much battery power is left" and "which radio station is this?" Let's go. In automobiles, service domain 48 will have a grammar and will provide services such as "open the sunroof" and "turn up the radio volume". These exemplary information sources and services are car-local and do not require access to the network.

Some embodiments have configurations that allow a combination of domains that require network access and domains that do not require network access. In such an embodiment, the domain responds successfully if network access is available, and the domain responds unsuccessfully if network access is not available.

Information Flow and Configuration Figure 5 shows a diagram of the data flow through the platform at runtime according to the embodiment. User 51 issues a natural language query that the platform processes. The interpreter 52 interprets this query and provides a response to the user 51. Depending on the query, the response may be a simple indication that the interpreter 52 cannot interpret the query. In some embodiments, the interpreter 52 alone can provide a response to a query that it interprets successfully.

Domain providers provide data from their own servers such as music or other sources such as weather sensors, or provide services such as sending messages or controlling vehicles. Generally, such data or services are known as domain 53b. To know when the user intends to call the domain, the user can have the corresponding syntax code 53a. The grammar code contains a phrase that calls the domain and a slot that can be filled with words from the user expression. For example, a grammar code with the phrase "how is the weather?" Calls the weather domain, and the domain code contains slots for where and when to get the weather forecast. The code for the weather domain phrases and slots recognizes that the phrase "what weather is Timbuktu tomorrow?" Should make a request to the domain provider for tomorrow's weather in Timbuktu. Allows you to. The code associated with the domain is, in many embodiments, registered, ingested, and stored by the platform.

If the interpreter 52 interprets the query according to the syntax code 53a associated with the domain 53b and determines that the query requests information from the domain 53b, the interpreter 52 is suitable provided by one of a plurality of domain providers. Request information from the domain 53b. The interpreter 52 uses this information to form a response to the user 51.

FIG. 6 shows a diagram of data flow via the platform at the time of configuration according to the embodiment. Among many app integrators, developer 66 provides one or more test query inputs 64 to the platform. Developer 66 also provides a selection of one or more domains 65. The interpreter 62 interprets the test query according to the syntax code 63a of the selected domain 63b to determine which of the selected domains 63b can interpret the query. The display shows the result from the interpreter 62 to the developer 66 (67). Many display views are suitable and useful for various embodiments. Some display views are illustrated and described below.

The embodiment of FIG. 6 further comprises promoting one or more domains based on the results of interpreter 62 (68). For example, if the test query can be successfully interpreted according to the promotion domain, the platform will show developer 66 the results that the interpreter would show to the user if the promotion domain was enabled. In some embodiments, the name of the promotion domain is indicated even if they are not enabled. In some embodiments, as an alternative to the response that would be generated by the information from the highest ranked alternative domain, the response that would be generated by the information from the promotion domain is shown.

For example, a weather domain provider with more accurate and detailed information may charge a higher price and offer itself as a promotional domain. The default weather domain can generally answer weather queries. The platform according to the embodiment will respond according to the information from the default weather domain and will indicate the response that will be generated by the information from the promotional weather domain. This can be done in combination with candidates for selecting a promotional domain and means such as button icons for selecting a promotional domain to include in the virtual assistant.

FIG. 7 shows a diagram of data flow through the platform. Developer 76 provides one or more test query inputs 74 to the platform. Developer 76 also provides a selection of one or more domains 75. The interpreter 72 interprets the test query according to the syntax code 73a of the selected domain 73b to determine which of the selected domains can interpret the query. The display shows the result of the interpreter 72 to the developer 76 (77). The embodiment of FIG. 7 further comprises promoting one or more domains 78 based on the results of the interpreter 72. After the configuration time, during runtime, user 71 issues a query interpreted by interpreter 72 to provide a response to user 71.

In the embodiment of FIG. 7, the platform calculates a charge 79 in response to receiving information from domain 73. This fee is calculated according to the pricing model. The pricing model is a domain usage-based feature. Some embodiments have a simple pricing model, such as a single charge for each piece of information access. Such charges are usually a fraction of US $ 1. Some embodiments support a more complex pricing model that is a feature based on the number of previous information accesses or the type of information accessed within a time window. Equally, some domains charge for the provision of services when others provide information.

In the embodiment of FIG. 7, the platform provides information to the virtual assistant to pay for each domain provider that constitutes a response to the user. This provides an incentive for the domain provider to want the developer to choose a domain to include in the virtual assistant, and thus an incentive for the platform provider to promote the domain of the domain provider with which the platform provider has a close business relationship.

Domain Table FIG. 8 shows a display view of the domain table 80 according to the embodiment. Table 80 has one row per domain. The Enabled column indicates the domain enabled status. In some embodiments, the cells in this column can be used to enable or disable the corresponding domain. Various inputs for selecting or deselecting a domain, such as a mouse for clicking, a keyboard with arrow or shortcut keys, a touch screen, or other suitable means for selecting objects in the user interface. Means are possible.

Many different graphic layouts are possible. In general, visually organize a large number of domains into rows and a relatively small number of columns, the name, whether the domain is enabled, the number or percentage of queries that the domain can interpret (from the test set), and the price. Grids are useful for displaying relevant information about a domain, such as a configuration model (or cost per query in the simplest case). Visual objects that show relevant information, such as rectangles that show that domains are aligned with respect to the grid, also work. It is also possible to arbitrarily drag and drop objects on the desktop of a personal computer.

Table 80 has a display view to show the VA developer which domain is selected to be included in the virtual assistant from the domain menu. The display view 80 has five columns. The heading row gives each column a title depending on its intended use. These columns indicate the name of each domain listed from the menu of available domains, whether the domain is enabled for the virtual assistant, and the domain can interpret each query in the test set. Counting the number of queries, which is the domain that the interpreter will choose among all enabled domains, pricing model for responding to queries,
And the total cost for the domain to provide the information needed for the virtual assistant to provide a response when the domain is selected to respond to each query in the test set.

Following the heading line is a line for each domain in the menu. Table 80 contains 7 domains in the menu. Some embodiments support a much larger number of domains, a variable number of domains, and domains that change dynamically as new and existing domain providers create or remove offerings. The first column shows a text string with the name of each domain in the menu. The domains shown are weather, food, sports, maps, information, timers and calculations.

The second column has cells for each domain and is checked when the domain is selected to be included in the virtual assistant. FIG. 8 shows a pointer that can be controlled by a mouse or finger on the touch screen. Clicking or double-clicking the mouse button, tapping or double-tapping, or long-pressing the finger changes the state of whether or not the domain at the pointer position is selected. This state is indicated by the presence or absence of a check mark in the cell.

The third column shows the cells for each domain, which contains a count of the number of queries that are responded to by information from the domains given the selected domain. In some embodiments, different columns are used for the number of queries recognized and the number of queries answered correctly if the exact query response is known. In some embodiments, percentages are displayed in addition to absolute numbers using alternatives or even other columns. In addition, some queries are ambiguous with respect to the selected domain set, i.e. can be interpreted by more than one domain. This is most often seen in short queries. For example, the query "turkey" can be interpreted by the weather domain, food domain, sports domain, map domain, and information domain. Some embodiments handle ambiguous queries by providing a labeled response to all ambiguous interpretations of the query.

In another embodiment, domain priority is used to select the highest priority domain that matches the query to more force disambiguation. In another variant, the query interpretation score is used and the domain with the highest scoring interpretation score from the corresponding grammar wins and forms the basis of the response. You can combine the two ideas. In all cases, ambiguity is removed by more compulsory choice of one or a few interpretations, which contributes to the number of single domains. As the state of domain selection changes, this can change the number of queries that other domains will respond to. In some embodiments, the query count is dynamically recalculated each time the domain selection is modified. Alternatively or in addition to the query count column in Figure 8, in some embodiments, the total number of test queries that each domain can provide information on is also a test query that can be answered accurately with the domain enabled. It also shows the increment of the number.

The fourth column shows a pricing model for explaining domain queries. Table 80 shows a simple kind of pricing model based on a fixed price per query. Prices shown range from 0 (free domain) to 5 ¢ ($ .05 US $) per query. Some embodiments support more complex pricing models, such as piecewise linear or expression-based models depending on the number of hits in a time window. A cyclic time window may be used as well as a sliding time window (eg, the last 30 days) (eg, a calendar month). Embodiments that support more complex pricing models do not have to show the pricing model itself in the pricing model column, instead by clicking or tapping the domain's pricing model cell to price the domain. You may be given access to a different display view, such as a pop-up window that shows the configuration model and allows you to edit it.

The fifth column shows, for each domain, the amount that will be paid to the corresponding domain provider based on the response to every query in the query's test set. You will be charged for any domain whose information is used in response. In the simple pricing model shown in Table 80, the domain cost for each domain is the number of queries times the cost per query. In embodiments that allow compound queries, a single query may hit multiple domains. For example, "How old is the President of the United States?" May hit both the domain that provides political information and the domain that provides personal information. In embodiments that allow ambiguity in the response, a single query may hit multiple domains for different reasons. Multiple answers are given according to conflicting interpretations. For the same reason that the number of queries can change each time a domain is selected or deselected, the domain cost can change accordingly.

The bottom complete row in Table 80 represents the number of queries that do not match any domain. Such queries can be counted, but the other columns are not.

In the line below it, two quantities are listed. The first quantity indicates the percentage of queries that receive a response in the current domain selection. In Table 80, the food domain and the map domain are not enabled. They would each respond to 1955 and 764 queries, respectively. They result in 633 queries that cannot be interpreted by any domain, and 73% of the test queries will receive a response successfully.

The last quantity cell shows the total domain cost for the enabled domain. This number changes dynamically as the VA developer enables or disables the domain. In general, the more domains a VA developer enables, the more successful the query will be and the higher the total cost will be. In general, the higher the query success rate, the more satisfying the user experience, so these dynamically informative display views give VA developers information between user satisfaction and domain usage budgets. You can compromise on the basis.

In all of the above methods, a test query or a group of test queries can be given duplication. Returning to Figure 2, which shows the test query set, note that, for example, the query "How is the weather?" Is made three times. In a production environment, VA developers will have access to field statistics across much larger datasets, with hundreds of months for "how is the weather?" Having multiple queries for the same domain provides a non-linear quantity discount for reasonable business arrangements. This is especially true if the platform caches results for data that is static or changes infrequently. Alternatively, in the absence of full-fledged statistics from the field, the small test set is curated (1) statistics from a small sample of queries, or (2) perhaps partly derived from data from the field. It may be based on the query set. In any case, the number of test sets can be multiplied by a variable element that can be modified by the VA developer to observe the effect of the non-linear quantity discount.

Some embodiments, especially those with a large number of domains, provide scroll bars that allow VA developers to view different domains within a limited viewing space. Some embodiments allow VA developers to sort the list of domains according to one or any number of column criteria. One way to do this is by receiving clicks or taps on the column headings and sorting the list. In addition, the list is sorted in reverse order by clicking on a column where the list is already sorted. Some embodiments provide a box for the VA developer to enter filtering criteria such as whether each domain is enabled, filtering the list of domains by the range of the pricing model and the range of domain costs. do.

Every virtual assistant is a different project for VA developers. In some embodiments, each project will be opened and viewed separately and will display a domain table, especially for certain virtual assistants. Some embodiments of the platform allow VA developers to create accounts and log in to configure their virtual assistant projects. Platform administrators can control what domains different VA developers can see, and what controls and tools give VA developers access to work on projects depending on the terms of their account. Can be controlled.

Cost Stack Chart FIG. 9 shows a display view of the cost stack chart 90 according to the embodiment. This chart has the title "Cumulative amount invested per domain". In this chart, the vertical axis is the cost and the horizontal axis is the domain. In some embodiments, the vertical axis indicates the domain and the horizontal axis indicates the cost. The former is generally preferred for virtual assistants with a small number of domains accounting for most of the cost, while the latter is generally preferred for virtual assistants with a large number of domains.

There are many useful ways to present visual data. For example, Edward Tufte's books, such as information envisioning and a visual display of quantitative information, provide information on this subject. Black-and-white line drawings are not ideal, but may be sufficient for explanation. In the embodiment shown in the black-and-white line drawing of FIG. 9, seven domains are drawn on the horizontal axis. To improve readability without duplicating words, domain names are right-aligned at an angle to the axis. The name of the enabled domain is shown in bold font, and the name of the disabled domain is shown between parentheses and written in a smaller font. Free domains are indicated with a "(f)" after the domain name. Domains that do not require access to external dynamic information are generally free. This is because platform providers can cache information and retrieve it at low cost.

In each domain column, a bar with a height proportional to the domain cost to respond to the test query set indicates the domain's contribution to the total cost of responding to all queries in this set. The bottom of the bar for each domain is located at the top height of the bar for the previous domain. Domains with no cost have zero-height bars and are displayed as horizontal lines. The vertical axis is represented by the cumulative cost value, and the horizontal dashed line at the top height of each bar extends from the vertical axis to the bar at the display height. The platform reorders domains by allowing VA developers to automatically sort available domains by various criteria such as alphabetical order of names, number of matching queries, cost per query, or total cost. Make it easy to do. The platform interface also allows the user to freeze the order from a particular sort while the selection changes and otherwise the order will change. You can also sort the domains manually. Also, some embodiments provide filtering the list of domains by various criteria, such as showing only enabled domains or selected disabled domains.

The display view of FIG. 9 also includes a pointer that toggles the state of whether the domain is enabled or not when clicked or tapped or otherwise properly launched. Therefore, VA developers can quickly and easily enable and disable domains to see their effect on the total cost of responding to queries in the test set. FIG. 10 shows a display view of the same virtual assistant cost stack chart 100 as that of FIG. 9 after the disabled food domain is enabled by clicking ((Food)) with the pointer. ..

In FIG. 10, domains are sorted by the number of queries that hit the domain. The food domain contributes more to the total cost of responding to the test set than the weather domain (because it has a higher cost pricing model), but it is shown second in this list. ing. This is because the test set has fewer food queries than the weather queries.

In some embodiments, the cost axis is displayed on a linear scale, and in some embodiments it is shown logarithmically. In some embodiments, an axis showing the cumulative number of queries (or test set percentages or percentages) responded by the enabled domain is shown on the right side of the chart. Such an axis will have an irregular scale whenever the cost scale is regular (linear or logarithmic). Alternatively, the regular (linear or logarithmic) representation of the replied query can have corresponding parallel cost axes with irregular intervals.

Query Hit Histogram FIG. 11 shows a display view having a histogram 110 of query hits per domain according to the embodiment. This histogram has the title "Histogram of queries that hit the domain", the vertical axis shows the hits, and the horizontal axis shows the domain. A domain hit refers to a query that is answered by information from the domain. In the histogram 110 of FIG. 11, the domain axis is not represented by the name of the domain. Because there are too many domains that can fit the name. Alternatively, the vertical axis may indicate the domain and the horizontal axis may indicate the hit. If the histogram chart is long enough or allows the VA developer to scroll through the inputs, it is possible to list the names of the domains and display the number of hits.

Histogram 110 has smoothed lines, but a staircase representation across domains will also work. Histogram 110 does not represent the horizontal axis by the name of the domain, but shows a "$" symbol to indicate a domain that has a pricing model that exceeds the threshold. In some embodiments, the threshold is zero (to indicate free and paid domains). In some embodiments, the VA developer sets a threshold to see affordable domains vs. expensive domains.

In some cases, even a domain with a small number of hits can bring disproportionate benefits to the user experience. For example, the domains available in the Automotive Virtual Assistant, which can locate lost phones, are rarely used, but will be greatly appreciated by users when rarely needed. That would be a strong selling point in automobiles. Even if access to that domain information would be very expensive to use for a VA developer, it would be well worth the cost to sell more cars.

Some embodiments allow VA developers to tag specific domains, such as with a star, as being of particular value to the user experience. In some embodiments, the histogram, as seen in FIG. 11, is shown so that the high value domains are highlighted with prominent colors or icons.

Since many queries can be interpreted by more than one domain, enabling or disabling a single domain changes the number of queries that are answered by information from all other domains. obtain. This can have the effect of reordering the domains in the histogram. For example, two competing weather domains are enabled, the first domain can provide all weather information, including barometric pressure, and the second domain is the default source of information for all weather questions, but barometric pressure. If no information can be provided, with both domains enabled, the first domain has a very small number of hits (shown near the right side of the histogram 110), but the second domain is disabled. In such a state, the first domain will have a very large number of hits.

FIG. 12 is a histogram 110 as seen in FIG. 11, but shows a display view of the histogram 110 with pointers. If you stay on top of the domain column in the histogram for 1 second, the platform enables the domain name, the number of hits it has for the test query set, and the domain when responding to the test set query. Pops up an information box showing the total cost for.

Domain Promotion FIG. 13 shows a display view having a domain table 130 according to an embodiment. Like the domain table 80 in FIG. 8, the domain table 130 in FIG. 13 lists the domains. The difference is that the domain table 130 shows two map domains, one named "map" and the other named "map PRO". The latter is a promoted domain. It appears to provide a better user experience than map domains, at least because it can serve 805 queries instead of 764 queries in the test set. The platform shows the map PRO domain and its line text in bold font, with emphasis around its enable check box. As a result, the promoted domain will be prominently displayed to the VA developer for consideration.

Although the embodiment of FIG. 13 uses bold fonts and highlighted boxes, there are many possible ways to highlight the promoted domain. For example, one or more promoted domains may be listed separately, sorted to be at the top of the list, shown using different colors or shading, and stars. It may be indicated by an icon such as, may be indicated with an image such as a company logo, or may be animated. Some of the means of emphasizing these domains may be used in combination.

FIG. 14 shows a display view 140 for inputting a specific test query and viewing the result according to the embodiment. The display view 140 includes a text input box 141. Virtual assistant developers can enter text using a physical or virtual keyboard or other suitable text input box. The entered text is displayed at the position of the cursor position indicator 142. On user interface devices with a microphone, the VA developer can tap or click the microphone button 143 to say a test query. Guidance items are displayed on the display when first called to inform the VA developer that it is possible to "speak out the query".

The display view 140 also includes an upload button 144. When launched, it calls a dialog box for the user to browse the file and choose to upload it to the platform. The file may contain a single query or may contain a delimited list of any number of queries with a complete test set. In some embodiments, the VA developer drags and drops a file from the graphical operating system or browser display into the query box and causes it to be automatically uploaded to the platform. Various implementations of upload buttons and drag-and-drop uploads are well known and readily available in browser client-side script templates for languages such as JavaScript®.

The display view 140 further includes a result box 145. For a single input query, the result box displays a list of domains that can provide the information needed to respond to the query, which is the name of the domain, the pricing model ( It has a column of responses (denoted as "cost"), and which the virtual assistant will provide to the user for that query with the domain enabled. In some embodiments, multiple domain providers can provide source information and syntax to form very different responses to queries. The result box 145 shows the virtual assistant response using the grammar and information from each of the four travel domain providers. The "Trip Booker" domain is relatively low cost (only 1 ¢ per query), but it's a fairly private response in that it recommends exactly one hotel brand by citing its own name "Trip Booker". Is. As you can see, the Trip Booker domain provider and this hotel brand have a lucrative business relationship. The "Travel Mate" domain is moderately costly (3 ¢ per query), but provides a fairly useful response showing the large number of results available and a valid list of the top five most likely to be of interest. .. The "Travel Hound" domain is expensive (8 ¢ per query), but users have found a certain number of hotels, some lists, and users by sorting or filtering the lists using an intuitive voice interface. Provides much more useful results with recommendations for getting far more satisfactory results. The "Chee-po-tels" domain is cheaper (only 1 ¢ per query), but instead of recognizing that the word Paris refers to the big cities of France, the hotel Imagine a ridiculously small town in Idaho, USA.

Regardless of the sort order of domains (pricing model, number of query hits, star selection by VA developers, query response type, etc.), the platform can show results in two sections. The first section, indicated by the subheading 146, entitled "Editor's Pick-Sponsor", indicates one or more domains that the platform provider wants the VA developer to choose. In general, an "editor's pick" domain is either sponsored by a domain provider or selected by a platform provider. Because they encourage more use or satisfactory results with virtual assistants. The rest of the domain that can provide information and complete the response to the test query is shown in the section indicated by the subheading 147 named "Other".

Pricing Model The example used above shows a very simple pricing model with a linear rate of US cents or a few US cents per information request. In some embodiments, the cost per request for information that is cheap to procure and deliver will be much lower than 1 US cent per request. In FIG. 15A, the price is $ per request. An example of a linear pricing model of 0005 (equal to .05 ¢ per request) is shown.

It is common in the industry to use a pricing model in which the domain is non-linear with respect to the number of queries with the highest interpretation score. An example of a non-linear type pricing model is a gradual pricing model. FIG. 15B shows such a pricing model. The cost per request decreases as the number of requests crosses the threshold. Specifically, the cost is $ 1. per query for the first 10,000 queries within a month. 0010. If the number of requests is 10,001 to 50,000, the cost is $ 1 per request. 0005. If the number of requests is 50,001 to 250,000, the cost is $ 1 per request. 0002. For 250,001 requests and others, the cost is $ per request. It is 0001. The number of queries is reset once a month.

In some embodiments, the domain provider sends a monthly invoice for the information request provided to the platform provider. In some embodiments, the platform provider purchases credits prior to requesting information. Credit charges are set at gradual pricing intervals, depending on the amount purchased. In various embodiments, the user contracts, the app integrator purchases credits from a platform or domain provider, any recipient pays by invoice, or other payments for services. There can be different combinations of suitable methods.

FIG. 15C shows a non-linear, expression-based pricing model. Pricing per request is based on an expression rather than a gradual interval of numerous information requests. Many equations are possible, but the equation in FIG. 16 is as follows.

The platform provider and / or domain provider maintain the number of information requests provided for the sliding window over the last 30 days. The price per request is $ per request. It is a cost that changes inversely with the minimum value of 0001 + the number of requests. This effectively provides quantity discounts to those who use it frequently.

Various other non-linear pricing models are also possible, limited only by the creativity of the person negotiating the business contract.

CRM
FIG. 16A shows an example of a non-temporary computer-readable medium 161 that is a rotary magnetic disk. Data centers typically use magnetic disks to store data and code with instructions for server processors. The non-temporary computer-readable medium 161 stores a code comprising an instruction, which, when executed by one or more computers, performs the steps of the method described herein on the computer. Will let you. Rotating optical discs and other mechanically movable storage media are also possible.

FIG. 16B shows an example of a non-temporary computer-readable medium 162 that is a flash random access memory (RAM) chip. Data centers typically use flash memory to store data and code for server processors. Mobile devices typically use flash memory to store data and code for processors in system-on-chip devices. A non-temporary computer-readable medium 162 stores a code comprising an instruction, which, when executed by one or more computers, performs the steps of the method described herein on the computer. Will let you. Other non-moving storage media packaged with leads or solder balls are also possible.

Any type of computer-readable medium is suitable for storing code with instructions according to various embodiments.

Server The server stores a database of domains provided to VA developers in the platform menu. The server also stores a database of code for the grammar associated with the domain. The server also stores a database of pricing models associated with the domain.

FIG. 17A shows a rack-mounted server blade multiprocessor server system 170 according to some embodiments. It comprises a number of networked computer processors running software in parallel.

FIG. 17B shows a block diagram of the server system 170. It comprises a multi-core cluster of computer processor (CPU) cores 171 and a multi-core cluster of graphics processor (GPU) cores 172. These processors connect to a random access memory (RAM) device 174 via a board-level interconnect 173 for storing program code and data. The server system 170 also includes a network interface 178 to allow the processor to access the Internet. By executing the instructions stored in the RAM device 174, the CPU 171 and GPU 172 perform the steps of the method described herein.

SoC
FIG. 18A shows the bottom side of a packaged system-on-chip device 180 with a ball grid array for surface mount soldering to a printed circuit board. Different package shapes and sizes are possible with different chip implementations. System-on-chip (SoC) devices control many of the embedded systems and IoT device embodiments described herein.

FIG. 18B shows a block diagram of the system on chip 180. It comprises a multi-core cluster of computer processor (CPU) cores 181 and a multi-core cluster of graphics processor (GPU) cores 182. These processors connect to an off-chip Dynamic Random Access Memory (DRAM) interface 184 via a network-on-chip 183 for storing volatile programs and data, and a computer to a flash RAM non-temporary computer readable medium. Connect to flash interface 185 for non-volatile storage of program code. The SoC 180 also has a display interface 186 for displaying the GUI and an I / O interface module 187 for connecting to various I / O interface devices as needed for various peripheral devices. The I / O interface enables, among other things, sensors (such as touch screen sensors), geolocation receivers, microphones, speakers, Bluetooth® peripherals, and USB devices (such as keyboards and mice). The SoC180 also has a processor that connects to the Internet via wired or wireless connections such as Wi-Fi, 3G, 4G Long Term Evolution (LTE), 5G, and other wireless interface standard wireless and Ethernet (registered trademark) connection hardware. It includes a network interface 188 to allow access. The CPU 181 and GPU 182 are described herein by executing instructions stored in a RAM device through interface 184 or by executing instructions stored in a flash device through interface 185. Perform the method steps.

Yet other considerations One of ordinary skill in the art will recognize many variations and modifications. Modifications and modifications include any relevant combination of disclosed features.

Various embodiments are methods that use human or machine behavior or human-machine combination behavior. The embodiment of the method is complete wherever most of the configuration steps are performed in the world. Some embodiments are one or more non-transitory computer-readable media arranged to store such instructions for the methods described herein. It is a complete embodiment of any machine holding a non-temporary computer-readable medium with any of the required codes. Some embodiments are arranged to store physical devices such as semiconductor chips, hardware description language representations of the logical or functional behavior of such devices, and such hardware description language representations. Only one or more non-transitory computer-readable media.

The description herein describing principles, features and embodiments includes their structural and functional equivalents. The elements described herein as coupled have a valid relationship that can be achieved by direct or indirect connection with one or more other intervening elements.

The examples shown and described use specific spoken language. Various embodiments work similarly with other languages or combinations of languages. The examples shown and described use a specific knowledge domain. Various embodiments work similarly with other domains or combinations of domains.

Some embodiments are screenless, such as earpieces that do not have a display screen. Some embodiments are stationary, such as vending machines. Some embodiments are mobile, such as in automobiles. Some embodiments are portable, such as mobile phones. Some embodiments include a manual interface, such as a keyboard or touch screen. Some embodiments include a neural interface that uses human thinking as a form of natural language expression.

Claims (17)

A platform for configuring natural language virtual assistants
A means for application developers of natural language virtual assistants to enter test queries,
Each of the plurality of competent domains is capable of interpreting the test query, and the platform provides means for filtering a large set of domains into a subset of the plurality of competent domains. Moreover,
It comprises means for presenting a list of domains, said means.
A list of said domains is presented by searching each of the plurality of competent domains in a number of promoted domains, each promoted domain having an associated pricing model, said means. Moreover,
Presenting a list of said domains by discovering at least one competent domain in said many promoted domains.
A list of said domains is presented by presenting a list of competent domains that include said said promoted domains, and said promoted domains are highlighted within said list.
A platform that allows app developers to identify promoted domains for inclusion in the Natural Language Virtual Assistant. Means for receiving user queries and
A means for interpreting the user query according to the promoted domain and providing a response.
The fee is calculated according to the pricing model associated with the promoted domain, further comprising means for calculating the fee associated with providing the response, according to claim 1. platform. A way to integrate a virtual assistant domain into your app
Steps to receive a test query from the app integrator,
With the steps of interpreting the test query in multiple domains, some domains charge the app integrator a cost, and the method further
Show the app developer a subset of the domains that can interpret the test query, and for each of the indicated domains, the app integrator will charge the cost of responding to the test query using the indicated domain. A method comprising the steps shown in. The step of receiving a second test query from the app integrator,
The step of interpreting the second test query in the plurality of domains,
The method of claim 3, further comprising calculating the cost of responding to the test query and the second test query using each domain according to a non-linear pricing model. A way to integrate a virtual assistant domain into your app
Steps to receive a large number of test queries from the app integrator,
Each of the plurality of domains comprises a step of interpreting each test query from the large number of test queries to determine the domain having the highest interpretation score for each test query, and at least some of the plurality of domains. The domain charges the app integrator according to the pricing model, and the method further
A method comprising the step of applying the pricing model to the query for which the costing domain had the highest interpretation score in order to calculate the total cost of the large number of test queries. The method of claim 5, further comprising the step of calculating the cost per domain. The method of claim 5, wherein the pricing model for a particular domain is non-linear with respect to the number of queries in which the particular domain has the highest interpretation score. A platform for configuring domains for inclusion in the Natural Language Virtual Assistant
A way to enter a large number of test queries,
A means for entering a domain selection to be included in the Natural Language Virtual Assistant,
A display view that shows the app developer which domain was selected to be included in the natural language virtual assistant from the domain menu.
With a display view showing the cost component of the total cost that would be levied on the natural language virtual assistant to respond to the large number of test queries with the domain selected to be included in the natural language virtual assistant. Each cost component is due to the domain selected from the domain menu.
The app developer is allowed to configure the selection of domain combinations to be included in the natural language assistant so that the total cost of responding to the test query is within the desired budget. platform. The platform of claim 8, further comprising a display view for the app developer to view a histogram showing how many of the test queries are answered by each domain selected to be included in the natural language virtual assistant. .. The display view, which shows the app developer which domain was selected to be included in the natural language virtual assistant from the domain menu, displays at least one promoted domain more prominently than the other domains. The platform according to claim 8. 8. The cost component of at least one domain varies non-linearly with respect to the number of test queries that the domain will respond to with the domain selected to be included in the natural language virtual assistant. Described platform. A platform for configuring a virtual assistant, the system
A server-hosted platform for VA (Virtual Assistant) developers to select domains for inclusion in Virtual Assistant, and
A grammar for requesting a large number of domains and domain providers available for selection,
A platform that, when executed by a processor in an embedded system, provides an interpreter function with means for exporting code that responds to queries against the local domain, independent of said platform server. A means to receive a set of test queries,
12. The platform of claim 12, further comprising a display view that allows the cost of responding to the query in the set of test queries for a given domain selection to be displayed. 12. The platform of claim 12, further comprising a number of pricing models associated with said domains available for selection. It's a method
When the platform receives the domain,
A step in which the platform generates a first domain grammar that identifies a grammatical phrase in the received domain.
A method in which the platform comprises associating the domain grammar with the domain before the VA developer selects the domain for inclusion in the virtual assistant. 15. The method of claim 15, further comprising associating a pricing model with the domain to identify the cost per query for using the domain. The step of receiving a set of test queries from the VA developer,
The step of receiving the domain selection to be included in the virtual assistant from the VA developer, and
15. The method of claim 15, further comprising generating a display view showing the cost of responding to the query in the set with the domain selected.

Images