JP2020091834A - Virtual assistant domain selection analysis - Google Patents

Abstract

To provide a user interface for a virtual assistant platform to configure a domain enable for a virtual assistant by an application developer.SOLUTION: A test query set may be uploaded, and statistical analysis may be displayed for the number of test query provided by each selected domain and the use cost of each domain. Cost may vary according to a complex price setting model. A user interface provides a table, a cost stack chart, and a display view of a histogram to notify a decision to trade off cost and benefit to virtual assistant user experience. A platform interface shows possible responses expected from various domains relative to each query. A platform provider promotes a particular selected domain.SELECTED DRAWING: Figure 1

Description

Background An increasing number of people are using virtual assistants such as Soundhound's Hound, Google Assistant, Amazon Alexa, Baidu Haku, Apple Siri, Line/Naver Clover, Microsoft Cortana, KT Gigajiny, SK Telecom NUGU and Open Source Microft. ing. First generation virtual assistants were only available on corporate brand computers, smartphones or home speaker devices that provided virtual assistant functionality. They were not configurable and only supported custom features that the company developed in-house.

Current virtual assistants may be text-enabled. They are capable of understanding written natural language text and making written natural language responses. Some embodiments are voice enabled. They understand the spoken natural language and may respond in the manner of text, synthetic speech, images, graphic animations, audio and video clips and the like.

Overview Companies that manufacture and sell applications (devices) such as cars, robots, shops and equipment, including virtual assistants (“VAs”), do not necessarily want to invest in creating VAs “from scratch” on their own. It does not mean that These companies want the flexibility to incorporate existing natural language features as part of their custom virtual assistant. These companies find it frustrating not to configure virtual assistants specifically to their requirements.

A domain is a unit of natural language function. Domains are characterized by a set of sentences that the domain can interpret and the ability to respond and act in interpreting recognized sentences. Examples of domains include a weather domain that answers questions about the weather, and a calendar domain that allows users to coordinate and manage appointments, that is, request actions and ask questions.

Some VA development platforms (“platforms”), such as Soundhound's Houndify, develop virtual assistant developers (“VA developers”) by providing a menu of domains available on the development platform. ) Support. On such platforms, VA developers choose domains for inclusion in their own custom virtual assistant, which includes an array of domains already known to the platform. Some embodiments provide for configuring a virtual assistant using such a platform. Some embodiments include such a platform. In some embodiments, the platform provides an interface for the domain provider to provide the domain to be integrated with the virtual assistant. The information provided by the domain provider is sufficient to support the functionality of the domain in the virtual assistant that selects the domain. In some embodiments, some domains are free. In some embodiments, the provided domain charges an associated fee for its use. In various embodiments, payment is made to the domain provider by the platform provider or by another party such as a VA developer, a user or a VA developer through the platform provider. In some embodiments, platform providers promote certain domains over other domains.

Some embodiments provide a VA developer to enter test queries for virtual assistants and see how the virtual assistant behaves in response to these queries. Some embodiments provide for showing and comparing the behavior of virtual assistants under different conditions in different domains selected from a menu of domains.

Some embodiments provide VA developers to enter large test query sets. Some embodiments display a view of an analysis of virtual assistant behavior in response to a test query set and domain selection. Some embodiments provide for analyzing the costs associated with responding to all queries in the set. Some embodiments provide for analyzing a portion of the test queries in the set that the virtual assistant can interpret.

Some embodiments provide a display view for analyzing the interpretation success rate and cost of queries within a set under conditions of different selection of domains. Some embodiments provide a tabular view of domain results. Some embodiments provide a pricing model for each domain. Some embodiments provide typical queries recognized by each domain. Some embodiments provide a test query set and a total cost of responding to queries from the test query set. Some embodiments provide a cost stack view of the cost per domain of responding to queries in the test set. Some embodiments provide a histogram view of queries answered per domain. Some embodiments indicate that some domains are free and others charge costs.

Some embodiments provide a graphical user interface with a pointer icon. In some such embodiments, when the pointer is over a particular portion of the display view, the corresponding information is displayed in the pop-up message.

Some embodiments provide for sorting or filtering the menu of domains according to useful criteria. Some embodiments provide for displaying different domains in a menu in different ways such as background color, text color, font and typeface.

Some embodiments provide parameterized pricing models for domains, such as linear, piecewise linear, or expression-based pricing models. Some such embodiments provide tools to help VA developers make cost-effective domain selections.

Some embodiments are computer-readable media. Some embodiments are cloud servers. Some embodiments are mobile devices. Some embodiments are systems of devices that communicate with a cloud server. Some embodiments are self-contained devices that implement a natural language virtual assistant configured by the platform.

5 illustrates an information flow from a domain provider to a platform provider to a user via a VA developer according to an exemplary embodiment. Indicates a test query set. 1 illustrates a cloud-based system consisting of domains, platforms and virtual assistants in an embodiment. 1 illustrates a single device system consisting of a virtual assistant configured from domains and platforms in an embodiment. In an embodiment, a user is shown interacting with a virtual assistant that interprets a query and provides a response according to a domain. In an embodiment, we show that a VA developer uses test queries to guide the configuration of a natural language virtual assistant through a domain promotion platform. 1 illustrates a complete system in which a user queries, a VA developer configures, and a domain provider provides information to users in an embodiment. In an embodiment, select which domains to enable, show how many test query sets are answered by each domain, and set the pricing model and per domain it takes to respond to queries in the test set. Figure 9 shows a display of a menu of domains that provides showing costs. 6 illustrates a cost stack chart of a set of enabled domains for a test query set in an embodiment. 6 illustrates a cost stack chart of a set of enabled domains for a test query set in an embodiment. 6 illustrates a histogram of queries from a test query set responded by each of a set of enabled domains in an embodiment. 6 illustrates a histogram of query responses per domain highlighted under hover pointers and indicating domain-specific information in an embodiment. 6 illustrates a display of a menu of domains in which the promoted domain is shown prominently, in an embodiment. Enter a test query to show the display to see the domains that can respond to this query and a filtered menu of those responses. A linear pricing model is shown. A piecewise linear pricing model is shown. An expression-based pricing model is shown. 1 illustrates a rotating non-transitory computer-readable medium. 1 illustrates a packaged solid state non-transitory computer readable medium. Shows a rack of servers. The block diagram of a server is shown. Figure 3 shows a packaged system-on-chip. A block diagram of a system-on-chip is shown.

DETAILED DESCRIPTION Various embodiments of the invention are described below that describe various features. In general, the described features can be used in any combination in embodiments.

Many companies manufacture or sell devices or apps that can be enhanced by integrating virtual assistants. Many virtual assistants are voice-based. Examples of voice-enabled devices are automobiles, robots, shops, appliances and smartphones. They can all have an integrated virtual assistant. A user communicates with a virtual assistant by issuing a natural language query, which is typically either a request for information or a request for action. The query may be a spoken query or a written query. The response from the virtual assistant may include voice, text, graphics, audio or video recordings, and actions.

The development of intelligent multifunctional virtual assistants is complex and requires, in addition to the common architecture, expertise and investment for each domain that the virtual assistant can handle. When developing a virtual assistant, it makes sense for a developer to purchase or in-license domain software from a virtual assistant development platform provider. This allows platform providers to hire enough specialists to build a good platform and allow various VA developers to share costs.

In such an ecosystem, the logic or software for implementing a virtual assistant is stored on a computer-readable medium in a network-attached server operated by a platform provider or in a device that may not be connected to the network. , Device logic or software is configured using the platform. In a typical networked embodiment, a client app or remote device uses an application programming interface (“API”) to access a server over the Internet. Some single-device virtual assistants operate autonomously, some only operate when you have network access, and the combination of local and remote virtual assistants provides intermittent network Some work with access. Whether connected to the network or not, the virtual assistant receives a natural language query from the user, interprets the query, and provides a response to the user. Zero or more than one domain may recognize each query. In the case of ambiguity, the virtual assistant selects one of the conflicting domains and determines its response.

A virtual assistant developer is a person (most often a trained technician) working in a company that integrates a virtual assistant into a particular device and app. VA developers use the platform to configure virtual assistants to specific requirements. Virtual assistants generally support multiple domains. Some domains provide information such as weather, news, trivia, restaurant searches, simple calculations and Wikipedia searches. Some domains provide services such as thermostat control or lighting, music streaming, sending SMS text messages, and managing calendar appointments. Some domains may be as simple as just telling the time. Some domains may be complex, such as providing travel agency services, and some may be very beneficial to the user, such as the ability to locate a child. Many domains access web APIs to access specific or dynamic information or to cause a requested action. Some domains offer information and services from third party providers in exchange for payment. For example, some weather domains access the National Weather Service API to obtain weather forecast information.

The platform provides VA developers with a way to select domains for inclusion in a virtual assistant, among other capabilities. Generally, the domain is provided by a domain provider entity. Some platforms support thousands of third-party domain providers. Some domain providers are small businesses or even individual programmers.

In some embodiments, the query set recognized by the domain is defined by a semantic grammar code. Such grammar code is written specifically for each domain by a domain developer (most often a trained technician) who creates the rules for interpreting the query as a specific information request or specific action request. To be done. In such an embodiment, the natural language interpretation is based on the domain grammar code. The domain grammar comprises the syntax rules used by the natural language processing system to parse the query. In domain grammar, syntax rules are extended by semantic extensions. Extension is the ability to construct a component interpretation from zero or more subcomponent interpretations, where the subcomponents are determined by parsing. In some embodiments, a domain grammar is used to recognize and interpret queries for domains. In some embodiments, at least some of the interpreters include neural networks that are trained using machine learning. In any case, the query's interpretation assigns values to semantic slots based on information from the query. In some embodiments, an intent slot is encoded that represents the user's overall intent for the query, such as meeting scheduling or flight booking.

In an exemplary embodiment, a semantic grammar for requesting a weather forecast may have (1) a syntactic component having a component named <location> and a corresponding semantic slot named “location”. , Its value must be considered as a location in every example, and (2) the syntactic component may have another component named <time> and the corresponding semantic slot named “time”. And its value must be considered as time. When the user asks, “What's the weather in Timbuktu tomorrow?”, this embodiment uses a weather domain grammar with <location> and <time> components and sets the intent slot to the value “weather_info”. You can fill in with, the place slot with the value "timbuktou" and the time slot with the value "tomorrow" or possibly the corresponding calendar date.

Similarly, if the user says, "Where is the closest French restaurant with at least 4 stars?", the system can use a restaurant domain grammar with syntax rules for <Cooking_Type> and <Rating> components. Corresponding slots for food type and rating will be filled with values that convey "French food" and "at least 4 stars" respectively. In another example, when the user says "text: I love mommy", the system recognizes a query with SMS texting domain grammar and fills the intent slot with "send_text", corresponding to "mama". Fill the recipient slot with reference to the contact in the user's contact list and fill the message slot with "love". The interpretation of the query is formed by these slots, and the completion of the interpretation of the query specified by the intent would be to send the textual content of the message to the recipient's mobile phone. When the user says, "How many claws does a cat have?" The system recognizes the knowledge domain query, fills the intent slot with "Wikipedia search" (say), and the kind slot with "cat". Fill the attribute slot with "nail" and the question slot with "how many". Other natural language approaches are possible.

Some queries can be interpreted by multiple domains. It is common to have such overlapping grammars when different domain providers compete to provide the same domain functionality. For example, the platform may provide four competing domains for booking a hotel, all four of which may be able to interpret the query "Please tell me the hotel in Paris". To help VA developers deal with this situation, the platform can configure the virtual assistant to include certain domains in its functionality and not other domains, or other domains. The virtual assistant can be configured to select a particular domain in preference to. Such selections can be made in "hard" or "soft" ways.

Hard choices are made at configuration time. Some domains are enabled in the virtual assistant, all other domains are disabled. This configuration step allows the VA developer to control which domains are involved in query interpretation at runtime. In production, query parsing only considers enabled domains.

Soft selections are made at runtime. The choice between query interpretations of competing domains (all of which are enabled) is generally made by choosing the highest scoring interpretation based on the scoring scheme. Scoring techniques can be used to prioritize domains. That is, if enabled domains A and B conflict to interpret a query, domain A can always be prioritized over B. Scoring can also address the issue of ambiguity between unrelated domains, but such semantic conflicts are less frequent. For example, the query "How high is Denver?" can be interpreted by a geographic domain that knows the elevation of a city, but also by the weather domain where "height" refers to temperature. be able to. The second interpretation may be unlikely in the absence of context, but "What is the weather like in Chicago?" [Answer: 56 degrees] How high is Denver? Is a very likely interpretation in the context of weather and temperature in conversations such as. Parsing and interpretation scores may be context sensitive, and scoring may take into account all of these and domain priorities.

For a valid configuration choice, the platform can provide empirical tools intended to help VA developers make wise decisions. In some embodiments, the cost-effectiveness of a domain is (1) coverage, that is, what part of the query is recognized by the domain, and (2) accuracy, that is, what part of the recognized query is recognized by the domain. It can be measured against the test set of queries in terms of correct interpretation, and (3) cost considerations.

Regarding costs, according to some embodiments, the platform is aware of the pricing model associated with each domain and can provide pricing information to VA developers. Many platforms will offer a specific domain for free, such as a domain for signaling time or a domain for performing simple calculations. However, many domain providers want to offer high value domains and be compensated for their use of their data and services. For example, a domain for trading stocks may charge a fee for each stock trade requested by the user. In some virtual assistants, the user pays the VA developer to use the domain, and the VA developer pays the domain provider either directly or via the platform. In some cases, the platform may raise the cost of the domain to recover the value added by the platform.

In practice, a platform or VA developer may discount a particular domain or subsidize to attract more customers. In pricing models, the price per query is often subject to volume discounts. All such considerations can be part of the domain pricing model.

Coverage and accuracy are measured based on a test query set that the platform receives from VA developers. A "hit" is registered when the enabled domain recognizes the query. A test query is "covered" if it hits at least one enabled domain. Coverage is measured (absolutely) by the number of test queries covered or (relatively) by the corresponding part of the test query. The accuracy is measured (absolutely) by the number of test queries processed correctly or (relatively) by the corresponding part of the test query covered. Accuracy can be measured if the test set contains query interpretation data or if the test set contains query data.

In some embodiments, testing for correctness requires interpreting the test query that would be made to the user query in production, but not performing (performing) the test query. do not do. Correctness is based on the equivalence (or matching consistency) of interpretations (which are internal representations of the meaning of the query). In some embodiments, testing for accuracy requires interpretation and fulfillment of test queries. And the accuracy is based on the equivalence (or matching consistency) of the responses.

In some embodiments, a VA developer can enter a test query as if it was received from a user, and the platform can (1) determine which domains can recognize the query. One or more of (2) query interpretation, (3) information needed to perform the interpretation and respond to the query, and (4) the cost of providing the response can be shown to the VA developer. ..

In some embodiments, the VA developer enters a large test query set, and the platform provides statistics about the domains and cumulative cost of the test set that this query set will hit. In some embodiments, the query may hit multiple domains. In other embodiments, the system ensures that there are no more than one hit per query. One type of statistic is a histogram of hits per domain across the query set. Another type of statistic is the domain name, whether or not each domain is enabled for virtual assistants, the pricing model for accessing each domain (such as the cost per simple query), and for each domain. The number of queries in the test set that are hit, the resulting cost of enabling the domain in the virtual assistant, the portion of the queries in the test set that will hit at least one of the enabled domains , And the total cost to process all queries in the test set using the enabled domains.

It is possible that some queries require information from more than one domain. For example, the query "What's the weather in the home base of the team that won the last Super Bowl game?" uses a sports information domain to help the virtual assistant determine which team won the last Super Bowl game and It would be necessary to locate the home and then use the city information to look up the weather information using the weather domain. Therefore, the number of domains hit by the virtual assistant responding to the test query set will be greater than the number of queries, and the cost of the queries will be higher than if each test query hits only one domain.

Some queries are not recognized by any domain. In some embodiments, such non-working queries are answered with default information, such as web search results. As a result, the number of hits will be less than the number of queries.

In general, the more domains the virtual assistant has, the more queries the virtual assistant can respond to satisfactorily. The more satisfied the user is with the virtual assistant, the more users will use the virtual assistant. The more users who use virtual assistants, the more incentives domain providers have to create and improve domains. This creates a virtuous cycle of day-to-day domains and virtual assistants, benefiting the user and benefiting all participating companies.

The following is a description of specific exemplary embodiments shown in the drawings.
Stakeholders Figure 1 shows a diagram of the virtual assistant ecosystem and its operation. Various domain providers provide information and services to platform 12. Specifically, the weather information domain provider provides the weather domain 11a to the platform 12, the map information domain provider provides the map domain 11b to the platform 12, and the text messaging domain provider provides the text messaging domain 11c to the platform 12. To provide. In addition to the domain grammar, the information provided to the platform may include pricing models and test data.

The VA developer uses the platform 12 to create virtual assistants such as the device 13a, the mobile phone 13b, and the car 13c, which all deliver VA functionality to multiple users 14. Some users use more than one type of virtual assistant. Information and services typically flow from the domain through the platform, through the VA, from left to right, and reach the user.

Generally, payments flow in the opposite direction. In various embodiments, the user pays directly to the VA developer, platform provider or domain provider. In various embodiments, the VA developer pays directly to the platform provider or domain provider. In some embodiments, the platform provider pays the domain provider. In some embodiments, platform providers offer virtual assistant developers discounts or access to several domains that are losing in order to win business or increase total usage. provide. In some embodiments, the domain provider pays or provides a discount to the platform provider in exchange for the platform provider promoting the domain. The techniques disclosed herein do not limit the business relationships possible using this framework.

In some embodiments, the enterprise is both a domain provider and a virtual assistant developer. For example, a virtual assistant for a car needs its own local domain to answer status queries about the car, such as fuel level, or to request actions such as turning on headlights. .. In addition to this, in some cases, an external domain such as a domain having weather information is included. In another example, a video player set-top box sold by a particular video content provider needs to have access to a domain to obtain its video content as part of its custom virtual assistant. This domain may be local or external. In some embodiments, some domains are dedicated to some virtual assistants and not available to other virtual assistants.

FIG. 2 shows an example of a query set specific to a general purpose virtual assistant. Queries about the weather are the most frequent, but other common queries are local business searches, map directions and traffic conditions, queries for news or sports, queries for trivia information, and a variety of other types. Is the query. Since each query may appear multiple times in the test set, the order is irrelevant, and an alternative expression known as a bag is to associate each unique query with a count. For example, there are three queries "How is the weather?" in this set. Extending this further, each query can be given a real-valued weight (not just a frequency count) that indicates to the experimenter its importance to other queries. Such weights can be normalized so that the sum is one.

Cloud and Device FIG. 3 shows a user 31 in communication with a first app 32 and a second app 33. These apps communicate with the platform 35 via the network 34. The app developer 36 has previously used the platform 35 to configure a first virtual assistant for the first app 32 and a second virtual assistant for the second app 33. In some embodiments, different developers configure each app.

On platform 35, multiple domains are available for configuration selection. These multiple available domains may be in the hundreds, thousands, or even more. Domains are provided by domain providers. They are made available on the platform via a domain registration or ingestion process. During registration, the information provided about the domain includes the unique ID, simple name, description, examples of recognized queries, and domains in executable or interpretable form (source code, object code, neural networks or semantic grammars, etc. ) May be included. In this disclosure, the term "grammar" or "semantic grammar" generally refers to a description of code, such as a source, object, neural network, or any data structure instance that can be executed to parse or recognize a query.

The virtual assistant configuration created by the developer 36 includes a selection of a first domain 37 and a second domain 38. When the user 31 makes a query to any of the apps, the user 31 sends the query to the platform 35, and the platform 35 interprets the query and outputs the web API from the first domain 37 or the second domain 38. Use to get the right information.

FIG. 4 shows a user 41 in communication with an app 42 integrated in a device 44 that operates independently of a platform 45. This figure is not drawn to scale. The app developer 46 has previously used the platform 45 to configure a virtual assistant for the app. An app integrator is a company that designs, manufactures, or sells products and services that have a virtual assistant app integrated inside. The platform provided the software that the app integrator needed to integrate the configured virtual assistant into the app 42.

Upon receiving the queries from the user 41, the app 42 uses the integrated interpreter to interpret them and request information from the local information domain 47 accordingly or a service action from the local service domain 48. .. For example, in a car device, the information domain 47 would have a grammar and provide information for queries such as "how much battery power is left" and "which radio station is this?" Let's do it. In a car, the service domain 48 would have a grammar and provide services such as "Please open sunroof" and "Turn on radio". These exemplary information sources and services are local to the vehicle and do not require access to the network.

Some embodiments have configurations that allow a combination of domains that require network access and domains that do not require network access. In such an embodiment, the domain responds successfully if network access is available, and fails the domain's response if network access is unavailable.

Information Flow and Configuration FIG. 5 shows a diagram of data flow through the platform at runtime according to an embodiment. The user 51 issues a natural language query that the platform processes. Interpreter 52 interprets this query and provides a response to user 51. For some queries, the response may be a simple indication that interpreter 52 cannot interpret the query. In some embodiments, interpreter 52 alone can provide a response to a query that it successfully interprets.

Domain providers provide data from their servers such as music or other sources such as weather sensors, or provide services such as sending messages or controlling vehicles. Generally, such data or services are known as domains 53b. In order to know when the user wants to call the domain, the user can have the corresponding grammar code 53a. The grammar code includes phrases that call the domain and slots that can be filled with words from the user expression. For example, a grammar code with the phrase "how is the weather?" calls the weather domain, and the domain code contains slots for where and when to get the weather forecast. The code for the weather domain phrases and slots tells the interpreter that the phrase "What's the weather in Timbuktu tomorrow?" should make a request to the domain provider about the weather in Timbuktu tomorrow. To be able to do. The code associated with the domain is, in many embodiments, registered, ingested, and stored by the platform.

When interpreter 52 interprets the query according to the grammar code 53a associated with domain 53b and determines that the query requires information from domain 53b, interpreter 52 determines that the appropriate query is provided by one of the domain providers. Request information from a different domain 53b. Interpreter 52 uses this information to form a response for user 51.

FIG. 6 shows a diagram of the data flow through the platform during configuration according to the embodiment. Among many app integrators, the developer 66 provides one or more test query inputs 64 to the platform. The developer 66 also provides a selection 65 of one or more domains. The interpreter 62 interprets the test query according to the grammar code 63a of the selected domain 63b and determines which of the selected domains 63b can interpret the query. The display displays the results from interpreter 62 to developer 66 (67). Many display views are suitable and useful for various embodiments. Some display views are illustrated and described below.

The embodiment of FIG. 6 further includes promoting (68) one or more domains based on the results of interpreter 62. For example, if the test query can be successfully interpreted according to the promotion domain, the platform shows the developer 66 the results that the interpreter would show to the user if the promotion domain was enabled. In some embodiments, the names of promotion domains are indicated even if they are not enabled. Some embodiments show responses that would be generated by information from the promotion domain as an alternative to responses that would be generated by information from the highest ranked alternative domains.

For example, a weather domain provider with more accurate and detailed information may charge a higher price and offer itself as a promotional domain. The default weather domain can generally answer weather queries. The platform according to the embodiment will show the response according to the information from the default weather domain, and the response that will be generated by the information from the promotion weather domain. This can be done in conjunction with candidates for selecting a promotion domain, and means such as button icons for selecting a promotion domain for inclusion in the virtual assistant.

FIG. 7 shows a diagram of the data flow through the platform. Developer 76 provides one or more test query inputs 74 to the platform. The developer 76 also provides a selection 75 of one or more domains. The interpreter 72 interprets the test query according to the grammar code 73a of the selected domain 73b to determine which of the selected domains can interpret the query. The display displays the results of interpreter 72 to developer 76 (77). The embodiment of FIG. 7 further includes promoting one or more domains 78 based on the results of interpreter 72. After the configured time, during runtime, user 71 issues a query that interpreter 72 interprets and provides the response to user 71.

In the embodiment of FIG. 7, in response to receiving information from domain 73, the platform calculates a charge 79. This fee is calculated according to the pricing model. The pricing model is a function based on domain usage. Some embodiments have a simple pricing model, such as a single fee for each information access. Such fees are typically a fraction of US$1. Some embodiments support more complex pricing models that are features based on the number of previous information accesses within a time window or the type of information accessed. Equivalently, some domains charge for the provision of services when other domains provide information.

In the embodiment of FIG. 7, the platform pays each domain provider that provides information to the virtual assistant and configures the response to the user. This creates an incentive for the domain provider to inspire the developer to choose a domain for inclusion in the virtual assistant, and thus for the platform provider to promote the domain of the domain provider with which the platform provider has a close business relationship.

Domain Table FIG. 8 shows a display view of the domain table 80 according to the embodiment. The table 80 has one row per domain. The "enabled" column indicates the state of domain enable. In some embodiments, the cells in this column can be used to enable or disable the corresponding domain. Various inputs for selecting or deselecting domains, such as a mouse to click, a keyboard with arrow or shortcut keys, a touch screen, or other suitable means for making selections of objects in the user interface. Means are possible.

Many different graphic layouts are possible. In general, visually organize a large number of domains into rows and a relatively small number of columns to name, whether or not the domain is enabled, the number or percentage of queries (from the test set) that the domain can interpret, and the price. The grid is useful for displaying relevant information about the domain, such as the configuration model (in the simplest case, cost per query, etc.). Visual objects that show relevant information, such as rectangles that indicate that the domains are aligned to the grid, also work. It is also possible to arbitrarily drag and drop an object on the desktop of a personal computer.

Table 80 has a display view to show the VA developer which domains are selected for inclusion in the virtual assistant from the menu of domains. The display view 80 has 5 columns. The heading row has a title in each column depending on its use. These columns are the name of each domain listed from the menu of available domains, whether the domain is enabled for virtual assistants, the domain can interpret each query in the test set. A count of the number of queries, which is the domain that the interpreter will choose from among all possible enabled domains, the pricing model for responding to the query, and the domain to respond to each query in the test set. The total cost for the domain to provide the information needed by the virtual assistant to provide a response when selected.

Following the heading line is a line for each domain in the menu. Table 80 contains seven domains in the menu. Some embodiments support a much larger number of domains, a variable number of domains, and domains that dynamically change as new and existing domain providers create or remove offerings. The first column shows a text string with the name of each domain in the menu. The domains shown are weather, food, sports, maps, information, timers and calculators.

The second column has a cell for each domain and is checked when the domain is selected for inclusion in the virtual assistant. FIG. 8 shows a pointer that can be controlled with a mouse or finger on a touch screen. Clicking or double-clicking the mouse button, tapping or double-tapping, or long-pressing the finger changes the state of whether or not the domain at the pointer position is selected. This state is indicated by the presence or absence of a check mark in the cell.

The third column shows the cell for each domain, which contains a count of the number of queries replied with information from the domain given the selected domain. In some embodiments, different columns are used for the number of recognized queries and the number of correctly answered queries when the exact query response is known. In some embodiments, percentages are displayed instead or in addition to absolute numbers using other columns. Furthermore, some queries are ambiguous with respect to the selected set of domains, ie can be interpreted by more than one domain. This is most often seen in short queries. For example, the query "turkey" can be interpreted by the weather domain, food domain, sports domain, map domain, information domain. In some embodiments, an ambiguous query is processed by providing a tagged response for every ambiguous interpretation of the query.

In other embodiments, domain priorities are used to select the highest priority domain that matches the query to more strongly disambiguate. In another variation, query interpretation scores are used and the domain with the highest scoring interpretation score from the corresponding grammar wins to form the basis of the response. You can combine the two ideas. In all cases, the ambiguity is eliminated by choosing one or a few interpretations more coercively, which contributes to the number of single domains. As the state of domain selection changes, this can change the number of queries that other domains will respond to. In some embodiments, the query count is dynamically recalculated each time the domain selection is modified. Alternatively or in addition to the query count column of FIG. 8, in some embodiments, the total number of test queries that each domain can provide information on is also the number of test queries that can be accurately answered with the domain enabled. Number increments are also shown.

The fourth column shows the pricing model to explain the domain query. Table 80 shows a simple type of pricing model based on a fixed price per query. Prices shown range from 0 (free domain) to 5¢ ($.05 US$) per query. Some embodiments support more complex pricing models such as piecewise linear or expression based models depending on the number of hits in a time window. Circular time windows may be used (eg calendar months) as well as sliding time windows (eg past 30 days). Embodiments that support more complex pricing models may not show the pricing model itself in the pricing model column, instead, by clicking or tapping on the domain's pricing model cell Access may be provided to different display views, such as pop-up windows that show the configuration model and allow it to be edited.

The fifth column shows, for each domain, the amount that would be paid to the corresponding domain provider based on the response to every query in the test set of queries. You will be charged for every domain where the information is used in the response. In the simple pricing model shown in Table 80, the domain cost for each domain is the number of queries times the cost per query. In embodiments that allow compound queries, a single query may hit multiple domains. For example, "How old is the President of the United States" may hit both domains that provide political information and domains that provide personal information. In embodiments that recognize response ambiguity, a single query may hit multiple domains for different reasons. Multiple answers are given according to competing interpretations. For the same reason that the number of queries can change each time a domain is selected or deselected, the domain cost can change accordingly.

The bottom complete row in table 80 represents the number of queries that did not match any domain. Such queries can be counted, but the other columns are not applicable.

Two quantities are listed in the line below. The first quantity indicates the percentage of queries that receive a response on the current domain selection. In table 80, the food domain and map domain are not enabled. They would each have responded to 1955 queries and 764 queries, respectively. They result in 633 queries that cannot be interpreted by any domain, and 73% of the test queries will receive a successful response.

The last quantity cell displays the total domain cost for the enabled domains. This number will change dynamically as the VA developer enables or disables the domain. In general, the more domains a VA developer enables, the greater the success of the query and the higher the total cost. In general, the higher the query success rate, the more satisfying the user experience, and such a dynamic and useful display view allows VA developers to keep track of information between user satisfaction and domain usage budget. You can compromise on the basis.

In all of the above methods, the test query or group of test queries may be given redundancy. Returning to FIG. 2, which shows the test query set, it should be noted that the query “How is the weather” has been made three times. In a production environment, VA developers would have access to field statistics across a much larger data set, and could have hundreds of "what's the weather" months. Having multiple queries for the same domain provides a non-linear quantity discount for valid business arrangements. This is especially true when the platform caches results for data that is static or changes only infrequently. Alternatively, in the absence of full-fledged statistics from the field, a small test set may be curated (1) from a small sample of the query, or (2) possibly derived in part from the data from the field. Based on the query set. In any case, the number of test sets can be multiplied by a variable factor that the VA developer can change to observe the effects of non-linear volume discounts.

Some embodiments, especially those with multiple domains, provide scrollbars to allow VA developers to view different domains within a limited display space. Some embodiments allow VA developers to sort the list of domains according to one or any number of column criteria. One way to do so is by receiving clicks or taps on column headings and sorting the list. In addition, by clicking on the column where the list is already sorted, the list is sorted in reverse order. Some embodiments provide a box for VA developers to enter filter criteria, such as whether or not each domain is enabled, filtering a list of domains by a range of pricing models and a range of domain costs. To do.

Every virtual assistant is a different project for VA developers. In some embodiments, each project will be opened and viewed separately, displaying a domain table, especially for certain virtual assistants. Some embodiments of the platform let VA developers create accounts and log in to configure their virtual assistant projects. Platform Administrators can control which domains various VA developers can see, and what controls and tools give VA developers access to work on projects depending on the terms of their account. Can be controlled.

Cost Stack Chart FIG. 9 shows a display view of a cost stack chart 90 according to an embodiment. This chart has the caption "Cumulative amount spent per domain". In this chart, the vertical axis is cost and the horizontal axis is domain. In some embodiments, the domains are shown along the vertical axis and the costs are shown along the horizontal axis. The former is generally desirable for virtual assistants, where a small number of domains generally dominate the cost, while the latter is generally desirable for virtual assistants with a large number of domains.

There are many useful ways to present visual data. For example, Edward Tafte's books, such as Information Envisioning and Quantitative Information Visual Display , provide information on this subject. The black and white line drawing is not ideal, but will be sufficient for explanation. In the embodiment shown in the black and white line drawing in FIG. 9, seven domains are drawn on the horizontal axis. To improve readability without duplicating words, domain names are written right-aligned at an angle to the axis. The names of enabled domains are shown in bold font, and the names of disabled domains are shown between brackets and written in a smaller font. Free domains are shown with a "(f)" after the domain name. Domains that do not require access to external dynamic information are generally free. This is because the platform provider can cache the information and get it at a low cost.

In each domain column, a bar with a height proportional to the domain cost to respond to the test query set indicates the domain's contribution to the total cost of responding to all queries in this set. The bottom surface of each domain bar is located at the top level of the previous domain bar. Domains with no cost have zero height bars and are displayed as horizontal lines. The vertical axis is represented by the value of the accumulated cost, and the horizontal broken line of the height at the top of each bar extends from the vertical axis to the bar at the indicated height. The platform reorders domains by allowing VA developers to automatically sort available domains by various criteria such as alphabetical order by name, number of matching queries, cost per query, or total cost. Easy to do. The platform's interface also allows the user to freeze the order from a particular sort while the selection will change and otherwise the order will change. You can also manually sort the domains. Also, some embodiments provide filtering the list of domains by various criteria, such as showing only enabled or selected disabled domains.

The display view of FIG. 9 also comprises a pointer that toggles the state of whether the domain is enabled or not when clicked or tapped or otherwise activated appropriately. Therefore, a VA developer can quickly and easily enable or disable a domain and see its effect on the total cost of responding to queries in the test set. FIG. 10 shows a display view of a cost stack chart 100 of the same virtual assistant as in FIG. 9 after the disabled food domain has been enabled by clicking ((food)) with the pointer. ..

In FIG. 10, the domains are sorted by the number of queries that hit the domain. The food domain contributes more to the total cost of responding to the test set than the weather domain (because it has a higher cost pricing model), but is shown second in this list. ing. This is because the test set has fewer food queries than weather queries.

In some embodiments, the cost axis is displayed on a linear scale, and in some embodiments it is shown logarithmically. In some embodiments, an axis showing the cumulative number of queries (or percentage or percentage of the test set) answered by the enabled domains is shown on the right side of the chart. Such an axis would have an irregular scale whenever the cost scale is regular (linear or logarithmic). Alternatively, the regular (linear or logarithmic) representation of the answered query may have corresponding parallel cost axes with irregular spacing.

Query Hit Histogram FIG. 11 shows a display view with a histogram 110 of query hits per domain according to an embodiment. This histogram has the title "Histogram of queries hitting the domain", with the vertical axis indicating hits and the horizontal axis indicating domains. A domain hit refers to a query that is answered with information from the domain. In the histogram 110 of FIG. 11, the domain axis is not represented by the domain name. This is because too many domains can fit the name. Alternatively, the vertical axis may show domains and the horizontal axis may show hits. If the histogram chart is long enough, or if it allows the VA developer to scroll the input, it is possible to list the names of the domains and display the number of hits.

The histogram 110 has smoothed lines, but a staircase representation across domains would also work. Histogram 110 does not have the horizontal axis labeled with the name of the domain, but does show the "$" symbol to indicate domains with pricing models that exceed the threshold. In some embodiments, the threshold is zero (to indicate free domains and paid domains). In some embodiments, a VA developer sets a threshold to see affordable vs. expensive domains.

In some cases, even domains with a small number of hits can bring disproportionate benefits to the user experience. For example, the domains available in a car virtual assistant that can locate a lost phone are rarely used, but will be greatly appreciated by users when rarely needed. That would be a strong selling point in automobiles. Even though access to that domain information would be very expensive to use for a VA developer, it would be well worth the cost to sell more cars.

Some embodiments allow VA developers to tag particular domains with an asterisk or the like as being of particular value to the user experience. In some embodiments, a histogram as seen in FIG. 11 is shown with high value domains highlighted in a prominent color or icon.

Since many queries can be interpreted by more than one domain, enabling or disabling a single domain changes the number of queries replied with information from all other domains. obtain. This can have the effect of reordering the domains in the histogram. For example, two competing weather domains are enabled, a first domain can provide all weather information including barometric pressure, and a second domain is the default source of information for all weather questions, but barometric pressure. If it cannot provide the information, with both domains enabled, the first domain has very few hits (shown near the right side of histogram 110), but the second domain is disabled. In the open state, the first domain will have a very large number of hits.

FIG. 12 shows a display view of the histogram 110 as seen in FIG. 11, but with a pointer 110. If you stay on top of the domain column in the histogram and do not move for 1 second, the platform will enable the domain in responding to the name of the domain, the number of hits it has for the test query set, and the test set query. Pop up an info box showing the total cost for.

Domain Promotion FIG. 13 shows a display view having a domain table 130 according to an embodiment. Like the domain table 80 of FIG. 8, the domain table 130 of FIG. 13 lists domains. The difference is that the domain table 130 indicates two map domains, one is named "map" and the other is named "map PRO". The latter is a promoted domain. It appears to provide a better user experience than the map domain, at least because it can provide information for 805 queries instead of 764 queries in the test set. The platform shows the map PRO domain and its line of text in bold font, with its enable check box highlighted. As a result, the promoted domain is conspicuously displayed to the VA developer for consideration.

Although the embodiment of FIG. 13 uses bold font and highlighted boxes, numerous methods for highlighting the promoted domain are possible. For example, one or more promoted domains may be listed separately, sorted to be at the top of the list, shown using different colors or shading, and starred. , May be shown with an image such as a company logo, or may be animated. Some of the means of highlighting these domains may be used in combination.

FIG. 14 illustrates a display view 140 for entering a particular test query and viewing the results, according to an embodiment. The display view 140 includes a text input box 141. The virtual assistant developer can enter text using a physical or virtual keyboard or other suitable text entry box. The entered text is displayed at the position of the cursor position indicator 142. For user interface devices that have a microphone, the VA developer can tap or click the microphone button 143 to speak out a test query. The guidance item is displayed on the display when it is first called to inform the VA developer that it is possible to "speak and say a query."

The display view 140 also includes an upload button 144. When invoked, it invokes a dialog box that allows the user to browse a file and select it for uploading to the platform. The file may contain a single query, or it may contain a delimited list of any number of queries with a complete test set. In some embodiments, the VA developer drags and drops a file from the graphical operating system or browser display into the query box to automatically upload it to the platform. Various implementations of upload buttons and drag-and-drop uploads are well known and readily available in browser client-side script templates in languages such as JavaScript®.

The display view 140 further comprises a results box 145. For a given single input query, the results box displays a list of domains that can provide the information needed to respond to the query, which is the name of the domain, the pricing model ( (Denoted as "cost"), and a column of responses that the virtual assistant would provide to the user for the query with the domain enabled. In some embodiments, multiple domain providers can provide source information and grammar to form very different responses to queries. Results box 145 shows a virtual assistant response using grammar and information from each of the four travel domain providers. The "Trip Booker" domain is relatively low cost (only 1¢ per query), but quite private in that it names itself "Trip Booker" and recommends exactly one hotel brand. Is. In fact, Trip Booker domain provider and this hotel brand have a profitable business relationship. The "Travel Mate" domain is moderately expensive (3¢ per query), but provides a fairly useful response with a large number of results available and a top 5 reasonable list of the ones that are most likely to be of interest. . The "TravelHound" domain is costly (8¢ per query), but users can find a certain number of hotels found, some lists, and by sorting or filtering the lists using an intuitive voice interface. Provides much more useful results with recommendations to help you get much more satisfying results. The "Chee-po-tels" domain is cheap (only 1¢ per query), but its grammar suggests that instead of recognizing the word Paris refers to the French metropolis Imagine a ridiculously small town in the state of Idaho, which is empty.

Regardless of the sort order of domains (pricing model, number of query hits, star selection by VA developers, query response types, etc.), the platform can show results in two sections. The first section, indicated by the sub-heading 146 named "Editor's Pick-Sponsor", shows the one or more domains that the platform provider wants the VA developer to select. Generally, an "Editor's Pick" domain is one sponsored by a domain provider or selected by a platform provider. Because they encourage more use or satisfactory results by virtual assistants. The rest of the domains that can provide information to complete the response to the test query is shown in the section indicated by subheading 147, labeled "Other."

Pricing Model The example used above shows a very simple pricing model with a linear fee of 1 US cent or several US cents per information request. In some embodiments, the cost per request for information that is cheap to source and deliver will be much less than 1 US cent per request. FIG. 15A shows that the price is $. An example of a linear pricing model is 0005 (equal to .05¢ per request).

It is common in the industry to use pricing models where the domain is non-linear with respect to the number of queries that have the highest interpretation score. An example of a non-linear type pricing model is a tiered pricing model. FIG. 15B shows such a pricing model. The cost per request decreases as the number of requests crosses the threshold. Specifically, the cost is $.50 per query for the first 10,000 queries within a month. It is 0010. For 10,000-50,000 requests, the cost is $. It is 0005. For 50,001 to 250,000 requests, the cost is $. 0002. For 250,001 requests and others, the cost is $.50 per request. It is 0001. The number of queries is reset once a month.

In some embodiments, the domain provider sends a monthly bill of information requests provided to the platform provider. In some embodiments, the platform provider purchases credit prior to requesting information. Credit charges are set at graduated pricing intervals depending on the amount purchased. In various embodiments, a user subscribes, an app integrator purchases credit from a platform provider or domain provider, any payee pays by invoice, or any other payment for service payment. There can be various combinations of suitable methods.

FIG. 15C shows a non-linear, expression-based pricing model. Pricing per request is based on formulas rather than graduated intervals of multiple information requests. Although many equations are possible, the equation in FIG. 16 is as follows.

The platform provider or domain provider or both maintain the number of information requests provided for a sliding window over the last 30 days. The price per request is $.per request. The minimum value of 0001+the cost that varies antilogarithmically with the number of requests. This effectively provides volume discounts for frequent users.

Various other non-linear pricing models are possible, limited only by the creativity of the person negotiating the business contract.

CRM
FIG. 16A illustrates an example of a non-transitory computer-readable medium 161 that is a rotating magnetic disk. Data centers typically use magnetic disks to store data and code comprising instructions for server processors. Non-transitory computer-readable medium 161 stores code that comprises instructions that, when executed by one or more computers, cause the computer to perform the method steps described herein. Will let you. Spinning optical disks and other mechanically moving storage media are also possible.

FIG. 16B illustrates an example of a non-transitory computer readable medium 162 that is a flash random access memory (RAM) chip. Data centers typically use flash memory to store data and code for server processors. Mobile devices typically use flash memory to store data and code for a processor in a system-on-chip device. Non-transitory computer readable medium 162 stores code that comprises instructions that, when executed by one or more computers, cause the computer to perform the steps of the methods described herein. Will let you. Other non-moving storage media packaged with leads or solder balls are also possible.

Any type of computer readable medium is suitable for storing code with instructions in accordance with various embodiments.

Server The server stores the database of the domain provided to the VA developer in the platform menu. The server also stores a database of code for grammars associated with the domain. The server also stores a database of pricing models associated with the domain.

FIG. 17A illustrates a rack-mounted server blade multiprocessor server system 170 according to some embodiments. It comprises multiple networked computer processors that execute software in parallel.

FIG. 17B shows a block diagram of the server system 170. It comprises a multi-core cluster of computer processor (CPU) cores 171 and a multi-core cluster of graphics processor (GPU) cores 172. These processors connect to a random access memory (RAM) device 174 via a board level interconnect 173 for storage of program code and data. The server system 170 also includes a network interface 178 to allow the processor to access the Internet. By executing the instructions stored in RAM device 174, CPU 171 and GPU 172 perform the method steps described herein.

SoC
FIG. 18A shows the bottom side of a packaged system-on-chip device 180 having a ball grid array for surface mount soldering to a printed circuit board. Different package shapes and sizes are possible with different chip implementations. A system-on-chip (SoC) device controls many embedded system and IoT device embodiments described herein.

FIG. 18B shows a block diagram of the system on chip 180. It comprises a multi-core cluster of computer processor (CPU) cores 181 and a multi-core cluster of graphics processor (GPU) cores 182. These processors connect to an off-chip dynamic random access memory (DRAM) interface 184 via a network on-chip 183 for storage of volatile programs and data, and a computer to a flash RAM non-transitory computer-readable medium. Connect to flash interface 185 for non-volatile storage of program code. The SoC 180 also has a display interface 186 for displaying a GUI and an I/O interface module 187 for connecting to various I/O interface devices as needed for various peripheral devices. The I/O interface enables sensors (such as touch screen sensors), geolocation receivers, microphones, speakers, Bluetooth peripherals, and USB devices (such as keyboards and mice), among others. The SoC 180 also allows the processor to connect to the Internet via wired or wireless connections such as Wi-Fi, 3G, 4G Long Term Evolution (LTE), 5G, and other wireless interface standard wireless and Ethernet connection hardware. A network interface 188 is provided to allow access. CPU 181 and GPU 182 are described herein by executing instructions stored in a RAM device via interface 184 or by executing instructions stored in a flash device via interface 185. Perform the method steps.

Still other considerations One of ordinary skill in the art will recognize many variations and modifications. Variations and modifications include any relevant combination of the disclosed features.

Various embodiments are methods that use human or machine behavior or combined human and machine behavior. Embodiments of the method are complete where most configuration steps are performed anywhere in the world. Some embodiments are one or more non-transitory computer-readable media arranged to store such instructions for the methods described herein. Any machine that holds a non-transitory computer readable medium having any of the required code is a perfect embodiment. Some embodiments are arranged to store a physical device, such as a semiconductor chip, a hardware description language representation of the logical or functional behavior of such device, and a hardware description language representation of such. And one or more non-transitory computer-readable media.

The description herein, which describes principles, features and embodiments, includes both structural and functional equivalents thereof. The elements described herein as coupled have the effective relationship that can be achieved by a direct or indirect connection with one or more other intervening elements.

The examples shown and described use specific spoken language. Various embodiments operate in other languages or combinations of languages as well. The examples shown and described use specific knowledge domains. Various embodiments operate similarly with other domains or combinations of domains.

Some embodiments are screenless, such as earpieces without a display screen. Some embodiments are stationary, such as vending machines. Some embodiments are mobile, such as a car. Some embodiments are portable, such as a cell phone. Some embodiments include a manual interface such as a keyboard or touch screen. Some embodiments include a neural interface that uses human thinking as a form of natural language expression.

Claims (17)

A platform for configuring a natural language virtual assistant,
A means for a developer of a natural language virtual assistant to enter a test query,
Means for filtering a set of multiple domains into a subset of multiple competent domains, each of the plurality of competent domains capable of interpreting the test query, the platform further,
A means for presenting a list of domains, said means comprising:
Presenting the list of domains by searching each of the plurality of competent domains in a number of promoted domains, each promoted domain having an associated pricing model, the means comprising: further,
Presenting the list of domains by discovering at least one competent domain in the number of promoted domains,
Presenting the list of domains by presenting a list of competent domains including the promoted domain, the promoted domain being highlighted in the list,
A platform that allows app developers to identify promoted domains for inclusion in the natural language virtual assistant. Means for receiving user queries,
Means for interpreting the user query according to the promoted domain and providing a response;
Means for calculating a charge associated with providing the response, the charge being calculated according to the pricing model associated with the promoted domain. platform. A method for integrating a virtual assistant domain into an app,
Receiving a test query from the app integrator,
Interpreting the test query in multiple domains, some domains charging a cost to the app integrator, the method further comprising:
Indicate to the app developer a subset of domains that can interpret the test query, and for each listed domain, the cost of responding to the test query using the listed domain to the app integrator. A method comprising the steps of: Receiving a second test query from the app integrator,
Interpreting the second test query in the plurality of domains,
The method of claim 3, further comprising calculating the cost of responding to the test query and the second test query using each domain according to a non-linear pricing model. A method for integrating a virtual assistant domain into an app,
Receiving a number of test queries from the app integrator,
Interpreting each test query from the multiple test queries in each of the plurality of domains to determine the domain having the highest interpretation score for each test query, at least some of the plurality of domains. The domain charges the app integrator a cost according to a pricing model, the method further comprising:
Applying the pricing model to the query for which a cost-charging domain had the highest interpretation score to calculate a total cost for the multiple test queries. The method of claim 5, further comprising calculating a cost per domain. The method of claim 5, wherein the pricing model for a particular domain is non-linear with respect to the number of queries that the particular domain has the highest interpretation score. A platform for configuring domains for inclusion in a natural language virtual assistant,
A means for entering a large number of test queries,
Means for entering a selection of domains for inclusion in the natural language virtual assistant;
A display view showing the app developer which domains have been selected for inclusion in the natural language virtual assistant from the menu of domains;
A display view showing a cost component of the total cost that the natural language virtual assistant will be charged to respond to the number of test queries with the domain selected for inclusion in the natural language virtual assistant. Each cost component is due to the domain selected from the domain menu,
The app developer is allowed to configure a selection of domain combinations for inclusion in a natural language assistant such that the total cost of responding to the test query is within a desired budget. platform. 9. The platform of claim 8, further comprising a display view for an app developer to see a histogram showing how much the test query is answered by each domain selected for inclusion in the natural language virtual assistant. .. The display view indicating to an app developer which domains have been selected for inclusion in the natural language virtual assistant from the menu of domains displays at least one promoted domain more prominently than other domains; The platform according to claim 8. 9. The cost component of at least one domain varies non-linearly with respect to the number of test queries that the domain will respond to with the domain selected for inclusion in the natural language virtual assistant. Platforms listed. A platform for configuring a virtual assistant, the system comprising:
VA (Virtual Assistant) A server-hosted platform for developers to select domains for inclusion in a virtual assistant,
A syntax for requesting a number of domains and domain providers available for selection,
Means for exporting code that, when executed by a processor in an embedded system, causes the interpreter function to respond to a query for a local domain, independent of the platform server. Means for receiving a set of test queries,
13. The platform of claim 12, further comprising a display view enabled to display the cost of responding to the queries in the set of test queries for a given domain selection. 13. The platform of claim 12, further comprising a number of pricing models associated with the domains available for selection. Method,
The platform receives the domain,
The platform generating a first domain grammar that identifies grammatical phrases in the received domain;
The platform associating the domain grammar with the domain before a VA developer selects the domain for inclusion in a virtual assistant. 16. The method of claim 15, further comprising associating a pricing model that identifies a cost per query for using the domain with the domain. Receiving a set of test queries from the VA developer,
Receiving a selection of domains for inclusion in a virtual assistant from the VA developer,
16. Generating a display view showing the cost of responding to the queries in the set with the selection of the domain.

Images