WO2017059500A1 - Structures et méthodologies configurées pour permettre une intégration simplifiée d'une fonctionnalité de traitement de langage naturel avec un ou plusieurs environnements d'interface utilisateur, comprenant un processus d'apprentissage assisté - Google Patents
Structures et méthodologies configurées pour permettre une intégration simplifiée d'une fonctionnalité de traitement de langage naturel avec un ou plusieurs environnements d'interface utilisateur, comprenant un processus d'apprentissage assisté Download PDFInfo
- Publication number
- WO2017059500A1 WO2017059500A1 PCT/AU2016/050950 AU2016050950W WO2017059500A1 WO 2017059500 A1 WO2017059500 A1 WO 2017059500A1 AU 2016050950 W AU2016050950 W AU 2016050950W WO 2017059500 A1 WO2017059500 A1 WO 2017059500A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- natural language
- template
- data
- user
- action
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2452—Query translation
- G06F16/24522—Translation of natural language queries to structured queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
- G06F40/35—Discourse or dialogue representation
Definitions
- the present invention relates to frameworks and methodologies configured to enable streamlined integration of natural language processing functionality with one or more of user interface environments.
- Embodiments of the invention have been particularly developed for enabling providers of client-facing technology (such as websites, software applications, and controllable machinery) to implement a natural language processing functionality (including, but not limited to voice control), in a streamlined manner. While some embodiments will be described herein with particular reference to that application, it will be appreciated that the invention is not limited to such a field of use, and is applicable in broader contexts.
- One embodiment provides a computer implemented method for providing natural language processing functionality, the method including:
- the template management interface is configured to enable an administrator user to: (i) view data representative of the natural language based request; and (ii) define an instruction template and action template in relation to the natural language based request.
- One embodiment provides a computer implemented method wherein the template management interface is configured to:
- each instruction template provide data representative of one or more natural language based requests associated with the instruction template
- One embodiment provides a computer implemented method wherein enabling user-definition and/or modification of an action template includes: (i) defining a parametrized portion of one or more of the natural language based requests associated with the instruction template; and (ii) defining a corresponding parametrized portion of the action template.
- One embodiment provides a computer implemented method wherein the parametrized portion of the action template is a portion of HTTP data.
- One embodiment provides a computer implemented method wherein enabling user-definition and/or modification of an action template includes enabling the user to selectively define (i) a text-based response; or (ii) a HTTP based response.
- One embodiment provides a computer implemented method wherein enabling user-definition and/or modification of an action template includes enabling the user to selectively define (i) a text-based response; (ii) a HTTP link based response; or (iii) a HTTP post/put/delete (or other) based response.
- One embodiment provides a computer implemented method wherein enabling user-definition and/or modification of an action template includes enabling the user to selectively define a complex action template.
- One embodiment provides a computer implemented method wherein the user interface is configured to enable a user to: (i) select a natural language based request associated with a first instruction template; and (ii) re-associate that natural language-based request with a second template.
- One embodiment provides a computer implemented method wherein the second template is a new template.
- One embodiment provides a computer implemented method wherein the natural language based request is receivable in either text form or speech form.
- One embodiment provides a computer implemented method for providing natural language processing functionality to a plurality of technology environments, the method including:
- each instruction template is associated with a respective instruction template
- One embodiment provides a computer implemented method wherein the method of matching includes the consideration of any one or more of the following variations: grammatical variations; synonymous variations; hyponyms of hypernyms; hyponymous, hypernymous, meronymous, holonymous variations; instantiable variations; and variations based on liberal matching of adjectives and adverbs.
- One embodiment provides a computer implemented method wherein the matching includes a scoring process, wherein the scoring process is affected by the presence of one or more variations.
- One embodiment provides a computer implemented method wherein the degree to which the score is effected in response to the presence of one or more variations is based on a learning algorithm.
- One embodiment provides a computer implemented method wherein a plurality of actions are chained together.
- One embodiment provides a computer implemented method wherein one or more of the instruction templates are parametrised.
- One embodiment provides a computer implemented method wherein the natural language processing engine is configured to, in the case that the matched instruction template is a parametrised instruction template: process the natural language based request thereby to determine parameter values for the parametrised instruction template.
- One embodiment provides a computer implemented method wherein defining an executable instance for the identified action template associated with a parametrised instruction template includes applying the determined parameter values to corresponding parametrised portions of the identified action template.
- One embodiment provides a computer implemented method wherein each parameter has a parameter type, and wherein the parameter type is used in the matching.
- One embodiment provides a computer implemented method wherein queries which are imperfectly aligned with instruction templates are handled.
- One embodiment provides a computer implemented method wherein the technology environments include: (i) a plurality of websites; (ii) a plurality of software applications; and (iii) a plurality of controllable machines.
- One embodiment provides a computer implemented method including providing to the client device a signal configured to cause the client device to request further information from the user to support the natural language based request.
- One embodiment provides a computer implemented method wherein the executable instance of the action template defines a series of consecutive processes configured to be performed by the client device thereby to satisfy the natural language based request.
- One embodiment provides a computer implemented method wherein the data representative of the natural language request includes: (i) request data; and (ii) contextual data.
- One embodiment provides a computer implemented method wherein the request data includes either: (i) speech data received by the client device, which is converted to text by a speech recognition associated with the natural language processing engine; or (ii) text data inputted at the client device.
- One embodiment provides a computer implemented method wherein the contextual data includes, in the case of a technology environment in the form of a software application, one or more of: localisation information, available objects and any objects which are displayed to the user, the current state of such objects, etc.
- One embodiment provides a computer implemented method wherein the contextual data includes, in the case of a controllable machine, one or more of: position information, information coming from sensors, current actions in progress, world information such as location of objects, size and weight of objects, etc.
- One embodiment provides a computer implemented method wherein, for a given technology environment, one or more of the instruction templates and/or action templates is defined, in whole or in part, by an automated process.
- One embodiment provides a computer implemented method wherein the automated process includes analysing user interface artefacts thereby to identify instructions that are configured to be provided directly via the user interface.
- One embodiment provides a computer implemented method including, for each technology environment as a configuration step, performing an automated process in respect of one or more databases associated with the technology environment thereby to extract parameter contextual data.
- One embodiment provides a computer implemented method for providing natural language processing functionality, the method including:
- an instruction template which includes parametrized portions based on the identified portions of the data representative of the natural language based request ;
- an associated action template which includes parametrized portions based on the identified portions of the data associated with the manually defined response
- One embodiment provides a computer implemented method wherein the step of providing an interface that is configured to enable manual defining of a response to the natural language based request is performed only in the case that the natural language based request is not matched to an existing instruction template. [0067] One embodiment provides a computer implemented method wherein performing a comparison of: (i) the data representative of the natural language based request; and (ii) data associated with the manually defined response includes:
- One embodiment provides a computer implemented method the discrete grammatical artefacts include datives, accusatives, direct objects, and indirect objects.
- One embodiment provides a computer implemented method wherein performing a comparison of: (i) the data representative of the natural language based request; and (ii) data associated with the manually defined response additionally includes:
- One embodiment provides a computer implemented method wherein the data commonalities are identified in a URL portion of a web page associated with the manually defined response.
- One embodiment provides a computer implemented method wherein the data commonalities are identified in context of a web page associated with the manually defined response.
- One embodiment provides a computer implemented method wherein context of a web page associated with the manually defined response includes a manually defined field value.
- One embodiment provides a computer implemented method wherein identifying data commonalities includes normalizing a data portion from either or both of the data representative of the natural language based request and the data associated with the manually defined response based on a predicted data type.
- One embodiment provides a computer implemented method wherein, the instruction template includes data defining a plurality of combinations between : identifiable grammatical artefact types; and identifiable data types.
- One embodiment provides a computer program product for performing a method as described herein.
- One embodiment provides a non-transitory carrier medium for carrying computer executable code that, when executed on a processor, causes the processor to perform a method as described herein.
- One embodiment provides a system configured for performing a method as described herein.
- any one of the terms comprising, comprised of or which comprises is an open term that means including at least the elements/features that follow, but not excluding others.
- the term comprising, when used in the claims should not be interpreted as being limitative to the means or elements or steps listed thereafter.
- the scope of the expression a device comprising A and B should not be limited to devices consisting only of elements A and B.
- Any one of the terms including or which includes or that includes as used herein is also an open term that also means including at least the elements/features that follow the term, but not excluding others. Thus, including is synonymous with and means comprising.
- exemplary is used in the sense of providing examples, as opposed to indicating quality. That is, an "exemplary embodiment” is an embodiment provided as an example, as opposed to necessarily being an embodiment of exemplary quality.
- FIG. 1 schematically illustrates a framework according to one embodiment.
- FIG. 2 illustrates a method according to one embodiment.
- FIG. 3 illustrates a client-server framework leveraged by some embodiments.
- FIG. 4A to 4E illustrate exemplary partial screenshots according to an embodiment.
- Fig. 5 illustrates a method according to one embodiment.
- the present invention relates to frameworks and methodologies configured to enable streamlined integration of natural language processing functionality with user interface environments.
- embodiments provide technology that enable integration of a natural language query processing functionality into existing use interface environments, such as websites (although websites are used as an illustrative example below, the technology is applicable to a wide range of other user interface environments).
- the technology allows a query to be submitted in natural language form (for example via text input and/or voice input), and for that query to be processed thereby to trigger an action that is specific to the user interface environment.
- Some embodiments described below relate to technology whereby a learning interface is configured to enable automated and/or semi-automated generation of computer coded rules and logic that is able to perform defined actions based on analysis of natural language queries.
- a learning interface is configured to enable automated and/or semi-automated generation of computer coded rules and logic that is able to perform defined actions based on analysis of natural language queries.
- such embodiments make use of human-assisted learning, whereby a human operator defines an appropriate response to a first natural language query, and the learning interface induces from that human-defined response (either automatically or semi-automatically) a template that allows subsequent automated responding to a second natural language query with different parameters (but identifiable commonality in terms of particular structural elements).
- Some embodiments provide technology configured to enable providers of client-facing technology (such as websites, software applications, and controllable machinery) to implement a natural language processing functionality, such as voice control, in a streamlined manner. More specifically, in some embodiments a natural language processing engine is made available via a cloud service. The cloud service maintains access to, for each client-facing technology environment, a respective data set including predefined parameterised instructions and actions specific to the relevant environment. In this manner, only a relatively small amount of environment-specific data is required for each individual environment, whereas the underlying natural language processing engine (and associated technology) scales commonly across the plurality of environments.
- client-facing technology such as websites, software applications, and controllable machinery
- FIG. 1 An exemplary framework is illustrated in FIG. 1 .
- a natural language command processing server 100 is configured to interact with instances of a plurality of technology environments.
- FIG. 1 illustrates three exemplary technology environments:
- a website 130 which is executed at a plurality of client devices via website rendering instances 1 -n. It will be appreciated that each individual website has a unique set of functions that it is configured to provide. These may be somewhat generic in function (for example navigation and the like), but unique in terms of the specific website (for example by reference to menu item names and the like).
- a controllable equipment environment which is executed by way of a plurality of individual physical machines 1 -n.
- the technology environment is defined by reference to machine hardware and software, and in some cases user interface software that executes separately of the machine.
- a software application instances 1 -n of which execute at respective client terminals (or, in the case of cloud hosted software, user interfaces of which execute at the client terminals).
- each category is illustrated, in practice there are a plurality of environments of each category.
- multiple software developers and/or website administrators adapt the technology environments they provide for compatibility with server 100.
- the technology environments may include several hundred distinct software applications, and for each software application there may be hundreds (or thousands) of individual executing instances simultaneously.
- one or more intermediate servers are used to assist in coordinating requests from groups of client devices (for example a dedicated server to receive and optionally partially process requests from a common software application).
- these technology environments are configured to interact with server 100 thereby to enable the provision of a natural language based request functionality.
- this functionality may include either or both of: a speech-based request interface (whereby a user of a client device that presents an instance of the technology environment provides a request verbally) and a text-based interface (whereby a user of a client device that presents an instance of the technology environment provides a request by typing that request into a user interface object).
- the client device Upon a natural language based request being received at a client device in one of the supported technology environments, the client device communicates data representative of the natural language based request to server 100 (for example audio data and/or text data), where it is received by input modules 101 .
- server 100 for example audio data and/or text data
- Server 100 executes a natural language based request processing engine 102.
- Engine 102 is available to all of the supported technology environments, and makes use of a common natural language processing engine 103, text-to-speech engine 107, and global learning data 108 (comprising data developed over time thereby to tune the effectiveness of engines 102, 1 03 and/or 107).
- Server 100 is configured to maintain access to data that defines, for each of the plurality of technology environments:
- a given action template may include a plurality of steps and/or reference one or more further action templates
- a supported environment data store 1 10 which maintains instruction/action templates 1 1 1 for each of a plurality of n supported technology environments.
- data store 1 10 additionally maintains other contextual data for each supported technology environment, which may include parameter field types, context to parameter meanings, and the like.
- environment-specific learning data 1 13 enabling tuning of natural language processing both at a global cross-environment level via data 108, and at an environment- specific level via data 1 13.
- FIG. 2 illustrates a method performed by the framework of FIG. 1 .
- Block 201 represents a process including receiving, from a client device, data representative of a natural language based request submitted by a user of the client device. This is followed, in the case of a speech command, by speech-to- text conversion at block 202.
- Block 203 represents a process including identifying the technology environment being executed at the client device.
- the data received at 201 in some cases includes or is associated with information identifying the client device and the technology environment executing at the client device.
- Block 204 then includes executing a natural language processing engine, which is available across of plurality of technology environments, thereby to match the natural language based request to one of the plurality of instruction templates associated with the identified one of the technology environments.
- the request is matched to a plurality of instructions (for example where the request combines functionalities provided by discrete instructions).
- Block 205 represents, based on the matching, identifying the action template associated with the matched instruction template. Then, block 206 represents a process including defining an executable instance for the identified action template.
- one or more of the instruction templates are parametrised (that is, they are defined as a template including a plurality of parametric portions that are configured to be populated with actual parameter values). That is, they are defined by reference to functional components, and portions that are parametric.
- the method includes processing the natural language based request thereby to determine parameter values for the parametrised instruction template.
- defining executable instances for the identified action template associated with a parametrised instruction template includes applying the determined parameter values to corresponding parametrised portions of the identified action template. In FIG. 1 , this is performed using an action construction engine 104.
- the literal value of the parametric portions is typically not of importance to the function of server 100.
- attributes of the parametric portion are in some cases relevant to selection instructions/action templates. For instance, where a parameter is a name, the name itself is typically not relevant to the matching process, but the fact that the parameter is of the type "person" is of relevance.
- parameter types have additional downstream bearing in terms of defining an executable instance of an action.
- an action may be defined that replaces the literal value "Bruce” with an alternate term based on what is an acceptable parameter for the defined action template being used. For example, “Bruce”, “him”, “the tall cook” and “somebody” are all able to be matched with person-type parameters.
- contextual data is used to assist in resolving ambiguities with respect to parameters of known types.
- types may have properties, which may or may not be inherent to the object (such as having a handle, or such as being within-reach).
- Block 207 represents a process including transmitting data representative of the executable instances for the identified action template to the client device, thereby to cause the client device to perform functions based on the executable instances for the identified action template. In FIG. 1 this is achieved via action output modules 106.
- a given action template defines a series of consecutive processes configured to be performed by the client device thereby to satisfy the natural language based request.
- engine 1 02 determines that additional information is required to enable matching to an instruction template and/or constriction of an executable action.
- contact modules 105 are configured to enable two-way communication between server 100 and a client device thereby to request (and obtain) further information from the user to support the natural language based request.
- server 100 defines a question which is intended to seek clarification and/or input from a user of the client device.
- the data representative of the natural language request includes: (i) request data; and (ii) contextual data.
- the request data includes either: (i) speech data received by the client device, which is converted to text by a speech recognition associated with the natural language processing engine; or (ii) text data inputted at the client device.
- the contextual data includes, in the case of a technology environment in the form of a software application, one or more of: localisation information, available objects and any objects which are displayed to the user, the current state of such objects, etc.
- the contextual data includes one or more of: position information, information coming from sensors, current actions in progress, world information such as location of objects, size and weight of objects, etc.
- contextual data is used in the context of a reasoning process, which supports the natural language processing.
- the natural language processing results in an ambiguity - which is not able to be resolved without recourse to reasoning. This may relate to a parameter: a request "pick up that thing” is inherently unclear as to what is meant by "that thing”. Alternately, there may be two or more possible interpretations for a given word/phrase/concept.
- a reasoning engine is configured to apply a set of rules thereby to seek resolution of ambiguity based on static and/or the contextual data.
- Rules defined for execution by a reasoning engine may include either or both of (i) rules defined for a specific technology environment; (ii) rules that are common across the plurality of technology environments.
- the reasoning engine is implemented as a first stage of ambiguity resolution, with two-way communication with a user of the client device via contact modules 1 05 being a second stage that is utilised in the case that automated resolution via the reasoning engine is unsuccessful (or provides a result of below a threshold level of accuracy probability).
- the user query is submittable in either written or spoken form (and in some cases a user interface environment provides for both forms of input).
- the user query arrives in spoken form, it is preferably converted into a textual form, using a speech recognition engine.
- the speech recognition engine is configured to define multiple textual queries (or query portions), which are preferably each associated with probability data. For example, a given spoken word may be converted into two variant text forms, one being associated with an 80% probability and the other with a 60% probability. Making such variants available to downstream steps (which have increased knowledge of environment-specific terminology) leads to performance advantages.
- this preferably includes information representative of the state of the local environment, such as a history of recent user queries (for anaphora resolution and for subtopic identification), and restrictions on available actions. In some embodiments it includes:
- the contextual information also preferably includes (or is configured to reference) meta information describing relationships between objects, linguistic terminology, etc. For example, ontologies and other data sources representing facts and rules concerning real world objects.
- each technology environment is configured to be compatible with server 100 by a process including defining action and instruction templates. These are collectively referred to as "rule templates”.
- the instruction represents something which is searchable; the action is something which can be executed in some fashion to achieve some outcome.
- template refers to the fact that part of instruction may be specified in terms of a parameter which is allowed to vary. This allows many (sometimes infinitely many) related instructions to be represented succinctly. For example, "Move ⁇ amount> metres forward” might represent a movement template which can be matched against “Move 5.3 metres forward", but also with "Move 21 metres forward”.
- the corresponding action will typically be parameterised in the same way so that the information which is realised by the query may be utilised in the execution of the action. For example, a robot may be able to make use of an action described as a program (where amount gets instantiated to the corresponding value in the user query):
- the user query gets parsed to reveal the structure of the query.
- the parse structure is analysed to discover the semantic roles of various components of the query. (Optional - perhaps just the structure is used, or even direct matching will be used later).
- Hyponymous, hypernymous, meronymous, holonymous variations e.g. "Turn on the hose” - "Turn on the tap of the hose” - "Turn on the tap”
- the quality of the match may be determined with reference to various resources such as ontologies. These resources is in some embodiments general purpose, or may be specialised to the application, or even specialised to the context of the application, as supplied in the accompanying contextual information.
- the relative importance of the various factors determining the quality of the match may be determined manually, or may be estimated using some machine learning technique.
- the best match is determined.
- This "best match” may be non-unique if the best few matches have scores which are similar.
- the query is deemed to be ambiguous, and may be returned ambiguously, or more information may be sought from the user in order to resolve the ambiguity.
- Matches may be made on subparts of the query. For example, a user query, "Display sales figures for March” might match “Show the results” ("Display ... ”) and “values for a period” ("... sales figures for March”). This might result in two or more actions being chained together. Of particular note, are actions associated with noun phrases, which will often be find()-type actions. These find()-type actions then become inputs to other actions, as noun phrases are often arguments of verb phrases.
- the instruction-action template may have parts which represent parameters of the action.
- the template In order to instantiate the template:
- the user query and the instruction template are aligned. This is done by matching the (semantically) corresponding parts of the query with the instruction template. For example, this could be done by parsing both, recognising the semantic roles that each of the constituent parts play in the parse, and aligning common parts. Frame/slot representations may be useful intermediate structures for making such an alignment, but this could be done working more directly with the parse trees for example.
- the parameters of the template are grounded by copying the values in their corresponding structure from the user query.
- the instruction template may be represented using English words which represent an instance of the way in which the corresponding action template may be referenced. For example, "Lift up a mug.”
- an example request is a convenient form of representation. It has the advantages that it is easy to understand and easy to create and modify for non-specialists who may be responsible for the collection of rule templates. Whilst such an instruction may be able to deal with a large number of different mugs, it may prove even more useful to generalise it further.
- Action templates communicate the user query in a form in which the software which performs the action can understand. What is the most appropriate representation is largely a question of what makes sense to the client software. SQL, XML, JSON, executable code, references to menus or functions in the client software may all be appropriate representations.
- server code For example, a phrase such as "the pink cup” may be disambiguated between multiple such cups available in the context by considering the distance between the speaker and each cup, or the distance between the actor (i.e. the robot) and each cup - the closer cup normally would have greater salience.
- server code For example, a phrase such as "the pink cup” may be disambiguated between multiple such cups available in the context by considering the distance between the speaker and each cup, or the distance between the actor (i.e. the robot) and each cup - the closer cup normally would have greater salience.
- this calculation is best performed by reasoning on the service as part of its ambiguity resolution.
- the server code may be directly included in each action which requires it, or may be contained in the contextual information which accompanies the request, with only a reference to the code contained in the action itself.
- the natural language processing service may choose to hobble the generality of the execution of client code for security, stability, privacy and sandboxing considerations.
- the action is described in terms of three other actions: it is convenient to express the action in natural language processing, or however else the instructions are represented.
- the action may have components which refer to other actions, expressed in natural language processing, and other components which are represented in some other fashion, as described earlier. Note that some of the actions which are described elsewhere may be further described in terms of other actions. For example, “Pick up the thing” might match with “Pick up something", which is in some embodiments defined as another 3-step action:
- Actions may also have optional parts.
- an action associated with getting a robot to walk may have a speed parameter which may be optionally expressed.
- Such optional parts may be handled in server code, or the representation of the action template might may this optionality explicit, for instance defining values to be used in the case that the corresponding optional part is missing.
- Actions can be anything which can be executed to achieve some outcome. They may result in a different presentation of some data, they may add, delete or modify some data. They may result in a motor being activated. They may change a mode of operation of the application. They add, delete or modify the application's understanding of the world (e.g. giving names to objects in the world). They may even add, delete or modify the available actions which the natural language processing service may operate over. [00150] There may be special types of actions which do not require a representation like those described so far. For example, "searchQ" and "helpQ" type functions may be special-cased.
- search() Return a list of all objects which meet some criteria.
- search is relevant here in a couple of ways: the search facility may be most easily specified using natural language processing ("search”, “find”, “show me”, 7) and the search criteria may also be most easily specified using natural language processing.
- Many search facilities have complicated user interfaces to allow the user to indicate restrictions on the search ("whole word”, “at the start”, “within the current selection”, etc); these can often be readily expressed in natural language processing.
- the search facility is greatly improved particularly in non-text applications, where the entities in the application may be hard to find. For example, “find the best salesman for last month” or asking a robot "what cutlery can you see?" or "show me all communication I've had with Mr. Benson in the last week.”
- a natural language processing service may be able to understand entities, and their relationship between each other via supplied ontologies or other reasoning resources. In the case of categorising the results, type and other knowledge can again be useful.
- Search is also relevant as it may be the input to another part of a natural language processing query (see discussion on action chaining earlier).
- the presentation of the results would nonetheless require that the client application be informed of the results and asked to present them.
- One application might choose to present them as a table on the right-hand side of the application; a robot might speak the results or even point to each of the objects in turn.
- search is just a special type of rule, but one where a lot of the details may be assumed rather than specified. Note that as it is a type of rule, it is possible to have multiple search rules available, each which are applicable for different types of objects being searched, for example, so that they are presented in different ways, or so that the search orders the results differently etc.
- the collection of rules is the glue which links the capabilities of the client application with instructions; the natural language processing service generalises the invokability of these rules.
- These rules may need to be checked so that they are properly understood by the natural language processing service, and for consistency.
- the natural language processing service might produce example natural language processing queries to simulate what the user might have requested, in order to help the author of the rules understand possible actions which is in some embodiments undertaken if the user were to express this.
- the natural language processing service could try to generate examples which it found ambiguous to tease out the most appropriate action to undertake in such a situation.
- Some embodiments provide a method wherein, for a given technology environment, one or more of the instruction templates and/or action templates is defined, in whole or in part, by an automated process.
- the automated process includes analysing user interface artefacts thereby to identify instructions that are configured to be provided directly via the user interface.
- the rules may be automatically generated. This is particularly the case where the client application is particularly structured in some way. For example, consider an application which interacts heavily with a database. This database may be automatically inspected to determine the nature and relationships between tables and fields in the database. In some cases, this may be able to be translated into rules, ready for the natural language processing service. Other examples of automatic extraction of rules may include automatic analysis of the user-interface elements in a software (menus, and buttons on user interfaces often correspond to actions, and the way they are labelled correspond to associated instructions). This is in some embodiments achieved by scanning the software code which represents the application, looking for such objects and their associated actions.
- the natural language processing service only requires the output of this automatic extraction. This means that such extraction may be offered such that it is run behind a firewall (for instance on the client's computer) so that the natural language processing service does not have access to associated databases or software code. The client might then be able to manually inspect the extracted rules and other associated information, before submitting it to the natural language processing service. In this way, the client's privacy and security is respected.
- the client's software were so arranged, it is in some embodiments possible to get the software to learn from certain situations.
- a robot In the case of a robot equipped with object tracking and computer vision, for example, a robot is in some embodiments told to watch and learn an action. By associating the action with an instruction, a new rule could be created.
- a desktop software application is in some embodiments able to be put in a "record" mode, where a sequence of actions are able to be combined; if those actions are then associated with an instruction, this rule could then be added to the rules available for the natural language processing service to consider.
- learnt rules could be created by the authors of the software, or potentially by users of the software. Such rules could potentially be shared between different users, or made available only to the user who created the rule.
- actions may be restricted in their application either by the situation or because those actions are restricted for that user. Such restrictions can be modelled in two ways. Actions may be hidden from the user in a particular situation if the user does not have adequate permission, and this hiding may be done in a way in which the rule appears to be completely absent from consideration. Alternatively, actions may be restricted from the user in a particular situation such that the user query may still be matched against the restricted actions (for example, blocking the applicability of other matching actions which have a lower score) and optionally, the user may be notified that he or she does not have permission to perform that action at this time.
- the permission information is in some embodiments attached to each action, with the user's identity being passed in the contextual information which accompanies the user request. In some cases, it may be desirable to use some strategy such as signing the request to prevent malicious forgeries from accessing parts of the application which are restricted.
- the natural language processing service may provide feedback to the user to help resolve ambiguities in the natural language processing service's understanding and applicability of the user query. But feedback may be an inherent requirement of the fulfilment of the action.
- "wizards" in some software applications which guide the user through a series of steps to elicit the information required in order to complete some task.
- a user query might have requested some task which is best matched to a wizard in the software: however some of the information which the user provides in their query is in some embodiments not applicable until later steps of the wizard. In some cases, this is best handled by having the natural language processing service interact with the user, effectively creating a dialogue which achieves the same end as the wizard in the extraction of the relevant inputs required to complete some task.
- the natural language processing service could be configured in a way so as to:
- Exemplary software Many software applications could benefit by adding a natural language processing interface. However, the financial and time cost as well as the risks involved in developing such software may be prohibitive if done for each software product, and would require on-going development to stay cutting edge. However, whilst a natural language processing service is in some embodiments able to identify what action is required, there are still issues involving getting this action to be performed, fire-walling the interface between the software and the natural language processing service (so that clients' data is not given to the service) and yet still being context aware in order to resolve natural language processing ambiguities.
- some embodiments relate to technology whereby a learning interface is configured to enable automated and/or semi-automated generation of computer coded rules and logic that is able to perform defined actions based on analysis of natural language queries.
- a learning interface is configured to enable automated and/or semi-automated generation of computer coded rules and logic that is able to perform defined actions based on analysis of natural language queries.
- such embodiments make use of human-assisted learning, whereby a human operator defines an appropriate response to a first natural language query, and the learning interface induces from that human-defined response (either automatically or semi-automatically) a template that allows subsequent automated responding to a second natural language query with different parameters (but identifiable commonality in terms of particular structural elements).
- Some embodiments provide computer implemented frameworks and methods for providing natural language processing functionality, which implement an assisted learning process as described below. This may be provided via a framework which is available for integration across a plurality of separate implementation environments, as described further above (for example providing access to global learning advantages and the like). In some embodiments it is integrated locally at a single implementation environment.
- this method (and associated software platforms) provide a simplified and streamlined mechanism to enable construction of instruction and action templates for a given implementation environment. More specifically, some embodiments provide a software platform that tracks and records natural language based requests submitted via the implementation environment, and enables a user to associate those with action templates (for example newly created action templates). At a practical level, this enables one or more of the following:
- a user to generate instruction templates by, in effect, reviewing and responding to queries submitted via a natural language query interface (which become associated with instruction templates), and defining responses to those questions (which become action templates).
- the responses may include text- based responses (and/or images, videos, and the like), HTTP-based responses, and more complex responses.
- This may be performed in a "live" mode, whereby a back-end service provider manually generates a query response (where an existing instruction/action template pair is not able to be defined) and provides that to the query-submitting user, and/or a non-live mode whereby the manual generation of the query response is performed at a later time.
- a user to quickly generate instruction templates by, in effect, asking a plurality of common/relevant questions (which become associated with instruction templates), and defining responses to those questions (which become action templates).
- the responses may include text-based responses, HTTP-based responses, and more complex responses.
- the technology includes maintaining, for a given implementation environment, access to data that defines: (i) a plurality of instruction templates; and (ii) a plurality of action templates, wherein each instruction template is associated with a respective action template.
- data defines: (i) a plurality of instruction templates; and (ii) a plurality of action templates, wherein each instruction template is associated with a respective action template.
- the collection of instruction templates is built up based on receipt of natural language based requests. For example, each time a natural language based request is received and processed, it is either associated with an existing instruction template (i.e. based on natural language processing based matching), or flagged for potential generation of a new instruction template via a template management interface.
- the method includes receiving, from a client device, data representative of a natural language based request submitted by a user of the client device.
- This may be a "typical" client device that accesses the implementation environment and provides the request in that context, or an administrator client device that provides the request in the context of the template management interface.
- the method then includes executing a natural language processing engine, thereby to attempt to match the natural language based request to one of the plurality of instruction templates. As a result of this, the request is either: (i) matched; or (ii) unable to be matched.
- the associated action template is able to be executed (for example by being returned to the calling application for execution).
- a new instruction template is defined (in draft form, and typically with parameter identification), and a notification generated to inform one or more administrator users that a new instruction template requires attention (e.g. in the sense of defining a new action template) via the template management interface.
- an administrator user is available to review and define an action for the new instruction template substantially immediately (in which case the user receives a response to his/her request, albeit via manual intervention, and that response is used subsequently as a basis for a new action template).
- the method includes making data representative of the natural language based request available in the template management interface.
- the template management interface is configured to enable an administrator user to: (i) view data representative of the natural language based request; and (ii) define an instruction template and action template in relation to the natural language based request.
- an automated process is performed thereby to autonomously define an instruction template and action template in relation to the natural language based request. This automated process is in some cases supplemented by manual input (for example in the context of a "review and revise" approach, by which a user inspects and optionally adjusts an autonomously defined instruction/action template pair).
- FIG. 4A illustrates an exemplary partial screenshot from a template management interface according to one embodiment.
- This shows a plurality of instruction templates, each described in terms of a primary natural language based request associated with that instruction template.
- the requests are referred to here as "questions”.
- Each instruction is associated with a count of the number of times the request has been invoked (see the right hand column), and is able to be expanded to reveal the set of individual natural language based requests that were matched to the instruction template (as shown in FIG. 4E).
- FIG. 4B illustrates another exemplary partial screenshot from a template management interface according to one embodiment. This shows an "edit answer” interface.
- a user selects an instruction template ("question") from the list in FIG. 4A, and is enabled to define an action template (an “answer").
- the action template is able to take any of the following forms:
- a HTTP response which causes the accessing of data via HTTP (for example to cause a page redirect, or rendering in an on-page object).
- a HTTP (post) response which causes, for example, a back-end functionality (for example updating a database, automatically completing a web form, or the like).
- More complex forms of action templates may also be defined in further embodiments, for example using approaches described further above.
- Other forms, such as charts and tables are made available in some embodiments, as a convenient manner for defining an action template without needing to define HTTP page content to provide dynamic functionality.
- the user interface provides an administrator user with functionality to define an "answer" by: accessing a web page that provides and answer to the relevant query, and inputting into that page data relevant to the query (or, in some embodiments, accessing a web page that is a result of corresponding to the query), and causing that web page to be delivered to the request-submitting user's terminal with any associated additional data. That is described in additional detail further below.
- FIG. 4B also shows an "examples" interface, which displays a plurality of sample natural language based requests and how they would be answered based on the user's input via the edit answer" interface.
- the "edit answer" interface enables user-definition and/or modification of an action templates including: (i) defining a parameterized portion of one or more of the natural language based requests associated with the instruction template; and (ii) defining a corresponding parameterized portion of the action template.
- FIG. 4C and FIG. 4D a user is provide with an "edit variations" option, which allows a user to select a portion of question text and answer text (which may be text in a HTTP data), and apply a variation to those. As shown in FIG. 4D, this includes assigning a "type" to the variation.
- types include: general purpose descriptors such as dates, names, places, and times, and also application-specific types such as a direction or path (for example in a robotics application), or a disease/symptom (for example in a medical application).
- application-specific types such as a direction or path (for example in a robotics application), or a disease/symptom (for example in a medical application).
- There may be a format associated with the type such as a date format).
- the collection of available types may be expanded over time.
- the user selects a "date” type.
- the user also defines the format for the date value in the answer (which it will be appreciated is of particular importance where the answer includes HTTP data).
- the format may be different for the question as opposed to the answer. That is, in the context of a question, the format relates to a format in which a parameter value is expected to be supplied by a user. In the context of an answer, on the other hand, the format may be quite different (for example where the value defines part of a HTTP request) . In some cases a given parameter is used at multiple locations in an action template, optionally in different formats.
- an administrator user is also enabled to specify further data sources when defining an action template.
- a request may be converted into a form configured to cause a lookup of rows in a database or a spreadsheet.
- the values of parameterized portions of the instruction template when grounded by matching against the user request, are optionally fed into a WHERE clause of an SQL statement and used to identify rows in a database. The identified rows (and values defined therein) are then able to be used as parameter values in the action template when returning the template to the calling application for execution.
- contextual information is also available to an action template.
- the calling application in some embodiments is configured to pass contextual information such as: personal information (e.g. the user's name and address); information about the state of the implementation environment and its state (e.g. what information is being displayed, what is selected, a cursor position, etc); and information concerning previous interactions (so a dialog might be realized and ambiguities resolved).
- contextual information might be defines as inputs for an action template.
- contextual information is additionally associated with the instruction template.
- an administrator user is enabled to specify context-based constraints, thereby limiting applicability of a given instruction template.
- a given instruction template may only be considered for matching in the event that, for example: the requesting user has prescribed attributes of personal information (such as an age); the implementation environment is in a prescribed state (for example the user is on a particular web page); and the like.
- an administrator user registers to use the natural language processing technology via a signup webpage.
- This enables the administrator user to embed code in a website (or other implementation environment) which is configured to receive natural language based request.
- the administrator user is also provided with an account with the template management interface, to enable managing of instruction and action templates for their implementation environment.
- the administrator user performs initial configuration by asking a range of expected common questions relevant to the implementation environment, and via the template management interface defines answers to those questions (for example via the interface shown in FIG. 4A to FIG. 4E).
- functionality is provided to enable provision the questions and/or answers in speech form. This provides a mechanism to quickly and efficiently create an initial set of functionalities available via natural language requests.
- This set of functionalities is expanded over time, as unmatchable requests are submitted, flagged, and brought to the attention of the administrator user (for example via electronic notifications), who is enabled to review those, and either configure them to in future be matched to an existing instruction template, or define a new instruction template and action template.
- any users who made unmatched requests which would now match against these action templates are notified (and in some cases the action template executed or made available for execution at their request). In this manner, a user who asks a question which is initially unable to be matched to an instruction template is provided an answer at a later point in time upon (upon generation/modification of an instruction template having an associated action template).
- users and/or administrator users have control over settings affecting delivery of such notifications.
- a feedback mechanism is provided for users, such that users are able to identify instances where an executed action template is unhelpful/inappropriate to the users needs, thereby to assist an administrator user in modifying and/or fine tuning instruction and action templates over time (for example notifications of feedback are made available to administrator users via the template management interface).
- a further example embodiment is described below by reference to FIG. 5.
- This embodiment is described by reference to a relatively straightforward practical example, in the context of a website that is configured to enable party-to-party payments (among other functionalities).
- a user of that website wishing to make a party-to-party payment navigates, via menus, to a "payments" page, and inputs into that page details of a payee and a payment amount.
- a user is provided with an interface via which to provide a natural language query (for example as text or as speech). For the sake of example, assume the following query is provided: "pay John Smith $50", as represented by block 501 .
- Block 502 represents a matching process, whereby "pay John Smith $50" is processed to determine whether it has a threshold match with an existing instruction template. For the present example, we assume it does not. Therefore, at block 503, the natural language query ("pay John Smith $50") is presented to an administrative user, referred to in this example as a natural language query training user (being a back-end user tasked with reviewing and responding manually to natural language queries). For the sake of this example, we assume that one or more natural language query training users are on hand to manually respond to unmatched natural language queries on an as-required basis (i.e. substantially immediately).
- the training user prepares a web page that corresponds to query.
- the training interface provides user interface objects that allow the training user to access the website from which the natural language query was submitted, navigate to a desired page, and where relevant adjust that page context.
- the way in which the training user navigates and provides context to a web page varies between embodiments, and the following are optionally used:
- the training user navigates to a "payments" page, which provides a form for inputting details for a payment to be made, and inputs into relevant fields "John Smith” and "$50".
- the training user then causes that payments page with its defined context (i.e. completed fields) to be rendered at the user terminal in response to the natural language request (this may be rendered in the existing window, or in a separate window/object).
- the user is provided with a web page that provides a pre- filled form to pay $50 to John Smith (with "John Smith” in a name field, and "50.00” in a payment amount field), and is able to complete the transaction (for example by clicking a "perform transfer” button on the page).
- Block 510 represents an optional step of normalizing a natural language query thereby to remove redundant language and/or standardising parts of the query in a way in which the meaning of the query is not changed. For example, natural language expressions such as “I would like to”, “can you please” and the like are removed. Various tools for de-complicating natural language expressions in this manner are known in the art.
- Block 51 1 represents a process of query term tagging using NLP techniques (this is an optional selection of technology; other approaches are used in further examples) .
- Some embodiments make use of available tagging technologies, for example those provided by third parties such as SpaCy.
- This allows identification, in the query of grammatical artefact types such as direct and indirect objects and the like.
- This is followed by an optional tagging simplification step at 512, which is configured to consolidate tagging (such as SpaCy type tagging) thereby to provide a simplified degree of tagging (for example combination of two-part names into a single artefact).
- tagging simplification step at 512, which is configured to consolidate tagging (such as SpaCy type tagging) thereby to provide a simplified degree of tagging (for example combination of two-part names into a single artefact).
- tagging simplification step is configured to consolidate tagging (such as SpaCy type tagging) thereby to provide a simplified degree
- Block 513 represents a process including identifying artefacts associated with the web page (with context) prepared at 504. This may include data inputted into forms, portions of a page URL, displayed content, and the like.
- Block 514 represents a matching process, whereby tagged artefacts derived from the query are matched against artefacts associated with the web page. This preferably includes:
- Each artefact is analyzed to identify one or more predicted data types (for example a name, a date, a currency value, and so on), and the artefact is associated with a basic value based on that data type. For example, “$50.00”, “fifty dollars”, “50 dollars”, and other variations are all able to be associated with a currency value of 50.00. As another example, a date in the format "next Thursday" is associated with an actual date in a standardized date format. An artefact may have one or more types and one or more values.
- Matched artefacts may be determined to be parameter values.
- the one query artefact may correspond to more than one part of the action template, for example "John Smith” in the query may correspond to FIRST NAME: "John”, LAST NAME: "Smith” in the action template.
- the reverse is not true: so at most one query artefact may be deemed to correspond to part of the action template.
- the exploitation of the commonality enables the automated defining of instruction and action templates at 515:
- the instruction template includes data defining an action type, and data types as tagged language parts.
- the action template includes data defining a web page (with context) which uses values extracted from the tagged language parts of identified data types.
- the format of the associated artefact is analysed and recorded. This is not the format of the instruction, (that is allowed to vary according to the requesting user's preferences), but is to be used for the format of the action. For example, a date corresponding to 28 September 2009 might be represented as "2009-09-28” and have a format representation "YYYY-MM-DD”. By recording the format of the action template parameter, when "14/8/2016" is required to be rendered, it would be represented as "2016-08-14".
- the matching makes use of a wider collection of data, including data associated with the request-defining user's web page context (at the time of the query and/or based on the response page that is defined), data associated with the user (for example personal information, known contacts, location, and so on), general context data (for example a current data and time).
- data associated with the request-defining user's web page context at the time of the query and/or based on the response page that is defined
- data associated with the user for example personal information, known contacts, location, and so on
- general context data for example a current data and time
- the technology is configured to handle situations where there is ambiguity in a parameter value required for an instruction/action template. In some embodiments this includes asking a further question to resolve the ambiguity.
- a portion of a natural language request includes a parameterised that portion could have two potential values based on the wording of the request (for example "Sydney" could be in Australia, or Sydney in Canada).
- a question is defined to elicit a response from the user to resolve the ambiguity.
- Further embodiments apply alternate resolution techniques, for example using knowledge of the user, the user's location, and the like.
- the technology is configured to handle situations where there is optionality in at least one parameter value required for an instruction/action template.
- a request is able to be matched to an instruction template in spite of lacking parameterized portions for optional parameters.
- a date value may be optional, allowing "I want to fly to Paris on Thursday" and "I want to fly to Paris” to be matched to the same instruction template.
- the former will use the date value in the action template, the latter will not (and may optionally ask additional question of the user, or use default values) .
- a training user is enabled to define multiple conditional manual responses, and associate those with conditions (for example conditions based on context, user attributes, user login status, and so on).
- a web server 302 provides a web interface 303.
- This web interface is accessed by the parties by way of client terminals 304.
- users access interface 303 over the Internet by way of client terminals 304, which in various embodiments include the likes of personal computers, PDAs, cellular telephones, gaming consoles, and other Internet enabled devices.
- Server 303 includes a processor 305 coupled to a memory module 306 and a communications interface 307, such as an Internet connection, modem, Ethernet port, wireless network card, serial port, or the like.
- a communications interface 307 such as an Internet connection, modem, Ethernet port, wireless network card, serial port, or the like.
- distributed resources are used.
- server 302 includes a plurality of distributed servers having respective storage, processing and communications resources.
- Memory module 306 includes software instructions 308, which are executable on processor [00229]
- Server 302 is coupled to a database 310. In further embodiments the database leverages memory module 306.
- web interface 303 includes a website.
- the term "website” should be read broadly to cover substantially any source of information accessible over the Internet or another communications network (such as WAN, LAN or WLAN) via a browser application running on a client terminal.
- a website is a source of information made available by a server and accessible over the Internet by a web-browser application running on a client terminal.
- the web-browser application downloads code, such as HTML code, from the server. This code is executable through the web-browser on the client terminal for providing a graphical and often interactive representation of the website on the client terminal.
- a user of the client terminal is able to navigate between and throughout various web pages provided by the website, and access various functionalities that are provided.
- client terminals 304 maintain software instructions for a computer program product that essentially provides access to a portal via which framework 100 is accessed (for instance via an iPhone app or the like).
- each terminal 304 includes a processor 31 1 coupled to a memory module 313 and a communications interface 312, such as an internet connection, modem, Ethernet port, serial port, or the like.
- Memory module 313 includes software instructions 314, which are executable on processor 31 1 . These software instructions allow terminal 304 to execute a software application, such as a proprietary application or web browser application and thereby render on-screen a user interface and allow communication with server 302. This user interface allows for the creation, viewing and administration of profiles, access to the internal communications interface, and various other functionalities.
- processor may refer to any device or portion of a device that processes electronic data, e.g., from registers and/or memory to transform that electronic data into other electronic data that, e.g., may be stored in registers and/or memory.
- a "computer” or a “computing machine” or a “computing platform” may include one or more processors.
- the methodologies described herein are, in one embodiment, performable by one or more processors that accept computer-readable (also called machine-readable) code containing a set of instructions that when executed by one or more of the processors carry out at least one of the methods described herein.
- Any processor capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken are included.
- a typical processing system that includes one or more processors.
- Each processor may include one or more of a CPU, a graphics processing unit, and a programmable DSP unit.
- the processing system further may include a memory subsystem including main RAM and/or a static RAM, and/or ROM.
- a bus subsystem may be included for communicating between the components.
- the processing system further may be a distributed processing system with processors coupled by a network. If the processing system requires a display, such a display may be included, e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT) display. If manual data entry is required, the processing system also includes an input device such as one or more of an alphanumeric input unit such as a keyboard, a pointing control device such as a mouse, and so forth. Input devices may also include audio/video input devices, and/or devices configured to derive information relating to characteristics/attributes of a human user.
- the term memory unit as used herein, if clear from the context and unless explicitly stated otherwise, also encompasses a storage system such as a disk drive unit.
- the processing system in some configurations may include a sound output device, and a network interface device.
- the memory subsystem thus includes a computer-readable carrier medium that carries computer- readable code (e.g., software) including a set of instructions to cause performing, when executed by one or more processors, one of more of the methods described herein. Note that when the method includes several elements, e.g., several steps, no ordering of such elements is implied, unless specifically stated.
- the software may reside in the hard disk, or may also reside, completely or at least partially, within the RAM and/or within the processor during execution thereof by the computer system.
- the memory and the processor also constitute computer-readable carrier medium carrying computer-readable code.
- a computer-readable carrier medium may form, or be included in a computer program product.
- the one or more processors operate as a standalone device or may be connected, e.g., networked to other processor(s), in a networked deployment, the one or more processors may operate in the capacity of a server or a user machine in server-user network environment, or as a peer machine in a peer-to-peer or distributed network environment.
- the one or more processors may form a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.
- PC personal computer
- PDA Personal Digital Assistant
- each of the methods described herein is in the form of a computer- readable carrier medium carrying a set of instructions, e.g., a computer program that is for execution on one or more processors, e.g., one or more processors that are part of web server arrangement.
- a computer-readable carrier medium carrying computer readable code including a set of instructions that when executed on one or more processors cause the processor or processors to implement a method.
- aspects of the present invention may take the form of a method, an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects.
- the present invention may take the form of carrier medium (e.g., a computer program product on a computer-readable storage medium) carrying computer-readable program code embodied in the medium.
- the software may further be transmitted or received over a network via a network interface device.
- the carrier medium is shown in an exemplary embodiment to be a single medium, the term “carrier medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions.
- the term “carrier medium” shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by one or more of the processors and that cause the one or more processors to perform any one or more of the methodologies of the present invention.
- a carrier medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media.
- Non-volatile media includes, for example, optical, magnetic disks, and magneto- optical disks.
- Volatile media includes dynamic memory, such as main memory.
- Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise a bus subsystem. Transmission media also may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
- carrier medium shall accordingly be taken to included, but not be limited to, solid-state memories, a computer product embodied in optical and magnetic media; a medium bearing a propagated signal detectable by at least one processor of one or more processors and representing a set of instructions that, when executed, implement a method; and a transmission medium in a network bearing a propagated signal detectable by at least one processor of the one or more processors and representing the set of instructions.
- Coupled when used in the claims, should not be interpreted as being limited to direct connections only.
- the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other.
- the scope of the expression a device A coupled to a device B should not be limited to devices or systems wherein an output of device A is directly connected to an input of device B. It means that there exists a path between an output of A and an input of B which may be a path including other devices or means.
- Coupled may mean that two or more elements are either in direct physical or electrical contact, or that two or more elements are not in direct contact with each other but yet still co-operate or interact with each other.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Machine Translation (AREA)
Abstract
Applications Claiming Priority (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU2015904121 | 2015-10-09 | ||
AU2015904121A AU2015904121A0 (en) | 2015-10-09 | Frameworks and methodologies configured to enable streamlined integration of natural language processing functionality with a plurality of user interface environments | |
AU2015905046A AU2015905046A0 (en) | 2015-12-04 | Frameworks and methodologies configured to enable streamlined integration of natural language processing functionality with a plurality of user interface environments, including assisted learning process | |
AU2015905046 | 2015-12-04 | ||
AU2015905061A AU2015905061A0 (en) | 2015-12-07 | Frameworks and methodologies configured to enable streamlined integration of natural language processing functionality with a plurality of user interface environments, including assisted learning process | |
AU2015905061 | 2015-12-07 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2017059500A1 true WO2017059500A1 (fr) | 2017-04-13 |
Family
ID=58487121
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/AU2016/050950 WO2017059500A1 (fr) | 2015-10-09 | 2016-10-10 | Structures et méthodologies configurées pour permettre une intégration simplifiée d'une fonctionnalité de traitement de langage naturel avec un ou plusieurs environnements d'interface utilisateur, comprenant un processus d'apprentissage assisté |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2017059500A1 (fr) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111108477A (zh) * | 2017-07-15 | 2020-05-05 | Cytk有限责任公司 | 通用虚拟专业工具包 |
US20220350961A1 (en) * | 2021-04-30 | 2022-11-03 | Bank Of America Corporation | Systems and methods for tool integration using cross channel digital forms |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5454106A (en) * | 1993-05-17 | 1995-09-26 | International Business Machines Corporation | Database retrieval system using natural language for presenting understood components of an ambiguous query on a user interface |
US6766320B1 (en) * | 2000-08-24 | 2004-07-20 | Microsoft Corporation | Search engine with natural language-based robust parsing for user query and relevance feedback learning |
US20040220809A1 (en) * | 2003-05-01 | 2004-11-04 | Microsoft Corporation One Microsoft Way | System with composite statistical and rules-based grammar model for speech recognition and natural language understanding |
US7403938B2 (en) * | 2001-09-24 | 2008-07-22 | Iac Search & Media, Inc. | Natural language query processing |
-
2016
- 2016-10-10 WO PCT/AU2016/050950 patent/WO2017059500A1/fr active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5454106A (en) * | 1993-05-17 | 1995-09-26 | International Business Machines Corporation | Database retrieval system using natural language for presenting understood components of an ambiguous query on a user interface |
US6766320B1 (en) * | 2000-08-24 | 2004-07-20 | Microsoft Corporation | Search engine with natural language-based robust parsing for user query and relevance feedback learning |
US7403938B2 (en) * | 2001-09-24 | 2008-07-22 | Iac Search & Media, Inc. | Natural language query processing |
US20040220809A1 (en) * | 2003-05-01 | 2004-11-04 | Microsoft Corporation One Microsoft Way | System with composite statistical and rules-based grammar model for speech recognition and natural language understanding |
Non-Patent Citations (1)
Title |
---|
SNEIDERS: "Automated Question Answering: Template-Based Approach", DEPARTMENT OF COMPUTER SCIENCES ROYAL INSTITUTE OF TECHNOLOGY AND STOCKHOLM UNIVERSITY, February 2002 (2002-02-01), pages 1 - 279, XP055371157, Retrieved from the Internet <URL:https://www.researchgate.net/profile/Eriks_Sneiders/publication/2560669_Automated_Question_Answering/liks/53ecf6b40cf2981ada110bd3.pdf> [retrieved on 20161121] * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111108477A (zh) * | 2017-07-15 | 2020-05-05 | Cytk有限责任公司 | 通用虚拟专业工具包 |
US20220350961A1 (en) * | 2021-04-30 | 2022-11-03 | Bank Of America Corporation | Systems and methods for tool integration using cross channel digital forms |
US11645454B2 (en) | 2021-04-30 | 2023-05-09 | Bank Of America Corporation | Cross channel digital forms integration and presentation system |
US11704484B2 (en) | 2021-04-30 | 2023-07-18 | Bank Of America Corporation | Cross channel digital data parsing and generation system |
US11763074B2 (en) * | 2021-04-30 | 2023-09-19 | Bank Of America Corporation | Systems and methods for tool integration using cross channel digital forms |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11669918B2 (en) | Dialog session override policies for assistant systems | |
US20230206087A1 (en) | Techniques for building a knowledge graph in limited knowledge domains | |
US10771406B2 (en) | Providing and leveraging implicit signals reflecting user-to-BOT interaction | |
US10503767B2 (en) | Computerized natural language query intent dispatching | |
US20220138432A1 (en) | Relying on discourse analysis to answer complex questions by neural machine reading comprehension | |
US20070203869A1 (en) | Adaptive semantic platform architecture | |
US10977155B1 (en) | System for providing autonomous discovery of field or navigation constraints | |
US20230185799A1 (en) | Transforming natural language to structured query language based on multi-task learning and joint training | |
US20220075960A1 (en) | Interactive Communication System with Natural Language Adaptive Components | |
US20240126795A1 (en) | Conversational document question answering | |
US20240061832A1 (en) | Techniques for converting a natural language utterance to an intermediate database query representation | |
WO2017059500A1 (fr) | Structures et méthodologies configurées pour permettre une intégration simplifiée d'une fonctionnalité de traitement de langage naturel avec un ou plusieurs environnements d'interface utilisateur, comprenant un processus d'apprentissage assisté | |
US20230139397A1 (en) | Deep learning techniques for extraction of embedded data from documents | |
EP4281880A1 (fr) | Équilibrage de caractéristiques multiples pour processeurs en langage naturel | |
US20230134149A1 (en) | Rule-based techniques for extraction of question and answer pairs from data | |
US20240232187A9 (en) | Output interpretation for a meaning representation language system | |
US20230341998A1 (en) | Automated processing and dynamic filtering of content for display | |
JP2023112307A (ja) | 開発支援システム及び開発支援方法 | |
CN118235143A (zh) | 自然语言处理的路径失活 | |
CN113761143A (zh) | 为用户问题确定答案的方法、装置、设备和介质 | |
CN118215920A (zh) | 用于使用散列嵌入进行语言检测的宽深网络 | |
CN115297207A (zh) | 用于客户关系管理的通话准备引擎 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16852910 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 16852910 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205N DATED 15/06/2018) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 16852910 Country of ref document: EP Kind code of ref document: A1 |