AU777441B2 - A method of developing an interactive system - Google Patents

Description

WO 00/78022 PCT/AU00/0065! -1- A METHOD OF DEVELOPING AN INTERACTIVE SYSTEM The present invention relates to a method of developing an interactive system and, in particular to a development system and tools for generating an application for an interactive system.

Interactive systems, such as interactive voice response systems (IVRs), are able to communicate with other machines or humans using natural language dialogue. The systems are able to prompt a communicating party for data required to execute application tasks and need to cater for a wide variety of possible responses to the prompts, particularly when communicating with humans. Developing a set of rules which defines all of the possible answers or responses to the prompts is particularly problematic and labour intensive. Also developing a structure to manage the dialogue which occurs between the communicating parties, is complex. Accordingly, it is desired to provide a method and tools which facilitates application development or at least provides a useful alternative.

In accordance with the present invention there is provided a method of developing an interactive system, including: inputting application data representative of an application for said system, said application data including operations and parameters for said application; generating prompts on the basis of said application data; and generating grammar on the basis of said application data.

The present invention also provides a system for developing an interactive system, including: means for inputting application data representative of an application for said system, said application data including operations and parameters for said application; means for generating prompts on the basis of said application data; and means for generating grammar on the basis of said application data.

WO 00/78022 PCT/AU00/00651 -2- The present invention also provides development tools for an interactive system, including: code for inputting application data representative of an application for said system, said application data including operations and parameters for said application; code for generating prompts on the basis of said application data; and code for generating grammar on the basis of said application data.

The present invention also provides a grammatical inference method for developing grammar, including processing rules of the grammar, attaching slot specification rules representing meaning, creating additional rules representative of repeated phrases, and merging equivalent symbols of the grammar.

Preferred embodiments of the present invention are hereinafter described, by way of example only, with reference to the accompanying drawings, wherein: Figure 1 is a block diagram of a preferred embodiment of an interactive system connected to a communications network; Figure 2 is a more detailed block diagram of the interactive system; Figure 3 is a preferred embodiment of a development system for the interactive system; Figure 4 is a flow diagram of a model merging process of the development system; Figure 5 is a flow diagram of an incorporation phase of the model merging process; Figure 6 is a flow diagram of a slot specification rule substitution process of the model merging process; Figure 7 is a schematic diagram illustrating appending a grammar symbol; Figure 8 is a flow diagram of a recurrence procedure; and Figure 9 is a flow diagram of a traverse node procedure.

An interactive system, as shown in Figure 1, is able to communicate with another party, being a human or machine, using a natural language dialogue. A communication path is established with the party over a communications network 4, such as the PSTN and/or WO 0078022 PCTAU00/00651 Intemrnet. The path is between the system 2 and a communications terminal, such as a standard telephone 6 or computer system 8. When the communicating party is human and a voice terminal, such as the telephone 6, is used the system 2 converts prompts for the party to speech for delivery to the terminal, and interprets speech received from the terminal. When communicating with a machine, such as the computer 8, text data representative of the prompts and responses can be passed between the machines 2 and 8. The architecture of the system 2 can be considered to be divided into three distinct layers 10 to 14, an application layer 10, a natural language processing layer 12 and a signal processing layer 14. The application layer 10 defines an application for the system 2, such as a bill payment service, a customer service operation or a sales service. The application identifies the operations and transactions to be executed by the system 2. The natural language processing layer 12 defines the prompts to be generated by the system 2, the grammar which is acceptable in return and the different operation states and state transitions of the system 2. The signal processing layer 14 handles the communications interface with the network 4, the terminals 6 and 8, and the data processing required for the interface, such as speech generation and recognition.

The natural language process layer 12, as shown in Figure 2, includes a finite state machine (FSM) 20, the prompts 22 and the grammar 24. The FSM 20 controls the states for the system 2 and the state transition and accordingly controls the decisions and tasks executed by the system 2. The FSM 20 effectively performs dialogue management for the system 2, so as to control selective transmission of the prompts 22 and act in response to the grammar 24.

The prompts 22 are words of questions which can be sent by the system 2. The grammar 24 is a set of rules stipulating words and/or phrases which form acceptable answers to the prompts. The rules also define slots in the answers which constitute parameters required by the FSM 20 for execution of decisions and tasks. Accordingly, the slots are also defined in the FSM 20. The prompts, on instruction from the FSM 20, are selectively passed to a speech generator 26 for conversion into speech or voice and transmission to the terminals 6 or 8 using a communications interface 30 of the system 2. Responses received from a party are received by the communications interface 30 and are converted to representative data by a speech recognition module 28. The response data is passed by the module 28 to the grammar 24 for WO 00/78022 PCT/AU00/00651 -4processing. The grammar 24 interprets the responses and passes operation instructions and operation data from the slots to the FSM The interactive system 2 may take a number of forms, as will be understood by those skilled in the art, ranging from a hand held mobile device to a distributed computer system.

For simplicity, the preferred embodiments are described hereinafter with reference to the interactive system being an IVR. IVRs are produced by a number of telecommunications equipment providers, for example the Voice Processing Service (VPS) by Periphonics Inc. The parts 26, 28 and 30 of the signal processing layer are normally standard hardware and software components provided with the IVR. A developer would then be required to define at length program code, which can be compiled or interpreted, for the components 20, 22 and 24 of the prompt layer 12. This is an extremely onerous task which the preferred embodiments seek to alleviate. The speech recognition accuracy obtained by the system is also dependent on the coverage of the grammar. If it is too broad recognition may degrade, and if it is too narrow, recognition performance will also be degraded by trying to match an unexpected phrase against a list of encrypted phrases. The preferred embodiments also seek to obtain optimal speech recognition accuracy early in the applications life cycle by learning grammars from examples and using prior knowledge.

A development system 40, as shown in Figure 3, includes an application generator 42 which operates on an application file 44 to generate an FSM 20, prompts 22 and grammar 24.

The grammar 24 is refined by a grammatical inference engine 46. The application file 44 can be considered to define the application layer 10. The file 44 includes semantic data which describes semantics to be used in the IVR 2. The file defines the operations to be executed by the system 2 and the parameters required to execute the operations. An example of an application file 44 for a stock trading application is shown in Appendix 1. The operations defined are "buy", "sell" and "quote". The operations each have a number of input parameters defined by a parameter name and a number of output or return parameters defined by a return name, with all of the parameters being allocated a parameter type. The parameter types are predefined in an extendible type library. This library can be extended using a type definition WO 00/78022 PCT/AU00/00651 file, a grammar file and a slots file, as described hereinafter, and may include integers, real numbers, dates, times and money amounts. Parameters types can also be defined by the developer in a list of items. The example of Appendix 1 includes a list of stocks to be used by the application. A list of items may be products, companies, options or any item of interest.

The operations "sell" and "quote" are defined as requiring a user to be prompted to confirm that the input parameters are correct before proceeding with the operation. The application file can also be used to define additional information that can be obtained from a user of the IR, such as a user's account number and PIN. Initial data such as this can be considered to be preamble parameters which are collected before a top-level state of the FSM 20 is entered, and are used as global variables. Global variables for the IVR 2 are used in all operations of the IVR. The names of the operations are used in the grammar 24 as keywords to signify execution of the operation. For example, a user may respond to a prompt by saying "sell" or "sell all my holdings in BHP".

The finite state machine 20, the prompts 22 and the grammar 24 generated by the application generator 42 on the basis of the application file 44 of Appendix 1 are shown in Appendices 2. 3 and 5. respectively. A slot is defined for each input parameter name, as well as a slot for "operation". The slots are therefore operation, stockname, number and price.

The finite state machine 20 of the stock trading example is written in the ITUs Specification Description Language (SDL) of the ITU. The FSM 20 includes a number of procedures with variables, and are similar to subroutines in that a stack is used to control each procedure's behaviour, yet, a procedure can include states. An FSM 20 is generated by the application generator 42 executing the following steps: Create an initial top-level state.

For each operation of the application file 44 create a state transition from the top-level state.

For each parameter create a variable. The name of the variable is globally unique, which is achieved by pre-pending the operation name to the parameter name.

WO 00/78022 PCT/AUOO/00651 -6- For each state transition related to an operation, set all input parameters that may be obtained in a first response from a user to a value, otherwise reset the parameters to a value representing "null" or "unknown". A call is made to a nested state machine, such as an SDL procedure, to check each parameter in turn. If an operation needs to be confirmed, a procedure is established to ask the user for confirmation, otherwise the operation is executed. The results of the operation are passed to the output parameters and forwarded to the user prior to returning to the top-level state.

A number of procedures are established for generic operations for IVRs, such as help, cancel, repeat and operator.

For each input parameter that does not have a default, a nested procedure is established to determine if a parameter has been set to a value obtained by a response. If it is not set, a default is used.

For each input parameter a nested procedure is created to check if an incoming message or response sets a parameter. If the parameter is set in the incoming message, the variable relating to the parameter is set.

For each parameter that does not have a default, a nested procedure is created to see if the parameter has been already set. If it is not set, the procedure prompts the user for the parameter.

The grammar 24, shown in Appendix 5, is in a format used by the preferred embodiment. This grammar is also repeated in the Nuance format in Appendix 6. Appendix 7 shows an alternative grammar format using the Nuance format. The grammar has a hierarchical structure, similar to a tree structure, where the nodes can be considered to be symbols that are either terminal or non-terminal. A terminal symbol represents a specific word.

A non-terminal symbol represents other symbols and can be expanded into a set of symbols using a rule of the grammar. A rule defines a relationship between a non-terminal and symbols it can represent. For example, buy, sell, cancel, repeat, help and quit are terminals, whereas ConmlonStockCmds and WaitAskbuystockname are non-terminals. A terminal is shown in the Appendix and herein as any string of characters beginning with lowercase letter. A WO 00/78022 PCT/AU00/00651 -7- NonTerminal is any string of characters beginning with uppercase letter, whereas Feature Value is any string of alphanumerics including'.' and lines beginning with are comments.

The syntax of the grammar format used by the preferred embodiment, in Backaus Naur Form, is shown in Appendix 4, and an example of this is given below: !S from Location:xl on Date:x2 (10,1) operation=fly from=$xl.location date.month=$x2.date.month date.year=$x2.date.year date.dom=$x2.date.dom In this example the signifies that the rule is fixed and is not to be altered by any learning process. The symbol Location is a non-terminal and can be expanded into other symbols such as melbourne. This non-terminal Location returns a value. The returned value from the first instance of Location in the rule is stored in the xl variable and the second one into x2. There have been ten observations that use this rule and one other rule makes reference to this rule. The rule defines three slot specification rules that define the value the rule will return. The first states that the operation slot will always be set to the value 'fly'. The from slot is set the value of the location slot stored in the variable xl. The example above has its slot specification rules written in an absolute form. Alternatively a relative form can be used. The same rule written in relative form would be !S from Location:xl on Date:x2 (10,1) operation=fly from=$2.location date.month=S4.date.month date.year=$4.date.year date.dom=$4.date.dom In this form, instead of referencing variables such as xl or x2, reference is made to which symbol the slot value is extracted from. A third form is the non-terminal relative form which would be !S from Location:xl on Date:x2 (10,1) operation=fly from-#l.location date.month-#2.date.month date.year=#2.date.year date.dom=$2.date.dom In this case the number represents the non-terminal index in the rule rather than the symbol index.

WO 00/78022 PCT/AU00/00651 -8- The grammar of Appendix 6 is the same grammar written in the Nuance grammar format. This is generated by executing the following steps: For each state in the FSM 20 a top-level grammar is created.

For each top-level grammar a common set of commands are included, e.g.

cancel, help, repeat and quit.

A non-terminal is created for each enumerated item from the application file 44.

For each operation a non-terminal is created that represents that different words can be used to request that operation, and the operation name is placed into the non-terminal.

For the top-level grammar attached to the top-level state, e.g. TopLevelStock, a non-terminal is added for each operation, to which is added a slot specification rule containing the operation slot. A slot specification rule or attribute is a set of key value pairs defining a slot value. For example the attribute "{(operation buy)}" is added to the non-terminal "GR_buy".

For each top-level grammar that corresponds to a state requesting an input parameter, a non-terminal is added that returns to the FSM an element of the requested type. Predefined rules are established for non-terminals that represent predefined types of parameters, e.g. money and integer.

For each non-terminal that is added to a top-level grammar, slot specification rules are added that pass the slots that are filled in the rules below the nonterminal to the top level rule. Input parameters may be either enumerated types, integers or data structures. For instance the data type date, may have a day of week, day of month, year and month slot. Where a parameter is a structured type, one slot specification rule is required for each member in the structure.

For instance date.year-$x.date.year. To accommodate this conversion a type definition file is provided, an example of which is shown in Appendix 8. In addition to the type definition file, a grammar file is required to define a structured type.

WO 00/78022 PCT/AUOO/00651 -9- Predefined grammars are then attached to the generated grammar file to complete the application grammar file. These grammars are for the predefined types such as money or integer. An example grammar is shown in Appendix 9. In these rules there is a attached to the rules in the predefined grammar.

This is used to indicate that these rules are fixed and are not to be changed by subsequent grammatical inference.

Appendix 10 is the grammar generated by the process with the predefined grammars.

The prompts 22 of Appendix 3 are generated from the application file 44 and for the example are written in the Clips language. The application generator 42 generates the prompts by executing the following steps: The initial top-level prompt is set to be "Welcome to the stock application.

Please say one of the following". The prompt then lists the names of the operations, e.g. "buy, sell or quote".

For the prompts where a parameter is being prompted for, a template associated with the parameter type is called and used to generate the prompt. Most of these prompts are in the form of "Please say the where X is the parameter name.

For each state in the FSM, a help prompt can be played to the user if requested or if the FSM determines it is required. For prompts where the parameter being prompted for requires a corresponding help prompt, as determined by the parameter type, a template for help prompts associated with the parameter type is used to generate the prompt. These take the form of "Please say the X", where X is the parameter name and this is immediately followed by an example, such as "For example, three hundred and twenty five dollars and fifty cents". For enumerated parameters, the first three elements in the enumeration list can be used to form the example in the prompt.

WO 00/78022 PCT/AU00/00651 The grammar generated by the application generator 42 can be significantly enhanced to cater for a wide variety of responses. A grammatical inference engine 46 is able to operate on response examples or observations to create additional grammars to extend the grammar 24. The examples can be provided directly to the grammatical inference engine 46 from observations recorded by the IVR. Examples can also be provided from a symbolic inference engine 48 which can generates additional examples from the predefined grammar 24. The symbolic inference engine 48 uses a linguistic and/or symbolic manipulation to extend the examples, and can rely on synonyms or antonyms extracted from a thesaurus. The symbolic inference engine 48 may also accommodate cooperative responses, which may provide additional useful information to a prompt. For instance, the prompt may be "What time would you like to travel to Melbourne?", and a cooperative response would be "I want to go to Malvern not Melbourne". Another form of cooperative response is a pre-emptive response, such as to the prompt "Do you have a fax machine I could fax the information the response may be "Yes my fax number is 9254 2770". Whilst a number of different grammatical inference engines could be used to extend the grammar 24, described below is a new model merging process 50 for grammatical inference that is particularly advantageous and efficient, and is executed by the engine 46.

The model merging process 50 of the grammatical inference engine 46, as shown in Figure 4, operates on the grammar 24 and additional observations.

The model merging process 50 is based on the following principles: Whenever a high correlation between a sequence of two or more symbols and some slot specification rules is observed, a new non-terminal and rule are created. In representing the rule, the new non-terminal is placed on the left hand side of the rule and the observed repeated sequence of symbols on the right hand side, as shown below. The slot specification rules are attached to the new rule. All observed sequences of the symbols on the right hand side of the new rule in the grammar are replaced by the new non-terminal. This phrase creation will only occur when there is more than one instance of a repeated set WO 00/78022 PCT/AU00/00651 11 of phrases. The exception to this is for single words. If there is a high correlation between an attribute and a symbol, a new rule will be created. New rules are added in an order such that the rules with the most evidence are created first.

The new rule needs to be applied more than once. A rule which is created during a chunking process 52 is deleted, as discussed below, if it is not used more than once in parsing the observations and it is not a top-level rule. The exception to this rule is when the rule is of length one, and has slot specification rules attached to it. A top-level rule of a grammar has a nonterminal on the left hand side which represents the rule and resides at the highest level of the grammar.

In a merging phase 54, merging evidence patterns, described below, are used to identify symbols used interchangeably in the observations. The process seeks two symbols used interchangeably in the same place in a sequence of words. For merging to occur the rules need to have compatible slot specification rules, and the non-terminals have to return the same set of slots.

The rules are each allocated a hyperparameter which correlates to the number of times the rule is used for the observations parsed by the generated grammar.

Probabilities for the rules can then be determined from the hyperparameters.

The grammatical inference engine 46 continually changes the structure of the grammar 24, and the use of the hyperparameters significantly reduces the amount of computation required as opposed to having to calculate and store rule probabilities. The hyperparameters are also used to remove redundant rules, and ambiguity during a reestimation phase 302. A discussion on the use of hyperparameters to calculate rule probabilities for model merging is provided in Andreas Stolcke, "Bayesian Learning of Probabilistic Language Models", 1996, Doctoral Dissertation, Department of Electrical Engineering and Computer Science, University of California, Berkeley, available at ftp:!/ftp.icsi.berkeley.edu/pub!ai/Stolcke/thesis.ps.z.

WO 00/78022 PCT/AU00/00651 -12- To attach meaning to the phrases, the model merging process 50 is based on an assumption that all slot specification rules are assignment operators, and the result of applying slot specification rules to a production rule is visible on the observations. Also it is assumed that these slot specification rules are applied universally. Application of the assumptions enables the model merging process to learn beyond what it has seen or received and to generalise. Under the assumptions, each slot specification rule may contribute either a slotvalue pair, or a value to the observation. For instance given the two rules: S from Location:xl from=Sxl.from Location melbourne from=melbourne S to Location:xl (to=$xl.from I the observation "from melbourne" with the slots "from=melboume" can be generated. In this example the first rule contributes the slots "from=melbourne" while the second rule contributes the value "melbourne".

If the last two rules were used to generate the phrase "to melbourne" with the slots "to-melbourne" the second rule contributes the value "melbourne" while the third rule contributes the slots "to=melbourne".

There are a number of ways to determine correlation between the slots contributed by a rule and the slots of an observation. If the event A is the event that a rule contributes a particular slot value pair or value, and the event B is that an observation generated using that particular rule possesses that slot value pair or value, then for error free data, P(B/A) 1 because event A implies event B. If A implies B, then not B implies not A. Using this technique the list of possible slot value pairs or values a rule contributes can be reduced, once a candidate rule is given. To do this the notation f=v is used to imply that a rule contributes the slot value pair f=v and the notation to imply that a rule contributes the value v.

Given a particular set of rules, and observations generated using those rules, and the attributes attached to the rules a set of slot value pairs and slot values can be determined that a rule can contribute. This is done as follows: WO 00/78022 PCT/AU00/00651 -13- 1. For every slot value pair f-=v attached to an observation, add the set of potential slot values and values contributed by a rule to be This set is known as the set of attribute constraints of the observation.

2. The attribute constraints of a rule is the intersection of all of the attribute constraints of the observations that are generated by that rule. The attribute constraints can then be converted to slot specification rules. The actual slot specification rules will most likely be a subset of the slot specification rules obtained from the attribute constraints. This is because wvhen A implies B then not B implies not A but B does not imply A.

In the more general case when a grammar is learnt from examples, and there is no starting grammar, the actual rules are unknown. To overcome this problem candidate rules with right hand sides of length one and two can be considered. Once a grammar has been constructed with rules of length one and two, longer rules can be constructed by representing them as rules of length two, where one or more of the symbols on the right hand side can be expanded into more symbols.

This technique works best when there is no errors in the tagging of the data. An extension of the process would involve using P(A/B) 1, i.e. for instance P(AIB) 0.9.

The model merging process 50 has five distinct phases, an incorporation phase 300, a chunking phase 52, a pruning phase 301 ,a merging phase 54 and a reestimation phase 302, as shown in Figure 4. During the incorporation phase 300, as shown in Figure 5, the grammar is extended to include previously unpredicted phrases. A list of observed phrases (observations) is presented to the grammatical inference engine, at step 402, along with the dialog state that the phrases were collected in, plus a set of slots. The phrases are then parsed at step 404 by the grammatical inference engine using a bottom up chart parser. If the phrase can be parsed, the parser attaches a meaning to the phrase in the form of a set of attribute slots.

If the meaning attached to the observation is the same as that attached to it by the parser, the grammar does not need to be extended to incorporate the phrase. If the meaning is not the WO 00/78022 PCT/AU00/00651 -14same, then an error will be flagged, at step 406, the user alerted and the execution halted. If the observation cannot be parsed using the existing grammar, the grammar needs to be extended to incorporate it. For instance the phrase "buy three hundred shares of abador gold for three dollars fifty a share" was observed once in the dialog state "TopLevelStock" with the slots operation=buy, stockname="abador gold price.dollars=3 price.cents=0 price.modifer=per_share This could be added to the grammar as a new rule as follows: .TopLevelStock buy three hundred shares of abador gold for three dollars fifty a share operation=buy, stockname="abador gold price.dollars=3 price.cents-O price.modifer=per_share However it is more advantageous to first generalise the rule, using a bottom-up partial parser, so that a rule of the form.

.TopLevelStock buy Number:xl of StockName:x2 for Money:x3 a share .operation=buy, stockname=$x2.stockname price.dollars=$x3.price.dollars price.cents=$x3.price.cents price.modifer=per share The observation is partially parsed, at step 408. This creates a small number of parse trees, which return slot values. Where partial parses overlap preference is given to longer parse trees, with a left to right bias. These slot definitions are substituted into the slot definitions of the observations, one at a time, from the left to right, using the slot specification rule substitution process 410, as shown in Figure 6. This process is also used in the chunking phase.

The slot specification rule substitution process can be used to substitute slot specification rules attached to rules with one, two or zero symbols on the right hand side. In the case where it is being used for substituting in new rules, created from observations, the new rule will have only explicit slot specification rules, i.e. x=y and none of the form x=$y.z.

WO 00/78022 PCT/AU00/00651 The slot specification rule substitution process takes five parameters and makes reference to a type manager object, that defines the types of all of the slots used in the grammar.

The five parameters are the slot specification rules of the rule being substituted into, the slot specification rules of the rule that is being referenced, the symbol on the left hand side of the rule that is being referenced, and the variables attached to the first and second symbols on the right hand side of the rule that is being referenced in the rule that is being substituted into. Under certain circumstances these last three symbols can be undefined. For instance where the rule that is being referenced has only one symbol on it right hand side, the second symbol is marked as undefined.

For instance the rule being substituted into may be S I want to fly from CITY:x to CITY:y note-tellfrom from=$x.city to=$y.city} While the rule being referenced may be XI from CITY:x city=$x.city note-tellfrom In this case the slot specification rules of the rule that is being substituted into would be from=$x.city to=$y.city note=tellfrom the slot specification rules of the rule that is being referenced would be city=$x.city note-tellfrom the symbol on the left hand side of the rule would be XI the first symbol would be "from" and second symbols would be "CITY", therefore the two variables referencing these symbols are undefined and x respectively.

WO 00/78022 PCT/AU00/00651 -16- Each slot definition rule attached to the rule being substituted into is examined one at a time. If it refers to one of the symbols on the right hand side of the rule that is being referenced, it needs to be modified. For instance the slot specification rule from=$x.city makes reference to the variable and thus needs to be modified. The slot specification rule "city=$x.city" is examined. Because it returns a slot of type city, the slot specification rule is converted to "from=$Xl.city". If there had been no reference to the slot "city", the type manager would have been examined and an appropriate type defined. For instance if a reference was made to a "from" slot, and the rule did not define a "from" slot, the type manager would be referred to. The type of the "from" slot would be defined as "city", and the first slot associated with the type "city" would be used. In this case this would be the slot "city".

This would be repeated for all slot definition rules in the slot specification rules defined in the rule being substituted into.

The slot specification rules in the rule being substituted into are then checked for static slot specification rules. A static slot specification rule is one where the slot is explicitly defined such as note-tellfrom If slot specification rules of the rule being substituted into are then checked one at a time for static rules that exist in the rule being referenced. In this example the specification rule "note-tellfrom" is located in both sets of rules, and thus the reference to note-tellfrom in the rule being substituted into is replaced by note=$Xl.note.

At the end of the process these two rules would be as follows.

S I want to fly XI:X1 to CITY:y note=$Xl.note from=$Xl.city to=$y.city from CITY:x cityx.city noteellfrom Xl from CITY:x city-$x.city notetellfrom WO 00/78022 PCT/AU00/00651 -17- The model merging process 50 can also operate when there is no starting grammar.

When this is the case the observations are added to the grammar verbatim for instance the observation "buy three hundred shares ofabador gold for three dollars fifty a share" observed in the state "TopLevelStock" with the slots operation=buy, stockname="abador gold price.dollars=3 price.cents=0 price.modifer=per_share would result in the following rule being added to the grammar.

TopLevelStock buy three hundred shares of abador gold for three dollars fifty a share operation=buy, stockname="abador gold price.dollars=3 price.cents=0 price.modifer=per_share During the chunking phase 52 repeated sequences of words, i.e. phrases, in the grammar that appear in more than one rule are placed by a reference to a new rule which contains the repeated phrase. For instance prior to the chunking phase the rules for two nonterminals may be as follows: A c d e B x c d k After the chunking phase 52 three rules may be defined as follows: A C e B C k C c d This can be expressed as the new rule C c d being substituted into the rule A b cd e.

The chunking phase also needs to attach slot specification rules to the new phrases.

Likewise when a production rule is substituted into another production rule, the slot specification rules of the new production rule is substituted into the slot specification rules of the production rule that references it.

For instance prior to the chunking phase the rules for two non-terminals may be: .TopLevelStock buy Number:xl of StockName:x2 for Money:x3 a share operation=buy, stockname=$x2.stockname WO 00/78022 WO 0078022PCT/AU00100651 -18price.doliars=$x3.price.dollars price.cents=$x3.price.cents price.modifer=per_share .WaitAskbuyprice Money:xl a share operation-buvprice price.dollars=$xl.price.doilars price.cents=Sx2..price.cents price.mocifer-per_share After the chunking phase 52 three rules may be defined as follows: .TopLevelStock buy Number:xl of StockName:x2 for Money:x3 X1:x4 operation=buy, stocknarne=Sx2.stockname price.doliars=$x3.price.dollars price.cents=$x3.price.cents price .modifer=$x4.price .modifcr .WaitAskbuyprice Money:xl X1:x2 operation=buyprice price.cioiiars=$xl.price.doliars price.cents=Sxl.price.cents price .rodifer=$x2.Drice .modifer X1 a share price.modifer-per_share In this case the slot specification rule "price.modifer=per share" is substituted into the slot specification, operation-buy, stockname=Sx2.stockname price.doliars=$x3.price.doilars price.cents=$x3.price.cents price.znodifer=per-share The result of this is operation=buy, stockname=$x2.stockname price.doliars='Sx3.price.doliars price.cents=$x3.price.cents price.modifer=Sx4 .price.modifer The merging phase 54 is able to merge symbols which can be considered to be equivalent. The symbols may be terminal or non-terminal. For example, if A and B are assumed to be same in the following rules, X A b h Y >q B hk A -y u i B t y WO 00/78022 PCT/AU00/00651 -19- Then after merging A and B into C the grammar would be X C b h Y C h k C->yui C Z t y The symbols A and B can be merged into symbol C when the phrases are identified by merging evidence patterns, discussed below, as being interchangeable.

Merging reduces the complexity of a grammar and generalises the grammar so that it can handle additional phrases. For instance consider the following fragment of a grammar.

S from Xl:xl from=$xl.from S to X1:xl (l,1)to=$xl.from S from X2:xl from=$xl.from XI melbourne from=melbourne X2 sydney from=sydney In this example the symbols Xl and X2 are merged into symbol X3. Creating the following grammar.

S from X3:xl from=$xl.from S to X3:xl to=$xl.from X3 melbourne from=melbourne X3 sydney from=sydney This new grammar can generate the observation "to sydney" with the meaning to=sydney which the starting grammar could not.

The chunking process 52 does not generalise the grammar, but may create a new nonterminal that adds to the hierarchical structure of the grammar.

WO 00/78022 PCT/AU00/00651 The reestimation phase 302 uses statistical techniques to remove ambiguity, and to remove redundant rules.

For instance consider the observations Observation From melboume to melboume from-melboume to=melbourme Observation From melbourne to perth (1){from=melboume, to=perth} Observation From perth to melbourne (l){from=perth to=melboume} After the chunking, and merging phases the result may be the following grammar.

Rule S from Xl:xl to XI:x2 from=$xl.to to=$xl.to} Rule S from X1:xl to Xl:x2 from=$xl.to to=$x2.to} Rule Xl melboume {to=melboure} Rule Xl perth to=perth In this example there are two numbers in brackets prior to the slot specification rules.

The first number represents the number of observations that use this rule. The second number represents the number of rules that reference this rule. The reestimation phase reestimates the first number so that there are less rules.

Each observation in the training set is parsed by the grammar. Where more than one parse is possible, the parse with the highest probability that gives the correct meaning is considered to the correct parse. This is known as the Viterbi parse. If there are an equal number of possible parses, the observation is considered to be equally parsed by all of them.

Consider the following observation, from melbourne to melboure from=melbourne to=melbourne} It can be parsed using rules 1, 3 3. This would give the parse tree S from (X1 melbourne) to (XI melbourne)) with the slots from=melbourne to=melboume}and with probability 1/3 4/6 4/6 0.148 or it may be parsed using rules 2, 3 3. This would give the parse tree WO 00/78022 PCT/AU00/00651 -21 S from (XI melbourne) to (XI melbourne)) with the slots from=melbourne to=melbourne}and with probability 2/3 4/6 4/6 0.296 Both give the same outcome but it is more likely that rules 2 3 are used and thus for the purposes of reestimation this does not use rule 1.

Observations 2 can only be parsed using the second rule as follows.

(S from (Xl melbourne) to (Xl perth)) {from=melbourne, to-perth} using rules 2, 3 &4 An alternative parse using rules 1,3 4 (S from (XI melboure) to (Xl perth)) would attach the slots from=melboume to=melbourne} which is incorrect because it contradicts the training data.

For the same reasons the third observation can only be parsed correctly using rules 2, 4 3. Giving the parse tree, (S from (Xl perth) to (X1 melboume)) {from=perth to=melbourne} using rules 2,4 3 The hyperparameters are then re-estimated using the Viterbi parse. This is done one observation at a time. The hyperparameter of each rule is set to zero for most rules, and set to 1 for fixed rules. In the example above the hyperparameters would initially be set to Rule S from X1:xl to Xl:x2 from=$xl.to to=$xl.to} Rule S from X1:xl to Xl:x2 from=$xl.to to=$x2.to} Rule XI melbourne {to=melboure} Rule XI perth to=perth Using the Viterbi parse the rules are then incremented by the observation count. After considering observation 1 the grammar would become Rule S from X1:xl to X1:x2 from=$xl.to to=$xl.to} WO 00/78022 PCT/AU00/00651 -22- Rule S from X1:xl to Xl:x2 from=$xl.to to=$x2.to} Rule Xl melbourne {to=melboure} Rule Xl perth to=perth After considering observation 2 the grammar would become Rule S from X1:xl to X1:x2 from=$xl.to to=$xl.to} Rule S from X1:xl to Xl:x2 from=$xl.to to=$x2.to} Rule Xl melboure {to=melbourne} Rule XI perth to=perth After considering observation 3 the grammar would become Rule S from X1:xl to X1:x2 from=$xl.to to=$xl.to} Rule S from X1:xl to Xl:x2 from=$xl.to to=$x2.to} Rule Xl melbourne {to=melboure} Rule XI perth to=perth Rules that have a hyperparameter of zero would then be deleted. After reestimation the resulting grammar would be: Rule S from Xl:xl to Xl:x2 from=$xl.to to=$x2.to} Rule Xl melbourne {to-melboure} Rule XI perth to=perth The reestimation phase executes a variation of the grammatical inference insideoutside algorithm, and is used to remove ambiguity and to delete unnecessary rules.

The model merging process 50 is able to operate on a list of rules, such as the rules of the predefined grammar 24 and rules which represent observations. A rule is assigned a probability of occurring which can be calculated from the probabilities of the observations.

These probabilities can be estimated by counting the number of times the observation has occurred and dividing by the total number of observations. The merging process 50 does not WO 00/78022 PCT/AU00/00651 -23create or use cost functions, which have been used by some grammatical inference engines to decide steps to be executed and when execution should cease.

Rules are stored using a double linked list format where each symbol has pointers to the preceding and succeeding symbol. The format enables long rules to be shortened and lengthened without computationally expensive replication of data, and a sequence of symbols can be moved from one rule to another by changing the pointers at either end, without modifying data in between.

To meet principle of the merging process during the incorporation and chunking phases, two data structures are created. The first is a Monogram table and the second is a bigram table. The monogram table has an entry for every word type in the grammar, for instance the word melbourne. This entry has a pointer to every occurrence of this word in the grammar, plus a set of attribute constraints. The bigram table has an entry for every occurrence of two successive symbols. For instance "to melbourne" or "to XI". It also has a pointer to every occurrence of this bigram in the grammar, plus a set of attribute constraints, plus a set of slot specification rules.

New rules are created during the chunking phase by first examining the monogram table and then the bigram table.

The monogram table is sorted so that the first monogram to be pulled of it has the most number of non fixed rules. If there are two candidates with the same number of non-fixed rules, the one with the least number of attribute constraints is chosen. A new rule is then created with an unique symbol on its left hand side and the symbol on its right hand side. The hyperparameter of the new rule is set to the number of observations, while a reference count is set to one. The attribute constraints attached to the monogram are then converted to a slot specification rule.

WO 00/78022 PCT/AU00/00651 -24- All instances of this word in the grammar are then be substituted by the new nonterminal. The slot specification rules are then modified to derive slot values from the newly formed rule. This is achieved by substituting the attributes to=melbourne from the newly created rule into the existing slot specification rules using the slot specification substitution process. For instance two rules may exist as follows: S to melbourne {to=melbourne} S from melbourne {from=melbourne} A new rule is then created as follows: Xl melbourne {to=melboure} All instances of "melbourne" are then replaced by the non-terminal X1 as follows: S to X1 (to-melbourne) S from Xl {from=melbourne} The bigram table and monogram table are updated as this occurs. The slot specification rules are then modified, by attempting to substitute the to-melbourne returned by X1 into the slot specification rules attached to rule S. The resulting rules would become.

S to Xl:xl {to=$xl.to} S from Xl:xl {from=$xl.to} XI melbourne {to=melbourne} To accommodate prepositional phrases, a slot definition file is generated that assigns types to the slots to enable modification of slot specification rules. This file defines the slot name and its type. An example may be: location to location from location WO 00/78022 PCT/AU00/00651 Once there are no more rules that can be created using the monogram table, the bigram table is examined to create new rules. The bigram table is sorted so that the first bigram to be pulled of it will have the most number of occurrences, and the least number of attribute constraints, and doesn't hide any other begrimes. In addition to storing attribute constraints for each entry in the bigram table, slot specification rules are also be stored. Whenever one of the symbols in the bigram is referenced for the purposes of slot specification, these fragments of slot specification rules are stored in the bigram table. If two different slot specification rules are used in separate production rules, that conflict, the bigram is never chinked.

A new rule is then created with an unique symbol on its left hand side and the two symbols of the bigram on its right hand side. The slot specification rules attached to the bigram are added to the newly formed rule. In addition the attribute constraints attached to the bigram are then converted to a slot specification rules and attached to the newly formed rule where this does not conflict with the slot specification rules.

All instances of this bigram in the grammar are then be substituted by the new nonterminal. The slot specification rules are modified to derive slot values from the newly formed rule. This is achieved by substituting the attributes from the newly created rule into the existing slot specification rules. For instance two rules may exist as follows: S ->to Xl:xl {to=$xl.to} S from Xl:xl to Xl:x2 {from=$xl.to to=$x2.to} A new rule is then created as follows: X2 to Xl:xl {to=$xl.to} All instances of "to XI" are then replaced by the non-terminal X2 as follows: S X2 {to=$xl.to} S from XI:xl X2 {from=$xl.to to=$x2.to} WO 00/78022 PCT/AU00/00651 -26- The slot specification rules are then modified, by attempting to substitute the to=melboure returned by X1 into the slot specification rules attached to rule S. The resulting rules would become S X2:x2 {to=$x2.to} S from Xl:xl X2:x2 {from=$xl.to to=$x2.to} X2 to Xl:xl {to=$xl.to} Substituting slot specification rules from newly created rules into existing rules is achieved using the slot specification substitution procedure which executes the following steps: A function substitute(Attributes substitute_into,Attribute substitute_from, Symbol new_symbol, Symbol first_symbol, Symbol second_symbol) is called.

In this function all references to the bigram symbols (first_symbol and second_symbol) in the "substitute_into" set of attributes are replaced by references to the new_symbol. The slot referenced on the right hand side of the individual slot specification rule is then assigned. If the slot exists in the new rule, it is assigned to that. If it is not defined in the newly created rule, it is assigned to the first slot of that type listed in the slot definition file.

If the newly created rule defines a static slot value (x=y rather than and an identical static slot value exists in the rule being substituted into a reference is made from the slot specification rule to the slot returned by the newly created rule.

To meet principle a reference count is attached to each rule which contains the number of other rules that reference the rule, and the reference count is distinct from the hyperparameter. When a rule refers another rule, the reference count on the other rule is incremented. When a rule ceases to use another rule, the reference count on the other rule is decremented. When the reference count for a rule becomes zero, the symbols contained in that rule are placed back in the rule which previously referred to or referenced the rule that now has a reference count of zero.

WO 00/78022 PCT/AU00/00651 -27- Although the monogram list and the bigram lists make reference to n-grams of length I and 2 respectively, rules of arbitrary length can be created, by merging rules that are only referenced by one other rule into that rule.

Operation of the chunking procedure is illustrated with reference to the following example, where the observations are: i like coffee in the morning (10) operation=update_knowledge drink=coffee i like tea in the morning (20) operation=update_knowledge drink=tea i'd like a cup of coffee please operation=request_drink drink=coffee The number at the end of the observations above is the number of times that phrase has been observed. As defined previously, in the rules below there is a two dimensional vector (ij) at the end of the rule. The first element i of the vector is the reference count and the second element j of the vector is the hyperparameter. In the scenario where there is no starting grammar, and the top level grammar is the grammar at the end of the incorporation phase would be: S i like coffee in the morning (10,1) (drink=coffee operation=update_knowledge} S i like tea in the morning (20,1) {drink=tea operation=update_knowledge S i'd like a cup of coffee please {drink=coffee operation=request_drink} In addition the monogram table would be as listed below. In this listing the first number after the symbol being referenced, is the number of times that symbol appears in the grammar. The second number is the number of non-fixed rules that this symbol appears in. It is this second number that is used to determine which rule should be created first.

a 1 1 drink=coffee *=coffee *=requestdrink operation=request_drink coffee 2 2 drink=coffee *=coffee wo oon8022 WO 0078022PCT/AUOO/00651 -28cup 1 1 drinkcof fee *=coffee =request-drink operation-request-drink i 2 2 =-update knowledge operation=update knlowledge i d 1 1 drink=coffee -=coffee *=requJest_drink operation=request drinlk in 2 2 *=update knowledge operation=update-kowledge like 3 3 morning 2 2 *-update-knowledge operationupdate_knowledge of 1 1 drink~cof fee *=cof fee *=request drink operat-ion-requestdrink please 1 1 drink=cof fee -=cof fee =request-drink operation=request-drink tea 1 1 drink-tea *=tea *=update-knowledge operation=update -knowledge the 2 2 *=update knowledge operationupdate-knowledge The bigram table would be a cup 1 1 drink=coffee *=coffee *=request_drink operationrequest-drink coffee in 1 1 drink-cof fee *=cof fee '=update-knowledge operationuodate-knowledge coffee please 1 1 drink=cof fee *=cof fee *=request-drink operation=request-drink cup of 1 1 drink-coffee *=coffee *=request_drink operation-request-drink i like 2 2 *=update-knowledge operation-update-knowledge i'd like 1 1 drink=cof fee '=cof fee *=request-drink operat ion= reque st-drink in the 2 2 update knowledge operationupdate-knowledge like a 1 1 drink=cof fee *-cof fee *=request-drink operation-requeSt-drink like coffee 1 1 drink=coffee *=coffee *=update-knowledge operationupdate -knowledge like tea 1 1 drink-tea *-tea *=update-knowledge operation-update -knowledge of coffee 1 1 orinkcoffee *=cof fee -request drink operationrequest-drink tea in 1 1 drink=tea *=tea -=update-knowledge operat jon=updiate-knlJedge the morning 2 2 *=update-knowledge operationupdatejknowledge WO 00/78022 WO 0078022PCT/AUOOIOO651 -29- The chunking phase would then begin. The Monogram table would first be examined.

Based on the monogram table, the first rule to be created would be X21 1 (30,2) operation=update knowledge The name of the non-terminal (X2 1) is assigned arbitrarily. The slot specification rules are extracted from the monogram table. The hv.perparameter and reference or rule count are derived from thc rules into which this new rule is substituted. Thc resulting grammar would be.

Step 1. The rules are: S X21:X21 like coffee in the morning (10,1) 1 drink=coffee operation=$X21 .operation) S X21:X21 like tea in the morning (20,1) drink=tea operation=SX21. operation) S i'd like a cup of coffee please j drink-coffee operation=request drink) X21 i (30,2) operation=update-knowledge After this has occurred the monogram table becomes (the following steps 1 to step 11I are a continuation of this process): X21 2 2 -'update-knowledge operation=update knowledge a 1 1 drink-cof fee *=coffee -request-drink operationrequest -drink coffee 2 2 drink=coffee -=coffee cup 1 1 drink=coffee *=coffee *=request-drink operation=request-drink i 1 1 .=Update knowledge operation=update -knowledge i'd 1 1 drink-coffee *=coffee *=request-drink operationrequest -drink in 2 2 =update-knowledge operation=update-knowledge like 3 3 morning 2 2 *=update-knowledge operation=update knowledge of 1 1 drink=coffee *-coffee *=request-drink 3 5 cperationrequest drink WO 00/78022 PCT/AUOO/00651 please 1 1 drink=coffee *=coffee 4 =request-drink operationrequest-drink tea 1 1 drink=tea *=tea *=update-knowledge cperation=update-knowledge the 2 2 *=update-knowledge operation=update knowledge Step 2 S X21:X21 like X23:X23 in the morning (10,1) drink=$X23.drink operation=$X21.operation) S X21:X21 like tea in the morning (20,1) drink=tea operation=$X2.operation} S i'd like a cup of X23:X23 please drink=$X23.drink operation=requestdrink) X21 i (30,2) 1 operation=update_knowledge X23 coffee (15,2) 1 drink=coffee X21 2 2 *=update-knowledge operation-update knowledge X23 2 2 drink-coffee coffee a 1 1 drink=coffee *=coffee '=request-drink operation-request_drink coffee 1 1 drink=coffee *=coffee cup 1 1 drink=coffee *=coffee *irequestdrink operation~requestdrink i 1 1 *=update knowledge operation=updateknowledge i d 1 1 drink=coffee *=coffee *=request drink operation-request drink in 2 2 .=update knowledge operation=update knowledge like 3 3 morning 2 2 *=updateknowledge operation=update_knowledge of 1 1 drink~coffee *=coffee *=requestdrink operation-requestdrink please 1 1 drink-coffee *-coffee -=request-drink operation-recuestdrink tea 1 1 drink=tea '=tea *=update-knowledge operation=update_knowledge the 2 2 *=update-knowledge operation=update knowledge Step 3 S X21:X21 like X23:X23 X24:X24 the morning (10,1) 1 drink=$X23.drink operation=$X21.operation) WO 00/78022 WO 0078022PC/AUOO/00651 -31 S X21:X21 like tea X24:X24 the morning (20, 1) drink=tea operation=$X21 .operationl S i'd like a cup of X23:X23 please {drink=SX23.drink operation~request drink I X21 (30,2) operation=update knowledge X23 ->coffee (15,2) drink=coffee) X24 ->in (30,2) f operation=update_knowledgeI X21 2 2 *=update knowledge operation=update_knowledge X23 2 2 drink=coffee *=coffee X24 2 2 '=update-knowledge operation=update_knowledge a 1 1 drink=coffee *=coffee *=request_drink operation-request -drink coffee 1 1 drink-coffee -coffee cup 1 1 drink-coffee *=coffee *=request_drink operation=request drink i 1 1 *=update-knowledge olDeration=utodate knowledge i'd 1 1 drink-coffee *=coffee *=request_drink operation-request -drink in 1 1 *=update-knowledge operation=update knowledge like 33 morning 2 2 -update -knowledge operation-update_knowledge of 1 1 drink-coffee *-coffee *-request drink operation-request drink please 1 1 drink=cof fee *=cof fee *=requJest-drink operation=request drink tea 1 1 drink=tea *=tea *-update-knowledge operation-update_knowledge the 2 2 *=update-knowledge operation=update-knowledge Step 4 S X21:X21 like X23:X23 X24:X24 X25:X25 morning (10,1) drink=$X23. drink operation=$X21 .operationi S X21:X21 like tea X24:X24 X25:X25 morning (20,1) 1 drink=tea operation=SX21 .operation) S i'd like a cup of X23:X23 please (drink-$X23.drink operation-request -drinkI X21 i 30,2) 1 operation-update knowledge X23 coffee (15,2) drink-coffeeI X24 ->in (30,2) 1 operation=update_knowledge X25 ->the (30 2 operation-update_knowledge wo oonnn WO 0078022PCT/AUOO/00651 32 Slots are X21 2 2 *=update-knowledge operation=uodate-knowledge X23 2 2 drink=coffee *=coffee X24 2 2 *=update-knowledge operation=update-knowledge X25 2 2 *=update-knowledge operation-update-knowledge a 1 1 drink=coffee *=coffee *request-drink operationrequest -drink coffee 1 1 drink=coffee -coffee cuo 1 1 drink=cof fee *=coffee *=request drink operat-icn-request -drink i 1 1 *=update_knowledoe operation=update knowledge i d 1 1 drink=coffee -=coffee *=request_drink operaticn=request drink in 1 I *update-knowledge operation=update knowledge like 3 3 mocrning 2 2 *=update-knowledge operation=update_knowledge of I 1 drink=coffee *=coffee *=request-drink operation=request-drink please 1 1 drinkcoffee -coffee *=request-drink operation=request drink tea 1 1 drink=tea *=tea I=update_knowledge operation=update-knowledge the 1 1 *=update-knowl1edge operation-update-knowledge Step S X21:X21 like X23:X23 X24:X24 X25:X25 X26:X26 10,1) drink=$X23.drink operation=$X21.operation) S X21:X21 like tea X24:X24 X25:X25 X26:X26 20,1) drink-tea operation=$X21.operation) S i'd like a cup of X23:X23 please (drink=$X23.drink operation-request -drink) X21 (30,2) operation-update knowledge X23 ->coffee (15,2) drink=coffeeI X24 ->in (30,2) f operation=update_knowledge X25 ->the (30 2 H operation=update_knowledgeI X26 m> orning (30,2) operation=update knowledge X21 2 2 *=update-knowledge operation-update knowledge X23 2 2 drink-coffee coffee X24 2 2 *=update-knowledge operation=update knowledge X25 2 2 *=update-knowledge operation=update knowledge WO 00/78022 WO 0078022PC/AUOO/00651 -33- X26 2 2 *=update-knowledge operation=update -knowledge a 1 1 drink=coffee *=coffee *=request-drink operation=request drink coffee 1 1 drink=coffee *=coffee cup 1 1 drink=cof fee *=coffee *=request_drink operation=request -drink i 1 1 *=update-knowledge operation~update_knowledge i'd 1 1 drink=coffee *=coffee *=request-drink operat ion=request drink in 1 1 *=update-knowledge operation=update knowledge like 3 3 morning 1 1 *=update-knowledge operation-uodate-knowledge of 1 1 drink=coffee *=cof fee *=request-drink operation~request drink please 1 1 drink=cof fee *=cof fee *=request-drink operation~request-drink tea 1 1 drink=tea *=tea *=update-knowledge ooerat ion=update knowledge the 1 1 *=update-knowledge operation=update-knowledge Step 6 S X21:X21 like X23:X23 X24:X24 X25:X25 X26:X26 drink=$X23 .drink operation=$X21.operation) S X21:X21 like X27:X27 X24:X24 X25:X25 X26:X26 20,1) drink=SX27.drink operation=$X21.operation) S i'd like a cup of X23:X23 please f drink=$X23.drink operation-request -drink) X21 (30,2) f operation=update knowledgeI X23 ->coffee (15,2) drink=coffee X24 ->in (30,2) {operation~update -knowledge ->the (30 2 )(operation-update -knowledgeI X26 ->morning (30,2) operation=update -knowledge X27 ->tea (20,1) drink=tea operation=update knowledge X21 2 2 *=update -knowledge operation=update-knowledge X23 2 2 drink~coffee -=coffee X24 2 2 *=update_knowledge operation=update -knowledge 2 2 *=update-knowledge operation=update -knowledge X26 2 2 -=update -knowledge operation=update-knowledge X27 1 1 drink=tea *=tea *=update-knowledge operation=update knowledge WO 00n8022 WO 0078022PCT/AU00100651 -34a 1 1 drinkcof fee '=cof fee *=request-drink cperation=request drink coffee 1 1 drz4nk=coffee *-coffee cup 1 1 drink=coffee *=coffee *=request-drink operation~request -drink 1 1 1 *=update-knowledge ooeration~update -knowledge i'd 1 1 drink=cof fee *=cof fee *request-drink operation=request-drink in 1 1 *=update knowledge ope rat ion=update-knowledge like3 3 morning 1 1 *=update -knowledge operation=update_knowledge of 1 1 drink=coffee -coffee *=request-drink operat ion=request -drink please 1 1 drink=coffee *=coffee '=request-drink operation=request -drink tea 1 1 drink=tea *=tea *=update-knowledge operation=update_knowledge the 1 1 *-update knowledge operation=update_knowledge Step 7 S X21:X21 like X23:X23 X24:X24 X25:X25 X26:X26 10,1) drink=$X23.drink operation=$X21 .cperation) S X21:X2!1 like X27:X27 X24:X24 X25:X25 X26:X26 20,1) drink=$X27.drink operation=$X21.operation) S X28:X28 like a cup of X23:X23 please drink=SX23.drink operation=$X28.operation) X21 i (30,2) Joperation-update knowledge X23 coffee (15,2) 1 dirink=coffee) X24 ->in (30,2) (operationupdate -knowledge X25 ->the (30 2 )Iope rat ion=update_knowl edge X26 ->morning (30,2) operation-update -knowledge I X27 ->tea (20,1) 1 drink=tea operation=update knowledge X28 drink=coffee operation=request drink X21 2 2 '=update-knowledge operation=update-knowledge X23 2 2 drink=coffee -coffee X24 2 2 *=update-knowledge operation=update_knowledge 2 2 '=update-knowledge operation=update_knowledge X26 2 2 *=uodate knowledge operation=update knowledge X27 1 1 drink~tea *=tea *=update-knowledge operation=uzdate-knowledge WO OOn8O22 WO 0078022PCT/AUOO/00651 35 X28 1 1 drink-coffee *=coffee *=request-drink operation=request-drink a 1 1 drink=coffee '=cof fee *=request-drink operation=request_drink coffee 1 1 drink~coffee *=coffee cup 1 1 drink=cof fee *-coffee *=request_drink operation=request-drink 1 1 1 *-update-knowledge operation=update -knowledge i'd 1 1 drink=coffee *=coffee *request-drink operation=request -drink in 1 1 *=update-knowledge operatz on~updateknowledge like 3 3 morning 1 1 -update -knowledge operation-update_knowledge of 1 1 drink-coffee *-coffee *=request-drink operation~request drink please 1 1 drinkcof fee *=coffee '=request-drink operationrequest-drink tea 1 1 drink=tea *=tea *=update-knowledge operation-update_knowledge the 1 1 *=update-knowledge operation-update-knowledge Step 8 S X21:X21 like X23:X23 X24:X24 X25:X25 X26:X26 10,1) drink=$X23.drink operation-$X21. operation) S X21:X21 like X27:X27 X24:X24 X25;X25 X26:X26 20,l) 1 drink=SX27 .drink operation=$X21.operation) S X28:X28 like X29:X29 cup of X23:X23 please 1 drink=SX23.drink operation=$X28.operationl X21 (30,2) Joperation=update_knowledge X23 -coffee (15,2) drink=coffee X24 ->in (30,2) 1operation=update_knowledge ->the (30 2 )Ioperation-update_knowledge X26 ->morning (30,2) operation-update -knowledge X27 ->tea (20,1) 1drink=tea operation=update_knowledge X28 1drink=coffee operation-request drink X29 drink-coffee operation=request_drinkI slots X21 2 X23 2 X24 2 *=update-knowledge operation=update_knowledge drink-coffee *=coffee *=update-knowledge operation=update_knowledge WO 00/78022 WO 0078022PCT/AUOO/00651 36 2 2 *=update-knowledge operation~update knowledge X26 2 2 =update_knowledge operation=update knowledge X27 1 1 drinktea *=tea '=upcate-knowledge operat ion~update knowledge X28 1 1 drink=coffee -coffee *=request-drink operation=request -drink X29 1 1 drink=coffee *=coffee *=request_drink operation=request -drink a 1 1 drink=cof fee *=coffee *-request drink operation-request-drink coffee 1 1 drink=coffee *=coffee cup 1 1 drink=cof fee -=coffee *='equest_drink operation=request -drink i 1 1 *=update knowledge operation~update knowledge i'd 1 1 drink-coffee *=coffee *=request drink operation=request -drink in 1 1 4=uodate-knowledge operation=update knowledge like 3 3 morning 1 1 *-update-knowledge operation=update_knowledge of 1 1 drink=coffee *=coffee *=requ.est-drink operation=request drink please 1 1 drink=coffee *=cof fee *=request-drink operation=request drink tea 1 1 drink=tea *=tea *=update knowledge operation=update -knowledge the 1 1 *=update_knowledge operation=update-knowledge Step 9 S X21:X21 like X23:X23 X24:X24 X25:X25 X26:X26 (10,1) drink=$X23.drink operation=$X21.onerat ion) S X21:X21 like X27:X27 X24:X24 X25:X25 X26:X26 (20,1) drink-$X27 .drink operation=$X21 .operation) S X28:X28 like X29:X29 X30:X30 of X23:X23 please (5,1) drink=$X23.drink operation=SX28 .operation) X21 (30,2) J operation=update knowledgeI X23 ->coffee (15,2% drink=coffeeI X24 ->in (30,2) operation=update knowledge ->the (30 2 operarion=update -knowledge X26 ->morning (30,2) operation=update knowledge X27 ->tea (20,1) drink=tea operation~update knowledge WO OOr78022 WO 0078022PCT/AUOO/00651 -37- X28 i'd 1, drink=coffee operation-request drink I X29 drink~coffee operation=request_drink I ->cup drink=coffee operation=request-drink all X21 2 2 *=update-knowledige operation-update knowledge X23 2 2 drink-coffee *-coffee X24 2 2 *=update-knowledge operation=update knowledge 2 2 *=update-knowledge operation~update knowledge X26 2 2 *=update-knowledge operation=update knowledge X27 1 1 drink=tea *=tea *=update_knowledge operation~-update_knowledge X28 1 1 drink=coffee *=coffee '=request-drink operationrequest -drink X29 1 1 drink=cof fee -=coffee *=request drink operation-request -drink 1 1 drink=coffee *=coffee *=request_drink operation=request -drink a 1 1 drink=coffee *=coffee *=request_drink operation=request -drink coffee 1 1 drink-coffee *-coffee cup 1 1 drink~cof fee *=coffee 4=request drink operation=request -drink i 1 1 =update-knowledge operation=update knowledge i'd 1 1 drink=coffee *=coffee *=request-drink operation=request -drink in 1 1 =update-knowledge operation=update_knowledge like 3 3 morning 1 1 *-update-knowledge operation=update_knowledge of 1 1 drink-coffee *=coffee *-request-drink operation~request -drink please 1 1 drink=coffee 4=coffee -=request-drink operation=request drink tea 1 1 drink=tea *=tea *=update_knowledge operation-update eknowledge the 1 1 *=update knowledge operation=update knowledge Step S X21:X21 like X23:X23 X24:X24 X25:X25 X26:X26 10,1) drink=$X23 .drink operation-$X21 .operation} WO OOn8O22 WO 0078022PCT/AUOO/00651 38 S X21:X21 like X27:X27 X24:X24 X25:X25 X26:X26 drink=$X27.drink operation=SX21.operation) S X28:X28 like X29:X29 X3O:X30 X31:X3l X23:X23 please (5,1) drink=SX23.drink operation=SX28.operation) X21 i (30,2) operation~update_knowledgeI X23 coffee (15,2) drink=coffee I X24 ->in (30,2) f operation-update_knowledge ->the (30 2 operation=update_knowledge X26 ->morning (30,2) operation-update knowledgeI X27 ->tea (20,1) (drink=tea operation=update_knowledge X28 (drink=coffee operation=reouest drink X29 drink-coffee operation=request_cirink I ->cup 1drinkcoffee operatJon=reques .t drink X31 ->of drink=coffee oDeration=request drink slots all X21 2 2 -update-knowledge operation=update-knowledge X23 2 2 drink=coffee -coffee X24 2 2 '=update-knowledge cperation=update -knowledge X24 X25 2 2 =update-knowledge operation~uodate -knowledge 2 2 *=update-knowledge operation=update-knowledge X26 2 2 *-update-knowledge operat ion=update knowledge X27 1 1 drink~tea *=tea *=update-knowledge operation~update -knowledge X28 1 1 drink=cof fee *=cof fee *=request_drink operation-request -drink X29 1 1 drink=coffee '=cof fee -=request_drink operation=request -drink 1 1 drink=cof fee *=coffee 4=request drink operation-request-drink X31 1 1 drink-coffee *-cof fee -request-drink operationrequest-drink a 1 1 drink~cof fee -cof fee I=reouest-drink operation=request -drink coffee 1 1 drink=coffee *=coffee cup 1 1 drink=coffee *=coffee *=request_drink operation=recluest -drink i 1 1 *=update-knowledae operation=update knowledge i'd 1 1 drink=coffee -~coffee =request-drink.

operation=reauest -drink in 1 1 *=update-knowledge operation-update-knowledge WO OV78022 WO OO/84Y22PCT/AUOO/00651 39 like 3 3 morning 1 1 *=update-knowledge operation=update-knowledge of 1 1 drink=coffee -coffee *=request drink operation=request -drink please 1 1 drink-cof fee *-cof fee *=request-drink operation=request-drink tea 1 1 drink=tea *=tea *=update-knowledge operationupdat eknowledge the 1 1 *=update-knowledge operation-update-knowledge Step I1I S X21:X21 like X23:X23 X24:X24 X25:X25 X26:X26 drink=$X23.drink operation-SX21.operation) S X21:X21 like X27:X27 X24:X24 X25:X25 X26:X26 20,1) drink=$X27 .drink operation=SX21 .operation) S X28:X28 like X29:X29 X30:X30 X31:X31 X23:X23 X32:X32 5,1) drink=SX2] .drink operation=$X26 .operation) X21 (30,2) operation-update knowledgeI X23 ->coffee (15,2) drink=coffee) X24 ->in (30,2) operation=update knowledge I ->the (30 2 H operation~update_knowledge X26 ->morning (30,2) operation-update knowledge X27 ->tea (20,1) 1 drink-tea operation=update_knowledgeI X28 J drink=coffee operation=request drink X29 dxink=coffee operationrequest drink ->cup drink=coffee operation=request -drink X31 ->of 1 drink-coffee operation-request drink X32 ->please drink-coffee operation-request_drink This would then complete all of the chunking that is suggested from the monogram table. At this point the bigramn table would look as follows: X21 like 2 2 *=update-knowledge operation-update-knowledge operation-Si .operation X23 X24 1 1 drink=cof fee '=coffee *=update_knowledge operationupdate -knowledge drink=$1 .drink X23 X32 1 1 drink-coffee *=coffee *=request drink operation~request -drink drink=$1.drink X24 X25 2 2 *=uodate-knowledge operation~update knowledge WO OOn8O22 WO 0078022PCIAUOO/00651 40 X26 2 2 *=update-knowledge operation-updateknowledg 2 X27 X24 1 1 drink=tea *=tea *=update-knowledge operation=update knowledge drink=$1 drink X28 like 1 1 drink=coffee *=coffee *=request drink operationrequest-drink operation=$1 .operation X29 X30 1 1 drink=cof fee *=cof fee -=request drink operation=request-drink X31 1 1 drink=coffee '=coffee *=request drink onerarion-request-drink X31 X23 1 1 drink=coffee *=coffee *=request drink operation=request -drink drink=S2.drink like X23 1 1 drink=coffee -cof fee *=update knowledge operationupdate-kowledge drink=$2.drink like X27 1 1 drink=tea *=tea *=update-knowledge operationupdate knowledge drink$2 .drink like X29 1 1 drink=coffee *=coffee *=request drink operationrequest-drink The bigramn table includes fr-agments of slot specification rules. For instance the bigram X23 X32 includes the fragment drink=$ I .drink. This is derived from the rule: S X28:X28 like X29:X29 X30:X3O X31:X31 X23:X23 X32:X32 5,1) drink-SX2J .drink operation-$X28.operation) If there was a conflicting slot specification rule, defined in another production rule, the bigram would be marked as unchunkable and would not be chunked.

Step 12 Based on the bigram table the first bigram. rule to be created would be: X33 X21:X21 like (30,2) operation=$X21.operation Substituting this rule into the grammar would result in S X33:X33 X23:X23 X24:X24 X25:X25 X26:X26 10i1) I drink=SX23.cirink operation=SX33 .operat ion) S X33:X33 X27:X27 X24:X24 X25:X25 X26:X26 20,1) drink=$X27.drink operation SX33.operationj S X28:X28 like X29:X29 X30:X]0 X31:X31 X23:X23 X32:X32 5,1) dJrink=$X23.drink operation=$X28.operation} wo oon8o22 PCT/AJOO/00651 -41 X21 (30,1) f operation=uodate-knowledge X23 ->coffee (15,2) jdrink=coffee) X24 ->in (30,2) (operation=update_knowledge ->the (30 2 )(operation=update_knowledge X26 ->morning (30,2) operation=update -knowledge I X27 ->tea (20,1) 1 drink-tea operation=update_knowledge X28 drink-coffee operation-request drink X29 1 drink=coffee operation~request_drinkI ->cup J drink=coffee operation=request -drink I X31 ->of drink=coffee operation=request drinkI X32 please drink-coffee operation=request-drink X33 X21:X21 like (30,2) operation=$X21.operation1 The bigram. table would then become X21 like 1 I *=update-knowledge operation=update-knowledge operation$l .operation X23 X24 1 1 drink=coffee *=cof fee *=update_knowledge operation=update -knowledge drink=$1.drink X23 X32 1 1 drink-coffee *=coffee *-request drink operation=request rink drink=S1 .drink X24 X25 2 2 *=update-knowledge operation=update-knowledge X26 2 2 *=update-knowledge operation=update-knowledge X27 X24 1 1 drinktea '-tea *=update-knowledge operation-update -knowledge drink=$l .drink X28 like 1 1 drink=coffee *=cof fee *=request-drink operation-request drink operation=$1 .operation X29 X30 1 1 drink-coffee -coffee =-request drink operationrequest -drink X31 1 1 drink=cof fee *=cof fee 4=request drink operation-request -drink X31 X23 1 1 drink=cof fee *=cof fee *=request drink operation~request -drink drink=$2.drink X33 X23 1 1 drink=coffee *=coffee *=update_kcnowledge operation=updiate-knowledge drink=$2.drink operationS1l.operaltion X33 X27 1 1 drink=tea '=tea '=update-knowledge operation=update -knowledge dri-*nk=S2.drink operatcn-$1.operation like X29 1 1 drink=coffee -coffee request drink operationrequest drink WO OOn8O22 WO 0078022PCT/AUOO/00651 42 Steps 13 and 14 below show the subsequent step in the chunking phase. As new rules are created the bigram and monogram tables are updated. Although these tables cater for the creation of rules of length one and two respectively rules of longer length can be created from them, due to the fact that a symbol in the bigram table can be expanded into more than one symbol.

Step 13 S X33:X33 X23:X23 X34:X34 X26:X26 10,1) drink=$X23.drink operaticn=$x33.operation) S X33:X33 X27:X27 X34:X34 X26:X26 20,1) f drink=$X27.drink operation=$X33 .operation) S X28:X28 like X29:X29 X30:X30 X31:X31 X23:X23 X32:X32 5,1) drink=$X23.drink operation=$X28.operation) X21 (30,1) i operation-update-knowledge I X23 ->coffee (15,2) drink=coffee X24 ->in (30,1) 1operation=update knowledgeI ->the (30,1) operation-update knowledge X26 ->morning (30,2) 1 operation=update_knowledge X27 ->tea (20,1) j drink=tea operation=update-knowledgeI X28 1 drink=coffee operation=request drink X29 drink=cof fee operation=request-drink ->cup drink-coffee operation-request_drink X31 ->of 1drink=coffee operation=request_drink X32 ->please drink-coffee operation=request -drinkI X33 ->X21:X21 like (30,2) operation=$X21.operation X34 ->X24:X24 X25;X25 (30,2) The bigram tablc would then be: X21 like 1 1 *'=update-knowledge operation-update knowledge operation=$l .opezation X23 X32 1 1 drink=coffee *-coffee *=request_drink cperation=reouest-drink drink=$1.drink X23 X34 1 1 drink=coffee *=coffee A=update_knowledge operation~update_knowledge drink=S1.drink X24 X25 1 1 *=update-knowledge operation-update knowledge operation=$l1. operation wo oon8022 WO 0078022PCTAUOO/00651 -43- X27 X34 1 1 drink=tea *=tea *=update_knowledge operation=update knowledge drink=$l .drink X28 like 1 1 drink=coffee *=cof fee *=request drink operation=request_drink operation=$l operation X29 X30 1 1 drink~cof fee =coffee *=request_drink operation=request -drink X31 1 1 drink=cof fee *=coffee *=request_drink operation~request -drink X31 X23 1 1 drink=coffee -coffee *=request drink operation~request -drink drink=$2.drink X33 X23 1 1 drink=coffee =coffee *=update_knowledge operationupdate knowledge drink=S2.drink operation=$ 1 operazion X33 X27 1 1 drink-tea *=tea 1-update knowledge operation=update -knowledge drink=S2.drink operation=S1 .operation X34 X26 2 2 *=update knowledge operation=update knowledge like X29 1 1 drink=cof fee *=coffee *=request drink operat ion- request drink Step 14 S X33:X33 X23:X23 X35:X35 10,1) (-drink=$X23.drink operation=$X33 .operat ion) S X33:X33 X27:X27 X35:X351 20,1) drink=$X27.drink operation=$X33.operation) S X28:X28 like X29:X29 X30:X30 X31:X31 X23:X23 X32:X32 5,1) drink-SX23.drink operation=$X2B .operation) X21 (30,1) operation=update_knowledge X23 ->coffee (15,2) J drink=coffee I X24 ->in (30,1) operation=update knowledge X25 ->the (30,1) f operation-update-knowledgej X26 ->morning (30,1) operation-update knowledge X27 ->tea (20,1) jdrink-tea operation=update_knowledge X28 drink=coffee operation=request -drink X29 drink=coffee operation-request_drink X30 ->cup {drink=coffee operation=request -drink X31 ->of {drink=coffee operation-request_drink X32 ->please drink=ccffee operation=request drink X33 ->X21:X21 like (30,2) operation=SX2l.operation ->X24:X36 X25:X37 X26:X26 (30 2 WO 00/78022 WO 0078022PCT/AUOO/00651 -44- Step At this point the grammar is pruned of redundant slot specification rules. This is determined by checking every rule that uses the nule being pruned, and finding out what attributes it uses.

For instance if we examine the non-terminal X24 in the grammar above, It is used in the following rule: X24:X36 X25:X37 X26:X26 (30 2 This rule makes no reference to the "operation" slot therefore the slot specification rule (operation=update knowledge is pruned from the rule.

After the pruning phase which prunes slot specification rules the grammar is S X33:X33 X23:X23 X35:X35 10,1) (drink=SX23.drink operation-$X33 operation) S X33:X33 X27:X27 X35:X35 20,1) (drink=$X27.drink operation=SX33. operation) S X28:X28 like X29:X29 X30:X30 X31:X31 X23:X23 X32:X32 drink=$X23.drink operation=SX28.operation) X21 >i (30,1) operation=update knowledge X23 ->coffee (15,2) drink=coffee X24 ->in (30,1) ->the (30,1) X26 ->morning (30,1) X27 ->tea (20,1) {drink-tea operation=update_knowledge X28 (drinkcoffee operation=request drink 1 X29 a (5,1) ->cup (5,1) X31 ->of (5,1) X32 ->please (5,1) X33 ->X21:X21 like (30,2) operation=SX21.operation X35 ->X24:X36 X25:X37 X26:X26 (30 2 again, after pruning slot specification rules thc grammar is S X32:X32 X22:X22 X35:X35 10,1) I drink=$X22.drink operatioii=SX32.operation) S X32:X32 X26:X26 X35:X35 20,1) drink=$X26.drink operation=$X32.operation) wo oon8022 WO 0078022PCT/AUOO/00651 S X27:X27 like X28:X28 X29:X29 X30:X30 X22:X22 X31:X31 5,1) drinkS$X22. drink operation=$X27.operation) x21 (30,1) operation-update knowledge X22 ->coffee (15,2) drink-coffee X23 ->in (30,1) x24 ->the (30,1) ->morning (30,1) x26 ->tea (20,1) Cdrink-tea X27 Coperation=request drink X28 (5,1) X29 -?cup (5,1) ->of (5,1) X31 ->please (5,1) X32 ->X21:X21 like (30,2)1 operation=SX21.operation X35 ->X23:X36 X24:X37 X25:X25 (30,2) If the rule unity rule is then applied the grammar becomes Step 16 S X32:X32 X22:X22 X35:X35 (10,1) {drink=SX22.drink operation-$X32 .operat.ionI S X32:X32 X26:X26 X35:X35 (20,1) Cdrink=SX26.drink operation=SX32 .operation) S X27:X27 like a cup of X22;X22 please drink-SX22.drink operation-X27.operation) X21 (30,1) f operation=update_knowledgeI X22 ->coffee (15,2) drink=coffee X26 ->tea (20,1) drink-tea X27 1 operation-request drink X32 ->X21:X21 like operation-SX2.operation ->in the morning (30,2) The merging phase procedure 54, uses a list of rules that are examined for merging evidence patterns. This list is known as the Merge Agenda. At the beginning of the merging phase all rules are added to the Merge Agenda. Each rule on the Merge Agenda is examined and processed to determine when there is evidence for effecting a merger. Based on principle WO 00/78022 PCT/AU00/00651 -46a set of evidence patterns is established to determine when merging needs to occur. The four evidence patterns and the required merger action is recited in Table 1 below.

Evidence Action X A B Merge B and C A C

X->AY

B

Y C X AB Merge A and C X B Y B Y A Y C X A B C Merge B and D X->ADC

X->AYC

Y B Y D X AD Merge A and F E X->FED

X->YD

A

Y F E X AB Merge B and CO X->ACO

X->AY

B

Y->CO

X A Merge A and B, where both are non-terminals X

B

Table 1 For all of the actions described in the above table, with the exception of the last action involving deleting the evidence rule, the symbols which are merged may be non-terminals or terminals. For the merge to occur the slot specification rules when expressed in relative form for both rules need to be identical. In addition the terminals to be merged need to return identical types. If a symbol to be merged is a non-terminal, then it is not necessary to create a rule of the form Y A where only one symbol is on the right hand side. A rule Y a only needs to be created for terminals involved in the merger. If, for example, all of the symbols to be merged are non-terminals, then the symbols can be replaced on the right hand side and left hand side of the rules of the grammar in which they appear with a reference to a new nonterminal, and no additional rules need to be created.

WO 00/78022 WO 0078022PCT/AUOO/00651 47 If there is evidence for the merge, the merger is executed, as explained further below.

Any redundant rules are deleted and any rules changed as a result of the merging are added to the Merge Agenda. Operation then proceeds to determining whether any rules should remain on the Merge Agenda. If not, a rule is taken off the Merge Agenda. If the last rule has been reached, the procedure ends.

Continuing on from the previous example after the end of the pruning phase 301 the grammar expressed in relative format will be as follows: S X32:X32 X22:X22 X35:X35 (10,1) 1 drink=$2.drink operation=$ operation) S X32:X32 X26:X26 X35:X35 (20,1) 1 drink=$2.drink operation-$1.operation) S X27:X27 like a cup of X22:X22 please drink=$6.drink operazion=$1 .operation) X21 i (30,1) 1 operation=update_knowledge X22 ->coffee (15,2) 1 drinkcoffeeI X26 ->tea (20,1) 1 drink-tea) X27 >i'd operation=request drink X32 ->X21:X2. like (30,2)f( operation=$l.operationI X3 5 -in the morning (3 0,2) Using the four different merging patterns listed in Table 1, the grammar is altered as follows: Step 17 (merge X22 and X26) S X32:X32 X38:X26 X35:X35 (30,1) 1 drink=$2.drink operation=S operation) S X27:X27 like a cup of X38:X22 please drink=$6.drink operation=$l .operation) X21 i (30,1) operation=update_knowledge X38 ->coffee (15,2) drink=coffee X38 ->tea 120,1) 1drink=tea) X27 1operation~request-drinkI X32 ->X21:X21 like (30,1) operation=S1.operation 1 X35 r. the morning (30,1) WO 00/78022 PCT/AU00/00651 -48- Using the doubly linked list structure, only the symbol on the left hand side needs to be changed for all of the rules that use it. In the example given X22 and X26 are nonterminals. If one of the symbols to be merged is a terminal a rule is created with that symbol on the right hand side.

Principle is satisfied by the chunking and merge procedures 52 and 54 adjusting the hyperparameters. When a rule for a phrase is added, the hyperparameter of a rule is equal to the sum of the hyperparameters of the two rules that use it. When a phrase rule is added that uses an existing rule. the hyperparameter of the existing rule is increased by the hyperparameter of the new rule. When two rules are merged, the hyperparameter of the newly formed rule is the sum of the two previous hyperparameters.

Once the model merging process is completed principle can be applied to rules that have slot specification rules and rule counts of one. If this is done to the previous example the grammar in absolute mode becomes S I like X38:X26 in the morning (30,1) drink=$X26.drink operation=update_knowledge S i'd like a cup of X38:X22 please drink=$X22.drink operation=request_drink) X38 coffee (15,2) drink=coffee I X38 tea (20,1) drink=tea The grammar can then be made more human readable by performing the following: Removing all variables that are not referenced.

(ii) Non-terminals are assigned names that reflect the slots they return.

(iii) Non-terminals that do not return a value, but reflect synonyms are renamed.

For instance X23 the (1,2) X23 a(1,l) Can be renamed WO 00/78022 PCT/AU00/00651 -49- The (1,2) The a (1,1) Variables can also be assigned lower case names with small numbers. Using this technique the grammar becomes S I like Drink:xl in the morning (30,1) drink=Sxl.drink operation=update_knowledge S i'd like a cup of Drink:xl please drink=$xl.drink operation=request_drink Drink coffee (15,2) drink=coffee Drink tea (20,1) drink=tea After the model merging procedure is completed the probabilities of the rules can be calculated S I like Drink:xl in the morning (30,1) drink=Sxl.drink operation=update_knowledge prob (30/35) S i'd like a cup of Drink:xl please drink=$xl.drink operation=request_drink) prob (5/35) Drink coffee (15,2) drink=coffee prob (15/35) Drink tea (20,1) drink=tea prob (20/35) It can be seen that the new grammar is more general than the observations. For instance it can generate the phrase i'd like a cup of tea please According to the inferred grammar the meaning of this phrase is determined to be operation=request_drink drink=tea The probability of this phrase is calculated to be 5/35 20/35 0.08.

Merging nearly always reduces the number of rules as whenever a merge is identified either two rules are merged or one is deleted. When two rules are merged, one rule is deleted and the hyperparameter of one is increased by the hyperparameter of the deleted rule.

WO 00/78022 PCT/AU00/00651 In order to operate well, the model merging procedure 50 uses as a data structure to ensure the following operations are executed with efficiency: Appending a symbol to a rule.

Merging rules.

In the data structure: Each word is replaced by a single integer representing the word.

(ii) Each symbol is represented by a structure that contains a symbol identification (id) number, plus a pointer to the previous and subsequent symbol in a rule.

Each rule has guard symbol Q which points to both the first and the last symbols in the rule. This implements the doubly linked list.

(iii) Each rule is attached to a structure representing a non-terminal.

The data structure and its use is illustrated in Figure 7 which show how the data structure changes for appending a symbol.

All non-terminals are referenced in two global associative arrays, which associates the non-terminal id with the non-terminal. The first array is known as the rule array, and contains all of the rules attached to a given non-terminal. The second is known as the monogram table and contains references to all of the occurrences of that non-terminal on the right hand sides of rules. If states A and B are to be merged, then all non-terminal structures referenced in the rule array are accessed and the non-terminal id on each of these is changed. Occurrences of the merged symbols on the right hand side are modified by iterating through all references contained in the monogram table. To enable merges to be rolled back, copies are made of monogram table entries, and rule array entries prior to a merge. The merge is then performed and tests for problems such as recursion or added ambiguity are then undertaken. If these tests suggest that a merge should not occur then the merge is rolled back. All relevant associative arrays and hash tables are then updated.

WO 00/78022 PCT/AU00/00651 -51 During the merging phase a set of rules known as the merge agenda exists which contains all of the rules that should be examined for merging evidence. At the end of the chunking phase all rules are added to the merge agenda. They are removed from the list, either prior to being examined or when the rule is deleted. Rules are added to the merge agenda when they change, usually due to the merging of two non-terminals. The list of rules are iterated through, by the use of a pointing itertor, as shown in Figure 8. The rules are iterated through in a re-entrant fashion. In this context re-entrant means that a rule could be deleted, and the remaining rules could be iterated through without affect. This is achieved by storing the rules in the global doubly linked list format. As shown in Figure 8, initially the iterator points to rule 1, and then subsequently is incremented to point to rule 2. Then if rule 2 is to be deleted, prior to deletion of the rule, the rule is removed from the doubly linked list, and then the iterator is incremented to point to rule 3.

The model merging process 50 is able to infer grammars of arbitrary complexity for categorical data. It achieves this by assuming that the results of slot specification rules are visible on the end result. The inferred slot specification rules only have assignment operators, i.e. product=isdn or product=$x.product. This technique can be extended to both structured data and integers. Structured data can be included by considering the members of a structure as separate slots. For instance consider the structure date, with four members, year, month, day_of_month, and day_ofweek If during the model merging technique these are represented as four separate slots eg date.year, date.month, date.day_of_month, date.dayofweek} then the model merging process need not be modified.

Numerical data such as integers and floating point numbers can likewise be converted to categorical data. A more useful technique however is to use a predefined grammar for numbers, which can be included, for instance during the templating process. Such a grammar is defined in Appendix 9 for example. To accommodate this grammar the grammar defined in Appendix 10 can be extended to include mathematical operators, such as addition, subtraction, multiplication and division. During the incorporation phase of model merging, any observations that can be parsed by this grammar will be, and generalisation can continue on WO 00/78022 PCT/AU00/00651 -52those other parts of the grammar. To do this however the rules of the predefmed grammar need to be denoted as fixed. In the grammar format used in this document this is noted through the use of the exclamation at the beginning of the rule. Rules that are fixed cannot be altered in any way. To accommodate fixed rules the following modifications are required.

Ifa non-terminal has any fixed rules, then that non-terminal cannot be merged with any other non-terminal.

Fixed rules cannot be deleted, nor can their probability be set to zero. To prevent this occurring, during the reestimation phase all fixed rules have an additional count of 1 added to the count obtained by parsing the examples.

Fixed rules cannot be chunked. If the RHS of a new rule is contained in a fixed rule, it will be substituted. In addition the rule counts using during the chunking phase will not include fixed rules.

Another important feature of the model merging procedure 50 is where the generated grammar cannot be recursive.

During the merging phase 54 the merge is tested to see if it generates a recursive grammar. If it does the merge is reversed. Grammar can be tested for recursion using a number of techniques. A preferred method involves execution of a recursive procedure 200, as shown in Figure 9 which sets a variable current non-terminal to be the top-level grammar at step 202 and then calls a traverse node procedure 250, as shown in Figure 10, which initially operates on an empty list of non-terminals. The procedure 250 is used recursively to test all of the rules in the grammar that have a particular non-terminal on the left hand side of a rule. A list of nonterminals that have been previously checked is passed to the procedure 250 when subsequently called. The current non-terminal is added to the list of previously tested rules at step 252 and at step 254 the next rule with the current non-terminal on the left hand side is accessed. A test is then executed at step 256 to determine if the last rule has been reached. If yes then a variable WO 00/78022 PCT/AU00/00651 -53recursive is set to false at step 258 and the procedure completes. Otherwise if the last rule has not been reached then the next symbol in the rule is retrieved at step 260, and a determination made to see whether the symbol exists in the symbol list, i.e. if a non-terminal appearing on the right hand side already exists in the list of previously tested non-terminals. If so, the variable recursive is set to true at step 264 and the procedure completes. If not, another instance of the procedure 250 is executed on the non-terminal. When an instance of the procedure 250 generated in this manner completes a test is made at step 266 to determine if the variable recursive is true. If so, the procedure completes, otherwise a test is then made to determine whether the last symbol in the non-terminal list has been reached at step 268. If not, operation returns to step 260, otherwise operation will return to step 254 to retrieve the next rule.

To illustrate how the procedures 200 and 250 operate, Table 2 below sets out the sequence of the rules examined and the compilation of the non-terminal list for the following grammar: S a X b S ->de X1 a X2 b X1 S b X2 i Rule Being Examined Non-Terminal List S a X B S (ii) XI X2 b S,XI (iii) X2 i S,X1,X2 (iv) XI a S b S,X1 Grammar is recursive Table 2 The table shows that the top-level grammar S is checked first. When checking the first rule, the non-terminal list contains only the symbol S, and when the symbol Xl is checked in the first rule. the procedure 250 is called recursively to check if any of the rules attached to XI are recursive, and the symbol XI is added to the list. The symbol X2 is encountered and also added to the list. When the symbol S is encountered in the right hand side of the second rule QAOOPERDB W0545-00 spe doc-22O704 -54for XI, the procedure 250 identifies at step 262 that this symbol already exists in the list, and the grammar is then identified as recursive and the procedure completes. When a test is executed to determine whether a grammar is recursive, and the grammar is known not to be recursive prior to executing a merge, the test for recursion can start with the newly merged non-terminal rather than the top-level grammar. This will reduce execution time when the newly merged non-terminal is not a top-level non-terminal.

During the merging phase a test is undertaken to ensure the possibility of introducing ambiguity as a result of merging two symbols is reduced. If the merging of two symbols causes two rules to exist with the same syntax but with differing slot specification rules, where the slot specification rules of the form X=Y, then the merge is rolled back. For instance: XI ->Y:xl Z:x2 {op=a x=$xl.X y= $x2.Y} XI ->Y:X1 Z:X2 {op=a x=$xl.X y= $xl.Y} 15 is acceptable, but Xl Y:xl Z {op=a X=Y Y=$X2.Y} XI Y:xl Z {op=a X=Z Y=$X2.Y} is not acceptable.

20 All of the processes and components of the development system 40 are preferably executed by a computer program or programs. Alternatively the processes and components may be at least in part implemented by hardware circuits, such as by ASICs. The components may also be distributed or located in one location.

Many modifications will be apparent to those skilled in the art without departing from the scope of the present invention as herein described with reference to the accompanying drawings.

The reference to any prior art in this specification is not, and should not be taken as, an acknowledgment or any form of suggestion that that prior art forms part of the common general knowledge in Australia.

WO 00/78022 PCT/AU00/00651 APPENDIX 1 <?xml version='l.0' encoding='utf-8' <APPLICATION NAME="Stock"> <ENUMERATED NAME="stockname"> <ITEM NAME="philp burns"/> <ITEM NAME-"macquarie qanmacs"/> <ITEM NAME-"a i engineering"/> <ITEM NAME="a p eagers limited"/> <ITEM NAME="a a p c limited <ITEM NAME="a a p t limited"/> <ITEM NAME="abador gold <ITEM NAME="abednego nickel"/> <ITEM NAME="aberfoyle limited"/> <ITEM NAME="abigroup limited"/> <ITEM NAME="arthur yates <ITEM NAME="york securities"/> <ITEM NAME="zeolite australia"/> <ITEM NAME="zephyr minerals <ITEM NAME="zicom australia"/> <ITEM NAME="zimbabwe platinum"/> <ITEM NAME="zylotech limited"/>

</ENUMERATED>

<OPERATION NAME="buy" <PARAMETER NAME="stockname" TYPE="stockname"/> <PARAMETER NAME="number" TYPE="INTEGER"/> <PARAMETER NAME="price" TYPE="money"/>

</OPERATION>

<OPERATION NAME="sell" CONFIRM="yes"> <PARAMETER NAME="stockname" TYPE="stockname"/> <PARAMETER NAME="number" TYPE="INTEGER"/> <PARAMETER NAME="price" TYPE="money"/>

</OPERATION>

<OPERATION NAME="quote" CONFIRM= "yes"> <PARAMETER NAME-"stockname" TYPE-"stockname"/> <RETURN NAME="stockname" TYPE="stockname"/> <RETURN NAME="date" TYPE="date"/> <RETURN NAME="price" TYPE="money"/>

</OPERATION>

</APPLICATION>

wo oon8022 WO 0078022PCT/AUOO/00651 56 APPENDIX 2 PROCESS stock; DCL buystocknane STOCKNAME, buynwnber INTEGER, buyprice MONEY, selistockname STOCKNAME, selinumber INTEGER, seliprice MONEY, quotes tockname STOCKNA4E, quotedate DATE, quoteprice MONEY;

START;

OUTPUT TopLeveiStock; NEXTSTATE TopLeveiStock; STATE TopLeveilStock; INPUT buy~stockname, number, price); TASK buystocklae stockname; TASK buynumber =numbier; TASK buyprice =price; CALL Askbuystockname; CALL Askbuynumber; CALL Askbuyprice; CALL Completebuy; NEXTSTATE TooLeveiStock; INPUT sell(stockname, number, price); TASK selistockname stockname; TASK selinumber =number; TASK seliprice =price; CALL Asksellstocknane; CALL Asksellnumber; CALL Askseilprice; CALL ConfirmCompletesell; NEXTSTATE TopLeveiStock; INPUT quote (stockname); TASK quotestockname =stockname; CALL Askquotestockname; CALL Completequote; NEXTSTATE TopLeveiStock; STATE This applies to all states INPUT cancel; OUTPUT operationcancel2led; NEXTSTATE TopLevel; INPUT help; OUTPUT helpstate*; The name of the state is appended to the prompt name1 NEXTSTATE /4 return to the same state 4 INPUT quit; OUTPUT quit;

STOP;

ENDPROCESS Stock; PROCEDURE Askbuystockname;

START;

DECISION buystockname; WO 00/78022 PCTAUO/00651 -57- OUTPUT Askbuystockname; NEXTSTATE WaitReplyAskbuystockname; .11)

RETURN;

ENDDECISION;

STATE WaitReplyAskbuystockname; INPUT buystockname(stockname, number, price); CALL setbuystocknamelfNotNil (stocknarne); CALL setbuynumberlfNotNil (number); CALL setbuypricelfNotNil (price);

RETURN;

ENDPROCEDURE Askbuystocknane; PROCEDURE setbuystocknamelfNotNiil; FPAR IN/OUT stockname STOCKNANE;

START;

DECISION stockname;

RETURN;

TASK buystockname =stockname;

ENDDECISION;

ENDPROCEDURE setbuystocknamelfNotNil; PROCEDURE Askbuynunber;

START;

DECISION buynuinber; OUTPUT As kbuynumber; NEXTSTATE WaitReplyAskbuynunber;

RETURN;

ENDDECISION;

STATE WaitReplyAskbuynumber; INPUT buynumber(nunber, price); CALL setbuynumberlfNotNil (number); CALL setbuypricelfNotNil (price);

RETURN;

ENOPROCEDURE Askbuynumber; PROCEDURE setbuynurnberlfNotNil; FPAR IN/OUT number INTEGER;

START;

DECISION number; 11 11)

RETURN;

111) TASK buynumber =number; WO OOn8O22 WO 0078022PCIT/AUOO/00651 58

ENDDECISION;

ENDPROCEDURE setbuynumberlfNotNil; PROCEDURE As kbuyprice; S TART; DECISION buypkice; OUTPUT Askbuyprice; NEXTSTATE WaitRepiyAskbuyprice; 1111))

RETURN;

ENDDECISION;

STATE WaitReplyAskbuyprice; INPUT buyprice(price); CALL setbuypricelfNotNil (price);

RETURN;

ENDPROCEDURE Askbuyprice; PROCEDURE setbuypricelfNotNil; FPAR IN/OUT price MONEY;

START;

DECISION price;

RETURN;

TASK buyprice price;

ENDDECISION;

ENDPROCEDURE setbuypricelfNotNil; PROCEDURE Completebuy;

START;

CALL buy(buystockname, buynumber, buyprice); OUTPUT buyConfirmed; NEXTSTATE WaitConfirrnbuyEnd; STATE waitConfirmbuyEnd; INPUT continue;

RETURN;

ENDPROCEDURE Completebuy; PROCEDURE Askselistockname;

START;

DECISION sellstockname; OUTPUJT Askseistocknarnc; NEXTSTATE WaitReplyAsksellstockname; 11)

RETURN;

wo oon8022 WO 0078022PCT/AUOO/00651 59

ENDDECISION;

STATE WaitReplyAsksellstocknane; INPUT selistockname(stockname, number, price); CALL setsellstocknamelfNotNii (stockname); CALL setsellnumberlfNotNil (number); CALL setsellpricelfNotNil (price);

RETURN;

ENDPROCEDURE Asksellstockname; PROCEDURE setsellstocknamelfNotNil; FPAR IN/OUT stockname STOCKNAME;

START;

DECISION stockname;

RETURN;

TASK selistockname stockname;

ENDDECISION;

ENDPROCEDURE setsellstocknamelfNotNil; PROCEDURE Asksellnumber;

START;

DECISION selJlnumber; 11) OUTPUT Asksellnumber; NEXTSTATE WaitReplyAsksellnunber;

RETURN;

ENDDECISION;

STATE WaitReplyAsksellnumber; INPUT sellnuiuber(number, price); CALL setsellnumberlfNotNil(number); CALL setsellpricelfNotNil (price);

RETURN;

ENDPROCEDURE Asksellnumber; PROCEDURE setsellnumberlfNotNil; FPAR IN/OUT number INTEGER;

START;

DECISION number;

RETURN;

111 TASK selinumber number;

ENDDECISION;

ENDPROCEDURE setsellnumberlfNotNil; PROCEDURE Asksellprice; WO 00/78022 WO 0078022PCT/AUOO/00651 60

START;

DECISION seilprice; OUTPUT Askseilprice; NEXTSTATE WaitReplyAsksellprice;

RETURN;

ENDDECISION;

STATE WaitReplyAskseliprice; INPUT seliprice (price); CALL setseilpricelfNotNil (price);

RETURN;

ENDPROCEDJRE Askseliprice; PROCEDURE setseiiprice~fNotNil; EPAR IN/OUT price MONEY;

START;

DECISION price;

RETURN;

111) TASK seliprice price;

ENDDECISION;

ENDPROCEDURE setsellpricelfNotNii; PROCEDURE ConfirmCompleteseil;

START;

OUTPUT AskConfirnsell.(selilstockname, seilnuinber, seliprice); NEXTSTATE WaitAskConfirmsell; STATE WaitAskConfirmseil; INPUT yes; CALL seli(seiistockname, selinumber, seliprice); OUTPUT seilConfirmed; NEXTSTATE WaitAskConfirmseilEnd; INPUT no; OUTPUT seliCancelled; NEXTSTATE WaitAskConfirnsellEnd; STATE WaitAskConfirmsellEnd; INPUT continue;

RETURN;

ENDPROCEDURE ConfirmCompletesell; PROCEDURE Askquotestockname;

START;

DECISION quotestocknane; OUTPUT Askquctestockname; NEXTSTATE WaitReplyAskquotestockname; 1.) WO OOf78O22 WO 0078022PCTAUOO/00651 61

RETURN;

ENDOECISION;

STATE WaitReplyAskquotestockname; INPUT quotestocknane (stockname); CALL setquotestocknamelfNotNil (stockname);

RETURN;

ENDPROCEDURE Askquotestockname; PROCEDURE setquotestocknamelfNatNil; FPAR IN/OUT stockname STOCKNAME;

START;

DECISION stockname;

RETURN;

TASK quotestocknane -stockname; ENDDECIS ION; ENDPROCEDURE setquotestocknamelfNotNil; PROCEDURE Compietequote;

START;

CALL quote(quotestockname, quotedate, quoteprice OUTPUT quoteConfirmed(quotestockname, quotedate, quoteprice NEXTSTATE WaitConfirmquoteEnd; STATE waitConfirmquoteEnd; INPUT continue;

RETURN;

ENDPROCEDURE Compietequote; WO 00/78022 PCT/AU00/00651 -62- APPENDIX 3 Clips output (deffacts Stock-questions (question (name "Askbuystockname") (interactiontype "ask") (prompt "Please say the stockname") (grammar ".WaitAskbuystockname") (question (name "helpstateAskbuystockname") (interactiontype "ask") (prompt "Please say the stockname. For example, b h p, commonwealth bank You can say cancel to go to the top level.") (grammar ".WaitAskbuystockname") (question (name "Askbuynumber") (interactiontype "ask") (prompt "Please say the number") (grammar ".WaitAskbuynumber") (question (name "helpstateAskbuynumber") (interactiontype "ask") (prompt "Please say the number. For example, three hundred and twenty five You can say cancel to go to the top level.") (grammar ".WaitAskbuynumber") (question (name "Askbuyprice") (interactiontype "ask") (prompt "Please say the price") (grammar ".WaitAskbuyprice") (question (name "helpstateAskbuyprice") (interactiontype "ask") (prompt "Please say the price. For example, three dollars and fifty cents. You can say cancel to go to the top level.") (grammar ".WaitAskbuyprice") (question (name "buyConfirmed") (interactiontype "tell") (prompt "buy operaticn confirmed") (grammar ".none") (question (name "Asksellstockname") (interactiontype "ask") (prompt "Please say the stockname") (grammar ".WaitAsksellstockname") (question (name "helpstateAsksellstockname") (interactiontype "ask") WO 00/78022 PCT/AU00/00651 -63- (prompt "Please say the stockname. For example, b h p, commonwealth bank You can say cancel to go to the top level.") (grammar ".WaitAsksellstockname") (question (name "Asksellnumber") (interactiontype "ask") (prompt "Please say the number") (grammar ".WaitAsksellnumber") (question (name "helpstateAsksellnumber") (interactiontype "ask") (prompt "Please say the number. For example, three hundred and twenty five You can say cancel to go to the top level.") (grammar ".WaitAsksellnumber") (question (name "Asksellprice") (interactiontype "ask") (prompt "Please say the price") (grammar ".WaitAsksellprice") (question (name "helpstateAsksellprice") (interactiontype "ask") (prompt "Please say the price. For example, three dollars and fifty cents. You can say cancel to go to the top level.") (grammar ".WaitAsksellprice") (question (name "AskConfirmsell") (interactiontype "ask") (prompt "According to the system you want to sell. The stockname is *stockname* The number is *number, The price is *price* Is this correct?") (grammar ".WaitAskConfirmsell") (question (name "helpstateAskConfirmsell") (interactiontype "ask") (prompt "Please say yes or no. You can say cancel to go to the top level.") (grammar ".WaitAskConfirmsell") (question (name "sellCancelled") (interactiontype "tell") (prompt "sell operation cancelled") (grammar ".none") (question (name "sellConfirmed") (interactiontype "tell") (prompt "sell operation confirmed") (grammar ".none") (question (name "TopLevelStock") (interactiontype "ask") WO 00/78022 WO 0078022PCT/AUOO/00651 64 (prompt "Welcome to the Stock application. Please say one of the following buy, sell, quote") (grammar ".TopLevelStock") (question (name "helpstateTopLevelStock") (interactiontype "ask") (prompt "Please say one of the following options buy, sell, quote") (grammar Topbevel Stock") (question (name "operationcancelled") (interactiontype "ask") (prompt "operation cancelled. Please say one of the following options buy, sell, quote") (grammar ".TopLevelStock") (question name quit) interactiontype "end") (prompt "goodbye") WO 00/78022 PCT/AU00/00651 APPENDIX 4 Rule IsFixed LHS RHS HyperParam RefCount FeatureSpecRule IsFixed I FeatureSpecRule FeatureSpecifications I LHS NonTerminal ProductRule RHS I RHS ProductRule; RHS NonTerminal I NonTerminal Variable I Terminal; FeatureSpecifications FeatureSpecification I FeatureSpecification FeatureSpecifications; FeatureSpecification AsignmentRule I FunctionRule I FunctionRule Function '('AsignmentRule AsignmentRule Function 'add' I 'mul' I 'div' I 'sub'; AsignmentRule Feature Value I Feature Variable Feature I Feature Integer Feature I Feature Integer Feature; WO 00/78022 WO 0078022PCT/AUOO/00651 66 APPENDIX !CommonStockCmds ->CancelActions (1000,I) loperation=cancel I !CommonStockCmds ->RepeatAction (1000,1) (operation=repeat !ComronStockCmds ->ShortHelpActions (1000,1) loperation-help !CommonStockCmds ->QuitActions (1000,1) foperation=quitI Stockname philp burns stockname="philp burns" Stockname ->philp burns I stockname="philp burns" Stockname ->macquarie qanmacs I stockname="macquarie qanmacs" I Stocknane i engineering stockname="a i engineering") Stockname p eagers limited I stockname="a p eagers limited" Stockname a p c limited (stockname="a a p c limited Stockname a p t limited {stockname="a a p t limited" Stockname ->abador gold J stockname="abador gold" S:ockname ->abednego nickel Istockname="abednego nickel" Stockname -~aberfoyle limited f stockname="aberfoyle limited" Stockname ->abigroup limited stockname="abigroup limited" Stockname ->arthur yates f stockname-="arthur yates I Stockname ->york securities stockname="york securities" Stockname ->zeolite australia (stockname="zeolite australia' I Stockname ->zephyr minerals (stockname="zephyr minerals Stockname ->zicom australia I stockname="zicom australia" Stockname ->zimbabwe platinum f stockname="zimbabwe platinum" I Stockname ->zylotech limited I stockname="zylotech limited" .WaitReplyAskbuystockname CominonStockCmds:c (1000,1) (operation-Sc. operationI .WaitReplyAskbuystockname Stockname:x (1000,1) (operation~buystockname stockname=$x.stockname) .WaitReplyAskbuyprice CormonStockCmds:c (1000,I2) loperation=Sc.operation 1 .WaitRep2.yAskbuyprice Money:x (1000,1) (operation-buyprice price.dollars-$x.price.dollars price.cents=Sx.price.cents GR -buy ->BuyNoun (1000,1) BuyNoun ->buy (1000,1) .WaitReolyAsksellstockname Cormmon t oc kCmds :c (100011) loperation=$c.operation) .WaitReplyAsksellstockname Stockname:x (1000,1) {operation=sellstockname stockname-Sx.stockname) .WaitReplyAskbuynumber CommonStockCmds:c (1000,I.) loperation=Sc.operation 1 .WaitReplyAskbuynumber Number:x (1000, 1) (number=$x.nwnber operation=buynunber) .WaitReplyAsksellnumber CommonStockCmds:c (1000,1) (operation=$c.operation .waitReplyAsksellnumber Number-x (1000,1) {number=$x.number operation~sellnumber) .WaitReplyAsksellprice CornmonStockCmds:c (1000,1) (operation=$c.operatiofl .WaitReplyAsksellprice oney:x (1000,1) (operation=sellprice price.dollars=$x.price.dollars price.cents=$x.price.cents I GR sell ->SellNoun (1000,1) SeilNoun ->sell (1000,1) .WaitAskConfirmbuy CommonStockCmds:c (1000,1) (operation=Sc.operation .W..aitAskCcnfirmrbuy Confirmation~c (1000,1) (operation=$c.conf'irm I .WaitReplyAsicConfirmsell ->CommonS toc kCmcs :c (1000,1) (operation=$c.operation .WaitReplyAskConfirmsell ->Confirmation:c (1000,1) (operation=$c.confirm WO 00178022 PCT/AUOO/00651 67 .TopLevelStock ->CornmonStockCrnds:c (1000,1) {operation-$c.operation .TopLevelStock ->GR_buy (1000,1) toperation=buy .TopLevelStock ->GR-sell (1000,1) loperation=sell wo oon8022 WO 0078022PCT/AUOO/00651 68 APPENDIX 6 CommonStockCmds CancelActions return <operation cancel>])) (RepeatAction )-0.25f return H <operation repeat>]f ShortHelpActions return <operation help>]))l QuitActions return U( <operation quit>])) Stockname philp burns )-0.05555S6( return ([<stockname "philp burns">]))I ?hilp burns ]-0.0555556( return ([<stockname "philp burns">])) -nacquarie qanmacs )-0.0555556( return (f <stockname "macquarie qanmacs 1 a i engineering )-0.0555556( return (f <stockname "a i engineering">))) a p eagers limited )-0.05555561 return (f <stockname "a p eagers limited">] M a a p c limited )-0.0555556{ return (f<stockname "a a p c limited a a p t limited )-0.05555561 return ([<stockname "a a p t limited">])) abador gold )-0.0555556( return <stockname "abador gold (abednego nickel )-0.0555556[ return <stockname "abednego nickel">]) aberfoyle limited )-0.05555561 return (f <stockname "aberfoyle limited">] (abigroup limited )-0.0555556( return (f <stocknane "abigroup limited">]) Carthur yates )-0.0555556( return <stockname "arthur yates york securities )-0.0555556( return U<stockname "york securities">]))) zeolite australia )-0.0555556( return Cf <stockname "zeolite australia">]) zephyr minerals )-0.0555556{ return Cf<stockname "zephyr minerals (zicom australia )-0.05555561 return (f <stockname "zicom australia">])) zimbabwe platinum )-0.0555556i return ([<stocknane "zimbabwe platinum">]) (zylotech limited )-0.0555556{ return U <stockname "zylotech limited">] M .TopLevelStock CommonStockCmds:c )-0.333333( return <operation $c.operation>]) buy )-0.3333331 return (U <operation buy>])) sell )-0.3333331 return <operation sell>))l .WaitReplyAskbuystockname CommonStockCmds:c )-0.51 return U( <operation $c.operation>])} Stockname:xl return <operation buystockname> <stockname $xl .stockname>])) 1 WaitReplyAskbuyprice CommonStockCmds:c return <operation $c.operation>])l Money:xl return <operation buyprice> <price.dollars Sxl.price.dollars> <price.cents $xl.price.cents>))) WaitReplyAsksellstockname CCommonStockCmds:c return (C <operation Sc.operation>])} Stockname.-xl 1-0.5( return Uf <operation selistockname> <stockname $xl.stockname>]) 1 WaitReplyAskbuynumber ComxnonStockCmds:c )-0.51 return (f <operation $c.operation>]))) CNuznber:xl return <number Sxl.nuniber> <operation buynuxnber>])l wo oon8022 WO 0078022PCTAUOOIOO651 69 .WaitReplvAsksellnumbe rI Commonst-ockCrds:c )-0.51 return U( <operation $c.operation>])l Number:xl return <number Sxl.number> <operation sellnuxnber>])1 .WaitReplyAsksellprice CommonStockCnds:c return (f <operation $c.operation>])l Money:xl )-0.51 return (f <operation seliprice> <price.dollars Sxl.price.doilars> <price.cents Sxl.price.cents>J)l .WaitAskConfirmbuy CommonStockCmds:c return (f <operation $c.operation>])1 Confirmation:c return <operation $c.confiru>l)) .WaitReoiyAskConfirmsell cConmonStockCnds:c return (f <operaticn $c.operation>]fl Confirmation:c return H[ <operation Sc.confirm>])) wo oon8022 WO 0078022PCT/AUOO/00651 APPENDIX 7 commonStockCmdsf CancelActions (<operation cancel>) RepeatAction [<operation repeat>) ShortHelpActions (<operation help>) QuitActions (<operation quit>) Stockname (philp burns) Jreturn("philp burns")) (macquarie qanmacs) {return("macquarie carnacs") I (a i engineering) (return("a i engineering")) (a p eaqers limited) freturn("a p eagers l1imited")) (a a p c limited (return("a a p c limited ")I fa a p t limited) (return("a a p t limited')) (abador gold freturn("abador gold (abednego nickel) (return("abednego nickel")) (aberfoyle limited) {return("aberfoyle limited")) (abigroup limited) (return("abigroup limited")) (arthur yates freturn("arthur yates (york securities) freturn("york securities")) (zeolite australia) ireturn("zeoiite australia")) (zephyr minerals (return("zephyr minerals (zicom australia) {returnk"zicom australia")) (zimbabwe platinum) (return("zimbabwe platinum")} (zylotech limited) (return("zylotech limited")) .WaitAskbuystockname CommonStockCmds (stockname:x) f <operation buystockname> <stockname .WaitAskbuynumber CornmonStockCmds (INTEGER:x) I <operation buynumber> <number .WaitAskbuyprice CommonStockCmds (money:x) f <operation buyprice> <price GR buy bGuyNoun BuyNoun buy .WaitAsksellstockname I CommonStockCmds (stockname:x) <operat-ion sellstockname> <stockname .WaitAskselinumber CornmonStockCmds wo oon8022 WO 0078022PCT/AUOO/00651 71 (INTEGER:x) J <operation seliriumber> <number $x>l .WaitAsksellprice CommronStockCmds (money:x) I <operation seliprice> <price GR-sell seliNoun 1 sellNoun sell .WaitAskCon-firmsell ComrnonStockCmds LOOSE CONFIRMATION:c {<operation Sc>} .TopLevelStock[ CommonStockCmds GR buy {<operation buy>) GR sell (<operation sell>) j WO OOn8O22 WO 0078022PCT/AUOO/00651 72 APPENDIX 8 <STRUCTURE NAME=date"> <MEMBER NANE='month"/> <MEMBER NAVE="dayl" TYPE=integer" <MEMBER NAME='year 'rYPE="integer"/> <MEMBER NAME="dayof -week"!> <MEMBER NAME="moiifier"/>

</STRUCTUJRE>

<STRUCTURE NAE="price"> <MEMBER NAME="dollars" TYPE='integer"/> <MEMBER NAME-"cents" TYPE="integer"/> <MEMBER NAME="rnocifier'

</STRUCTURE>

<STRUCTURE NAME="price'> <MEMBER NAME="hours "1> <MEMBER NAME="minutes"/> <MEMBER NANE-"seconds"/> <MEMBER NAME="am or mI/> <MEMBER NAME="mojifler"/>

</STRUCTURE>

WO OOn8O22 WO 0078022PCTAUOOIOO651 73 APPENDIX 9 Number tpToHundred:n {number=Sn.number !UpToHundred ->Digit:n {nunber=Sn.number 'UpToHundred ->Teens:n (number=$n.number I Number Stricti-undredToThousand:n tnumber=Sn.number Zero zero {number=O) Zero nought {number=0 Zero ->oh Inumber=0 'Zero ->no {number=O NonZero ->one (number-1I NonZero ->two (number-2I NonZero ->three (nuxnber=3 NonZero ->four Jnumber=4 'Nonzero ->five 'Nonzero ->six Inumber=6 NonZero ->seven Inumber=7 I 'NonZero ->eight Inumber=8 NonZero ->nine Inumber=9 !Digit Zero {number=0 Digit NonZero:n {nuiber=$n.number TwentyToNinety ->twenty TwentyToNinety ->thirty 'TwentyToNiflety -forty !TwentyToNiflety ->fifty (nuinber=50 'TwentyToNinety ->sixty (number=60 TwentyToNiflety seventy TwentyToNinety ->eighty TwentyToNinety ->ninety !Teen ->eleven {number=11 Teen ->twelve {nuinber=12 Teen ->thirteen (number=13 'Teen ->fourteen {number=14 Teen ->fifteen 'Teen ->sixteen {number=16 Teen ->seventeen {nurnber=17I Teen ->eighteen fnumber=18I Teen ->nineteen {number=19 Teens ->ten inumber=10 !Teens ->TwentyToNinety:i (number=$n.number !Teens ->Teen:n (number-Sn.numberI !TwentySome TwentyToNinety:nl NonZero:n2 1 number-acd(Snl. number Sn2.number) Teens TwentySome:n inumber=$n.number !UpToThousand Teens:n (number=$n.numberI UpToThousand -NonZero:n {number=Sn.numberI 'HundredSlot ->hundred (number=100I HundredSlot hundred {number=100 HundredSlot ->NonZero:n hundred (number=Tnul($n.number 100) 'HundredToThousand HundredSlot:nl and UpToThousand:fl2 (1,1) (number=add(Snl.number $n2.number) !HundredToThousand HundredSlot:nl UpToThousand:n2 (1,1) (number=add(Snl.number $n2.number) 11StrictHundreciToThousand HundredSlot:n (number='$n.number :strictmundred[ToThousarld HundredToThousand:n Inumber=$n.nunber Number ThousandToMillion:n (number=$n.number I ThousandToMillion a thousand {number=1000 I ThousandSlot UpToTenThousand:n thousand (number-mul number 1000) wo oon8022 WO 0078022PCT/AU00100651 74 ThousandToMillion ->ThousandSlot:n Inumber=Sn.number !ThousandToMillion ->ThousandAndSome:n {number=Sn.numberI !ThousandAndSome ThousandSlot:nl tpToTenThousald:fl2 (111) (number=add($nl.number $n2.number) !ThousandAndSome ThousandSlot:nl and UpToTenThousand:n2 (141) (number=add(Snl.number Sn2.number) !tpToTenThousand HundredSlot:n (number=Sn.number !UpToTenThousand HundredToThousand:n (number=$n.numberI !UpToTenThousand UpToThousand:n (number=$n.number !ThousandToMillion ->ManyHundred:n [nuxnber=$n.flulber !ThousandToMillion ->ManyHundred and some:n Inumber=$n.nuinber I Manyflundred TwentySome:n hundred (nuiber=mul($n.number 100) 'ManyHundred Teen:n hundred (number=mul($n.number 100) 1 ManyHundred_and some Many~undred:nl UpToThousand:n2 (1,1) (number=add(Snl.nunber $n2.nuniber)) ManyHundred_and -some ManyHundred:nl and UpToThousand:n2 (1,1) {number=add(Snl.number $n2.number)) -The following rules set the number modified.

-They float in this grammar but can be included as detected 1 ->by the model merging algorithm during the incorporation phase Number modifier ->less than {number.modifier=less-than Number modifier ->at most Inumber.modifier=less-than) Number modifier ->no more than (number.modifier=less-than Number modifier ->greater than {number.modifier=greater than 'Number-modifier ->at least {number.modifier-greater -than I Number modifier ->no less than (number.modifier-greater than Number modifier ->all inumber.modifier~all Number-modifier ->every {number.modifier~every WO 00178022 WO 0078022PCT/AUOO/00651 75 APPENDIX Grammar generated by tools with predefined grammars included !CommonStockCmds ->CancelActions {operationcancel !CommonStockCmds ->RepeatAction (operation=repeat !CommonStockCmds ->ShortHelpActions {operation=help !CommonStockCmds ->QuitActions (operation=quit) Stocknarne ->philp burns (stockname-"philp burns" Stockname ->philp burns Istockname-1philp burns") Stocknarne ->macquarie qanmacs istockname-"macquarie qanmacs"1 Stockname i engineering {stockname="a i engineering" Stockname p eagers limited (stockname="a p eagers limited" I Stockname a p c limited il stockname="a a p c limited I Stockname a p t limited {stockname="a a p t limited" Stockname ->abador gold (1 stockname="abador gold Stockname ->abednego nickel {stockname="abednego nickel"I Stockname ->aberfoyle limited istockname="aberfoyle limited" I Stockname ->abigroup limited (14l) {stockname="abigroup limited" I Stockname ->arthur yates 11 stockname="arthur yates" Stockname ->york securities {stockname="york securities" Stockname ->zeolite australia (stockname="zeolite australia", I Stockname ->zephyr minerals (1 stockname="zephyr minerals" Stocknane ->zicom australia (stockname="zicom australia" Stocknazue ->zimbabwe platinum {stockname="zimbabwe platinum" 1 Stockname ->zylotech limited {stockname="zylotech limited" .Wait Repl yAs kbuystocklwfe CornmonStockCmds:c (operation-$ c.ope ration .WaitReplyAskbuystockname Stockname:x (operationbuystocklame stockname=$x. stockname) .WaitReplyAskbuyprice CommonStockCmds:c loperation=$c. operation JdaitReplyAskbuyprice Money:x {operation-buyprice price.dollars=$x.price.dollars price.cents=$x.price.cents GR buy ->BuyNoun (1,1) BuyNoun ->buy (1,1) .WaitReplyAsksellstocklame ->CommonStockCmds: c (1,1) operation=$c. operationI .WaitReplyAsksellstockname Stockname:x (ope rat ion=se llstockname stockname=$x.stockname .WaitReplyAskbuynumber ->ComxnonStockCmds:c foperation=$c.operatiofl .WaitReplyAskbuynumber Number:x (number=Sx.number operation=buynumberI .WaitReplyAsksellnumber ->CommonStockCmds:c (operation=$c.operatiol .WaitReplyAsksellnunber Number:x inumber=Sx.number operation=sellnumber I .WaitReplyAsksellprice CommonStockCmds:c {operation=$c.operatioTn .WaitReplyAskseliprice Money:x (operation=sellprice price-dollars=$x.price.dollars price.cents=$x.price.celtsI GR sell ->SellNoun (1,1) SellNoun ->sell (1,I) .WaitAskConfirmbuy CommonStockCmds:c {operation=$c.operation .WairAskConfirmbuy Confirmation:c (operation=$c.confirm .WaitReplyAskConfirnsell ->CoronStockCmds:c {operation=$c.operation laitReplyAskConfirmsell Confirmation:c {operation=$c.confirm I .TopLevelStock CornmonStockCmds:c toperation=Sc.operation

I

WO OOnSO22 WO 0078022PCT/AUOO/00651 76 .TopLeve!Stock ->GRbuy {operation=buy .TopLeveiStock -GR -sell (operation~'sellI Number UpToHundred:n (14l) {nuinber=$n.nuxnber !UpToHundred ->Digit:n (141) Inurber=Sn.nurnber !UpToHundred ->Teens:n Inumber=$n.number Number StrictHundredToThousand:n Inuxber=$n.number Zero ->zero {number-O I Zero ->nought inumber=0 Zero ->oh 1) 1 number=0 I !Zero ->no Inum~ber=0 I !NonZero one Inuxnber=1 NonZero two Inuxnber=2 !NonZero ->three (number-3 NonZero ->four (number-4 !Nonzero ->five NonZero ->six {number=6 NonZero ->seven {nurnber=7 NonZero ->eight {number=8I NonZero nine (number=9 !Digit ->Zero Inumber=0 I Digit ->NonZero:n (number-$n.number TwentyToNinety twenty !TwentyToNinety thirty !TwentyToNinety forty (number=40 !TwentyToNinety fifty !TwentyToNinety sixty (number=60 !TwentyToNinety ->seventy !TwentyToNinety ->eighty TwentyToNinety ->ninety !Teen ->eleven Inumber=11 !Teen ->twelve Inumber=12 !Teen ->thirteen (nuznber=13 Teen ->fourteen inumber=14 I !Teen ->fifteen !Teen ->sixteen (number=16 Teen ->seventeen Inumber=17 Teen ->eighteen inumber=18 Teen ->nineteen {number=19 Teens ->ten !Teens ->TwentyToNinety:n (14l) (nuiber-$n.number Teens ->Teen:n (number-Sn.number 'TwentySome TwentyToNinety:nl NonZero:n2 {number=add number $n2. number)I Teens TwentySome:n inumber=$n.nurnber !UpToThousand Teens:n fnumber=Sn.number I UpToThousand ->NonZero:n Inumber=$n.number HundredSlot ->hundred (number=100) HundredSlot hundred (number-l00 !HundredSlot ->NonZero:n hundred (number=mul($n.number 100) !HundredToThousand HundredSlot:nl and UpToThousand:n2 (1,1) (number-add(Snl.number Sn2.number) I !HundredToThcusand HundredSlot~nl UpToThousand:n2 (111) (number=add ($nl number Sn2.number)I !StrictHundredToThousand ->HundredSlot:n {number=$n.number I !StrictHundredToThousand ->HundredToThousand:n {number=$n.number 1 Number ThousandToMillion:n (number=Sn.nunber I !ThousandToMillion a thousand 1rnumber=1000 I !ThousandSlot UpToTenThousand:n thousand (number=mul number 1000) !ThousandToMillion ->ThousandSlot:n *{number=$n.number I !ThousandToMillion ->ThousandAndSome:n inumber=Sn.number I WO 00/78022 WO 007R022PCT/AUOO/00651 77 !ThousandAndSome ThousandSlot:nl UpToTenThousand:n2 (1,1) [number=add($nl.number $n2.number)) !ThousafldAr'dSome ThousandSlot:nl and UpToTenThousand:n2 (1,1) (number=add(Snl.number $n2.number)) !UpToTenThousand ->HundredSlot:n (14l) {number=$n.number UpToTenThousand ->HundredToThousand:n (number=$n.number !UpToTenThousand ->UpToThousand:n {number=$n.numberI !ThousandToMillion ->ManyHundred:n [number=$Sn-number 'ThousandToMi4llion ->ManyHundred_and some:n inumber=$n.nunber !Many~undred -TwentySome:n hundred (number=mul($n.number 100)1 'ManyHundred ->Teen:n hundred inumber-mul($n.number 100)1 ManyHundred_and -some ManyHundred:nl UpToThousand:n2 (1,1) (number=add($nl-number Sn2.number) I ManyHundred_and -some ManyHundred:nl and UpToThousand:n2 (1,1) {number=add($nl.nunber Sn2.number)I -The following rules set the number modified.

-They float in this grammar but can be included as detected -by the model merging algorithm during the incorporation phaseI Number modifier ->less than (number.modifier-less than I !Number-modifier ->at most {number.modifier=less than I Number modifier ->no more than {number.modifier=less thanI Number-modifier ->greater than {number.modifier=greater -than Number modifier ->at least {number.modifier=greater -than I Number modifier -no less than (14l) {number.modifier=greater-than !Number-modifier ->all mnumber.modifier=all !Number modifier ->every inumber.modifier-every 'Money Number:d DOLLARS UpToHundred:c (price. cents=$ c. number price. dollars=$d .numberI !Money Number:d Money2:c {price.cents=$c.price.cents price.dollars='$d.number) !Money Number:d Money2:c CENTS {price.cents=Sc.price.cents price. dollars=$d. number I !Money ->UpToHundred:c CENTS (price.cents=$c.number price.dollars-0 !Money ->Number:d DOLLARS (price.cents=O Drice.dollars=$d.number Money2 ->DOLLARS and UpToHundred:c fprice.cents=$c- numberI DOLLARS -dollars (1,1) DOLLARS ->dollar (1,1) CENTS ->cent (1,1) !CENTS ->cents (1,1)

Claims (30)

1. A method of developing an interactive system, including: inputting application data representative of an application for said system, said application data including operations and parameters for said application; generating prompts on the basis of said application data; and generating grammar on the basis of said application data.

2. A method as claimed in claim 1, wherein said prompts and grammar are generated on the basis of a predetermined pattern or structure for said prompts and grammar.

3. A method as claimed in claim 2 wherein said grammar includes predefined grammar.

4. A method as claimed in claim 1, including generating a dialogue state machine on the basis of said application data. A method as claimed in claim 4, wherein said grammar and said state machine include slots defining data on which said interactive system executes the operations.

6. A method as claimed in claim 5, wherein said slots include value data representing the meaning of phrase or term of a slot.

7. A method as claimed in claim 6, wherein said application data is inputted in an application file, and the operations include a number of input and return parameters with parameter types.

8. A method as claimed in any one of claims 3 and 5 to 7, including executing grammatical inference to enhance the grammar.

9. A method as claimed in claim 8, wherein executing said grammatical inference includes executing a model merging process, including: WO 00/7802 PCT/AU00/00651 -79- processing rules of the grammar; generating additional rules representative of repeated phrases; and merging equivalent symbols of the grammar; wherein said rules define said slots and include said symbols. A method as claimed in claim 9, wherein said rules include slot specification rules including key value data representing the meaning of a phrase or term for a slot.

11. A method as claimed in claim 9 or 10, wherein said grammar is hierarchical and said rules include terminal and/or non-terminal symbols, whereby said rules refer to lower level rules to resolve non-terminal symbols.

12. A method as claimed in claim 1 1, wherein said rule generating step includes generating a non-terminal symbol rule from correlated symbols and slot specification rules.

13. A method as claimed in claim 12, wherein said merging step includes identifying interchangeable symbols on the basis of predetermined merging evidence patterns.

14. A method as claimed in claim 13, wherein said merging step includes determining whether symbols to be merged have compatible slot specification rules and return corresponding slots. A method as claimed in claim 14, wherein said rules include a hyperparameter representing use of the rule in observations parsed during said grammatical inference.

16. A method as claimed in claim 15, wherein said evidence patterns represent relationships between rules indicating a merger and corresponding rule formats to be generated when one of said relationships exist between said rules. WO 00/78022 PCT/AU00/00651

17. A method as claimed in claim 9 or 16, wherein said rules include a reference count representing the number of other rules that reference the rule.

18. A method as claimed in claim 9 or 17, wherein said rules are determined on the basis of attribute constraints during said creating step.

19. A grammatical inference method for developing grammar, including processing rules of the grammar, creating additional rules representative of repeated phrases, and merging equivalent symbols of the grammar, wherein said rules define slots to represent data on which an interactive system executes operations and include symbols representing at least a phrase or term. A method as claimed in claim 19, wherein said rules include slot specification rules including key value data representing the meaning of a phrase or term for a slot.

21. A method as claimed in claim 19 or 20, wherein said grammar is hierarchical and said rules include terminal and/or non-terminal symbols, whereby said rules refer to lower level rules to resolve non-terminal symbols.

22. A method as claimed in claim 21, wherein said rule generating step includes generating a non-terminal symbol rule from correlated symbols and slot specification rules.

23. A method as claimed in claim 21, wherein said merging step includes identifying interchangeable symbols on the basis of predetermined merging evidence patterns.

24. A method as claimed in claim 23, wherein said merging step includes determining whether symbols to be merged have compatible slot specification rules and return corresponding slots. WO 00/78022 PCT/AU00/00651 -81 A method as claimed in claim 21, wherein said rules include a hyperparameter representing use of the rule in observations parsed during said grammatical inference.

26. A method as claimed in claim 23, wherein said evidence patterns represent relationships between rules indicating a merger and corresponding rule formats to be generated when one of said relationships exist between said rules.

27. A method as claimed in claim 19, wherein said rules include a reference count representing the number of other rules that reference the rule.

28. A method as claimed in claim 19, wherein said rules are determined on the basis of attribute constraints during said creating step.

29. A development tool for an interactive system having code for executing the steps of the method as claimed in any one of claims 1 to 28. An interactive system developed using the method as claimed in any one of claims 1 to 28.

31. A system for developing an interactive system, including: means for inputting application data representative of an application for said system, said application data including operations and parameters for said application; means for generating prompts on the basis of said application data; and means for generating grammar on the basis of said application data.

32. A development tool for an interactive system, including: code for inputting application data representative of an application for said system, said application data including operations and parameters for said application; code for generating prompts on the basis of said application data; and code for generating grammar on the basis of said application data. Q \OPER\DBWVO5S45-0 spe doc-22704 -82-

33. A method of developing an interactive system substantially as hereinbefore described with reference to the accompanying drawings.

34. A grammatical inference method for developing grammar substantially as hereinbefore described with reference to the accompanying drawings. A development tool for an interactive system substantially as hereinbefore described with reference to the accompanying drawings.

36. An interactive system substantially as hereinbefore described with reference to the accompanying drawings.

37. A system for developing an interactive system substantially as hereinbefore described with reference to the accompanying drawings. 20 DATED this 22nd day of July 2004 TELSTRA NEW WAVE PTY LTD By its Patent Attorneys DAVIES COLLISON CAVE o00o0 **0 0 @0* 0 0 000 0 0 0 0 0 0 00 0 0 0 oooe o oo* o o *oo oo* *oo oo