WO2010076327A1

WO2010076327A1 - Method for analysing an interactive voice response system

Info

Publication number: WO2010076327A1
Application number: PCT/EP2009/068007
Authority: WO
Inventors: John Costello; Tony Mccormack; John Yoakum
Original assignee: Nortel Networks Ltd
Current assignee: Nortel Networks Ltd
Priority date: 2008-12-29
Filing date: 2009-12-29
Publication date: 2010-07-08
Anticipated expiration: 2011-06-29
Also published as: US20100166158A1; US8934618B2

Abstract

Methods for analysing an interactive voice response (IVR) system are disclosed. In the methods, prompts are issued from the IVR system and user input made in response to the prompts from the IVR system is received. One method involves analysing conversation between the caller and the agent with which the caller has been connected whereby to diagnose faults in the IVR system. Another method involves displaying the prompts issued by the IVR system to an operator; and receiving input data from the operator, the data defining whether any change to the IVR system is recommended and/or any recommended changes to the IVR system. Another method involves detecting the occurrence of an unexpected event by comparing the user input against a predefined set of rules, each rule being associated with one or more of the unexpected events; and taking a predefined action in response to the detection of one or more of the unexpected events.

Description

METHOD FOR ANALYSING AN INTERACTIVE VOICE RESPONSE SYSTEM Field of the Invention

[0001] This invention relates to a method for analysing an interactive voice response (IVR) system, and to an IVR system on which the method can be performed.

Background of the Invention

[0002] IVR systems are well known, particularly for the automated handling of telephone calls by large corporations. Often such systems are used to handle simple queries (for example, to tell a customer the balance of his account or to book cinema tickets) or to handle initial call filtering to ensure that a customer is transferred to an operator or agent with the knowledge and skills required to handle a particular type of query.

[0003] A typical IVR system comprises a set of options which are presented to a user in spoken form. The set of options is often defined using a suitable programming language such as VoiceXML. A VoiceXML file can be parsed by a voice browser, which plays the sound associated with each option when it is invoked. The VoiceXML also defines the hierarchy of options so that the voice browser knows which sound it is appropriate to play to the user. The sounds that are played to the user may simply be pre-recorded audio files (for example, MP3 files) or they may be generated by text-to-speech (TTS) software.

[0004] Speech recognition software analyses the responses made by the user to the sounds played by the IVR system and takes appropriate action. This action may be to transfer the user to the correct operator or agent or to cause another sound or set of sounds to be played to the user (for example, if the user has entered a sub-menu).

[0005] Whilst IVR systems are clearly very useful for handling large volumes of simple queries in a timely and cost-effective manner and for handling routing of calls to the appropriate agent, they are notorious for frustrating users. Often users find themselves in the wrong queue and then end up being connected to the wrong agent. Alternatively, users may become so frustrated that they simply hang up. [0006] These problems generally stem from poor design of the menu system used by the IVR, .which may be difficult for a user to understand or be ambiguous or simply too long-winded for a user to listen attentively to.

[0007] Prior solutions have involved using after-call surveys or scorecards to gather the user's opinion on the IVR system, but users are generally reluctant to participate and are not often reliable in the answers they provide. Other options include listening to call recordings to determine whether any problems with the IVR menu options exist and what such problems are, but this is a very labour- intensive and hence expensive process. Summary of the Invention

[0008] According to one aspect of the invention there is provided a method of analysing an interactive voice response (IVR) system for routing voice calls to agents in which at least one prompt is issued from the IVR system, caller input is received in response to the or each prompt, and the call is connected to an agent, the method comprising analysing conversation between the caller and the agent with which the caller has been connected whereby to diagnose faults in the IVR system.

[0009] A speech recognition system may be used to analyse the conversation between the caller and the agent. The speech recognition system may identify utterances, key words, or phrases in the conversation. Keywords could include "transfer" indicating that the agent is having to transfer the caller. As will be discussed in more detail below, the system might simply detect that the caller is uttering sound of disgust or profanities as a result of frustration with the system. Things that can be detected beyond keywords and language indicating frustration would be discussions around 'I need to transfer you to' or 'I want to talk about X' (check to see if X is something the agent the call has been routed to is skilled in), 'You don't seam to be able to help me' and similar things that indicate the true topic of the discussion compared to the skills based routing that occurred. Speech recognition can be phonetic in nature and multilingual or somewhat language independent including recognizing utterances not commonly found in a dictionary. Any set of phonetics recognized can be utilized in analysis and IVR adaptation. [0010] It is important to note here that it is usual in an IVR system for the speech analysis to cease as soon as the caller has been routed to an agent. Typically a speech analytics engine forming part of the IVR system or even the entire IVR system may be disconnected from the caller and agent as soon as the caller and agent are connected.

[0011] According to this aspect of the invention, either the same speech analytics engine that instructed the IVR system or a new speech analytics engine at a call center (or otherwise associated with the agent) analyses the conversation between the caller and the agent after the call has been routed. Either way there are two versions of speech analytics logic being operated, the first instructing the routing of the call and the second analyzing the conversation after the call has been routed in order to determine whether the call has been correctly routed.

[0012] Thus in a preferred method in which the agents are accessed via a contact centre, upon detection of a call to the contact centre the call is connected to the IVR system, the call is then connected to an agent at the contact centre in response to an instruction from the IVR system. Then, a connection between the IVR and the caller is maintained whereby to enable a speech recognition system associated with the IVR to analyse the conversation between the caller and the agent. Alternatively or additionally a connection between the IVR and the agent may be maintained or newly established in order for the conversation to be analysed. This is advantageously achieved with the use of a conference bridge. A conference bridge would enable multiple connections between any of: one or more callers, the IVR and one or more agents.

[0013] In an alternative method, a different speech recognition system associated with the agent analyses the conversation between the caller and the agent after the caller has been routed to the agent. This method comprises upon detection of a call to the contact centre, connecting the call to the IVR system, connecting the call to an agent at the contact centre in response to an instruction from the IVR system and connecting the IVR and/or the caller to a speech recognition system for analysing the conversation between the caller and the agent. A conference bridge may also be used to make the connections in this method. [0014] If a fault in the routing of the call is suspected, it is usually advantageous to display the or each prompt issued by the system.

[0015] In accordance with another aspect of the invention, there is provided a method for analysing an interactive voice response (IVR) system, the method comprising issuing at least one prompt from the IVR system; receiving user input made in response to the or each prompt from the IVR system; displaying the or each prompt issued by the IVR system to an operator; and receiving input data from the operator, the data defining whether any change to the IVR system is recommended and/or any recommended changes to the IVR system.

[0016] By displaying the prompts issued by the IVR system to the operator it is easy for the operator to see the nature of the interaction between the IVR system and the user and hence to determine where any problems may have occurred. This knowledge is key to the suitable reconfiguration of the system to eliminate such problems.

[0017] The display of the prompts may be made either during or at the end of the call or in response to a request made by the operator (for example, if the user explained to the operator during conversation that a problem occurred with the IVR). [0018] By analysing the problems as they occur it is possible to refine the system in an expedient manner, thereby reducing the level of any future problems.

[0019] The or each prompt issued by the IVR system is typically a spoken prompt. It may be generated by playing a pre-recorded sound file to the user or by text-to-speech software.

[0020] The user input is preferably received by a speech recognition system adapted to convert spoken user input into a form understandable by the IVR system. For example, the speech recognition system may convert the detected speech input from the user into text data. [0021] However, in other embodiments the user input may be received in the form of dual-tone multi-frequency (DTMF) signalling. In this case, the prompts will typically request a user to press one of the buttons on their telephone keypad to select an associated option. If a user presses a button on their keypad a DTMF signal is generated which can be detected by the IVR system and decoded by a DTMF decoder. The output from the DTMF decoder indicates to the IVR system which button the user has pressed on their keypad, and hence which option has been selected.

[0022] Typically, the or each prompt forms part of a set of predefined prompts, which collectively define a menu of options for selection by a user. The set of predefined prompts typically includes one or more sub-menus. [0023] In a preferred embodiment, the step of displaying the or each prompt issued by the IVR system to the operator comprises displaying a visual representation of the menu of options and highlighting the options selected by the user. This visual representation may be a textual representation or, preferably, a graphical representation in which the highlighting is performed by displaying the options selected by the user in a brighter colour than the unselected options. This makes the path chosen by the user through the menu system particularly apparent to the operator.

[0024] Preferably, the step of receiving input data from the operator comprises receiving user input defining alterations to the menu of options. These alterations represent the recommended changes to the IVR system and they may be implemented immediately by the IVR system or after review and approval by a supervisor.

[0025] Typically, the step of displaying the or each prompt issued by the

IVR system to the operator comprises displaying a visual representation of the user input made in response to the or each prompt from the IVR system. This is particularly helpful in analysing the nature of any problems as it indicates quite how the user encountered a problem and why. For example, the problem may be a unique problem caused by a user making an irrational or deliberately unhelpful response to a prompt. In such a case, the operator may choose not to recommend any changes. [0026] In accordance with a second aspect of the invention, there is provided a method for analysing an interactive voice response (IVR) system, the method comprising issuing at least one prompt from the IVR system; receiving user input made in response to the or each prompt from the IVR system; detecting the occurrence of an unexpected event by comparing the user input against a predefined set of rules, each rule being associated with one or more of the unexpected events; and taking a predefined action in response to the detection of one or more of the unexpected events.

[0027] This aspect of the invention represents an alternative solution to that presented by the first aspect. It overcomes the problems with existing IVR systems by detecting when unexpected or undesired events have occurred (for example, a user ending up in the wrong sub-menu on an IVR system) by comparing the user input against a predefined set of rules. These rules are constructed to detect when problems occur and have an action associated with them that can be taken. The action may help to ameliorate the situation.

[0028] The or each prompt issued by the IVR system is typically a spoken prompt. It may be generated by playing a pre-recorded sound file to the user or by text-to-speech software.

[0029] The user input is preferably received by a speech recognition system adapted to convert spoken user input into a form understandable by the IVR system. For example, the speech recognition system may convert the detected speech input form the user into text data.

[0030] However, in other embodiments the user input may be received in the form of dual-tone multi-frequency (DTMF) signalling. In this case, the prompts will typically request a user to press one of the buttons on their telephone keypad to select an associated option. If a user presses a button on their keypad a DTMF signal is generated which can be detected by the IVR system and decoded by a DTMF decoder. The output from the DTMF decoder indicates to the IVR system which button the user has pressed on their keypad, and hence which option has been selected. [0031] Typically, the or each prompt forms part of a set of predefined prompts, which collectively define a menu of options for selection by a user. The set of predefined prompts typically includes one or more sub-menus.

[0032] In one embodiment, the set of rules includes one or more rules which analyse whether the received user input is related to a sub-menu other than the current sub-menu. In this case, the predefined action can advantageously be to cause the IVR system to issue the set of predefined prompts which includes the sub-menu related to the received user input. Thus a user that has ended up in an undesired sub-menu can be redirected to the desired sub-menu if he makes an input associated with the desired sub-menu.

[0033] The set of rules may include one or more rules which analyse whether the received user input is recognisable by the IVR system. If it is not then the user may be routed to an agent to handle the query, for example.

[0034] The set of rules may include one or more rules which analyse whether a particular prompt has been issued already to the user and/or whether user input has already been received from the user in response to an issued prompt. Thus, if the user ends up in a loop between sub-menus this can be detected and appropriate remedial action taken.

[0035] The predefined action may be to cause the IVR system to redirect the user to an operator for assistance.

[0036] The predefined action may be to store the prompts issued by the

IVR system and the user input received in response to the prompts for analysis by an operator. This allows an operator to analyse any problems that may have occurred and which the IVR system cannot handle automatically so that appropriate action (for example, redesigning the IVR menus) can be taken.

[0037] In a third aspect of the invention, an IVR system comprises a processor adapted to perform the method of any of the preceding claims.

[0038] In a fourth aspect of the invention, a computer program comprises a set of instructions for performing the method of the first aspect when executed on a processor. [0039] In a fifth aspect of the invention, a computer-readable medium upon which a computer program comprises a set of instructions is stored for performing the method of the first aspect when the computer program is executed on a processor. [0040] Other aspects and features of the present invention will become apparent to those of ordinary skill in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying drawings.

Description of the Embodiments [0041] Embodiments of the invention will now be described with reference to the accompanying drawings, in which:

[0042] Figure 1 shows an IVR system on which the method may be performed.

[0043] Figure 2 shows an example menu structure for the IVR system. [0044] Figures 3, 4 and 5 show flow charts of steps performed by methods of the invention.

[0045] Figure 1 shows an IVR system on which the method of the invention may be performed. The items 1 , 6, 7, 8 and 9 collectively form the IVR system. The IVR system comprises a voice switch 1 which is used for routing incoming voice calls (for example, from a customer's telephone 2). Voice switch 1 may use any conventional call routing technology to transport the voice traffic, for example it may be a circuit switched system or a Voice over Internet Protocol (VoIP) system. The voice switch can route an incoming call to any one of the call centre agents 3, 4 or 5. However, new calls are firstly routed to a media application server 6 by voice switch 1.

[0046] The media application server 6 forms the core of the IVR system and runs voice browser software, which parses a VoiceXML file and issues spoken prompts or requests to a caller. As already explained, the VoiceXML player may either configure the voice browser to replay pre-recorded files which contain the spoken prompts or request, or alternatively may pass character data from the tags in the VoiceXML file to text-to-speech converter software that generates the spoken prompts or requests on the fly.

[0047] The media application server 6 can issue a command to voice switch 1 to cause it to route a call onwards, if appropriate, to one of the call centre agents 3, 4 or 5. This may be done because the caller has selected an option that requires the call to be routed to one of the agents 3, 4 or 5 or because the IVR system cannot understand a response and defaults to routing the call to one of the agents 3, 4 or 5.

[0048] Voice responses made by the caller from telephone 1 are captured by a speech analytics engine 7 connected to the media application server 6. This comprises speech recognition software that converts the words spoken by the caller into machine-readable input (for example, character data). Alternatively the speech recognition software may analyse phonetic sounds in the voice responses rather than comparing or converting them into words or characters. [0049] Although the embodiments described herein refer exclusively to the use of a speech recognition system (i.e. speech analytics engine 7), it is possible to construct a system which comprises a DTMF decoder for detecting DTMF signalling responses made by the user either instead of or in addition to the speech analytics engine 7 in relation to initially determining how to route the call. [0050] The speech analytics engine 7 may refer to a thesaurus 8 which comprises a list of predefined synonyms for certain words. For example, if a caller is asked to choose from a list of options, one of which is "Customer Support" and they respond with "Repairs" (perhaps because they did not hear the options correctly and merely respond with an option they would prefer) then the speech analytics engine 7 may refer to the thesaurus 8 to see if there are any synonyms for "Repairs". If the thesaurus is configured appropriately then it can indicate that "Repairs" is a synonym for "Customer Support", and the appropriate action can be taken in response to the previously unrecognisable user input.

[0051] In addition, the speech analytics engine 7 may comprise software that can analyse the speech of the caller and identify the topic that they are talking about, their emotional state and the ratio of silence to speech. These items may form part of the machine-readable input derived from the captured speech by the speech analytics engine 7.

[0052] The speech analytics engine 7 provides the machine-readable input to the voice browser software running on the media application server to influence the flow of parsing of the VoiceXML file. For example, if "Customer Support" is selected then the voice browser executing on media application server 6 can be instructed to follow an associated link to the correct portion of the VoiceXML file for handling issues related to customer support.

[0053] The speech analytics engine 7 also provides the machine-readable input to an artificial intelligence engine 9, which compares the machine-readable input against a set of rules and if a match occurs takes a predefined action associated with the rules. Examples of the rules and predefined actions will be provided later.

[0054] Nowadays many call centres operate using a conference bridge for connecting a call to an appropriate agent. Thus in the diagram of figure 1 the media application server could implement such a conference bridge (not shown). In this situation, an incoming call would be routed by the voice switch to a conference bridge. The IVR logic would instruct the conference bridge as to which agent or agents the call should be directed to and would connect them to the conference bridge as appropriate.

[0055] An example application will now be described with reference to

Figure 2 and the flow charts of Figures 3 and 4.

[0056] Figure 2 shows an example of a menu structure for use in an IVR system employed by an insurance company. The menu structure comprises a top- level menu with four options "Home Insurance", "Car Insurance", "Customer Services" and "Accounts". Under each of the "Home Insurance" and "Car Insurance" options is a respective sub-menu, each having the options "Request Quote", "Renew Policy", "Cancel Policy" or "Make a Claim".

[0057] The menu structure can be implemented by creating a suitable VoiceXML file. The VoiceXML file will cause the voice browser software running in media application server 6, on answering their call, to play an introductory welcome message and then to ask the caller whether their call relates to a query to do with Home Insurance or Car Insurance or whether they would like to be transferred to the Customer Services or Accounts departments. The voice browser will then wait for further input. [0058] When the caller responds the speech analytics engine 7 will determine the nature of their response and provide the further input to the voice browser, causing it to parse the portion of the VoiceXML file related to the caller's response. For example, if the caller responds that their query relates to car or home insurance then the voice browser will parse the portion of the VoiceXML file that relates to the appropriate sub-menu for either Home Insurance or Car

Insurance. The voice browser will then issue the prompts associated with the submenu, i.e. it will ask the caller whether they would like a quote for insurance, wish to renew their policy, wish to cancel their policy or to make a claim. Again, it will await further input. [0059] When the caller responds the speech analytics engine 7 will determine the nature of the caller's response and provide the further input to the voice browser. The call will then be transferred to one of the agents 3, 4 or 5 with the appropriate skills to deal with the caller's query.

[0060] Figure 3 shows a flow chart which explains how a first embodiment of the invention works in practice with reference to the menu structure of Figure 2. In step 10, the voice browser, having answered a call, issues the welcome message and the voice prompts for the top-level menu, i.e. it asks the caller whether the call is to do with home insurance or car insurance or whether they wish to be transferred to either of the customer services or accounts departments. [0061] In step 11 , the voice browser software running on media application server 6 awaits input from the speech analytics engine 7. The speech analytics engine 7 analyses the voice response from the user and provides machine- readable input to the voice browser software. If the caller has responded that their query is to do with car insurance or home insurance then the voice browser will parse the appropriate section of the VoiceXML file, causing it to issue the voice prompts for the appropriate sub-menu, i.e. for either the car insurance or home insurance sub-menu. [0062] On the other hand, if the speech analytics engine 7 determines that the caller wishes to be transferred to the customer services or accounts department then the voice browser will parse the appropriate portion of the VoiceXML file which will cause the voice switch 1 to transfer the call to an agent 3, 4 or 5 in the customer services or accounts department.

[0063] If the voice browser issued the voice prompts in step 1 1 for either the car or home insurance sub-menus then in step 12 it waits for a response from the caller and then transfers the call to the appropriate agent 3, 4 or 5 to handle the query. [0064] In step 13, the agent 3, 4 or 5 is asked whether he believes the customer's experience with the IVR system was acceptable. If he answers "Yes" then no further action is taken. However, if he answers "No" then in step 14, the menu structure of Figure 2 is displayed to the agent 3, 4 or 5 and the route through the menu structure chosen by the caller highlighted (for example, by displaying it in a different or brighter colour).

[0065] In addition to highlighting the route through the menu structure that the caller has chosen, the voice responses made by the caller can also be replayed to the agent (or if DTMF input is received, the keypad button pressed by the caller can be displayed), for example, by clicking on the appropriate highlighted menu options. This helps the agent to see whether the user's response was appropriate to the circumstances or not. Furthermore, the machine- readable input generated by the speech analytics engine 7 can be displayed to the agent so that any possible problems with analysing the caller's voice can be picked up. [0066] One example of a problem that could occur in the example of Figure

2 is when a customer with a home insurance policy who wishes to check that their payment has been received responds that their query relates to home insurance before listening to all the options in the top-level menu. Thus, they will not realise that they should have requested to be transferred to the accounts department. However, once they have selected the sub-menu for home insurance there is no appropriate option for them to select. They are therefore likely to simply select a random one of the four inappropriate options and will be transferred to the wrong agent. The agent will quickly realise that a problem has occurred and can answer "Yes" in response to the question posed in step 14.

[0067] A second possible problem is that a caller may not realise that to ask a general query related a car insurance policy he need to select the Customer Services option from the top-level menu and instead selects the car insurance option. However, there is no option for asking a general query under the car insurance sub-menu. Again, the caller is likely to select a random one of the four inappropriate options and will be transferred to the wrong agent.

[0068] In step 15, the agent 3, 4 or 5 is asked whether they wish to recommend a change to the menu structure having analysed the route chosen and the voice responses made by the user (and possibly the machine-readable input). If they do not then no further action is taken.

[0069] However, if they do wish to make a change then step 16 captures the details of the changes. These may be made in a graphical environment by dragging and dropping the elements of the existing menu, deleting undesired elements and adding any new elements required. The modified menu structure can then be submitted for review in step 17.

[0070] For example, one modification that would obviate both of the abovementioned problems is to move the replicated elements in each sub-menu (i.e. "Request Quote", "Renew Policy", Cancel Policy", and "Make a Claim") into the top -level menu along with the "Customer Services" and "Accounts" options. Each of the "Request Quote", "Renew Policy", Cancel Policy", and "Make a Claim" options could then have a sub-menu where the caller can select either "Car Insurance" or "Home Insurance". This modification will prevent the caller pre- emptively selecting "Car Insurance" or "Home Insurance" as in the first problem and should prevent the caller from ending up in the sub-menu without an appropriate option to select as in the second problem.

[0071] Figure 4 shows a flow chart for another embodiment of the invention for automatic detection of problems occurring with the usage of the IVR system. The embodiment of Figure 4 may be used alone or in combination with the Figure 3 embodiment. [0072] The Figure 4 embodiment deals with the processing performed by the artificial intelligence engine 9 shown in Figure 1. Steps 18 and 19 illustrate the issue of a voice prompt and a response received from a caller.

[0073] In step 20, the artificial intelligence engine 9 compares the machine- readable input generated by the speech analytics engine 7 with a set of predefined rules. Step 21 determines whether any of the rules is satisfied. If not then no further action is performed. However, if one of the rules is satisfied then a predefined action associated with that rule is taken in step 22.

[0074] The artificial intelligence engine 9 comprises a set of rules, each of which has an associated predefined action. The rules are written to detect the occurrence of an unexpected or undesired event and the predefined actions attempt to ameliorate the occurrence of the event if possible. The rules may also have a priority order, either by including them in the rule set in the order in which they should be processed and always dealing only with the first rule which is satisfied or by associating a priority field with each rule, the priority field indicating the relative priority of the rule.

[0075] Clearly, a large variety of rules may be created and these will need to be made to suit a particular situation. However, some examples of rules and associated predefined actions that may be found in typical situations follow. [0076] In a first example, a rule is defined which detects whether the caller has spoken an unrecognised word. The artificial intelligence engine 7 can look in the thesaurus to see if the unrecognised word is a synonym for a recognised word. If the word is not recognisable then the predefined action can be to route the call to an agent 3, 4 or 5. This will be done by issuing a suitable command to the media application server 6 to cause it to request the voice switch 1 to reroute the call to the agent 3, 4 or 5.

[0077] A modification to this rule deals with the situation where a user has become annoyed with the IVR system to the extent that they are swearing. The modified rule can detect whether the caller has spoken any one of a number of profane words, and if so transfer the call to an agent. [0078] As already mentioned, the speech analytics engine 7 may also be configured to detect the ratio of silence to speech and/or the emotional state of the caller. A rule may therefore be defined that causes the call to be routed to an agent if the caller is silent for an extended period of time (perhaps indicating they are confused by the IVR menu options) or if they are detected to have become irate.

[0079] It should be realised that whilst the flow charts of Figures 3 and 4 show the receipt of voice input from a caller coming after the issue of voice prompts from the IVR system, it is possible that the caller could start to respond before the voice prompt has completely finished. This pre-emptive response will be monitored and processed by the IVR system as though it had come at the expected stage after the voice prompt has completely finished.

[0080] In a second example, a rule can be defined to simply make a record of the call (including details of the voice prompts and responses by the caller) and flag the record for analysis (for example, by e-mailing it to an operator) if a caller appears to become irate or swears at the IVR system. This might be useful in circumstances where a recorded message (for example, indicating that the office called is now closed) is played after a protracted period of navigating through an IVR menu. By monitoring the records of such calls, an operator can quickly determine that the message should be played at the start of the IVR interaction rather than just before a caller believes that he is about to be transferred to an agent after a prolonged period navigating through the IVR menus.

[0081] In a third example, a rule can be defined which looks for words that are not expected in the context of a list of options associated with the voice prompts given, but which are understood in a different context. For example, in the case of the Figure 2 example a user may have chosen the home insurance option in the top-level menu because they have a home insurance policy about which they have a general query. There is no option for general queries in the sub-menu for home insurance in the menu structure of Figure 2, leaving the caller with no apparent options. However, the rule can be configured to detect words such as "Customer Services" or "General Query" and take action by routing the call to an appropriate agent, in this case a customer services agent. A record of the call may also be made as in the second example. This allows the operator to get a good idea of the frequency of such occurrences so that they can consider whether a redesign to avoid such a problem is worthwhile. Before routing the caller to the appropriate agent, the IVR system could be configured to issue a voice prompt to check that this is what the caller desires.

[0082] In a fourth example, a rule can be defined to detect when the caller has simply ended up at the end of the IVR menu. For example, in the Figure 2 menu structure, a caller could end with no options left after having selected either of the Car Insurance or Home Insurance options when they wanted to make a general query or accounts query on either type of insurance policy because the options for these are in the top-level menu. This type of problem can be detected by a prolonged silence or an irate outburst from the caller or alternatively if they choose a random one of the available options and then have to be transferred to the appropriate agent for their query. A record of the call may also be made as in the second and third examples.

[0083] In a fifth example, a rule can be defined to analyse whether a particular prompt has been issued already to the caller and/or whether the caller's input has already been received in response to an issued prompt. This is useful for detecting if the caller is jumping backwards and forwards between different sub-menus, which could occur in more complicated IVR menu structures. The predefined action would typically be to route the call to an agent to intervene and/or to make a record of the call as in the second to fourth examples.

[0084] Figure 5 shows another method according to the invention in which the conversation between the agent and the caller, which takes place after the caller has been routed by the IVR system, is analysed in order to diagnose faults in the IVR system.

[0085] Steps 10 to 12 of figure 5 are the same as steps 10 to 12 of figure 3.

In this method, following the call being routed at step12, at step 25 instead of the IVR being disconnected, the conversation between the caller and the agent to which the caller has been routed is analysed by the speech analytics engine 7 of figure 1. Alternatively a different speech analytics engine with associated thesaurus and artificial intelligence may "listen in" to and analyse the conversation. This analysis is carried out in order to determine whether the caller has been routed to an appropriate agent. If it has not, the nature of the problem and the appropriate remedy may be immediately apparent from the conversation.

[0086] For example if the agent says "I need to transfer you to X" and "X" is not an option in the menu, the appropriate remedy would be to add this option to the menu. This is an alteration that might be performed automatically under certain circumstances without an operator having to re-design the menu. In such a situation it would be usual to require such changes to be signed off by an operator before being implemented. Indeed even with a menu structure involving one or more sub-menus, it may be possible to alter the menu automatically after the analysis of the conversation indicates a fault with the IVR system. However it is more likely in the case of an IVR having one or more sub-menus that it is necessary to investigate the route through the menu taken by the caller before the appropriate remedy can be determined. [0087] At step 26 the speech analytics engine is queried to determine whether any potential problems with the menu have been identified in the analysis of step 25. If yes, the route taken through the menu is investigated at step 27 by displaying the menu structure and indicating the options selected by the caller, for example by highlighting. It is likely that this menu flow would be presented to someone other than the actual agent that was involved in the call such as an IVR manager or call centre supervisor.

[0088] At step 28 the route taken by the caller is analysed and any necessary changes to put right any potential problems with the IVR system are made. [0089] References in the foregoing to "problems" and "faults" are intended to encompass any indications that the system is not operating optimally.

[0090] The same changes to the menu structure discussed in connection with the method of figure 3 are equally possible in connection with the method of figure 5. [0091] The embodiments described herein set out ways in which problems with the design of an IVR system can be detected (either by manual means or automatically). In some instances, automatic intervention is possible whilst in other instance manual redesign of the IVR system will be determined to be appropriate without expending large amounts of labour.

Claims

1. A method of analysing an interactive voice response (IVR) system for connecting voice calls to agents in which at least one prompt is issued from the IVR system, caller input is received in response to the or each prompt and the call is connected to an agent, the method comprising analysing conversation between the caller and the agent with which the caller has been connected whereby to diagnose faults in the IVR system.

2. The method of claim 1 comprising using a speech recognition system to analyse the conversation between the caller and the agent.

3. The method of claim 2 in which the speech recognition system identifies utterances, key words, or phrases in the conversation.

4. The method of claim 2 or claim 3 in which a speech recognition system is used to analyse caller input received in response to the or each prompt and the same speech recognition system analyses the conversation between the caller and the agent after the caller has been connected the agent.

5. The method of claim 4 in which the agents are accessed via a contact centre comprising, upon detection of a call to the contact centre, connecting the call to the IVR system, connecting the call to an agent at the contact centre in response to an instruction from the IVR system and maintaining a connection between the IVR and the caller and/or the agent whereby to enable a speech recognition system associated with the IVR to analyse the conversation between the caller and the agent.

6. The method of claim 2 or claim 3 in which a speech recognition system is used to analyse caller input received in response to the or each prompt and a different speech recognition system associated with the agent analyses the conversation between the caller and the agent after the caller has been routed to the agent.

7. The method of claim 6 in which the agents are accessed via a contact centre comprising, upon detection of a call to the contact centre, connecting the call to the IVR system, connecting the call to an agent at the contact centre in response to an instruction from the IVR system and connecting the IVR and/or the caller to a speech recognition system for analysing the conversation between the caller and the agent.

8. The method of any preceding claim comprising, in the event of diagnosis of a fault, displaying the or each prompt issued by the IVR system.

9. A method for analysing an interactive voice response (IVR) system, the method comprising issuing at least one prompt from the IVR system; receiving user input made in response to the or each prompt from the IVR system; displaying the or each prompt issued by the IVR system to an operator; and receiving input data from the operator, the data defining whether any change to the IVR system is recommended and/or any recommended changes to the IVR system.

10. A method according to claim 8 or 9, wherein the step of displaying the or each prompt issued by the IVR system to the operator comprises displaying a visual representation of the user input made in response to the or each prompt from the IVR system.

1 1. A method for analysing an interactive voice response (IVR) system, the method comprising issuing at least one prompt from the IVR system; receiving user input made in response to the or each prompt from the IVR system; detecting the occurrence of an unexpected event by comparing the user input against a predefined set of rules, each rule being associated with one or more of the unexpected events; and taking a predefined action in response to the detection of one or more of the unexpected events.

12. A method according to claim 1 1 , wherein the set of rules includes one or more rules which analyse whether the received user input is recognisable by the IVR system.

13. A method according to claim 1 1 or 12, wherein the set of rules includes one or more rules which analyse whether a particular prompt has been issued already to the user and/or whether user input has already been received from the user in response to an issued prompt.

14. A method according to any of claims 1 1 to 13, wherein the predefined action is to cause the IVR system to redirect the user to an operator for assistance.

15. A method according to any of claims 1 1 to 14, wherein the predefined action is to store the prompts issued by the IVR system and the user input received in response to the prompts for analysis by an operator.

16. A method according to any preceding claim, wherein the or each prompt issued by the IVR system is a spoken prompt.

17. A method according to any preceding claim, wherein the user input is received by a speech recognition system adapted to convert spoken user input into a form understandable by the IVR system.

18. A method according to any preceding claim, wherein the or each prompt forms part of a set of predefined prompts, which collectively define a menu of options for selection by a user.

19. A method according to claim 18, wherein the set of predefined prompts includes one or more sub-menus.

20. A method according to claim 19 when dependent on claim 1 1 , wherein the set of rules includes one or more rules which analyse whether the received user input is related to a sub-menu other than the current sub-menu.

21. A method according to claim 20, wherein the predefined action is to cause the IVR system to issue the set of predefined prompts which includes the sub-menu related to the received user input.

22. A method according to claim 8, 9 or 10, or any of claims 16 to 21 when dependent on claim 4, 5 or 6 wherein the step of displaying the or each prompt issued by the IVR system to the operator comprises displaying a visual representation of the menu of options and indicating the options selected by the user.

23. A method according to any of claims 18 to 20 when dependent on claim 4, wherein the step of receiving input data from the operator comprises receiving user input defining alterations to the menu of options.

24. A method for analysing an interactive voice response (IVR) system, the method comprising a combination of the steps of two or more of: the method of any of claims 1 to 8; the method of claim 9 or 10; and the method of any of claims 1 1 to 24.

25. An IVR system comprising one or more processors adapted to perform the method of any of the preceding claims.

26. A computer program comprising a set of instructions for performing the method of any of claims 1 to 24 when executed on a processor.

27. One or more computer-readable media upon which one or more computer programs comprising a set of instructions is stored which when executed by a processor of a computing device cause the processor to perform the method of any preceding claim.