WO2020242383A1 - Système et procédé de dialogue conversationnel - Google Patents

Système et procédé de dialogue conversationnel Download PDF

Info

Publication number
WO2020242383A1
WO2020242383A1 PCT/SG2020/050310 SG2020050310W WO2020242383A1 WO 2020242383 A1 WO2020242383 A1 WO 2020242383A1 SG 2020050310 W SG2020050310 W SG 2020050310W WO 2020242383 A1 WO2020242383 A1 WO 2020242383A1
Authority
WO
WIPO (PCT)
Prior art keywords
query
semantically
expanded
queries
determining
Prior art date
Application number
PCT/SG2020/050310
Other languages
English (en)
Other versions
WO2020242383A9 (fr
Inventor
Avinash PAUL A
Saurabh Jain
Shashank Sharma
Original Assignee
Active Intelligence Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Active Intelligence Pte Ltd filed Critical Active Intelligence Pte Ltd
Priority to SG11202111971PA priority Critical patent/SG11202111971PA/en
Publication of WO2020242383A1 publication Critical patent/WO2020242383A1/fr
Publication of WO2020242383A9 publication Critical patent/WO2020242383A9/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G06F40/35—Discourse or dialogue representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • the present invention relates to the field of automated conversational dialogue, particularly to handling semantic context in automated conversational dialogue.
  • Conversational dialogue systems or "chatbots" as they are sometimes known provide an automated response to user queries.
  • Typical uses of such systems include customer service provision.
  • a consumer or potential consumer will either speak into a microphone or type into a dialogue box and the dialogue system will determine an appropriate response to the customer's query. For example, a customer may ask "What are the premiums for life insurance?" and the dialogue system will provide an appropriate response, consisting of the cost of the premium of the life insurance product offered by the company to which the query is being made.
  • Some sophisticated conversational dialogues systems employ natural language processing in combination with artificial intelligence, whereas others may simply perform a database search based on identified keywords in the user's query. Typically, such systems are designed to mimic human dialogue as closely as possible.
  • a user may ask "What are the premiums for life insurance” and then subsequently ask "What about car insurance?”.
  • the user is asking for information regarding the premiums of car insurance in the second query.
  • the semantic context of the query is incomplete - not all the information required to understand the query is present.
  • automated conversational dialogue systems will address the issue of incomplete semantic context by asking follow-up questions in order to clarify the second query, such as "What would you like to know about car insurance?".
  • queries interrupt the natural flow of dialogue and make it clear to the user that they are not interacting with a human. Also, they require the conversational dialogue system to process significantly more queries than would otherwise be necessary.
  • a method of expanding a user query with contextual information for use with an automatic conversational dialogue system comprising: receiving text data indicating a first query at a first time; receiving text data indicating a second query at a second time, wherein the second time is later than the first time; categorizing the second query as semantically complete or semantically incomplete; when the second query is categorised as semantically incomplete, determining an expanded query based on information derived from the first query and the second query; and outputting the expanded query as text data.
  • the query may be received a speech data and then converted to text data using speech recognition software or a speech recognition system.
  • the text data may be received from a speech recognition module or as an output of a method of speech recognition.
  • the output text data may be converted to speech data using text to speech.
  • the output text data may comprise an input to a text to speech module or to a method of text to speech.
  • two or more queries from a single user may be received.
  • information from two or more received queries may be employed in order to create the expanded query.
  • semantically complete or equivalently contextually complete
  • semantically incomplete or equivalently, contextually incomplete it is meant that the query cannot be understood on its own, i.e. without additional context.
  • semantically incomplete or equivalently, contextually incomplete it is meant that the query cannot be understood on its own, i.e. without additional context.
  • completeness/incompleteness may be defined relative to what an automatic conversational dialogue system, with which the expanded query is intended to be used or otherwise, can understand without additional contextual information. Semantically complete may mean that all information required to understand each lexical unit in the query is present in the query itself.
  • automatic conversational dialogue system it is meant any automated system which determines responses to queries. The term is not intended to limit the disclosure to any specific system or any method of determining a response to a query. The system may or may not use methods based on artificial intelligence to determine a response to a query.
  • Determining an expanded query based on information derived from the first query and the second query may comprise identifying one or more of an action, a item name and an item attribute in the first and/or second queries.
  • the item may be tangible or non-tangible. It is not intended to be limited to an object.
  • the item may be a product and the item name may be a product name and an item attribute may be a product attribute.
  • item names, or equivalently products may comprise insurance, loans, a current account, etc.
  • Item attributes, or equivalently product attributes may comprise cost, account name, availability, etc.
  • An item may equivalently or alternatively refer to a noun or a proper name.
  • a method of determining a response to a user query comprising: receiving text data indicating a first query at a first time; receiving text data indicating a second query at a second time, wherein the second time is later than the first time; categorizing the second query as semantically complete or semantically incomplete; when the second query is categorised as semantically incomplete, determining an expanded query based on information derived from the first query and the second query; determining a response to the expanded query using an automated conversational dialogue system; and outputting the response to the expanded query as text data.
  • a system for expanding a user query with contextual information for use with an automatic conversational dialogue system comprising, the system comprising: an input for receiving text data indicating a first and second query; a processor configured to perform a method of expanding a user query with contextual information for use with an automatic conversational dialogue system is provided, the method comprising: receiving text data indicating a first query at a first time; receiving text data indicating a second query at a second time, wherein the second time is later than the first time; categorizing the second query as semantically complete or semantically incomplete; when the second query is categorised as semantically incomplete, determining an expanded query based on information derived from the first query and the second query; and outputting the expanded query as text data; and an output configured to output the expanded query as text data.
  • a carrier medium comprising computer readable code configured when performed by a processor of a computer to cause the processor to perform a method of expanding a user query with contextual information for use with an automatic conversational dialogue system, the method comprising: receiving text data indicating a first query at a first time; receiving text data indicating a second query at a second time, wherein the second time is later than the first time; categorizing the second query as semantically complete or semantically incomplete; when the second query is categorised as semantically incomplete, determining an expanded query based on information derived from the first query and the second query; and outputting the expanded query as text data.
  • the carrier medium may be tangible or non-tangible.
  • a carrier medium comprising computer readable code configured when performed by a processor of a computer to cause the processor to perform a method of determining a response to a user query, the method comprising: receiving text data indicating a first query at a first time; receiving text data indicating a second query at a second time, wherein the second time is later than the first time; categorizing the second query as semantically complete or semantically incomplete; when the second query is categorised as semantically incomplete, determining an expanded query based on information derived from the first query and the second query; determining a response to the expanded query using an automated conversational dialogue system; and outputting the response to the expanded query as text data.
  • the carrier medium may be tangible or non-tangible.
  • Figure 1 shows a system for performing query expansion according to an embodiment
  • Figure 2 shows a method of query expansion according to an embodiment
  • Figure 3 shows an automatic dialogue system according to an embodiment
  • Figure 4 shows a method of query expansion according to an embodiment
  • Figure 5 shows a method of query expansion according to an embodiment
  • Figure 6 shows a method of pre-processing according to an embodiment
  • Figure 7 shows a method of evaluating query completion according to an embodiment
  • Figure 8 shows an example of a universal sentence encoder
  • Figure 9 shows a method of generating query indices according to an embodiment
  • Figure 10 shows a method of query expansion using triplet validation according to an embodiment
  • Figure 11 shows an example of a neural network
  • Figure 12 shows an architecture of a neural network for query or product expansion according to an embodiment
  • Figure 13 shows a method of training a neural network for query or product expansion according to an embodiment
  • Figure 14 shows a method of triplet validation according to an embodiment
  • Figure 15 shows a method of training a neural network for triplet validation according to an embodiment
  • Figure 16 shows an example of query expansion according to an embodiment
  • Figure 17 shows a Seq2seq model according to an embodiment
  • Figure 18 shows a method of training a Seq2seq model according to an embodiment
  • Figure 19 shows a method of co reference resolution according to an embodiment
  • Figure 20 shows a method of candidate evaluation according to an embodiment.
  • references to "an embodiment / example”, “another embodiment / example”, “some embodiments / examples”, “some other embodiments / examples”, and so on, indicate that the embodiment(s) / example(s) so described may include a particular feature, structure, characteristic, property, element, or limitation, but that not every embodiment / example necessarily includes that particular feature, structure, characteristic, property, element or limitation. Furthermore, repeated use of the phrase “in an embodiment / example” or “in another embodiment / example” does not necessarily refer to the same embodiment / example.
  • a set as defined herein can correspond to a unit, singlet, or single-element set, or a multiple-element set), in accordance with known mathematical definitions.
  • the recitation of a particular numerical value or value range herein is understood to include or be a recitation of an approximate numerical value or value range.
  • the recitation of a particular numerical value or value range herein is understood to include or be a recitation of an approximate numerical value or value range.
  • FIG. 1 illustrates a computer system 380 suitable for implementing a context expansion method according one or more embodiments disclosed herein.
  • the computer system 380 includes a processor 382 (which may be referred to as a central processor unit or CPU) that is in communication with memory devices including secondary storage 384, read only memory (ROM) 386, random access memory (RAM)
  • processor 382 which may be referred to as a central processor unit or CPU
  • memory devices including secondary storage 384, read only memory (ROM) 386, random access memory (RAM)
  • the processor 382 may be implemented as one or more CPU chips. It is understood that by programming and/or loading executable instructions onto the computer system 380, at least one of the CPU 382, the RAM 388, and the ROM 386 are changed, transforming the computer system 380 in part into a particular machine or apparatus having the novel functionality taught by the present disclosure. It is fundamental to the electrical engineering and software engineering arts that functionality that can be implemented by loading executable software into a computer can be converted to a hardware implementation by well-known design rules. Decisions between implementing a concept in software versus hardware typically hinge on considerations of stability of the design and numbers of units to be produced rather than any issues involved in translating from the software domain to the hardware domain.
  • a design that is still subject to frequent change may be preferred to be implemented in software, because re-spinning a hardware implementation is more expensive than re-spinning a software design.
  • a design that is stable that will be produced in large volume may be preferred to be implemented in hardware, for example in an application specific integrated circuit (ASIC), because for large production runs the hardware implementation may be less expensive than the software
  • the CPU 382 may execute a computer program or application.
  • the CPU 382 may execute software or firmware stored in the ROM 386 or stored in the RAM 388.
  • the CPU 382 may copy the application or portions of the application from the secondary storage 384 to the RAM 388 or to memory space within the CPU 382 itself, and the CPU 382 may then execute instructions that the application is comprised of.
  • the CPU 382 may copy the application or portions of the application from memory accessed via the network connectivity devices 392 or via the I/O devices 390 to the RAM 388 or to memory space within the CPU 382, and the CPU 382 may then execute instructions that the application is comprised of.
  • an application may load instructions into the CPU 382, for example load some of the instructions of the application into a cache of the CPU
  • an application that is executed may be said to configure the CPU
  • the secondary storage 384 is typically comprised of one or more disk drives or tape drives and is used for non-volatile storage of data and as an over-flow data storage device if RAM 388 is not large enough to hold all working data. Secondary storage 384 may be used to store programs which are loaded into RAM 388 when such programs are selected for execution.
  • the ROM 386 is used to store instructions and perhaps data which are read during program execution.
  • ROM 386 is a non-volatile memory device which typically has a small memory capacity relative to the larger memory capacity of secondary storage 384.
  • the RAM 388 is used to store volatile data and perhaps to store instructions. Access to both ROM 386 and RAM 388 is typically faster than to secondary storage 384.
  • the secondary storage 384, the RAM 388, and/or the ROM 386 may be referred to in some contexts as computer readable storage media and/or non-transitory computer readable media.
  • I/O devices 390 may include printers, video monitors, liquid crystal displays (LCDs), plasma displays, touch screen displays, keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices.
  • LCDs liquid crystal displays
  • plasma displays plasma displays
  • touch screen displays touch screen displays
  • keyboards keypads
  • switches dials
  • mice track balls
  • voice recognizers card readers, paper tape readers, or other well-known input devices.
  • the network connectivity devices 392 may take the form of modems, modem banks, Ethernet cards, universal serial bus (USB) interface cards, serial interfaces, token ring cards, fiber distributed data interface (FDDI) cards, wireless local area network (WLAN) cards, radio transceiver cards that promote radio communications using protocols such as code division multiple access (CDMA), global system for mobile communications
  • GSM global system for microwave access
  • WiMAX WiMAX
  • NFC near field communications
  • RFID radio frequency identity
  • Other air interface protocol radio transceiver cards and other well-known network devices.
  • These network connectivity devices 392 may enable the processor 382 to communicate with the Internet or one or more intranets. With such a network connection, it is contemplated that the processor 382 might receive information from the network, or might output information to the network in the course of performing the below- described method steps. Such information, which is often represented as a sequence of instructions to be executed using processor 382, may be received from and outputted to the network, for example, in the form of a computer data signal embodied in a carrier wave.
  • Such information may be received from and outputted to the network, for example, in the form of a computer data baseband signal or signal embodied in a carrier wave.
  • the baseband signal or signal embedded in the carrier wave may be generated according to several methods well-known to one skilled in the art.
  • the baseband signal and/or signal embedded in the carrier wave may be referred to in some contexts as a transitory signal.
  • the processor 382 executes instructions, codes, computer programs, scripts which it accesses from hard disk, floppy disk, optical disk (these various disk based systems may all be considered secondary storage 384), flash drive, ROM 386, RAM 388, or the network connectivity devices 392.
  • processors While only one processor 382 is shown, multiple processors may be present. Thus, while instructions may be discussed as executed by a processor, the instructions may be executed simultaneously, serially, or otherwise executed by one or multiple processors. Instructions, codes, computer programs, scripts, and/or data that may be accessed from the secondary storage 384, for example, hard drives, floppy disks, optical disks, and/or other device, the ROM 386, and/or the RAM 388 may be referred to in some contexts as non-transitory instructions and/or non- transitory information.
  • the computer system 380 may comprise two or more computers in communication with each other that collaborate to perform a task.
  • an application may be partitioned in such a way as to permit concurrent and/or parallel processing of the instructions of the application.
  • the data processed by the application may be partitioned in such a way as to permit concurrent and/or parallel processing of different portions of a data set by the two or more computers.
  • virtualization software may be employed by the computer system 380 to provide the functionality of a number of servers that is not directly bound to the number of computers in the computer system 380.
  • virtualization software may provide twenty virtual servers on four physical computers.
  • the functionality disclosed above may be provided by executing the application and/or applications in a cloud computing environment.
  • Cloud computing may comprise providing computing services via a network connection using dynamically scalable computing resources.
  • Cloud computing may be supported, at least in part, by virtualization software.
  • a cloud computing environment may be established by an enterprise and/or may be hired on an as-needed basis from a third party provider.
  • Some cloud computing environments may comprise cloud computing resources owned and operated by the enterprise as well as cloud computing resources hired and/or leased from a third party provider.
  • the computer program product may comprise one or more computer readable storage medium having computer usable program code embodied therein to implement the functionality disclosed below.
  • the computer program product may comprise data structures, executable instructions, and other computer usable program code.
  • the computer program product may be embodied in removable computer storage media and/or non-removable computer storage media.
  • the removable computer readable storage medium may comprise, without limitation, a paper tape, a magnetic tape, magnetic disk, an optical disk, a solid state memory chip, for example analog magnetic tape, compact disk read only memory
  • CD-ROM compact discs
  • floppy disks floppy disks
  • jump drives digital cards, multimedia cards, and others.
  • the computer program product may be suitable for loading, by the computer system 380, at least portions of the contents of the computer program product to the secondary storage 384, to the ROM 386, to the RAM 388, and/or to other non-volatile memory and volatile memory of the computer system 380.
  • the processor 382 may process the executable instructions and/or data structures in part by directly accessing the computer program product, for example by reading from a CD-ROM disk inserted into a disk drive peripheral of the computer system 380.
  • the processor 382 may process the executable instructions and/or data structures by remotely accessing the computer program product, for example by downloading the executable instructions and/or data structures from a remote server through the network connectivity devices 392.
  • the computer program product may comprise instructions that promote the loading and/or copying of data, data structures, files, and/or executable instructions to the secondary storage 384, to the ROM 386, to the RAM 388, and/or to other non-volatile memory and volatile memory of the computer system 380.
  • the secondary storage 384, the ROM 386, and the RAM 388 may be referred to as a non-transitory computer readable medium or a computer readable storage media.
  • a dynamic RAM embodiment of the RAM 388 may be referred to as a non-transitory computer readable medium in that while the dynamic RAM receives electrical power and is operated in accordance with its design, for example during a period of time during which the computer system 380 is turned on and operational, the dynamic RAM stores information that is written to it.
  • the processor 382 may comprise an internal RAM, an internal ROM, a cache memory, and/or other internal non-transitory storage blocks, sections, or components that may be referred to in some contexts as non-transitory computer readable media or computer readable storage media.
  • the computer system 380 operates on a query in order to expand it.
  • the computer system 380 receives an input query.
  • the input query may be received via the input/output 390 or via the network 392.
  • the input query is received as text data.
  • the input query may be received as speech data which is subsequently converted to text data using speech recognition.
  • the CPU 382 is configured to perform speech recognition and thereby convert the input speech data to text. In another embodiment, this conversion may be performed before the data is received by the system 380.
  • speech recognition technology is provided by a third-party provider.
  • step S103 the CPU 382 categorizes the query as being semantically complete (i.e. a
  • the processor may use information stored in the ROM 386 or Secondary storage 384 to perform expansion in the query.
  • the processor may employ data derived from other queries received by the system including from the same user in order to perform the query expansion. This process will be discussed below according to embodiments.
  • step S109 the system outputs the expanded query directly via the output 390 or via the network 392, according to how the expanded query will be used.
  • the expanded query is output as text data.
  • the expanded query may be output in other forms, including but not limited to speech vectors.
  • the computer system 380 forms part of a system for generating responses to queries using an automatic conversational dialogue system.
  • a system 200 according to an embodiment is shown in Figure 3.
  • the system comprises a plurality of networked computing devices
  • These computing devices may have the same arrangement as the device shown in Figure 1 and described above or they may have a different
  • the network may comprise further devices which perform one or more of the functions of the devices shown in Figure 2.
  • the network may comprise one or more further components (not shown) with load balancing functionality or load balancing may be performed by one or more of the nodes shown, for example the server 207.
  • the devices are connected via the Internet, however, the system may include various other configurations and protocols such as Bluetooth, an intranet, a wide area network, etc. or various combinations of the above for connection some or all of the devices shown.
  • the system may include various other configurations and protocols such as Bluetooth, an intranet, a wide area network, etc. or various combinations of the above for connection some or all of the devices shown.
  • Personal user devices such as a PC 201 or mobile device 203 may be connected to the system via the network.
  • the user devices 201 and 203 may have the same arrangement as the device shown in Figure 1 and described above or they may have a different arrangement.
  • User devices 201 and 203 may be configured to enable the user to enter a query for processing by the system.
  • the system may be configured to receive inputs from many such devices.
  • the system may be configured to receive inputs from many such devices simultaneously.
  • the network may include a server 207 which hosts a webpage into which the user inputs the query via the device 201, 203 which they are using which is then passed from the server 207 to the other devices in the network.
  • the server may also display receive the answer to the query and display it on the webpage for the user to view with their device 201, 203.
  • the user may submit the query verbally as speech into a microphone incorporated into their device 201 or 203.
  • one of the devices shown in Figure 3, or another networked device, not shown may comprise speech recognition functionality and be configured to convert the input speech into text data. The conversion from input speech into text data may be carried out by a third- party provider.
  • the system 3 comprises at least one component, or node 209 functioning as conversational middleware. In this embodiment, this node 209 is configured to receive queries from elsewhere in the network and to distribute them accordingly.
  • the server 207 is configured to transmit raw input (i.e. unexpanded or otherwise unprocessed) queries received from a user to the conversational middleware 209.
  • the text data may be end-to-end encrypted and the middleware 209 may be configured to perform decryption.
  • the conversational middleware 209 then, in turn, distributes the raw query to a query expander device 380 for processing according to the method of Figure 3.
  • the query expander 380 was described above in association with Figures 1 and 2.
  • the query expander processes the query according to the method of Figure 2 and then returns the processed query to the conversational middleware 209.
  • the conversational middleware 209 then directs the processed query to an automatic conversational dialogue system 205.
  • automatic conversational dialogue systems 205 are configured to retrieve an answer to a query. Suitable automatic conversational dialogue systems are well known in the art and readily available. Examples include Siri (Apple), Google assistant, cortarna, etc. Existing automatic dialogue systems may take a number of forms. Such systems may be highly complex such as those based on artificial intelligence or may comprise systems which perform keyword searches of a database stored in memory. In an embodiment, the automatic conversational dialogue system 205 is accessed via an Automatic
  • the automatic conversational dialogue system 205 is domain specific, that is, it is designed to respond to queries relating to a specific field, for example banking products if the dialogue system is hosted by a financial institution. Once the answered query is output from the automatic conversational dialogue module
  • some or all of the queries entered via a user device 201 or 203 may be passed directly by the conversational middlware 209 to the conversational dialogue system 205 without sending it via the query expander 308.
  • the server 207 receives a first query from a user, or equivalently, a session starting sentence from the user via PC 201 or mobile 203 and the middleware 209 directs this starting sentence directly to the conversational dialogue system 205 in order to obtain a response without sending it to the query expander 380.
  • the server 207 receives a first query from a user, or equivalently, a session starting sentence from the user via PC 201 or mobile 203 and the middleware 209 directs this starting sentence directly to the conversational dialogue system 205 in order to obtain a response without sending it to the query expander 380.
  • the server 207 or other node in the system 200 may store a copy of this starting sentence in memory 386 for use with semantic expansion of subsequent queries.
  • the server 207 (or other node in the system 200) then may flag any subsequent queries received by the same user in the same session such that they are directed to the query expander
  • the stored session starting query is also passed to the query expander 308 to enable expansion of the input query.
  • a unique hash is created.
  • this hash token is exchanged between the query expansion system and the user device, e.g. the browser, thereby enabling the system to recognise that the queries belong to the same user session.
  • the token may expire due to inactivity or due to time. For example, the token may expire after 20 minutes. Other factors may determine the expiry of the token according to embodiments.
  • a query is stored until either one or more subsequent queries are expanded and output by the query expander, or until the user ends their session (for example, by logging out), whichever occurs first. In another embodiment, they are stored for a predetermined time. A method according to this embodiment is shown in Figure 4.
  • step S301 an input query is received from a user, for example via a PC 201 or Mobile 203.
  • step S303 it is determined if the query is a session starting query, i.e. the first query entered by the user in the session or a subsequent query, i.e. a second or later query entered by the user.
  • step S305 a copy of the query is stored for possible use the in the expansion of later queries.
  • step S307 an answer to the query is determined by the automatic conversational dialogue system 205.
  • step S307 the answer to the query is communicated to the user.
  • step S311 the query is flagged for processing by the query expander.
  • step S313 the query is processed by the query expander according to the method of
  • step S315 a copy of the query output by the query expander is stored by the system. This may be an expanded query or a raw input query depending on whether expansion was determined to be necessary according to the method of Figure 2.
  • the output query replaces in memory any previously stored queries from the same user, for example the session starting query.
  • the output query may be stored in addition to one or more previous queries from the same user.
  • Steps S307 and S309 are then performed for the output query as described above. Note that once the expanded query is determined according to embodiments described above, the expanded query is then passed to back to the middleware which then directs it to the conversational Al system. Thus, the conversational Al system can be switched out/upgraded as needed without impacting the query expander and vice- versa.
  • system may be configured to store second or later queries without first performing processing by the query expander.
  • system may be configured to process all queries, including the session starting query using the query expander.
  • step S101 one or more queries is input by a user.
  • step S103 namely that of determining if a query is contextually complete, comprises two sub-steps as shown in Figure 5: namely 1) step S502 of preprocessing the query and 2) step S503 of determining if the query requires expansion.
  • Preprocessing essentially comprises standardizing the language of the input queries in order to enable the subsequent steps of the process and in order to allow meaningful comparison with the vector indexes employed in step S503 of the process (see Figure 7 and associated description).
  • the skilled person will appreciate that a wide range of methods of pre-processing text data are well known in the art and that the methods described below are intended only to be one example of preprocessing that could be implemented according to
  • Figure 6 shows one example of a method for prepressing queries according to an embodiment of the invention with preprocessing of the example query "I Wana chek my
  • preprocessing of queries comprises two distinct parts:
  • the language of the query is standardized. Processes that may be carried out may include but are not limited to: tokenization or segmentation based on the script of the language; lemmatization; duplicate removal; case
  • Natural Language Toolkit (NLTK) package (https://www.nltk.org/) may be employed for sentence segmentation.
  • the TreebankWordTokenizer from the NLTK package may be employed for tokenization.
  • Lemmatization tools are provided as part of the open- source spaCy library (https://spacv.io/models). Other tools may be employed according to embodiments.
  • string matches and/or cosine similarity are employed to remove duplicates. These processes are well known in the art.
  • NPL Natural Language Processing
  • step S401 language-based processing takes place in step S401.
  • the input query "I Wana chek my , savngs Acc transaction for last ten months” is converted to "i wana chek my savngs acc transaction for last ten months”.
  • steps S403 and S405 domain based pre-processing is performed on the sentence output from step S401.
  • Domain based language processing comprises domain specific processing once the language has been standardized.
  • domain specific processing it is meant that processing in line with language particularities according to the specific context in which queries are being made is performed. For example, the language typically employed as part of queries relating to banking or financial services may be different from the language employed in other industries such as travel and holiday services.
  • domain-based language processing examples include but are not limited to domain specific normalization of dates into the format required by the database from which the response to the query will be derived, contextual spell correction including acronym expansion, root word normalization and slang correction.
  • Slang correction enables algorithms to be run on well-formed English sentences and reduces part of speech (POS) tagging errors. This is important for candidate according to the embodiments described below. Examples of slang correction include
  • slang correction is achieved by performing a keyword search of a corpus of regional slang and profanities to identify slang words and replace them appropriately, according to the user's location.
  • Date normalization may be important where the middleware connects to a bank/financial institution Application Programming Interface (API) which typically consume certain dates in certain formats and therefore requires the query to be input in a particular format.
  • date normalization comprises first cleaning clubbed words followed by converting the date format.
  • cleaning clubbed words includes: • 2months converted to 2 months
  • Examples of converting date formats include:
  • dates are classified into categories according to those which require special preprocessing later in the pipeline.
  • Date range e.g. transaction of last 15 days
  • Period e.g. interest rate of 1 year; open 2 year fixed deposit.
  • a Support Vector Machine (SVM) classifier is employed to identify different types of dates which require normalization.
  • SVM Support Vector Machine
  • the use and development of SVM classifiers is well known in the art. Details are given in, for example Scikit-learn:
  • the dates are converted into the desired date format using a custom wrapper.
  • Suitable methods of date parsing are well known in the art.
  • One example according to an embodiment comprises employing the duckling library available at https://pypi.org/project/duckling/ and then converting using a custom wrapper to a desired date format.
  • Generic wrappers may be employed according to embodiments. Such generic date wrappers are well known in the art.
  • Root word normalization comprises converting meaning according to context.
  • Examples for financial services may include:
  • contextual spell correction and root word normalization is achieved using an n-gram based model created using domain-specific data and comparing words in the input sentence with the domain-specific data using Levenshtein distance.
  • Such techniques are well known in the art.
  • Number processing may also be carried out according to embodiments.
  • embodiments for example:
  • step S403 domain-specific slang, spelling and acronym correction is carried out in step S403, resulting in generation of the query "i want to check my savings account transaction for last ten months" Finally, in step S405, domain or use specific date normalization is performed resulting in
  • Figure 5 is step S503, evaluate query completion. In an embodiment, this comprises determining if the input query requires expansion or if it is already semantically complete.
  • the flow chart of Figure 7 shows this process according to an embodiment for an example query "how do I buy it”.
  • step S201 the pre-processed query is converted into a vector.
  • the query is converted into a 512-dimensional vector using a universal sentence encoder.
  • Universal Sentence Encoders encode text into high-dimensional vectors that can be used for text classification, semantic similarity, clustering and other natural language tasks.
  • a schematic of the process performed by a Universal Sentence Encoder is shown in Figure 8.
  • a sentence 2001 (in this case a query) is input into the Universal Sentence Encoder.
  • the Universal Sentence Encoder converts the query into the vector 2005.
  • step S703 the calculated vector is employed to determine the semantic similarity between the query vector and an index of domain based primary queries and to determine a score representing such similarity.
  • primary query it is meant that the query is semantically complete, i.e. does not require expansion in order to be understood and answered by the automatic dialogue system.
  • adaptive models trained with machine learning techniques such as neural networks may be employed to search the index and determine a score.
  • Suitable examples include neural networks based on the transformer architecture (see e.g. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N
  • step S705 the score is output.
  • the score gives a likelihood that the input query is a primary query, i.e. it does not follow on from an earlier query.
  • the query is a primary query, i.e. it does not follow on from an earlier query.
  • the query vector is employed to determine the semantic similarity between the query vector and index of domain based secondary queries.
  • secondary query it is meant that the query is contextually incomplete, i.e. its meaning is unclear without additional contextual information.
  • a score is determined for the semantic similarity between the input query and the secondary query vector index.
  • this score is determined in the same way as described above in relation to the primary query index. In essence, this score gives a likelihood that the input query is a secondary query, i.e. it follows on from an earlier query. In the example given in Figure 7, the query "how to buy it", which is a contextually incomplete secondary query, obtains a score of 0.9 when compared with the secondary query vector index. As one would expect, this is higher than the primary index score indicating that the query is more likely to be a secondary query than a primary query.
  • steps S703 and S705 may be performed in parallel with steps S707 and S709.
  • Step S711 the scores are compared. If Score 2 (i.e. the secondary query index score) exceeds that of Score 1 (i.e. the primary query index score) then it is determined that expansion is required and the method proceeds to step S107 in the method of Figure
  • this step determines if the query is more likely to be a secondary query than a primary query.
  • score 1 exceeds score 2 then it is determined that expansion of the query is not required, and the query is left unamended and output in step S105 of Figure 2.
  • the primary query index score or the secondary query index score may be determined.
  • the categorisation of the query as a primary or secondary query and the decision to expand the query or not will be simply based on whether the score exceeds a predetermined threshold.
  • the index of domain-based primary queries is created by encoding a number of contextually complete queries containing domain-specific references and storing them in an index.
  • the secondary query vector index is created by encoding a number of contextually incomplete queries containing domain-specific references and creating storing them in an index.
  • the indices may be stored in the memory 386 of the query expander 380. In an embodiment, one or more of the indices is an annoy
  • an annoy index comprises C++ library with Python bindings to search for points in space that are close to a given query point. They also create large read only filed-based data structures that are mapped into memory so that many processes may share the same data.
  • Figure 9 shows a method of creating primary and secondary query vector indices according to an embodiment.
  • step S901 a series of primary queries is compiled, i.e. queries which are semantically complete.
  • queries which are semantically complete.
  • the skilled person will appreciate that a greater number of input training sentences can be employed in order to generate the index.
  • the accuracy of query categorisation will be improved the more training examples are included in the index.
  • the query data is domain-specific in order to improve accuracy of scoring.
  • the number of queries employed as training data may be based on the number of use cases (e.g. product categories) of the conversational dialogue system that the query expansion is intended to support. In an embodiment, around 1000 training queries are employed per use case.
  • the input training queries are queries which do not produce a follow up question from the automatic conversational system with which the query expander will be employed.
  • the training queries will be domain or project specific, according to requirements.
  • step S901 each training sentence is converted into a vector and the vectors are output in step S903.
  • step S905 the vectors are compiled as an index.
  • the index is an annoy index which is arranged based on nearest-neighbour relationships, as described above. Analogous steps are performed using secondary, or semantically incomplete, queries.
  • step S711 if, as a result of the comparison of scores in step S711, it is determined that the query is not semantically complete, the systems moves on to step
  • this step comprises two steps: step S505 of candidate generation and then step S507 of candidate evaluation. These steps will now be described in detail.
  • one or more expanded queries are generated as candidates to replace a contextually incomplete query.
  • natural language processing techniques are employed in order to generate viable candidates.
  • several candidates are generated using a plurality of techniques.
  • NLP Natural Language Processing
  • information from two queries is employed to generate an expanded query, namely the query determined to be semantically incomplete according to the method of Figure 7 and a previous query input by the user.
  • the previous query may either be a semantically complete query, such as the session starting query or it may itself be an expanded query generated by methods according to embodiments.
  • the previous query from the user is the query immediately preceding the query determined to be semantically incomplete or it may be an earlier query.
  • information from three or more queries may be employed to determine an expanded query, i.e. the query determined to be semantically incomplete and two or more input or expanded queries.
  • triplet validation is employed in order to generate an expansion candidate.
  • Triplet validation is a technique which exploits the assumption that simple sentences can be generated by using action(verb), attribute and product(nouns).
  • a valid action, attribute and product are identified from the primary and secondary query and employed to generate an expansion candidate.
  • the triplet may be identified from three or more queries.
  • step S1001 both sentences are input into a sentence selector module.
  • This module is identifies whether or not the sentences contain action words and therefore which sentence should be used for expansion of the identified triplet as well as the action for the triplet. This will be discussed further below.
  • step S1003 both sentences are checked for action words. In an embodiment this is done by employing a syntax tree. Suitable syntax parsers such as the open source spaCy syntax parser (https://spacy.io/models) are well known in the art.
  • the sentences are dealt with in parallel in order to ensure minimal runtime.
  • step S1005 the action words for both sentences are output (if available).
  • Step S1007 the action word which is to be employed in the triplet is selected.
  • the action word from the most recent query i.e. the secondary query
  • (S2)) is always selected for candidate expansion. If the secondary query does not contain an action word, then the action word from the primary query is selected instead.
  • step S1009 both SI and S2 are analysed for product and attribute.
  • step S1011 adaptive models trained by machine learning methods, e.g. neural networks, to perform both product and attribute extraction are employed in order to determine whether an attribute and/or product is present in either query. In an embodiment, this extraction is performed together for both queries.
  • neural networks are sets of algorithms configured to map inputs to outputs.
  • the neural network comprises an input layer 1901 where the input data is input into the network, one or more hidden layers 1903 where inputs are combined and an output layer 1905 at which the output is received.
  • the hidden layer 1903 comprises a series of biased nodes 1907. Each input to each hidden layer is weighted and combined at a node with a non-linear activation function.
  • the parameters of the neural network including those
  • the neural network may comprise a plurality of hidden layers, according to the architecture employed.
  • training data in the form of inputs and
  • Figure 12 shows a flowchart for product or attribute extraction using a neural network according to an embodiment.
  • step S1101 the primary query SI and the secondary query S2 are encoded as vectors using a universal sentence encoder, as described above according to embodiments.
  • An extraction query q is also encoded as a vector.
  • this extraction query may comprise, for example, "what is the attribute", in the case of attribute extraction and "what is the product” in the case of product expansion.
  • step S1103 SI and S2 are input into the neural network along with the extraction query vector.
  • the neural network is a multiclass function and the output layer
  • 1905 is softmax layer enabling the output to be determined as a probability over products or attributes, as appropriate.
  • the architecture of the neural network is configured to perform the following steps:
  • step 1103 the distribution of products or attributes in each individual sentence is calculated by processing each sentence with the query vector:
  • step S1105 the average of is taken to give:
  • step S1107 the average is processed with the extraction query vector in order to obtain the distributed probability of all answer phrases to the input extraction query, as follows: IVis a matrix of dimension (d * V) where V is the total number of expected answers.
  • the product or attribute with greatest probability is taken as the product or attribute, respectively, of the triplet.
  • Figure 13 shows a method of training the product and attribute extraction models according to an embodiment.
  • step S1109 the neural network (NN) is initialised.
  • the skilled person will appreciate that a number of standard methods may be employed to initialize the neural network.
  • random initialization is employed, i.e. the weights and biases that characterise the neural network as described above are chosen at random.
  • step Sllll the neural network (NN) error is calculated from the training data used for training the NN.
  • the training data comprises SI, S2, the expansion query and the product or attribute to be extracted and sentence to which the product or attribute corresponds.
  • the training data may take the following form according to an embodiment: SI what is my maturity instruction for fixed deposits
  • the training data may take the following form according to an embodiment:
  • the neural network error is then calculated by comparing the output distribution with the target output.
  • the NN error is calculated as the WARP loss or the max-margin loss. Details of WARP loss are given in
  • step S1113 the NN is modified (i.e. the weights and biases, as discussed above, are adjusted) in order to reduce the error calculated in step Sllll. In an embodiment, this is done using stochastic gradient descent. The method then returns to step Sllll and repeats until the NN error reaches convergence.
  • the parameters of the product and attribute neural networks are stored in the memory 386 of the query expander 380.
  • step S1017 the triplet, i.e. the action, product and attribute selected for candidate expansion is compiled.
  • the triplet then undergoes a validation process, "triplet validation".
  • Triplet validation according to an embodiment is shown in Figure 14.
  • Step S801 the triplet is input.
  • step S803 each member of the triplet is converted to a vector. In an embodiment, this is done using a universal sentence encoder as described above in relation to Figure
  • an output confidence is determined for the triplet.
  • this is done by inputting the vectors into an adaptive model trained by a machine learning method, e.g. a neural network.
  • a machine learning method e.g. a neural network.
  • Neural networks are described above in association with Figure 11. Essentially, the output of the neural network gives a likelihood that the triplet is mutually compatible. For example, this means that the attribute is an attribute which validly exists in association with the product.
  • the neural network is a neural tensor network. Details of such networks is given in, for example, Socher, Richard et al. "Reasoning With Neural Tensor
  • the triplet (offers(attribute), available(action), credit card(product)) is valid because offers are available in association with credit cards.
  • a triplet such as (pin(attribute),reset(action), travel insurance(product)) may not be valid if a reset pin option is not available for travel insurance.
  • the neural network is trained using a series of training sentences with the training sentence as the input and the target output as 1 for a valid triplet and
  • step S1301 the NN is initialised.
  • the skilled person will appreciate that a number of standard methods may be employed to initialize the neural network. In an
  • random initialization is employed, i.e. random values are chosen for the weights and biases of the neural network.
  • step S1301 the neural network (NN) error is calculated.
  • the training triplets are used as an input for the NN.
  • the neural network then outputs scores for the triplets.
  • the neural network error is then calculated by comparing the output scores with the score of 1 or 0 assigned to the triplets.
  • step S1305 the NN is modified in order to reduce the error calculated in step S1305, i.e. to minimize the objective function. In an embodiment, this is done using stochastic gradient descent. The method then returns to step S1303 and repeats until the NN error reaches convergence.
  • the parameters of the neural network determined during training such as the weights and biases of the network are stored in the memory 386 of the query expander 380.
  • step S807 if the output confidence exceeds a predetermined threshold then the triplet is considered valid and is therefore employed for candidate generation in S1021.
  • a predetermined threshold in the range of 0.88-0.92 was found to obtain accurate results using a standard data set.
  • the predetermined threshold may be adjusted and is determined heuristically according to the automatic conversational dialogue system with which the query expander is to be employed in order to obtain optimal results according to the organisation to which queries are being directed.
  • the triplet score is not greater than the predetermined threshold, the triplet is discarded and not employed to generate an expansion candidate.
  • Step S1021 an expansion candidate is generated based on the validated triplet.
  • a number of methods of generating the expansion candidate based on a validated triplet may be employed according to embodiments.
  • the expanded candidate is generated on the basis of the identified action/verb. If the latest sentence or current sentence contains an action then it is employed for expansion otherwise, an action is identified in the first query, SI. For example, in the example of Figure 16 the following sentences are processed: s1 what are the offers available on my credit card? s2 what about debit card?
  • SI an action
  • SI is present only in the first query (SI).
  • SI is therefore employed for expansion.
  • the product and attribute are replaced or append in the sentence employed for expansion.
  • the product and attribute validated in the triplet are
  • the second query (S2) may contain an action and consequently be employed for candidate expansion, for example: s1 what is travel insurance? s2 is that applicable for damage to luggage also?
  • the triplet identified for the above according to the embodiments described above "is damage to luggage"(attribute), “applicable”(action), “travel insurance”(product)).
  • S2 contains the action (applicable) and is therefore employed for expansion, with the latest product ("travel insurance") inserted into the appropriate position in S2, i.e. replacing "that".
  • the generated expansion candidate consequently becomes: "is travel insurance applicable for damage to luggage also?"
  • the triplet may be employed as an input into an adaptive model trained by a machine learning method, e.g. a SeqZseq model.
  • Seq2seq models are well known in the art and convert a sequence of text into another. Details of a suitable Seq2Seq model are given in Wu, Yonghui et al. "Google's Neural Machine
  • the Seq2seq model is trained to convert an input triplet into a query, as shown in Figure 17.
  • the triplet 1501 comprising "Nominee name, tell, savings account” is input into the Seq2seq model
  • the output 1505 is "Tell nominee name on my savings account.”
  • step S1601 the Seq2seq model is initialised.
  • the skilled person will appreciate that a number of standard methods may be employed to initialize the Seq2seq model. In an embodiment, random initialization is employed.
  • step S1601 the Seq2seq model error is calculated.
  • a number of training triplets, represented as vectors are used as an input for the Seq2seq model.
  • the Seq2seq model then outputs query vectors corresponding to the triplets.
  • the model error is then calculated by comparing the output vectors with the training queries
  • this is done using cross entropy.
  • step S1605 the Seq2seq model is modified in order to reduce the error calculated in step S1605. In an embodiment, this is done using stochastic gradient descent. The method then returns to step S1603 and repeats until the Seq2seq model error reaches convergence.
  • the parameters determining during the training of the Seq2Seq model are stored in the memory 386 of the query expander 380.
  • both Seq2seq models and creating expansion candidates on the basis of identified actions as described above are employed in order to generate expansion candidates from the validated triplet.
  • further methods which employ a validated triplet may be employed in order to generate expansion candidates.
  • a triplet is identified from two sentences, SI and
  • the triplet may be identified from three or more sentences according to embodiments.
  • the neural network described above in relation to Figure 8 would be trained using three or more sentences.
  • system may be configured to attempt to generate a triplet using two sentences and if no valid triplet can be identified, to attempt to generate a valid triplet using three or more sentences.
  • an expansion candidate is generated using coreference resolution.
  • Coreference resolution relies on the fact that typically about half of all incomplete query cases, the latest product/attribute is referenced using pronouns.
  • the input queries are si: do you cover theft under travel insurance and s2: how long is it valid for. These are input into the coreference module.
  • a pronoun and a reference noun are identified in the two sentences.
  • the pronoun is replaced in S2 in order to obtain the expansion candidate.
  • an adaptive model trained by a machine learning method, such as a neural network is employed in order to perform the coreference resolution.
  • a machine learning method such as a neural network
  • Neural networks are described above in association with Figure 11.
  • An example of a suitable coreference resolution model for use in this embodiment is given in Clark, Kevin.
  • the coreference resolution model is trained on domain specific data.
  • this will improve accuracy of the coreference resolution.
  • the generated candidates are evaluated and the most suitable candidate is selected.
  • this is done by comparing the encoded vector corresponding to the expanded query with the primary query index discussed above.
  • primary query it is mean that the query is semantically complete, i.e. does not require expansion in order to be understood.
  • the semantic similarity between the primary query vector index and the expanded query is determined analogously to that described in relation to flow chart of Figure 7.
  • the candidate selected as the expanded query is the candidate with the highest score for semantic similarity between the primary query vector index and the expanded query, i.e. the candidate with the greatest likelihood of being a contextually complete (i.e. primary) query.
  • comparison with both the primary query index and secondary query index are employed in order to select the best candidate.
  • Expanded candidates may or may not be valid as per the premises of the use cases being handled in the domain. In this step, therefore, each of the candidates are scored against the ground truth data of the domain to estimate the validity of the expansion.
  • the validation methods include semantic similarity of the expanded queries to the queries present in the domain corpus.
  • Figure 19 shows a method of candidate evaluation using the primary query index according to an embodiment.
  • triplet validation and coreference resolution are employed according to embodiments described above in step to determined expansion candidates for the query "what about debit card”.
  • triplet validation produces a contextually complete query "what are the offers available on my debit card” in step S1803, while coreference resolution provides no change in the original sentence, i.e. the expansion candidate remains "what about debit card”.
  • step S1805 both candidate queries are evaluated according to the primary query index as described above.
  • step S1807 "what are the offers available on my debit card” achieves a score of 0.95 whereas “what about debit card” achieves a score of 0.35.
  • step S1809 the query with the highest score, i.e. "what are the offers available on my debit card" is selected as the expanded query for input to the automated dialogue system.
  • Step S109 the selected expanded query is output from the system.
  • this expanded query then serves as an input to an automatic conversational dialogue system 205.
  • the systems and methods according to embodiments described above may be employed for use with any conversational dialogue system. Because it replaces a semantically incomplete query with a semantically complete query it is adaptable and provides an out of the box solution for the problem of dealing with queries which are missing context.
  • the query expander provides a solution which interacts only with the API (application programming interface) of an organisation's systems therefore changes to the API can be made without affecting the query expander.
  • API application programming interface
  • TPS Transactions Per Second
  • methods and systems of query expansion described above allow domain specificity to be incorporated into the query response at the expansion stage, before the query is passed to the automated dialogue system.
  • This may permit a generalised automatic dialogue system which is not-domain specific to be employed for generating the response to the query as the domain specific processing has already been performed before the query reaches that stage.
  • this may reduce memory requirements and processing by the automated dialogue system and enables organisations to implement off the shelf dialogue systems without requiring complex alteration which is both expensive and time-consuming.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé d'expansion d'une requête d'utilisateur avec des informations contextuelles destiné à être utilisé avec un système de dialogue conversationnel automatique (205), le procédé consistant à : recevoir des données de texte indiquant une première requête à un premier instant ; recevoir des données de texte indiquant une seconde requête à un second instant, le second instant étant ultérieur au premier instant ; catégoriser la seconde requête comme étant sémantiquement complète ou sémantiquement incomplète ; lorsque la seconde requête est catégorisée comme étant sémantiquement incomplète, déterminer une requête étendue sur la base d'informations dérivées de la première requête et de la seconde requête ; et fournir la requête étendue sous la forme de données de texte.
PCT/SG2020/050310 2019-05-28 2020-05-27 Système et procédé de dialogue conversationnel WO2020242383A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
SG11202111971PA SG11202111971PA (en) 2019-05-28 2020-05-27 Conversational dialogue system and method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
SG10201904801U 2019-05-28
SG10201904801U 2019-05-28

Publications (2)

Publication Number Publication Date
WO2020242383A1 true WO2020242383A1 (fr) 2020-12-03
WO2020242383A9 WO2020242383A9 (fr) 2021-11-25

Family

ID=73552992

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2020/050310 WO2020242383A1 (fr) 2019-05-28 2020-05-27 Système et procédé de dialogue conversationnel

Country Status (2)

Country Link
SG (1) SG11202111971PA (fr)
WO (1) WO2020242383A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114357231A (zh) * 2022-03-09 2022-04-15 城云科技（中国）有限公司 一种基于文本的图像检索方法、装置及可读存储介质
US20220207284A1 (en) * 2020-12-31 2022-06-30 Oracle International Corporation Content targeting using content context and user propensity

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120016678A1 (en) * 2010-01-18 2012-01-19 Apple Inc. Intelligent Automated Assistant
CN108334487A (zh) * 2017-07-14 2018-07-27 腾讯科技（深圳）有限公司 缺失语意信息补全方法、装置、计算机设备和存储介质
CN108829682A (zh) * 2016-04-08 2018-11-16 上海智臻智能网络科技股份有限公司 计算机可读存储介质、智能问答方法及智能问答装置
CN109522395A (zh) * 2018-10-12 2019-03-26 平安科技（深圳）有限公司 自动问答方法及装置
CN109697282A (zh) * 2017-10-20 2019-04-30 阿里巴巴集团控股有限公司 一种语句的用户意图识别方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120016678A1 (en) * 2010-01-18 2012-01-19 Apple Inc. Intelligent Automated Assistant
CN108829682A (zh) * 2016-04-08 2018-11-16 上海智臻智能网络科技股份有限公司 计算机可读存储介质、智能问答方法及智能问答装置
CN108334487A (zh) * 2017-07-14 2018-07-27 腾讯科技（深圳）有限公司 缺失语意信息补全方法、装置、计算机设备和存储介质
CN109697282A (zh) * 2017-10-20 2019-04-30 阿里巴巴集团控股有限公司 一种语句的用户意图识别方法和装置
CN109522395A (zh) * 2018-10-12 2019-03-26 平安科技（深圳）有限公司 自动问答方法及装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220207284A1 (en) * 2020-12-31 2022-06-30 Oracle International Corporation Content targeting using content context and user propensity
CN114357231A (zh) * 2022-03-09 2022-04-15 城云科技（中国）有限公司 一种基于文本的图像检索方法、装置及可读存储介质
CN114357231B (zh) * 2022-03-09 2022-06-28 城云科技（中国）有限公司 一种基于文本的图像检索方法、装置及可读存储介质

Also Published As

Publication number Publication date
WO2020242383A9 (fr) 2021-11-25
SG11202111971PA (en) 2021-12-30

Similar Documents

Publication Publication Date Title
US11250033B2 (en) Methods, systems, and computer program product for implementing real-time classification and recommendations
US11086601B2 (en) Methods, systems, and computer program product for automatic generation of software application code
US10705796B1 (en) Methods, systems, and computer program product for implementing real-time or near real-time classification of digital data
US10832006B2 (en) Responding to an indirect utterance by a conversational system
US10303683B2 (en) Translation of natural language questions and requests to a structured query format
US11954613B2 (en) Establishing a logical connection between an indirect utterance and a transaction
US10467122B1 (en) Methods, systems, and computer program product for capturing and classification of real-time data and performing post-classification tasks
US11783131B2 (en) Knowledge graph fusion
US10754886B2 (en) Using multiple natural language classifier to associate a generic query with a structured question type
US11164564B2 (en) Augmented intent and entity extraction using pattern recognition interstitial regular expressions
US10977155B1 (en) System for providing autonomous discovery of field or navigation constraints
US11727266B2 (en) Annotating customer data
Yu et al. AVA: A financial service chatbot based on deep bidirectional transformers
US20220350970A1 (en) Intent determination in a messaging dialog manager system
WO2020139865A1 (fr) Systèmes et procédés pour des conversations automatisées améliorées
CN116547676A (zh) 用于自然语言处理的增强型logit
WO2020242383A1 (fr) Système et procédé de dialogue conversationnel
US11797842B2 (en) Identifying friction points in customer data
US20220245361A1 (en) System and method for managing and optimizing lookup source templates in a natural language understanding (nlu) framework
US20230351121A1 (en) Method and system for generating conversation flows
WO2021118462A1 (fr) Détection de contexte
US20220237383A1 (en) Concept system for a natural language understanding (nlu) framework
US20220229986A1 (en) System and method for compiling and using taxonomy lookup sources in a natural language understanding (nlu) framework
WO2022048535A1 (fr) Interprétation de langage naturel basée sur un raisonnement
Palm End-to-end information extraction from business documents.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20814301

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 22.03.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20814301

Country of ref document: EP

Kind code of ref document: A1