DE102019219406A1 - CONTEXT-SENSITIVE VOICE DIALOGUE SYSTEM - Google Patents

Abstract

A dialog system (100) set up to receive a dialog input is set up to identify at least one intention in the dialog input (206) and to determine an object or an object class linked to it in a standardized knowledge modeling. The dialogue system (100) searches (214) in a memory or a database (116) in which status information or states of a spatial and / or temporal environment of a user of the dialogue system (100) recorded or controlled by a device (118) are used as Properties or attributes of objects or object classes of the standardized knowledge modeling are provided in a retrievable manner, according to properties or attributes of the objects or object classes for which no such properties or attributes are contained in the dialog input, creates (222) a dialog response based on the identified intention of the at least one object or the object class and the associated properties or attributes, and outputs the dialog response (224).

Description

FIELD

The present invention relates to speech dialog systems, particularly those for use in high-context defined environments.

BACKGROUND

With the increasing spread of question-and-answer technologies such as chatbots, digital personal assistants, speech dialog systems, smart speakers and the like, the users of computer devices increasingly interact with their computers using natural language.

For example, users can use a digital personal assistant implemented in a smartphone to set appointments and reminders by voice command, or control the light or other electrical loads in a smart home by voice command. In motor vehicles, functions are also increasingly controllable by voice command.

Also, when searching for information, users are increasingly turning to conversational search instead of traditional search methods that require the entry of keywords. In a conversational search, a user can phrase a question or query in such a way that the intent of the user is explicitly defined. For example, a user can ask: "What is the weather forecast for today in Berlin?" In this question, there is no ambiguity in identifying the relevant objects or content "weather forecast", "today", "Berlin", or in understanding the Intent behind the question.

Alternatively, a user's question may be contextual, with the user asking a question in such a way that context information is needed to infer the user's intent. For example, a user can try to find information on a topic using only a limited number of terms. The dialogue system is then required to understand the intention behind the user's question and to try to find web pages or other results as an answer using databases or search engines. For example, a user can ask the dialog system: "Will it rain tomorrow?" In this example, the system receiving the query may need to use contextual information, such as the current location of the user or entries in a schedule that suggest a different location. to understand the intent of the user.

As another example, a user may ask a question that contains an indefinite pronoun that refers to one or more objects, beings, or locations unspecified in the query, and the entity to which the indefinite pronoun refers can be in a current Query not specified, but may have been mentioned in a previous query or response. For example, a user can ask: "Who played in the Champions League yesterday", whereupon the dialogue system names two teams, followed by "Who won", whereby the user wants to know "who won the Champions League game yesterday" . People are usually able to relate to contextual information that has already been mentioned. However, search engines or other dialogue systems have so far had difficulties in determining or selecting a correct context for a query and often treat each query as if it were not connected to previous queries.

Dialog systems available today are usually trained with the help of exemplary utterances. These utterances are enriched with annotations that indicate what purpose they fulfill, so-called "intents", which meaningful elements or "entities" they contain and how these elements are mapped to the relevant fields, which are also known as "slots". For the exemplary utterance “looking for an Italian restaurant”, the purpose is a restaurant search or “restaurant search”. The meaningful element, the entity, is the word "Italian", which is shown on the slot or the relevant field "cuisine".

The disadvantage of the conventional approach is the isolated observation of explicitly made utterances. Only content that the user has already expressed is taken into account by the speech dialogue system. Other information, which can further enrich the annotations without any action on the part of the user, improve the quality of the query and thus also the quality of the response, has so far not been used or has only been used insufficiently. One of the reasons for this is that each speech dialogue system is individually designed by the respective provider, with each provider making its own, limited and static selection of possible intents, entities and slots. The enrichment of the dialog request with temporal and spatial context information depends accordingly on whether a provider of a dialog system can provide suitable modeling of a current context and access to corresponding information.

It is easy to see that dialogue systems, which with each request or with each request to control functions require complete sets of information that are rarely accepted by users. If a system cannot understand a user's intention behind a question or utterance to control a function, the user may need to re-ask a question in another way in order to get the desired answer. This is inefficient for the user.

It is therefore an object of the present invention to propose an improved dialog system and method for controlling the same, which in particular reduces the need to input or repeat context information that is obvious to a user.

DESCRIPTION

Such a dialog system is proposed in claim 1 and a corresponding method is proposed in claim 6. Refinements or further developments are specified in the respective dependent claims.

In the following description, the terms dialog input, question, query, inquiry or query are used synonymously, if the context does not result in a different meaning. These terms do not relate exclusively to questions, but are also intended to include statements by a user that are used to control functions or devices. The present invention can be used equally for speech dialogue systems as well as text-based dialogue systems. In the following, the terms dialogue system and speech dialogue system are used synonymously if the respective context does not result in a different meaning. The same applies to the terms input or voice input and output or voice output. If reference is made to different interfaces in the present description, this can mean separate physical interfaces, but also separate logical interfaces or ports which are assigned to the same physical interface.

A dialog system according to the invention comprises a first device for receiving a dialog input, a second device for outputting a dialog response, a processor unit and volatile and / or non-volatile memory. The dialog system also includes a first interface for data communication with a memory or a database in which or in which state information or states of a spatial and / or temporal environment of a user of the dialog system as properties or attributes captured or controlled by a third device of objects or object classes of a standardized knowledge modeling are made available in a database so that they can be called up.

States of the spatial environment can be represented, for example, by sensor data or state data from actuators and other devices, but also include transient phenomena such as acoustic or visual information that a user can perceive at a point in time or during a period of time. The visual or acoustic information can be recorded via cameras or microphones, but it is also possible to receive metadata belonging to the information. An example of acoustic information is a radio program heard by a user, for which meta information such as a title of a program or an artist of a piece of music is available. An example of visual information is, for example, image data from a camera around a vehicle, which records and recognizes other road users or fixed installations that are relevant for driving the vehicle. Other states or state information can relate to operating modes or parameters of devices and devices in the environment of the user, or environmental information, or the like. A state of a temporal environment can relate, for example, to information about a location or an activity that is in the future or in the past.

The non-volatile memory contains program instructions that can be read and executed by the processor unit, which set up the system to receive dialog input and to identify at least one intention therein. The intention can, for example, represent a request for information, or a request to a device controllable by the dialog system or the like.

Receiving the dialog input can include receiving a voice input or a text input. Correspondingly, the first device can comprise a microphone or a keyboard.

Identifiable intentions can be linked to one or more objects or one or more object classes of the standardized knowledge modeling. The standardized knowledge modeling can be present as a knowledge graph or a structured database and represents a cornerstone of the “Semantic Web”. Well-known examples of standardized knowledge modeling are Wikidata, DBpedia, Freebase or YAGO.

The program instructions also set up the dialog system to determine possible objects or object classes linked to the identified intention in the standardized knowledge modeling.

The program instructions also set up the dialog system to search the context of the dialog input for properties or attributes that match the objects or object classes determined. The context includes, in particular, utterances made in a previous time period and, if applicable, reactions to them, as well as links with objects or object classes of knowledge modeling determined in the previous time period. If suitable properties or attributes are not found in the dialog input for each of the determined objects or object classes - or if such properties or attributes are to be updated, for example due to specified rules - the program instructions for those objects or object classes are used to search for properties or attributes are triggered in the database for which no suitable information was found in the dialog input or which should be updated. The properties or attributes retrievable from the database preferably relate to status information or statuses of the spatial and / or temporal environment of the user recorded or controlled by the third device within a time window that includes at least the time of the beginning of the dialog input and extends into the past of the dialogue system. The time window can be different for different objects or object classes, depending on whether and how quickly a property or an attribute of the object or object class in question changes. In most cases, however, it can be assumed that a state relevant for the dialog input was present at least at the time of the dialog input, so that information valid for this time should always be retrievable from the database. To what extent and how far back previous states can also be retrieved from the database depends, among other things, on the available storage space and decisions made by an implementer, but also on the content of the dialog. If, for example, the user repeats a previously expressed dialog input with the same identifiable intention, and a reaction has already taken place to the previous dialog input, it can be assumed that at least parts of the content of the previous dialog are no longer relevant. In this case it may be appropriate to search the database for changed properties or attributes. If, for example, the user repeats a previously uttered dialogue input directed towards a piece of music with the same identifiable intention, to which a reaction has already taken place, but the piece of music has changed in the meantime, it can be assumed that the previous dialogue has been completed and a new one Dialogue begins. On the other hand, in the case of a dialogue relating to a piece of music played on the radio, for example, an update of the information about a piece of music that is subsequently played may be unsuitable if the original dialogue has not yet ended, i.e. if further dialogue entries clearly relate to the dialogue that was started previously.

The program instructions also set up the dialog system to create a dialog response based on the identified intention and the associated properties or attributes, and to output the dialog response.

The creation of the dialog response can include sending a request to the identified intention, the at least one object or the object class and the associated properties or attributes to a database or an Internet search engine.

The output of the dialogue response can include an acoustic and / or visual output or an output of a control signal. Correspondingly, the second device can have a loudspeaker, a screen and / or a control device. The control device can feed the control signal to an actuator of a further device located in the spatial environment, the reaction of which to the control signal can be sensed by the user. In the case of embodiments of the dialog system, the control device can also be set up to feed a control signal to an actuator or to another device that is not in the immediate spatial vicinity of the user. In this case, the control device can be set up to receive a confirmation signal from the actuator or the other device or a monitoring device arranged in the vicinity thereof and to translate it into a dialog response.

A method according to the invention for controlling a dialog system comprises receiving a dialog input and identifying an intention in the dialog input. For this purpose, the dialog input can be searched for certain key terms, for example. The key terms can in particular contain question words such as “when”, “where”, “who” etc., and can enable the intention to be identified via a further term that can be linked to them. The further term can also be recognized in the dialog input, from previous dialog inputs, from a spatial and / or temporal context or the like.

The method also includes determining at least one possible object or an object class that is linked to the identified intention in a standardized knowledge modeling. The standardized knowledge modeling can be in the form of a knowledge graph or a structured database, for example Wikidata, DBpedia, Freebase or YAGO. The dialog input is then searched for properties or attributes that match the objects or object classes that have been determined. If suitable properties or attributes were not found in the dialog input for each of the determined objects or object classes - or if such properties or attributes are to be updated, for example due to specified rules - a database searches for properties or attributes for those objects or object classes searched for for which no properties or attributes were found in the dialog input or which should be updated. The properties or attributes that can be called up from the database relate to state information or states of the spatial and / or temporal environment of the device for generating context information recorded or controlled within a time window that includes at least the time of the beginning of the dialog input and extends into the past User of the dialog system.

Based on the identified intention and the properties or attributes found in the dialog input or the search, a dialog response is created and output.

Generating the dialog response may include sending a query on the identified intent and properties or attributes to a database or an Internet search engine.

A device for generating context information for a dialog system comprises a processor unit, volatile and non-volatile memory, a first interface for data communication with a database and at least one second interface for communication with sensors or other devices, which state information or states of a spatial and / or temporal Capture or control the environment of a user. The non-volatile memory contains program instructions that can be read and executed by the processor unit, which set up the device to record or control status information or statuses of a spatial and / or temporal environment of a user of the dialog system as properties or attributes of objects or to provide object classes of standardized knowledge modeling in the database so that they can be called up.

The dialog system described above, the associated control method and the device for generating context information can be used in a particularly advantageous manner in context-rich environments in which a large amount of information about the states of devices or systems and other environmental parameters are available via corresponding sensors, interfaces, etc. . Such context-rich environments are a vehicle, a smart home, a digital factory or the like.

In an exemplary application, a user listens to music by the musician William Fitzsimmons via an audio system, which can provide additional information about the program being played. This additional information can include RDS data for radio playback via an FM receiver, for example, it can be contained in the received data as metadata when audio content is played back via a streaming service, or it can be obtained in another way, e.g. via a query from a music information service or the like .

A device for generating context information cyclically or continuously records the information available for the music. In addition, the device continuously records further context information, such as information on the current weather, a weather forecast and the like, and stores this as properties or attributes for corresponding objects or object classes of a standardized knowledge database, which can be called up in a memory or a database.

While the music is playing, the user utters the dialogue input “When is the next concert?”. The dialog system analyzes the dialog input and tries to identify an intention of the user. Among other things, the question word "when" and the further term "concert" help. The dialog system searches the dialog input for further information that may be helpful in answering the request, but does not find any.

Therefore, the dialog system now searches a database with context information that can possibly provide useful information for answering the request. For this purpose, the dialogue system uses the term “concert” contained in a standardized knowledge database as an object or English “entity”, which is listed, for example, in the wikidata.org database under the reference Q182832. Finally, an entry for the class “musician” with reference Q639669 is linked to this entry in the knowledge database via further intermediate links. The database with context information does not contain any entries for the intermediate links of the knowledge database, but the device for generating context information for the music played at the time of the dialogue request has stored an entry for the reference Q639669 in the database with context information. This entry is now used as context information to send a query to an Internet search engine with the content " Concert - William Fitzsimmons ”. The result found by the Internet search engine is output as a dialog response.

In another exemplary use case, the context rich environment is a vehicle that has thermometers to measure indoor and outdoor temperatures. If the user asks the question "How many degrees are there?" Without prior interaction with the dialog system, the dialog input only contains the question word "How many" and the additional term "Degree". In the standardized knowledge modeling, the "Degree" object is entered under reference Q25267 and linked to the physical variable temperature (reference Q11466). A mere search in an Internet search engine would not necessarily produce meaningful results, if only because of the missing location. However, the dialog system finds entries on the inside and outside temperature in the database with context information under the reference Q11466 and can use this information to create a dialog response that with some probability provides a satisfactory answer to the user's question.

The cyclical or continuous storage of context information as referenceable objects or object classes of one or more standardized knowledge models by the device for generating context information enables the dialog system to extensively supplement even seemingly incomplete dialog entries by a user, so that queries to supplement the dialog entry have to be made less frequently. The more natural dialog guidance based on the context information obtained automatically by the dialog system increases the acceptance of the dialog system on the part of the user considerably.

The use of standardized knowledge modeling facilitates the integration of dialogue systems in a large number of different context situations.

The dialog system described above and the method for controlling it can be used advantageously in many applications, for example in a motor vehicle, a smartphone or a smart home.

Figure list

• 1 ein exemplarisches Blockschaltbild eines erfindungsgemäßen Dialogsystems,
• 2 ein exemplarisches Flussdiagramm eines von dem erfindungsgemäßen Dialogsystem ausgeführten Verfahrens,
• 3 eine vereinfachte Darstellung des Informationsflusses einer beispielhaften Dialogeingabe in einer kontextreichen Umgebung, und
• 4 ein exemplarisches Blockschaltbild einer Vorrichtung zur Generierung von Kontextinformationen für das Dialogsystem.

In the following, the invention is explained by way of example with reference to the drawing. In the drawing shows:

• 1 an exemplary block diagram of a dialog system according to the invention,
• 2 an exemplary flowchart of a method executed by the dialog system according to the invention,
• 3 a simplified representation of the information flow of an exemplary dialog input in a context-rich environment, and
• 4th an exemplary block diagram of a device for generating context information for the dialog system.

Identical or similar elements can be referenced in the figures with the same reference symbols.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

1 shows an exemplary block diagram of a dialog system according to the invention 100 . A first device 102 for receiving dialog input, a second device 104 for outputting a dialog response, a processor unit 106 , Storage 108 and a first interface 110 are via one or more data links and / or data buses 112 communicatively connected to each other. The first interface 110 is via a communication link 114 with a memory or a database 116 connected, in which a third device 118 , the status information or statuses of a spatial and / or temporal environment of a user of the dialog system 100 detects or controls, this status information or status is made available as contents of objects or object classes of a standardized knowledge modeling. The status information or statuses can from the third device 118 cyclically or continuously recorded and updated, so that status information or statuses for a time window reaching back into the past are available at all times. Each status information or status can have its own time window within which status information or statuses are valid. The third device can be connected to further devices in the spatial and / or temporal environment of the user via sensors, control outputs and communication interfaces (not shown in the figure). Storage 108 may comprise volatile and non-volatile memory, program instructions readable and executable by the processor unit are stored in the non-volatile memory, which the dialog system for executing the with reference to 2 described procedure 200 set up.

2 Fig. 10 shows an exemplary flow diagram of one of the present invention Dialogue system 100 executed procedure 200 . After starting it will be in step 204 receive a dialog input, e.g. as a voice input. In step 206 the dialog system identifies at least one intention of the dialog input. For this purpose, the dialog input can be searched for certain key terms, for example. The key terms can in particular contain question words such as “when”, “where”, “who” etc., and can enable the intention to be identified via a further term that can be linked to them. In step 208 at least one possible object or object class linked to the identified intention in a standardized knowledge modeling is determined. Then in step 210 searches the dialog input to find properties or attributes that match the identified objects or object classes. If properties or attributes that are not suitable for all objects or object classes can be found in the dialog input, "n" branch from step 212 , will be in step 214 database 116 searches for suitable properties or attributes for the objects or object classes. If this is not the case, "n" branch of step 216 , a query can be output as a dialog response, step 218 whereupon the procedure will step the previous information 220 saves as partial information of a dialog and for dialog input 204 returns, or an attempt can be made with the incomplete information in step 222 create a dialog response in step 224 is issued. If the properties or attributes are missing in the database 116 are present, "j" branch from step 216 , or if suitable properties or attributes can be found for all objects or object classes in the dialog input, "j" branch from step 212 , will be in step 222 a corresponding dialog output is created and in step 224 issued.

3 shows a simplified representation of the information flow of an exemplary dialog input in a context-rich environment. A device for generating context information has determined the name of the artist for a currently played music reproduction, for example from information associated with a music file, and stored this under the reference Q6942562 in the database for context information. The dialogue system DS first receives the dialogue input “When is the next concert?” And determines that information for the creation of a meaningful dialogue response is missing. In addition to the question word “when”, the dialogue request also contains the term “concert”, which is included in the standardized knowledge modeling with reference Q182832. The combination of the two terms together with the more or less indefinite time “next” suggests that a concert event in the future is wanted. The dialog system DS now searches the database with context information DB for content relating to reference Q182832. The database with context information DB has no content stored for this, so the dialog system DS searches for objects or object classes linked with reference Q182832 and finds the linked reference Q6942562 (“musical performance”). The dialog system DS again sends a query to the database with context information DB, this time for content with the reference Q6942562. Again, there are no entries and the dialog system DS searches for objects or object classes linked with reference Q6942562 and finds reference Q639669 (“musician”). The dialog system DS makes a further query to the database with context information DB, this time for content with the reference Q639669. The database with context information DB has saved the entry “William Fitzsimmons” for this purpose and supplies this information to the dialog system DS. The dialog system DS now uses the original search term “concert” and the context information “William Fitzsimmons” to create a search query for an Internet search engine, and generates a dialog output from the response (not shown in the figure). The dialog system can also exclude search results that are in the past as additional information.

4th shows an exemplary block diagram of a device 118 to generate context information for the dialog system 100 . A processing unit 406 is with a memory 408 , a first interface 410 and at least one second interface 420 via one or more data links and / or data buses 412 communicatively connected. The first interface 410 is over a data connection 414 with a memory or a database 116 connected in which the device 118 State information or states of a spatial and / or temporal environment of a user of the dialog system 100 as content of objects or object classes of a standardized knowledge modeling makes available. The device 118 is for this purpose via at least one of the second interface 420 outgoing data connection 430 with sensors or other devices 440 communicatively connected which state information or states of a spatial and / or temporal environment of a user of the dialog system record or control.

List of reference symbols

100

Dialogue system

102

first device

104

second device

106

Processor unit

108

Storage

110

first interface

112

Data connection / bus

114

Communication link

116

Database

118

third device

200

Procedure

204

Dialog input received

206

Identify intention

208

Determine object

210

Search property / attribute for object in dialog input

212

Property / attribute found for object in dialog input?

214

Search database

216

Database search successful?

218

Inquiry

220

Save query content as partial information

222

Create dialog response

224

Output dialog response

406

Processor unit

408

Storage

410

first interface

412

Data connection / bus

414

Communication link

420

second interface

430

Communication link

440

Sensor / further device

Claims (10)

Dialog system (100) with a first device (102) for receiving a dialog input, a second device (104) for outputting a dialog response, a processor unit (106) and a memory (108) which contains program instructions that can be read and executed by the processor unit, wherein the dialog system (100) also comprises a first interface (110) for data communication with a memory or a database (116) in which or which by a third device (118) acquired or controlled state information or states of a spatial and / or the temporal environment of a user of the dialog system (100) can be called up as properties or attributes of objects or object classes of standardized knowledge modeling, the program instructions, when they are executed by the processor unit (106), setting up the dialog system (100) for: - receiving (204) the dialog input, - Identifying (206) at least one intention in the dialog input, - Determining (208) at least one possible object or an object class linked to the identified intention in the standardized knowledge modeling, Searching (210) the dialog input for properties or attributes that match the at least one determined object or the object class, with the program instructions, if the properties or attributes are not found in the dialog input for each of the determined objects or object classes, the program instructions the dialog system ( 100) also set up for: - Searching (214), in the database (116), of properties or attributes for those objects or object classes for which no properties or attributes were found in the dialog input, the properties or attributes retrievable from the database being within obtain a time window that includes at least the time of the beginning of the dialog input and extends into the past from the third device (118) recorded or controlled status information or statuses of the spatial and / or temporal environment of the user of the dialog system (100), - creating (222) a dialog response based on the identified intention, the at least one object or the object class and the associated properties or attributes, and - Outputting (224) the dialog response. Dialog system (100) according to Claim 1 wherein receiving (204) the dialog input comprises receiving a voice input or a text input, and wherein the first device (100) comprises a microphone or a keyboard. Dialog system (100) according to Claim 1 or 2 , wherein the output (224) of the dialogue response comprises an acoustic or visual output or an output of a control signal, wherein the second device (104) comprises a loudspeaker, a screen or a control device, the control device sending the control signal to an actuator of one located in the spatial environment feeds further device, the reaction of which to the control signal can be sensed by the user. Dialog system (100) according to Claim 1 , 2 or 3 wherein the creation (222) of the dialogue response comprises: sending a request on the identified intention, the at least one object or the object class and associated properties or attributes to a database or an Internet search engine. Device (118) for generating context information for a dialog system (100) with a processor unit (406) and a memory (408) which contains program instructions that can be read and executed by the processor unit, a first interface (410) for data communication with a memory or a database (116), at least one second interface (420) for communication with sensors or other devices (440) which record or control status information or statuses of a spatial and / or temporal environment of a user, the program instructions if they executed by the processor unit (406) set up the device (118) to use the sensors or other devices (440) detected or controlled state information or states of a spatial and / or temporal environment of a user of the dialog system (100) as properties or Attributes of objects or object classes of a standardized knowledge modeling to be made available in the database (116) so that it can be called up. A method (200) for controlling a dialog system (100), comprising: - receiving (204) a dialog input, - identifying (206) an intention in the dialogue input, - Determining (208) at least one object or an object class linked to the identified intention in a standardized knowledge modeling, - Searching (210) the dialog input for properties or attributes that match the at least one determined object or the object class, - Check (212) whether suitable properties or attributes were found in the dialog input for each of the determined objects or object classes and, if not, search (214) in a database (116) for properties or attributes for those objects or object classes for which in the dialog input no properties or attributes were found in the dialog input, the properties or attributes retrievable from the database being recorded or recorded by a device (118) within a time window that includes at least the time of the beginning of the dialog input and extends into the past. Obtain controlled status information or statuses of the spatial and / or temporal environment of the user of the dialog system (100), - creating (222) a dialog response based on the identified intention, the at least one object or the object class and the associated properties or attributes found in the dialog input or the search (214), and - Outputting (224) the dialog response. Method (200) according to Claim 6 wherein the creation (222) of the dialog response comprises: sending a request on the identified intention, the at least one object or the object class and the associated properties or attributes to a database or an Internet search engine. A computer program product comprising instructions which, when the program is executed by a processor unit (106) of a dialog system (100), cause the program according to the method (200) Claim 6 or 7th to execute. Computer-readable medium on which the computer program product is after Claim 8 is stored Vehicle with a dialog system (100) according to one of the Claims 1 - 4th and a device according to Claim 5 .

Images