CZ307132B6 - A method of managing a multi-user voice dialogue system using discrete simulation - Google Patents

Description

A method of controlling a multi-user voice dialogue system using discrete simulation

Technical field

The present invention is in the field of voice dialogue systems, that is, systems enabling human-computer communication in natural language. The invention is particularly useful for educational or training purposes in the field of air transport and the teaching of air traffic controllers. However, it is applicable in all areas where multiple voice dialogue with simulated counterparts is needed.

BACKGROUND OF THE INVENTION

In the field of air traffic control training, there is a demand for training of controllers in a simulated environment where controllers communicate with human counterparts (pseudopilots), who then control the air traffic generator (ATG). The output of ATG is then a radar image of generated (simulated) air traffic on the radar screen controlling in training.

Voice dialogue systems that could be used for this training allow the user to communicate via voice with computer or Internet applications. The main purpose of voice dialogue systems is to create an interface between a computer-controlled application and a user who communicates by voice. For a successful human interaction with the voice dialogue system, a well-designed dialogue management module, as well as speech recognition modules (ASR), speech synthesis (TTS) and a spoken language understanding module, should be available.

A device for multi-user communication is known, for example, from EP 2 189 199. However, its function is aimed at the mutual communication between the users themselves, that is to say human-to-human communication in a real world simulation environment and not communication of users with simulated counterparts and environment.

Another device is known from JP 2000/347558. Its function is focused on the training of air traffic controllers based on a voice dialogue system with the possibility of training trainer training and simulation of air traffic and pilots. However, it is a single-user voice dialogue.

CN 103310670 also deals with the issue of teaching air traffic controllers. However, this device is primarily used to simulate air traffic (display and behavior of aircraft) and communication devices used in it, but not to simulate the multi-user dialogue itself that takes place in this environment.

SUMMARY OF THE INVENTION

SUMMARY OF THE INVENTION The present invention provides a method for controlling a multi-user voice dialog system by discrete simulation. The method is performed on a computing device. The computing device includes a computing unit (a conventional PC), input and output devices (eg, keyboard, microphone, monitor, headphones), and a user interface. Discrete simulation includes simulation processes, simulation events, and shared resources. The discrete simulation is adapted to implement a multi-user voice dialogue system of a set of real users.

Discrete simulation also includes simulation processes for speech recognition and synthesis that generate simulation events.

- 1 GB 307132 B6

Discrete simulation creates connections to ensure that simulation processes have access to shared resources, especially speech recognition and synthesis.

A discrete simulation manages a set of scheduled events that is synchronized with real time.

The speech recognition process generates simulation events in the form of priority events. Priority events are handled at the time of occurrence, rather than scheduled events.

In execution, the computing device mediates communication between a plurality of real users and a plurality of simulated counterparts. There is at least one communication link between the sets.

The simulated counterpart is in the form of a simulation process and the communication link is mediated by speech recognition and synthesis processes.

The present invention is based on the use of the already known discrete simulation extended to control the multi-user voice dialogue system by special simulation processes representing speech recognition and synthesis together with their interconnection to shared resources and simulation events.

In contrast to the prior art approaches to controlling voice dialog systems, the invention allows multi-user communication where the dialog system serves more than one user. In existing dialogue systems, one user communicates with the dialogue management system via a speech recognition system. The speech recognition system processes the speech signal from the user and generates its meaning description. The dialogue management system updates its internal state based on the semantic description of the user input and generates a semantic description of the response. The semantic description of the response is then converted in the speech-to-text synthesis system into a speech signal which is played back to the user. The invention is a non-trivial generalization of a single-user voice dialogue system.

Clarification of drawings

An exemplary embodiment of the proposed solution is described with reference to Fig. 1, in which the algorithm and information flow diagram is shown:

1. Initialization:

Create an empty set of scheduled events

b. Create a set of resources

c. Creating a set of simulation processes

d. Creating at least one (1-n) speech synthesis simulation process (TTS)

e. Creating at least one (1-n) speech recognition simulation process (ASR).

f. Starting parallel running of ASR, TTS and event processing

2. Running the ASR process

a. if the speech recognition process detects the beginning of the utterance, it inserts a priority of the utterance of the utterance into the set of scheduled events,

b. if the speech recognition process detects the end of the utterance, it inserts a priority end of the utterance event into the set of scheduled events,

c. if the speech recognition process performs speech recognition and understanding, it inserts a priority event carrying the recognition result into the set of scheduled events.

-2GB 307132 B6

3. Running the TTS process

a. The speech synthesis process is waiting for the speech synthesis request event to be triggered and the required resources available

b. Perform speech synthesis

c. insert a speech synthesis end priority event into the set of scheduled events

d. release allocated resources

4. Event processing

a. If there is a priority event in the set of scheduled events, then activate that event U. For each process P waiting to trigger the event U perform:

i. activate process P; ii. wait for the set V to return the set V, iii. suspending process P, iv. add V events to the set of scheduled events.

Remove U from the set of scheduled events.

b. If there is an event in the set of scheduled events, the condition of which is met, then activate that event U. For each process P waiting to trigger the event U perform:

i. activate process P; ii. wait for the set V to return the set V, iii. suspending process P, iv. add V events to the set of scheduled events.

Remove U from the set of scheduled events.

DETAILED DESCRIPTION OF THE INVENTION

In an exemplary embodiment, a method of controlling a multi-user voice dialogue system by discrete simulation is implemented that is used for training air traffic controllers. The method of managing voice dialogue by means of discrete simulation is used to increase the realism of training.

In this case, the invention is practiced as follows:

The method is performed on a computing device comprising a computing unit, an input / output device, and a user interface.

Discrete simulation is a model of the world, and its activity consists of executing a sequence of events discrete over time. This model of the world changes its state only in these discrete moments. In a discrete simulation environment, there is a simulation process for each simulated flight in ATG. This simulation process receives commands from the controller and generates responses that are passed back to it. Commands and responses are represented by a semantic description. The discrete simulation is adapted to implement a multi-user voice dialog system. Discrete simulation includes:

• simulation processes - the simulation process is a model of the real world object. The simulation process is activated by simulation events. Once activated, the process changes its internal state and generates new events for which it waits again.

• simulation event - the condition during which the event is activated is connected to the simulation event. After the simulation event is activated, the simulation processes that are waiting for the event are gradually activated. Simulation events are inserted into the set of scheduled events.

• Shared Resources - Shared Resources are real-world object models that allow access to only a limited number of simulation processes. If a simulation process requires a shared resource to operate, it must first reserve it. When reserving a shared resource, it is checked that the condition for the number of simulation processes accessing the shared resource is met. If this condition is met, the simulation resource is reserved. If the condition is not met, the simulation process is paused and a new event is generated and the simulation process is waiting to be activated. The event fires when the number of simulation processes accessing the shared resource is reduced to meet the constraint condition. Activating the event activates the simulation process. When the simulation process ceases to require access to the shared resource, the shared resource reservation is released.

Shared resources in discrete simulation are used to model a shared transmission band for analog radiotelephony. This transmission band allows transmission of only one radio station at a time. Other radio stations are tuned to receive and receive all radio operations at a given frequency. The role of the receiving / sending party changes during communication. The shared "frequency broadcast" source may then have at most one speech-to-text synthesis process reserved, ensuring that there is no overlap between two simulated flights.

• set of scheduled events - is a set of pairs (condition, event), where the condition specifies when the corresponding event is activated. When an event is triggered, all the simulation processes that are waiting for the event are progressively scanned and these processes are activated one by one. Each activated process changes its internal state and generates a new event it is waiting for. Generating a new event adds it to the set of scheduled events. The set of scheduled events is synchronized with real time.

Discrete simulation also includes speech recognition and speech synthesis simulation processes that generate simulation events. The speech-to-speech synthesis process communicates with the simulated flight process. The speech-to-text synthesis process generates a speech signal that is played to the supervisor in training. It adds deterministic and / or random noise to the generated speech, representing noise and noise in the aircraft and in the communication channel, and thus significantly increases the credibility of the resulting simulation of air communication and thus brings closer the training controlling real traffic.

The speech recognition process recognizes voice commands from the controller and generates their meaning description. This semantic description is transmitted to the simulated flight process using simulation events. At the same time, it generates other simulation events such as the event indicating the beginning of the controller's speech, or the event indicating the end of the controller's speech. These events are inserted into the set of scheduled events as priority, ie they are not linked to any condition and are activated at the earliest possible opportunity.

Discrete simulation provides simulation processes with exclusive approaches to shared resources, especially speech recognition and synthesis. Exclusive access is a type of access where no more than one process can access the resource (eg, the use of communication frequency).

The speech recognition process generates priority events that are handled at the time of occurrence, rather than scheduled events. This brings more realism into the simulation, when new instructions can be introduced suddenly and the user is forced to respond immediately.

In performing the method, communication is provided between a plurality of users and a plurality of simulated counterparts. There is at least one communication link between the sets. The communication link ensures the transmission of necessary information between users and simulation processes. Communication

Binding is realized by passing information between processes within a single computing device, as well as between processes on different computing devices.

The simulated counterpart is in the form of a simulation process and the communication link is mediated by speech recognition and synthesis processes.

Thanks to the described method, discrete simulations allow training of multiple controllers at the same time each of the controllers is assigned one speech recognition process (corresponding to one communication frequency) and this speech recognition process passes the generated simulation events to simulated flight processes that have "tuned" this communication frequency. A set of simulated flights is shared across the simulation. It is thus possible to realize the transfer of flights between different frequencies (and therefore between different air traffic control areas and between different controllers), each frequency being modeled as a shared resource. The use of discrete simulation to control the voice dialogue system in this case alone makes it possible to realize a scenario in which the aircraft first communicates with the first controller, which can instruct it to retune to the new frequency, thereby transmitting the communication to the second controller.

Claims (1)

PATENT CLAIMS A method of controlling a multi-user voice dialog system by discrete simulation performed on a computing device, the computing device comprising a computing unit, an input / output device, and a user interface, wherein discrete simulation performs simulation processes, simulation events and shared resource modeling, the discrete simulation is adapted to implement a multi-user voice dialogue system of a plurality of real users, characterized in that the discrete simulation first creates a set of scheduled events, a set of shared resources, and a set of simulation processes that includes at least one simulation process of speech synthesis and at least one speech recognition simulation process, then the discrete simulation will start all simulation processes where the speech recognition process at the time the speech is detected is detected the user generates and inserts a priority of the utterance into the set of scheduled events, and at the moment of the utterance detection the user generates and inserts a priority of the utterance in the set of scheduled events, and the speech recognition process generates its meaning description a set of scheduled events as a priority event containing the result of the recognition, then the dialogue management system updates the internal state and generates a semantic response description based on the user's speech description, which is then converted from text to speech in the speech synthesis system and is output to the user by the output device; wherein the discrete simulation processes scheduled events by firing an event in a set of scheduled events, preferably firing a priority event, and processing the event by activating a process waiting for that event, processing the event, and the discrete simulation waiting for a set of new events to be returned, which then adds to the set of scheduled events, and the process pauses and removes the processed event from the set of scheduled events.