JP3299418B2 - Method and apparatus for speech recognition - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to the field of speech recognition, and more particularly to the recognition of unknown phrases. More specifically, the present invention relates to a method and an apparatus for speech recognition in consideration of background noise.

[0002]

2. Description of the Related Art Speech analysis and speech recognition algorithms,
Equipment and devices are generally well known. Such systems are becoming more powerful and cheaper. In recent years, the use of speech recognition systems has exploded. These systems allow users of data processing systems to use voice activation commands to direct various programs and applications. One of the goals of a speech recognition system is to provide a more human interface to the operation of a data processing system. Speech recognition system, mouse,
It is commonly used with other input devices such as a keyboard or printer. These input devices are often used to assist input / output (I / O) processing in speech recognition systems. Various known speech recognition systems typically include a set of recognizable phrases (ie, templates) from which a user can speak to use a voice activation command. The memory of a speech recognition system always contains a recognizable set. The recognizable set includes a set of digitized acoustic phrases from which to select recognizable phrases. For example, if there are 64 trained or trained phrases in the memory of the speech recognition system, the detected sounds that are background sounds or intended sounds are compared to this recognizable set. In this way, unintended background noise may create confidence factors that can be interpreted as recognizable phrases in the set.

[0003] Generally speaking, monitoring of the acoustic environment causes the speech recognition system to detect background noise. These background noises are often interpreted as recognizable inputs by the user. In such a situation, the background noise may cause a problem that the speech recognition system executes an operation or a command. Attempts have been made to solve this problem using calibration techniques. Such a method would essentially use a speech recognition system to monitor samples of background noise first. The sample serves as a collective factor when the speech recognition system actually tries to hear the recognizable phrase. These calibration techniques are often inefficient and determine that the sample of background noise detected during the calibration phase is the same or similar to the background noise that exists during the actual recognition phase. This often happens.

[0004] Another approach has allowed a user to manually deactivate the recognition mode of a speech recognition system. However, such an approach requires manually activating and deactivating the recognition mode when the user thinks that background noise will interfere with the operation of the speech recognition system. In this technique, the user must remember which mode the speech recognition system is operating in. In addition, activating and deactivating speech recognition can be very cumbersome. These background noises are often caused by the user. Examples of background noise caused by the user include sounds of peripheral devices such as a keyboard and a printer. These noises hinder the operation of the speech recognition system. That is, the system may recognize the background noise as a phrase corresponding to a command or a function.

[0005] The problem of inadvertently selecting background noise as a recognizable phrase occurs because the background noise is very similar to the phrases in the recognizable set in the memory of the speech recognition system. Therefore, it is convenient to have a method and apparatus that can recognize the operation of peripheral devices that generate background noise as background noise during the recognition mode of the speech recognition system.

[0006]

It is an object of the present invention to provide an improved method and apparatus for speech recognition.
Another object is to provide a method and an apparatus for recognizing an unknown phrase, and further to provide a speech recognition method and an apparatus in consideration of background noise.

[0007]

SUMMARY OF THE INVENTION The present invention provides a method and apparatus for analyzing a sound input event, and utilizes a template to analyze the sound input event. The speech input event is identified and the identified speech input event is recorded. The recorded speech input event is processed to create a first entry in the template. Selected non-speech input events occurring in the selected environment are identified and the identified non-speech input events are recorded. The recorded non-speech input event is then processed to create a second entry in the template. Thereafter, the sound input event is distinguished from the non-speech input event by comparing the sound input event with the template in which the non-speech input event is identified.

[0008] The foregoing has been described, by way of the following detailed description:
It will become apparent with other objects, features, and advantages of the present invention.

[0009]

FIG. 1 shows a multimedia data processing system 11 including a plurality of multimedia terminals 13 electrically connected to a computer 15. Computer 15 may be any personal computer system such as, for example, an IBM PS2 computer. The plurality of multimedia terminals 13 include all types of multimedia terminals that create or use real-time / asynchronous stream data, and include the video monitor 25 without limitation. Each of these multimedia terminals will be called by multimedia application software to create or use stream data.

For example, the operation of the CD-ROM player 17 resides in the computer 15 and is controlled by multimedia application software executed by the computer 15. The real-time digital data generated as the output of the CD-ROM player 17
The data stream is received and processed by computer 15 according to the instructions of the multimedia application in computer 15. For example, a real-time digital data stream may be compressed for storage on a floppy disk, or for transmission over a regular telephone line via a modem to a remote computer system. And the remote computer
When the system receives it, it decompresses and plays the digital stream data on the analog audio device.
Alternatively, the real-time data stream output from the CD-ROM player 17 is
After being received by a digital or analog filter, amplified and sound balanced, for output to acoustic speakers 31, 33,
Analog signal format, analog stereo amplifier 29
Sent to

A microphone 19 is used to receive an analog input signal corresponding to ambient sound. The real-time analog data stream is sent to computer 15, converted to digital form, and manipulated by multimedia application software, such as a speech recognition program. The digital data is stored, compressed, encrypted, filtered, transformed, and converted to an analog stereo amplifier 29.
Output in analog form to the telephone 23, sent as analog output to the telephone 23, output in digitized analog form as the output of a modem for transmission over the telephone line,
It is converted to a visual image for display on monitor 25, or undergoes various other different multimedia digital signal processing operations or normal multimedia digital signal processing operations.

Similarly, for the analog / digital input / output of the keyboard 21, the telephone 23 and the video monitor 25, ordinary multimedia operations are performed in the computer 15. In particular, the computer 15 is used as a speech recognition system that directs commands and functions to other applications running thereon. The microphone 19 is used to receive voice input events, i.e., human voices, which are processed using a multimedia application that recognizes voice by analyzing input from the microphone 19.

FIG. 2 is a block diagram illustrating the main hardware components used in the present invention to execute a multimedia application that controls the operation of the multimedia terminal device 13. A central processing unit (CPU) 33 is provided to computer 15 as is conventional in multimedia data processing operations. Multimedia application software, such as a voice recognition application, typically resides in RAM memory 35. The CPU 33 executes instructions included in the multimedia application. A digital signal processor 37 is provided as an auxiliary processor dedicated to performing operations on real-time / asynchronous stream data, as is usual in multimedia data processing operations. As is well known to those skilled in the art, digital signal processors are intended to perform operations on, or include, real-time data and provide real-time operability of multimedia terminals. A microprocessor device designed to operate very quickly and respond quickly. Digital signal processor 37
In general, a direct memory access (DMA) 39 is provided in order to increase the speed of the operation of the memory device, thereby realizing high-speed retrieval and storage of data.
In the present invention, an instruction memory (IM) 41 and a data memory (DM) 43 are provided to further speed the operation of the digital signal processor 37. Digital signal processor 37 and hardware interface 47
A bus 45 is provided for communicating data between and. The hardware interface 47 includes a digital-to-analog converter and an analog-to-digital converter. The inputs and outputs of the various multimedia terminals 13 are connected via digital-to-analog (D / A) converters and analog-to-digital (A / D) converters 47.
In FIG. 2, a telephone input / output 49, a microphone input 53 and stereo outputs 55 and 57 are shown as typical examples, and the A / D converter and D /
It is connected via an A converter. The MIDI input / output is also connected to the digital signal processor 37 via the hardware interface 47, but is not connected to the A / D converter and the D / A converter.

FIG. 3 illustrates a high-level flowchart of a process used by a user to prime an application to recognize speech recognition commands in accordance with a preferred embodiment of the present invention. The user must supply a set, a template of phrases for recognition, as shown in block 300.
The user also defines a set of actions in the form of a macro that reveals a predetermined action, as shown in block 302 according to a preferred embodiment of the present invention. Then, as shown at block 304, the user associates the particular phrase with the action. In other words, the user associates a speech phrase or ideal input event with the macro. Then, as shown in block 306, the user loads a template for the particular application. During this phase of the process, the speech recognition system encounters background noise, which may match an entry in the template, ie, a confidence factor for an entry in the set of phrases.

Also, the recognizable set in the template may not execute all of the commands desired by the user for a particular application. For example, the desired speech recognition template may not currently be in the memory of the speech recognition system. In such a case, the user can issue a voice command to swap the template from memory. This occurs when the user needs additional templates, as shown in block 308. In another situation, the user
Block 3 when loading a new application
As shown at 10, a new set of templates may need to be loaded.

The present invention uses a method and apparatus that allows a speech recognition system to automatically register background noise created by peripheral devices. The present invention may also automatically activate and deactivate the speech recognition mode based on interrupts from peripheral devices. The present invention involves a method and apparatus wherein background interrupt noise is not ignored and is dynamically added to the set of recognizable phrases. An empty command is registered in the background noise phrase according to the preferred embodiment of the present invention.

[0017] Alternatively, the command may be associated with a background noise phrase. For example, the command may deactivate the speech recognition system until some other event occurs. This approach allows the speech recognition system to be dynamically populated for background noise. The system may be trained to recognize different background noises, which reduces the likelihood that the background noise will be mistaken for a recognizable phrase in the set. That is, the confidence factor for the background noise is included in the recognizable set.

FIG. 4 shows a template 400 according to a preferred embodiment of the present invention. The phrase column is
Identify the textual representation of the recognizable phrase. The command column identifies a command, such as a keyboard macro, that runs on a speech recognition system that recognizes acoustic phrases. For example, in a speech recognition system, "PRINTD
When the phrase "OCUMENT" is recognized, the function key F7 is sent to the keyboard buffer,
Next, the word DOC is sent, and the ENTER key (@
E) enters the keyboard buffer. As a result, "P
On a speech recognition system that recognized the phrase "RINT DOCUMENT", the application
Receives the keystroke DOC ENTER.
These will be the commands that the application needs to print the document. The digitized format column shows a graphical representation of the acoustic samples for each phrase in the template. The display is for illustration purposes only and shows the average of how the user speaks a particular phrase, ie, the sampled phrase that was loaded.

The comparison between the digitized sound form and the digitized form in the template supplied by the user is made by a speech recognition system that detects sounds, ie, acoustic input events. Upon detecting background noise as defined by the interrupt criteria, an acoustic phrase is dynamically added to the template. The symbol ｛｝ indicates a phrase entry created according to the invention, as can be seen in the phrase column of FIG.

In accordance with a preferred embodiment of the present invention, a null command can be associated with the created phrase entry. When the speech recognition system detects an acoustic input event of a background sound or a human voice, it compares the sound with all entries in the template. Background sound is "non-voice input event"
The human voice is also called "speech input event".
Speech recognition systems have higher confidence factors for comparing acoustic input events to recognizable set entries in the template and for comparing background noise to samples of background noise.

FIG. 5 illustrates a preferred embodiment of the present invention.
FIG. 4 shows a flowchart of a process for registering peripheral devices that may create noise, ie, background sounds (non-voice input events). The process begins by receiving user input in the form of a peripheral device name, as shown in block 502. The process then receives a user input identifying the interrupt for each of the peripheral devices, as shown in block 504. Thereafter, the process receives user input indicating a communication port associated with the peripheral device, as shown in block 506. The process then proceeds as shown in block 508:
Receive user input regarding elapsed time of device recognition.

Next, as shown in block 510,
User input is received identifying any optional commands to be performed during recognition. And the process is
As shown at block 512, user input for a notification selection is received. The user may choose to be notified when the proper recognition has been made. The process then proceeds as shown in block 514:
It is determined whether the user wants to be notified while detecting noise from the peripheral device. If the user wishes to be notified, the process receives user input specifying an output device for the notification, as shown in block 516. The user will be notified via various output devices, such as a speaker for audio notification or a video monitor for video notification. The process then activates the notification flag, as shown in block 518.

Thereafter, the process ends, as indicated by block 520, after storing the information entered by the user in the device recognition table. Block 514
At, if the user does not want notification, the process proceeds to block 520. The device recognition table may take various forms, such as a file continuation field or a relational database.

FIG. 6 detects a sound (also called an "acoustic input event") that is a speech input event or a non-speech input event;
4 shows a flowchart of a process for provisioning and updating a speech recognition system to identify. The process begins by loading the device recognition table into active memory, as shown in block 600. The device recognition table is data input and stored by the user as shown in FIG. The process sets the interrupt vector to intercept before the interrupt from the peripheral device specified in the device recognition table reaches the target application, as shown in block 602. The process then activates the monitor service, as shown in block 604. The monitor services used to monitor interrupts are well known to those skilled in the art, and various methods may be used in accordance with the preferred embodiment of the present invention.

The process waits for an interrupt from a peripheral, as indicated by block 606. Then, as indicated at block 608, the process receives an interrupt from a peripheral device. Next, the process passes the interrupt to the finally communicating real application address to the interrupt to the target application, as shown in block 610. The process then records the time of receipt of the interrupt, as shown in block 612. Next, the process starts an expiration clock, as indicated by block 614. The expiration clock is basically a timer used in the preferred embodiment of the present invention to determine the time elapsed since the detection of the interrupt.

Then, as indicated by block 616, the process waits for acoustic recognition. In other words, the process waits to see if a recognition pattern that meets the trust threshold of an entry in the template is found. Upon recognizing the speech, the process determines whether an audible interrupt has been received, as indicated at block 618. An acoustic interrupt occurs when an input device, such as a microphone, detects an acoustic input event. If an acoustic interrupt has not been received, the process then proceeds to block 6
As shown at 20, it is determined whether the time for recognition has expired. If expired, the process clears the record of the time of the interrupt reception, as shown in block 622. Then, the process returns to block 606 to wait for an interrupt from the peripheral device. At block 620, if the time for recognition has not expired, the process proceeds to block 6
Return to step 16 and wait for acoustic recognition.

At block 618, if an acoustic interrupt has been received, the process proceeds to receive an acoustic pattern (acoustic input event), as indicated at block 624. The process records the time of receipt of the acoustic pattern, as indicated at block 626. Thereafter, the process determines whether the acoustic pattern is recognized, as shown at block 628. If no acoustic pattern is recognized, the process proceeds to block 630.
As shown in (1), the process proceeds to a process for clearing the record of the time of receiving the acoustic pattern. The process then proceeds to block 622.

At block 628, if an acoustic pattern is recognized, the process subtracts the peripheral interrupt time from the acoustic interrupt time to determine the elapsed time, as shown at block 632. Next, the process determines whether the calculated time is within an acceptable range, as indicated by block 634. If not, the process proceeds to block 630, as described above. On the other hand, if so, the process then determines whether the user should be notified of the recognition of the non-speech input event, as shown at block 636. If the user should not be notified, the process proceeds to block 63
As shown at 8, it is determined whether the command should be executed.

At block 636, if the user is to be notified, the process notifies the user according to the recognition table definition, as indicated at block 640. Thereafter, the process determines whether the command is to be executed, as indicated by block 638. If the command is to be executed, the process executes the command according to the recognition table, as shown in block 642. Thereafter, as shown in block 644, the noise (non-speech input event) is stored as a recognizable template pattern. In other words, non-voice input events are stored as template entries. Thereafter, the process proceeds to block 630, as described above. At block 638, if the command is not to be executed, the process proceeds directly to block 644.

In accordance with a preferred embodiment of the present invention, the process illustrated in FIG. 6 is implemented as a Terminate Resident (TSR) service that intercepts interrupts registered by a user or other entity, such as a person responsible for managing the application. Is done. An interrupt from a peripheral device is immediately sent to the interrupt vector table address associated with it, that is, the interrupt is forwarded to the appropriate device.
This ensures that the keyboard interrupt service receives the keyboard interrupt and the printer service receives its output. Detectable speech recognition phrases that do not match the confidence factors in the template, but are received within the specified time of the interrupt, are candidates for a null association according to a preferred embodiment of the present invention. Each peripheral has an associated interrupt defined for it. For example, a personal computer can use interrupt 14H for keyboard interrupts.

The present invention can be used to intercept hardware or operating system interrupts. The registration service shown in FIG. 5 allows a user to specify an interrupt at which recording should be activated for an acoustic input event. The user may adjust the sensitivity at which the interruption should be interpreted as background noise, ie, a non-speech input event. Certain default values will be set for existing devices. For example, printers are usually
Operates at interrupt 5H.

According to a preferred embodiment of the present invention, a pre-process is used to evaluate whether the detected acoustic input event should be compared with an existing empty phrase or whether a new phrase should be created. Sometimes. Such a process would require the continuous use of background noise to feed the system with a particular noise phrase. In addition, empty commands may take the place of user-supplied commands or system default commands. For example, the command could issue a SAVE command for the word processor. As noise activity increases, such as jamming noise, the user may want to save the work that has just been completed. In such a case, one of the background noises matches the executable phrase.

When the user creates an empty association,
It may be notified graphically. Such notifications are:
It is also possible through acoustic means. Further, the user could be able to change the empty command at the time of creation notification.

The present invention also provides for,
It would also allow switching the entire template based on the type of interrupt received. Such an option means that the detected interrupt will be an excellent signal for new applications that use a different set of speech recognition phrases, requiring a new set of templates.

In accordance with a preferred embodiment of the present invention, the fundamental problem of a speech recognition system involving distinguishing non-speech input events from speech input events is addressed. The present invention provides that the peripheral device will create background noise, and
Recognize that the system may inherently execute commands inappropriate for the application in response to this background noise. The present invention provides a method and apparatus that allows a peripheral device to add digitized speech phrases into a speech recognition set such as a template. The invention has further advantages over the prior art in that the invention need not always be activated. Once the background noise is "built in" and registered in the template, the invention may be deactivated or removed. This has the advantage of freeing up computer resources for other applications.

The invention has been described with reference to examples.
It will be understood by those skilled in the art that various changes may be made in form and detail without departing from the spirit and scope of the invention.

[0037]

According to the present invention, speech can be distinguished from non-speech such as background noise, and the accuracy of speech recognition is improved.

[Brief description of the drawings]

FIG. 1 is a multimedia data processing system according to a preferred embodiment of the present invention.

FIG. 2 is a block diagram of the main hardware components used to execute an application, such as a speech recognition system, according to a preferred embodiment of the present invention.

FIG. 3 is a high-level flowchart of a process used by a user to train an application to recognize speech recognition commands, in accordance with a preferred embodiment of the present invention.

FIG. 4 is a template according to a preferred embodiment of the present invention.

FIG. 5 is a flowchart of a process for registering peripheral devices that may create noise, ie, background sound, according to a preferred embodiment of the present invention.

FIG. 6 is a flowchart of a process for installing and updating a speech recognition system to detect and identify sounds of a speech input event or a non-speech input event.

[Description of sign]

 11 Multimedia Data Processing System 13 Multimedia Terminal Device 15 Computer 17 CD-ROM Player 19 Microphone 21 Keyboard 23 Telephone 25 Video Monitor 29 Analog Stereo Amplifier 31 Acoustic Speaker 33 Acoustic Speaker 45 Bus

Continuation of the front page (56) References JP-A-49-57702 (JP, A) JP-A-59-109095 (JP, A) JP-A-60-158494 (JP, A) JP-A-3-160499 (JP) JP-A-1-138595 (JP, A) JP-A-5-19781 (JP, A) JP-B-2-22960 (JP, B2) JP-B-4-49952 (JP, B2) JP-B 5-56512 (JP, B2) JP 61-49226 (JP, B1) (58) Fields investigated (Int. Cl. ⁷ , DB name) G10L 15/00-15/28 JICST file (JOIS)

Claims (8)

(57) [Claims] 1. A method for analyzing a sound input event using a template in a data processing system including a non-speech input event that becomes background noise and a peripheral device that generates an interrupt, comprising: identifying a speech input event; Recording the identified speech input event; processing the recorded speech input event to create a first entry in the template; identifying a selected non-speech input event occurring in the selected environment. Recording the identified non-speech input event; processing the recorded non-speech input event to create a second entry in the template; and detecting an interrupt from the peripheral device Detecting an audio input event occurring after the interrupt; and By comparing the template, acoustic input event analysis method comprising the distinguishing step the audio input event and non-speech input events. Determining, in response to the identification of the non-speech input event, whether a command is associated with the non-speech input event; and wherein the command is associated with the non-speech input event. Executing the command in response to the request. 3. A method for analyzing a sound input event using a template in a data processing system including a non-speech input event that becomes background noise and a peripheral device that generates an interrupt, comprising: identifying a speech input event; Recording the identified speech input event; processing the recorded speech input event to create a first entry in the template; identifying a selected non-speech input event occurring in the selected environment. Recording the identified non-speech input event; processing the recorded non-speech input event; and creating a second entry in the template for the processed non-speech input event; Detecting an interrupt from a peripheral device; and detecting an acoustic input event occurring after the interrupt. Identifying the non-voice input event by comparing the acoustic input event with the template; and responding to the identification of the non-voice input event occurring within a predetermined time after the interruption, Determining whether or not the command is associated with the non-speech input event; and executing the command in response to the command being associated with the non-speech input event. Event analysis method. 4. The method according to claim 1, further comprising the step of processing said identified non-speech input event occurring within a predetermined time after said interruption and replacing said second entry of said template for said processed non-speech input event. 4. The method of claim 3, comprising: 5. An apparatus for analyzing a sound input event using a template in a data processing system including a non-speech input event serving as background noise and a peripheral device generating an interrupt, wherein the first input device identifies the speech input event. Identification means, and first recording means for recording the identified speech input event, and processing the recorded speech input event, and
First processing means for creating an entry of the following, second identification means for identifying a selected non-speech input event occurring in a selected environment, and second recording means for recording the identified non-speech input event Second processing means for processing a recorded non-voice input event and creating a second entry of the template for the processed non-voice input event; and first detecting an interrupt from the peripheral device. Detecting means for detecting a sound input event occurring after the interruption, and comparing means for distinguishing a voice input event from a non-voice input event by comparing the sound input event with the template. Including equipment. 6. A means responsive to the identification of the non-voice input event to determine whether a command is associated with the non-voice input event, and wherein the command is associated with the non-voice input event. Means for executing the command in response to the request. 7. An apparatus for analyzing an audio input event using a template in a data processing system including a non-voice input event serving as background noise and a peripheral device for generating an interrupt, wherein the first input device identifies the voice input event. Identification means, and first recording means for recording the identified speech input event, and processing the recorded speech input event, and
First processing means for creating an entry of the following, second identification means for identifying a selected non-voice input event occurring in a selected environment, and second recording for recording the identified non-voice input event Means for processing the recorded non-speech input event, and second processing means for creating a second entry of the template for the processed non-speech input event; and detecting an interrupt from the peripheral device First detection means, second detection means for detecting an audio input event occurring after the interruption, and third identification means for identifying a non-voice input event by comparing the audio input event with the template. Determining, in response to identifying a non-voice input event occurring within a predetermined time after the interruption, whether or not a command is associated with the non-voice input event; Command is in response to being associated with a said non-speech input events, and an execution means for executing the command, the acoustic input event analyzer. 8. The apparatus further comprising: means for processing the identified non-voice input event occurring within a predetermined time after the interruption and replacing the second entry of the template for the processed non-voice input event. An apparatus according to claim 7.