KR102109876B1 - Privacy-preserving training corpus selection - Google Patents

Abstract

The present invention relates to training a speech recognition system. The system includes an automatic speech recognizer and receives data from the client device. The system determines that at least a portion of the received data is important data. Before at least a portion of the received data is deleted, the system provides at least a portion of the received data to a model training engine that trains a recognition model for an automatic speech recognizer. After at least a portion of the received data is provided, the system deletes at least a portion of the received data.

Description

Personal Information Protection Training Corpus Selection {PRIVACY-PRESERVING TRAINING CORPUS SELECTION}

This specification generally relates to automatic speech recognition.

Speech recognition is about the transcription of words spoken by humans. In performing speech recognition, acoustic and language models are used by an automatic speech recognition (ASR) engine to statistically analyze the encoded speech to generate one or more possible transcriptions that reflect the words spoken by the speaker.

According to an implementation example, the present specification identifies sensitive information among data that a user submits to a recognition system, for example, an automatic voice recognizer, and also allows the user to recognize the sensitive information before the sensitive information is deleted. Describes the process of using this information to train a recognizer model used by subsequent iterations, for example an optical recognition model, acoustic model or language model. In doing so, additional iterations of the recognition system can accurately recognize submitted data containing similar sensitive information, while complying with all appropriate data privacy policies regarding the collection and storage of sensitive information.

In some aspects, the invention described herein includes receiving data from a client device by a system comprising an automatic speech recognizer and determining that at least a portion of the received data is likely to be sensitive data, The important information is data that is not stored in a log of data received from client devices, but is deleted by the system or within a predetermined receiver between the systems. Additional operations may include providing at least a portion of the received data to a model training engine that trains recognition models for the automatic speech recognizer before at least a portion of the received data is deleted. Additional operations may include deleting at least a portion of the received data after at least a portion of the received data is provided.

Other versions include a computer program configured to perform operations of the above methods encoded on a corresponding system, apparatus, and computer storage devices.

Each of these and other versions may optionally include one or more of the following features. For example, in some implementations, the received data includes one or more of audio data, transcription data, and contextual metadata. In some aspects, the predetermined period is shorter than the default period, and after this period, data determined to be unlikely to be important is deleted.

In some aspects, the operations include determining that at least a portion of the received data is likely to be sensitive, the operation comprising at least a portion of the received data in a plurality of classifiers-each of which is different Associated with a type of sensitive data-an indication provided as an operation provided to and from a specific classifier associated with a specific type of important data, at least a portion of the received data is associated with the specific type of important data above the confidence threshold It includes the operation of receiving. In some implementations, each type of sensitive data has a predetermined associated period for deletion.

In some aspects, the operations include anonymizing at least a portion of the received data, and storing the anonymized data in a log of data received from client devices. In some aspects, operations include receiving an updated recognition model from the model training engine, providing the updated recognition model to the automatic speech recognizer, and subsequently using the updated recognition model. And recognizing the received audio data.

In some implementations, the important data includes one or more of contact information, background audio data, personal identification information, predetermined transcription keywords and geographic coordinates.

Advantageous implementations may include one or more of the following features. For example, instead of assuming that there is some static speech or data set, such as a new phone to which speech recognition models are applied, a new noise condition or a new accent, the speech recognition techniques described above always allow speech records to be recorded in the logs. When entering, it can be trained in a continuous loop. This makes it possible to cope with the challenge of requiring that any speech remarks and / or contextual information needed to accurately recognize the remarks is not logable and must be removed from the systems quickly, eg within seconds, minutes, days, etc. . Through the techniques described herein, the model training can follow a distributed backpropagation neural network training implementation. Selection and transcription algorithms can be integrated into the production setup, and when data points arrive in the log, they can be fed into training servers in real time. The described technique potentially allows large-scale speech recognition systems to dictate that demanding legal environments should keep logging to a minimum, or markets where logs must be implemented in ephemeral storage. Can make it work more successfully.

Details of one or more implementations of the invention described herein are submitted in the accompanying drawings and the description below. Other potential features, aspects and advantages of the invention will become apparent from the detailed description, drawings and claims.

1 is a diagram of an exemplary speech recognition system.
2 is a flow diagram illustrating an exemplary process for training a speech recognizer.
3 is a diagram of an exemplary personal information protection speech recognition system.
4 shows an example of a computing device and a mobile computing device that can be used to implement the techniques described herein.
The same reference numbers and names in the various drawings indicate the same elements.

Training of acoustic models for speech recognition can involve algorithms that consume significant computational resources, such as hundreds or thousands of machines, and it will take a very long time to achieve good quality, for example a week or more. Can be. Some training approaches assume that audio recordings of voice data can always be stored, which may not be due to user privacy restrictions. In a production environment where privacy is important, voice data may or may not be logged from time to time. Occasionally, additional contextual information needed to accurately transcribe a utterance cannot be logged.

As an example, contacts recognition for voice actions such as "send text message to John" may require that the user's contact must be known to the voice recognizer when recognition is complete. This information can be accessed when a production query comes in, but when storing a user's contact database has a variety of privacy implications. Such contextual information can sometimes enable large speech recognition quality enhancements. Therefore, it is desirable to use such non-logged context information to handle such information within a speech recognition and analysis environment while satisfying all legal and security standards. Accordingly, there is a need for improved techniques, systems and methods to provide high quality speech recognition for sensitive data subject to one or more limitations on how such data is handled, stored or processed within a speech recognition and analysis environment. exist.

Accordingly, the present invention describes techniques for using important data in training an acoustic model for automatic speech recognition. For example, important data may be data that is deleted by the system without being stored in a log of data received from client devices. As another example, the important data may be data that is deleted between receivers predetermined by the system. These techniques include identifying important data during speech recognition. Before the identified important data is deleted, this data can be used to train the automatic speech recognizer. The sensitive data may include, for example, important information in the audio content of the data or important information related to the context of the audio content.

1 is a diagram of an exemplary speech recognition system 100. User 102 submits speech 104 to speech processing system 106 for recognition and transcription. In this example, utterance 104 submitted by user 102 is the phrase “Bob Smith. My credit card number is 01-234”. User 102 speaks 104 to computing device 112, such as a mobile computing device, tablet computer, wearable computer, e-book reader, desktop computer, laptop computer, and the like. Computing device 112 may, for example, execute a text editor with speech recognition input, a search bar with free-form speech input, or any other suitable program. Computing device 112 may generate a speech recognition request 107 that includes a speech data item 108a (eg, a waveform data file, an audio file, or other data type encoding speech 104). .

The speech recognition request 107 may also include context data 108b. The context data 108b may include, for example, information related to the application or web page that the user 102 was accessing during the speech 104, geographic coordinates of the user 102, information related to the contact list of the user 102, and the like. The same may include such contextual information or other contextual information. Other examples of context data 108b include request 107 metadata, user 102 biological characteristics (eg, gender, age, accent or other characteristics), noise conditions, user 102's country of residence / region of residence, And a conversation state. In some cases, the speech recognition request 107 may include sensitive data. Computing device 112 may send request 107 to voice processing system 106.

The speech processing system 106 is a system that receives a speech recognition request 107 associated with the speech 104, analyzes the request 107, and generates a text transcription 110 of the speech 104. In general, the text transcription 110 may include a character string or other textual representation representing the top hypothesis of the speech recognizer for the content of the remarks of the request 107. The transcription 110 can be transmitted to the computing device 112 or other system or program. The voice processing system 106 may transmit the transcription 110 to the user 102 in real time or near real time (eg, tens of milliseconds, hundreds of milliseconds, or other duration of time).

In general, such a system, such as speech processing system 106, may receive a speech recognition request 107 from users on a large user basis. The voice recognition system may generate text results 110 of one or more recognition processes executed by the voice recognizer under each request 107 in response to each request 107. Although for simplicity, a single user 102 is shown, the voice recognizer receives requests 107 from a much larger number of users, eg tens of thousands, and can take some or all of the requests 107 received. By processing in parallel, it is possible to provide text results 110 to each user while other requests 107 are received and processed.

Further, the speech recognizer system can be any type of suitable system for performing speech recognition tasks. For example, the speech recognizer can be assigned to a group of virtual machines or tasks running in a massively distributed computing environment that also hosts virtual machines or tasks for various different services, tasks or applications. It can be configured by. In some implementations, the voice recognizer may be a component of a large system of servers or services or may be one of many services provided in connection with some product or service.

The exemplary speech processing system 106 includes a front end 114 that receives a speech recognition request 107 from the computing device 112. For example, the front end 114 can receive the speech recognition request 107 and prepare the request 107 data for processing. The front end 114 can send the request 107 to the critical information identifier 116. The critical information identifier 116 can analyze the request 107 to determine whether at least a portion of the data in the request 107 is likely data.

Important information identifier 116 can identify various types of data that are likely to be important within request 107. Data that may be important may be included in audio data 108a and / or context data 108b. The important information identifier 116 can analyze the background audio present in the audio data 108a for information that is likely to be important. For example, the background audio may include information identifying a user's location, information identifying speakers other than the user, or other important information.

The sensitive information identifier 116 can identify such sensitive financial information, such as credit card information, bank account information, monetary amounts, or other sensitive financial information. Important information identifier 116 may also identify such user contact information, such as people, phone numbers, e-mail addresses, or other potentially important contact information in the user's contact list. The sensitive information identifier 116 may identify sensitive personal information such as a name, social security number, address, and phone number or other sensitive personal information. These are exemplary types of sensitive data that can be identified, and in other implementations, other types of sensitive data can be identified. For example, other information may include location information (eg, Global Positioning System (GPS) coordinates, mobile phone information, radio access information, web page information) or other information or data.

The important information identifier 116 may include one or more important data classifiers 118a-e, each of the classifiers 118a-e being associated with different types of important data. For example, each type of important data described above can have an associated classifier. Each classifier can provide an indication that at least a portion of the request 107 contains sensitive data associated with the classifier. For example, the classifier can provide an indication that the request 107 is likely to include sensitive data above a confidence threshold associated with the relevant type of sensitive data.

Referring to the example shown in FIG. 1, important information identifier 116 has identified some of the most likely information in request 107. Background audio classifier 118a has confirmed that the background audio is likely to contain important information. For example, the background audio may include audio indicating that the user 102 is located in the hospital. The credit card classifier 118b has identified that the audio data 108a of the remark 104 contains credit card information that is likely to be important, that is, the credit card number “01-234”. However, the user contact list classifier 118c has determined that the audio data 108a does not contain important information about the user's contact list. The personal information classifier 118d identified personal information that is likely to be important, namely, "Bob Smith". Other classifiers represented by classifier 118e may determine other types of important data.

Important information identifier 116 may send some or all of the request 107 data to training engine 118. In some cases, some or all of the request 107 data is sent directly to the training engine 118 by the front end 114. Training engine 118 is a component that trains recognition models that can be used by an automatic speech recognizer (eg, automatic speech recognizer 120).

The training engine 118 may use some or all of the audio data 108a and context data 108b to create a speech recognition model or update an existing speech recognition model. In some cases, the use of context data 108b in conjunction with audio data 108a may enable a more accurate speech recognition model than using audio data 108a alone. The speech recognition model may be provided to the automatic speech recognizer illustrated in FIG. 1 with an arrow 119 connecting the training engine 118 to the automatic speech recognizer 120. In this way, the training engine 118 can use the data received from the request 107, including the data of interest before any data of interest is deleted.

Some or all of the request data 107 is also sent to the automatic speech recognizer 120. The automatic speech recognizer 120 generates a transcription 110 of the audio data 108a using the speech recognition model provided by the training engine 118. This transcription 110 is provided to the computing device 112.

Some or all of the request 107 data is also sent to critical information manager 122. In addition, the important information signal 123 may be transmitted from the important information identifier 116 to the important information manager 122. Transcription 110 may also be provided to critical information manager 122. For example, the critical information signal 123 may indicate which portion of the request 107 and / or the transcription 110 is likely to contain sensitive information. Important information signal 123 may be based on indications from classifiers 118a-e. Based on the critical information signal 123, the critical information manager 122 can determine which parts of the request 107 and / or the transcript 110 should be logged and which parts should not be logged. For example, portions of request 107 and / or transcription 110 that are not to be logged may be deleted. In some cases, critical information manager 122 may anonymize request data 107 and / or transcript 110. For example, the warrior "This is Bob Smith" can be anonymized as "This is <name>". Other anonymization techniques are possible. Important information manager 122 may provide portions of request 107 and / or transcription to logger 124 for storage.

2 is a flow diagram illustrating an example process 200 for training an automatic speech recognizer. The example process 200 can be implemented, for example, by some or all of the speech recognition system 100. In step 202, data from the client device is received by a system that includes an automatic speech recognizer. For example, the data may include a waveform data file, an audio file, or other data type encoding a utterance, meta data associated with the utterance, transcriptional data, and / or contextual data associated with the utterance. The client device may be a portable computer, smart phone, or other client device as described above. The voice recognizer can use one or more voice recognizer components in generating transcriptions of such voice data items, such as an acoustic model, a language model, or other type of voice recognizer component. The one or more speech recognizer components may be neural network based.

In step 204, it is determined that at least a portion of the received data is important data. In some cases, sensitive data is data that is deleted by the system without storing in a log of data received at the client device. In some cases, the sensitive data may include one or more of contact information, background audio data, personal identification information, predetermined transcription keywords or geographic coordinates. Important data may also be data deleted within a predetermined reception period by the system. For example, the predetermined period may be shorter than the default period-data determined to be insignificant after this default period is deleted. In some implementations, each type of sensitive data has a predetermined associated period for deletion.

Determining that at least a portion of the received data will be important data can include providing at least a portion of the received data to one or more classifiers. Each classifier can be associated with different types of important data. Certain classifiers associated with a particular type of sensitive data may generate an indication that the received data is associated with a particular type of sensitive data.

In step 206, before at least a portion of the received data is deleted, at least a portion of the received data is provided to a model training engine that trains recognition models for an automatic speech recognizer. At least a portion of the received data may include audio data and / or context data. In some implementations, the automatic speech recognizer can receive the updated recognition model from the model training engine. The updated recognition model can be used to automatically recognize the audio data that is subsequently received.

In step 208, after at least a portion of the received data is provided to a model training engine, at least a portion of the received data is deleted. In some implementations, at least a portion of the received data is anonymized, and the anonymized data can be stored in a log of data received from the client device.

3 is a diagram of an exemplary speech recognition system 300. The system 300 may be implemented by, for example, the speech recognition system 100 shown in FIG. 1 or the process shown in FIG. 2. The speech recognition system 300 includes privacy-sensitive data 302. The personal information-critical data 302 may be a voice recognition request. The personal information-critical data 302 may include context information 304 and audio data 306.

Some of the personal information-critical data 302 may be determined to be logable. For example, some of the personal information-critical data 302 may not contain sensitive information, and that portion may be logged. As another example, a portion of the personal information-critical data 302 may be processed anonymously. A logable portion of personal information-critical data can be provided to the logger 308 for storage. In some implementations, the logger 308 includes a separate audio storage 310a and a separate context storage 310b. For example, the audio storage device 310a can store the audio data 306, and the context storage unit can store the context data 304. In some implementations, audio data 306 is not stored separately from context data 304.

Some or all of the privacy-critical data can be provided to the exemplary model trainer 312. The model trainer 310 may be, for example, a training engine for a speech recognizer. In some implementations, the privacy-critical data 302 is used only once for training prior to deletion, and the new privacy-critical data 302 is continuously provided to the model trainer 312. Model trainer 312 may include one or more components. For example, the model trainer 312 may select training data from the personal information-important data 302 using heuristic-based data selection shown at 314a. The model trainer 312 may generate a candidate model shown by reference numeral 314b using, for example, a mini-block training technique and a training loop technique. As shown at 314c, candidate models can be investigated and analyzed to identify the model with the best performance. The model trainer 312 may include other components or use other techniques.

The speech model 316 can be provided from the model trainer 312 to the production speech recognition server 318. Production speech recognition server 318 may receive a speech recognition request from a client device, and may provide speech transcriptions back to the client device using one or more speech models 316. In this way, voice models 316 can be continuously updated using privacy-critical data 302 and model trainer 312 while maintaining any policies of the system related to privacy-critical data retention. .

4 shows an example of a computing device 400 and a mobile computing device that can be used to implement the techniques described herein. Computing device 400 is intended to represent various types of digital computers, such as laptops, desktops, workstations, PDAs, servers, blade servers, mainframes, and other suitable computers. Mobile computing devices are intended to represent various types of mobile devices, such as personal portable information terminals, cellular telephones, smart phones, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are meant to be exemplary only, and are not meant to limit the implementation of the inventions described herein and / or in the claims.

Computing device 400 includes processor 402, memory 404, storage device 406, memory 404, and high-speed interface 408 and low-speed interface 414 coupled to multiple high-speed expansion ports 410 and It is stored in the storage device 406. The processor 402, memory 404, storage device 406, high speed interface 408, high speed expansion port 410, and low speed interface 412 are interconnected using various buses, either on a common motherboard or It is mounted in any other suitable way. Processor 402 processes instructions for execution within computing device 400, including instructions stored in memory 404 or storage device 406, and displays graphical information for the GUI coupled to high-speed interface 408. It can be displayed on such an external input / output device such as (146). In other implementations, multiple processors and / or multiple buses can be used as appropriate with multiple memories and types of memory. In addition, multiple computing devices can be connected, each device providing portions of the necessary operations, such as a server bank, a group of blade servers, or a multiprocessor system.

Memory 404 stores information in computing device 400. In some implementations, the memory 404 is volatile memory unit (s). In some implementations, the memory 404 is non-volatile memory unit (s). The memory 404 can also be other types of computer readable media, such as magnetic or optical disks.

The storage device 406 can provide a mass storage device for the computing device 400. In some implementations, the storage device 406 is a device, including a floppy disk device, hard disk device, optical disk device or tape device, flash memory or other similar solid state memory device, or devices or other configurations in a storage area network. It can be or include such a computer readable medium, such as an array of. Computer program material may be embodied in an information carrier type. The computer program product may also contain instructions that, when executed, perform one or more methods as described above. Computer program material may also be tangibly embodied in such computer or machine readable media, such as memory on memory 404, storage device 406, or processor 402.

High-speed interface 408 manages bandwidth-intensive operations for computing device 400, while low-speed interface 412 manages low-bandwidth-intensive operations. This assignment of functions is merely exemplary. In some implementations, the fast interface 408 is connected to a memory 404, a display 416 (eg, via a graphics processor or accelerator), and a fast expansion port 410 that can accommodate various expansion cards (not shown). Connected. In these implementations, the slow interface 412 is connected to the storage device 406 and the slow expansion port 414. The low speed expansion port 414, which may include such various communication ports as USB, Bluetooth, Ethernet, wireless Ethernet or the like, is one or more such input / output devices such as a keyboard, pointing device, scanner, or network, for example It can be connected to a networking device such as a switch or router through an adapter.

The computing device 400 may be implemented in a number of different forms, as shown in the figure. For example, it can be implemented as a standard server 420, or multiple times as a group of such servers. Also, it may be implemented in a personal computer such as a laptop computer 422. Also, it may be implemented as part of the rack server system 424. Alternatively, components from computing device 400 may be combined with other components of such a mobile device (not shown), such as mobile computing device 450. Each of these devices may include one or more of the computing device 400 and the mobile computing device 450, and the entire system may be composed of multiple computing devices communicating with each other.

Mobile computing device 450 includes, among other components, such input / output devices, such as processor 452, memory 464, display 454, communication interface 466, and transceiver 468. The mobile computing device 450 may also be provided with such storage devices, such as micro-drives or other devices, to provide additional storage devices. Each of processor 452, memory 464, display 454, communication interface 466, and transceiver 468 are interconnected using various buses, some of these components on a common motherboard or other suitable Can be mounted in a manner.

The processor 452 can execute instructions in the mobile computing device 450, including instructions stored in the memory 464. The processor 452 may be implemented as a chipset of chips comprising individual or multiple analog and digital processors. Processor 452 coordinates other components of such mobile computing device 450, such as, for example, control of user interfaces, applications executed by mobile computing device 450, and wireless communication by mobile computing device 450. (coordination).

The processor 452 can communicate with the user through a control interface 458 and a display interface 456 connected to the display 454. The display 454 may be, for example, a TFT (Thin-Film-Semiconductor Liquid Crystal Display) display or an Organic Light Emitting Diode (OLED) display, or other suitable display technology. Display interface 456 may include suitable circuitry to drive display 454 to provide graphics and other information to the user. The control interface 458 can receive instructions from the user and convert them to submit to the processor 452. In addition, the external interface 462 may communicate with the processor 452 to enable short-range communication of the mobile computing device 450 with other devices. The external interface 462 can provide, for example, for wired communication in some implementations, or wireless communication in other implementations, and multiple interfaces can also be used.

Memory 464 stores information in mobile computing device 450. The memory 464 may be implemented as one or more of a computer readable medium or medium, a volatile memory unit, or a non-volatile memory unit. The extended memory 474 may also be provided and connected to the mobile computing device 450 through an extended interface 472, which may include, for example, a Single In Line Memory Module (SIMM) card interface. The extended memory 474 may provide extra storage space for the mobile computing device 450 or store applications or other information for the mobile computing device 450. In particular, the extended memory 474 may include instructions for performing or supplementing the aforementioned process, and may also include security information.

Thus, for example, the extended memory 474 can be provided as a security module for the mobile computing device 450 and can be programmed with instructions that allow the secure use of the mobile computing device 450. In addition, the SIMM card can be provided with a secure application along with additional information such as disposing the identification information on the SIMM card so that it cannot be hacked.

The memory may include, for example, flash memory and / or NVRAM memory (non-volatile random access memory), as described below. In some implementations, the computer program product is tangibly embodied in an information medium. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The computer program product may be a computer or machine readable medium, such as memory 464, extended memory 474, or memory on processor 452. In some implementations, the computer program product can be received, for example, as a signal propagated through the transceiver 468 or the external interface 462.

The mobile computing device 450 can communicate wirelessly through a communication interface 466, which may include digital signal processing circuitry, if desired. The communication interface 466 includes, among other things, GSM voice calls (global systems for mobile communication), SMS (Short Message Service), EMS (Enhanced Messaging Service) or MMS Messaging (Multimedia Messaging Service), CDMA (Code) Provides communications under such various modes or protocols, such as split multiple access (TDMA), time division multiple access (TDMA), personal digital cellular (PDC), wideband code division multiple access (WCDMA), CDMA2000 or GPRS (General Packet Radio Service) can do.

Such communication may occur, for example, via transceiver 468 using radio frequencies. In addition, short-range communication may occur, such as using Bluetooth, WiFi, or other transceiver (not shown). In addition, the GPS receiver module 470 may provide the mobile computing device 450 with additional navigation and location related wireless data that may be appropriately used by applications running on the mobile computing device 450.

The mobile computing device 450 can also aurally communicate using an audio codec 460 that can receive voice information from a user and convert it into available digital information. The audio codec 460 can likewise generate an audible sound for the user, for example through a speaker in the handset of the mobile computing device 450. Such sounds may include sounds from voice calls, may include recorded sounds such as voice messages, music files, or the like, and also include sounds generated by applications running on mobile computing device 450. can do.

The mobile computing device 450 can be implemented in a number of different forms, as shown in the figure. For example, it can be implemented as a cellular phone 380. It may also be implemented as part of a smart phone 382, PDA or other similar mobile device.

Various implementations of the systems and techniques described herein can be realized with digital electronic circuits, integrated circuits, specially designed ASICs (on-demand integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations include at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions and transmit the data and instructions to a storage system, at least one input device, and at least one output device. It may include an implementation in one or more computer programs executable and / or interpretable on a programmable system.

These computer programs, also known as programs, software, software applications or code, contain machine instructions for a programmable processor, and be implemented in a high level procedural and / or object oriented programming language and / or assembly / machine language. Can be. The terms "machine readable medium" and "computer readable medium" as used herein refer to a machine readable medium that receives machine instructions as a magnetic disk, optical disk, memory, programmable logic device (PLD) machine readable signal. Including providing machine instructions and / or data to a programmable processor. The term "machine readable signal" means any signal used to provide machine instructions and / or data to a programmable processor.

To provide interaction with a user, the systems and techniques described herein can be implemented on a display device, eg, a computer having a display device. It may include a cathode ray tube (CRT) or liquid crystal display (CRT) monitor for displaying information to the user, and a keyboard and pointing device, for example, a mouse or trackball, through which the user can provide input to the computer. . Other types of devices can be used to provide user interaction. For example, the feedback provided to the user may be any form of tactile feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user may be in any form, including acoustic, voice, or tactile input. Can be received.

The systems and techniques described herein can be implemented in a computing system that includes a backend component, such as a data server, or middleware component, for example. An application server or front-end component, for example a client computer with a graphical user interface or web browser through which the user can interact with the implementation of the systems and techniques described herein, or any of these back-end, middleware or front-end components Includes combinations. The components of the system can be interconnected by any form or medium of digital data communication, for example a communication network. Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), and the Internet.

The computing system can include a client and a server. The client and server are usually remote from each other and generally interact through a communication network. The relationship between the client and the server is caused by computer programs running on each computer and having a client-server relationship to each other.

A number of embodiments have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. Also, the logic flow shown in the figures does not require the specific order or sequential order shown to obtain the desired results. In addition, other steps can be provided or removed from the described flows, and other components can be added or removed from the described systems. Accordingly, other embodiments are within the scope of the following claims.

Claims (20)

A computer-implemented method for improving automatic speech recognition using sensitive data,
Receiving, by the automatic speech processor, specific audio data corresponding to the speech uttered by the user;
Determining, by the automatic speech processor, that at least a portion of the specific audio data is likely to include sensitive data to be deleted without being logged to a log of audio data, the sensitive data being selected from the type of sensitive data and Associated, and the determination is based on determining that at least a portion of the specific audio data has a possibility to include sensitive data above a confidence threshold associated with the type of sensitive data;
Providing, by the automatic speech processor, the specific audio data to an acoustic model training engine that trains acoustic models for the automatic speech processor before at least a portion of the specific audio data likely to contain important data is deleted. step;
Deleting, by the automatic speech processor, after the specific audio data is provided to the acoustic model training engine, at least a portion of the specific audio;
Receiving, by the automatic speech processor, from the acoustic model training engine, an updated acoustic model trained using at least a portion of the specific audio data; And
And using, by the automatic speech processor, the updated acoustic model to recognize subsequently received audio data. The method according to claim 1,
The method of claim 1, wherein the important data is data to be deleted after a period shorter than a basic period after the data determined to be not important data is deleted. The method according to claim 1,
Each type of sensitive data has a predetermined period associated with it for deletion. The method according to claim 1,
And anonymizing at least a portion of the received audio data. The method according to claim 1,
The method of claim 1, wherein the important data includes contact information. The method according to claim 1,
The method of claim 1, wherein the important data includes personal identification information. The method according to claim 1,
The method of claim 1, wherein the important data includes important financial information. The method according to claim 1,
Wherein the important data includes one or more predetermined keywords. A system for improving automatic speech recognition using important data,
And one or more computers and one or more storage devices storing executable instructions, the instructions, when executed by the one or more computers, cause the one or more computers to perform operations, the operations being:
Receiving, by the automatic speech processor, specific audio data corresponding to the speech uttered by the user;
Determining, by the automatic speech processor, that at least a portion of the specific audio data is likely to contain sensitive data to be deleted without being logged to the log of audio data, the sensitive data being selected from the type of sensitive data and Associated, and the determination is based on determining that at least a portion of the specific audio data has a possibility to include sensitive data above a confidence threshold associated with the type of sensitive data;
Providing, by the automatic speech processor, the specific audio data to an acoustic model training engine that trains acoustic models for the automatic speech processor before at least a portion of the specific audio data likely to contain important data is deleted. action;
Deleting, by the automatic speech processor, at least a portion of the specific audio after the specific audio data is provided to the acoustic model training engine;
Receiving, by the automatic speech processor, from the acoustic model training engine, an updated acoustic model trained using at least a portion of the specific audio data; And
And using, by the automatic speech processor, the updated acoustic model to recognize subsequently received audio data. The method according to claim 9,
The system is characterized in that the important data is data to be deleted after a period shorter than the basic period after the data determined to be not important data is deleted. The method according to claim 9,
Each type of sensitive data has a predetermined period associated with it for deletion. The method according to claim 9,
And anonymizing at least a portion of the received audio data. The method according to claim 9,
The important data comprises a contact information system. The method according to claim 9,
The important data system characterized in that it comprises personal identification information. The method according to claim 9,
The system, characterized in that the important data includes important financial information. The method according to claim 9,
Wherein the important data comprises one or more predetermined keywords. A computer-readable non-transitory medium storing software comprising instructions executable by one or more computers, the instructions causing the one or more computers to perform operations when executed, the operations:
Receiving, by the automatic speech processor, specific audio data corresponding to the speech uttered by the user;
Determining, by the automatic speech processor, that at least a portion of the specific audio data is likely to contain sensitive data to be deleted without being logged to the log of audio data, the sensitive data being selected from the type of sensitive data and Associated, and the determination is based on determining that at least a portion of the specific audio data has a possibility to include sensitive data above a confidence threshold associated with the type of sensitive data;
Providing, by the automatic speech processor, the specific audio data to an acoustic model training engine that trains acoustic models for the automatic speech processor before at least a portion of the specific audio data likely to contain important data is deleted. action;
Deleting, by the automatic speech processor, at least a portion of the specific audio after the specific audio data is provided to the acoustic model training engine;
Receiving, by the automatic speech processor, from the acoustic model training engine, an updated acoustic model trained using at least a portion of the specific audio data; And
And using the updated acoustic model to recognize subsequently received audio data by the automatic speech processor. The method according to claim 17,
And wherein the important data is data to be deleted after a period shorter than the basic period after the data determined not to be important data is deleted. The method according to claim 17,
Each type of sensitive data has a predetermined period associated with it for deletion. The method according to claim 17,
And anonymizing at least a portion of the received audio data.

Images