WO2016194740A1

WO2016194740A1 - Speech recognition device, speech recognition system, terminal used in said speech recognition system, and method for generating speaker identification model

Info

Publication number: WO2016194740A1
Application number: PCT/JP2016/065500
Authority: WO
Inventors: 泰貴畠山
Original assignee: シャープ株式会社
Priority date: 2015-06-04
Filing date: 2016-05-25
Publication date: 2016-12-08
Also published as: JP2017003611A; JP6084654B2

Abstract

Provided is a system capable of identifying speakers without a user being conscious of learning. A speech recognition system is provided with a terminal (300) and a server (350). The terminal (300) is provided with a speech input unit (31) for receiving speech input and a speech output unit (32) for outputting speech. The server (350) is provided with: a control unit (30) for controlling the operations thereof; a speaker identification unit (33) for identifying speakers on the basis of signals and data stored in the terminal (300); a speaker identification learning unit (34) for creating data (user profile) for each user using information (user ID and the like) for speakers identified by the speaker identification unit (33); a user management unit (35) for storing user information in the terminal (300); a speech recognition unit (36) for performing speech recognition processing; and a conversation analysis/generation unit (37) for analyzing messages for the terminal (300) on the basis of the results of speech recognition and generating responses according to those messages in accordance with the results of the analysis.

Description

Speech recognition apparatus, speech recognition system, terminal used in the speech recognition system, and method for generating a speaker identification model

This disclosure relates to speech recognition, and more specifically to technology for identifying a speaker.

A technology for identifying a speaker in speech recognition is known. For example, Japanese Patent Laying-Open No. 2010-217319 (Patent Document 1) discloses a technique for “increasing accuracy for speaker identification in a speaker identification device that identifies a speaker from an audio signal”. (See [Summary]). Japanese Patent Laid-Open No. 7-261781 (Patent Document 2) discloses a “learning method for creating a phoneme model for speaker recognition with high speaker recognition accuracy” (see [Summary]).

JP 2010-217319 A Japanese Patent Laid-Open No. 7-261781

In conventional speaker identification based on speech, a model for identifying a speaker is given in advance, and the goal is to build an efficient model with shorter user utterances. For this reason, the user is requested in advance to speak for about 1 to 2 minutes although it is short, and a speaker identification model is established from the obtained speech data.

The conventional technology requires the user to speak as preprocessing. However, in speaker identification in voice communication, in order to perform a more natural dialogue, it is necessary to acquire learning data without making the user feel that the user's speech is used for learning. Therefore, it is necessary to acquire voice data necessary for building a speaker identification model without imposing a load on the user in a state where the speaker identification model is not built.

The present disclosure has been made to solve the above-described problems, and an object in one aspect is to provide a speech recognition apparatus that can acquire speech data necessary for building a speaker identification model. It is to be.

An object in another aspect is to provide a speech recognition system that can acquire speech data necessary to build a speaker identification model.

The purpose in another aspect is to provide a terminal used in the speech recognition system.

Still another object of the present invention is to provide a method for generating a speaker identification model necessary for constructing a speaker identification model.

A speech recognition apparatus according to an embodiment includes a speech input unit for receiving an utterance including information for identifying a speaker and an utterance not including information for identifying a speaker, and a speech for performing speech recognition processing A recognition unit, a voice output unit for outputting a voice, and a control unit for controlling the voice recognition device based on a result of the voice recognition process. The control unit generates a speaker identification model for identifying the speaker by associating the information for identifying the speaker with the utterance not including the information for identifying the speaker.

In a certain aspect, the user can collect voice data necessary for learning only by performing a normal voice dialogue without being aware of preprocessing for learning.

The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the present invention which is to be understood in connection with the accompanying drawings.

It is a figure showing the exchange between the user 1 and the terminal 2 when a shiritori game is performed. It is a figure showing the outline | summary of a structure of the speech recognition system according to 1st Example which concerns on this indication. It is a figure showing the outline | summary of a structure of the speech recognition system according to 2nd Example which concerns on this indication. It is a figure showing the outline | summary of a structure of the speech recognition system according to 3rd Example which concerns on this indication. It is a figure showing the outline | summary of a structure of the speech recognition system according to 4th Example which concerns on this indication. It is a block diagram showing the structure of the function which implement | achieves the speech recognition system which concerns on this indication. It is a figure which represents notionally 1 aspect of the storage of the data hold | maintained in a speech recognition system. It is a figure showing the state by which the speaker model 80 is produced | generated by the dialogue between the user 1 and the terminal 2. It is a flowchart (the 1) showing a sequence in case a user becomes the starting point of speech. It is a flowchart (the 2) showing a sequence in case a user becomes the starting point of speech. It is a flowchart (the 3) showing a sequence in case a user becomes the starting point of speech. It is a figure showing the sequence of the exchange between the user 1 and the terminal 2 when a user is known to the voice recognition system. It is a sequence chart (the 1) showing the flow of the processing performed when a user is known. It is a sequence chart (the 2) showing the flow of processing performed when a user is known. It is a figure showing the case where talking to the user 1 from the terminal 2 serves as a trigger of a dialog. It is a sequence chart (the 1) showing a part of process performed with a speech recognition system. It is a sequence chart (the 2) showing a part of process performed with a speech recognition system.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<Technology>
According to the present disclosure, voice data (for example, voiceprint information) necessary for building a model for speaker identification is collected by asking a user name by voice conversation contents and classifying the user when the user is indefinite. For example, in a dialogue such as “Shiritori” or a quick phrase such as a game, a plurality of user utterances with respect to a game partner (for example, a terminal, a home appliance, etc.) is expected. In such a case, the device as the game opponent can make a series of user utterances as learning data by asking the user name in advance and playing the game. Alternatively, an unknown user name one hour before can be determined by asking the user name after a certain unknown user speaks.

In the present embodiment, for example, morphological analysis is used as an example of speech recognition. According to this analysis technique, proper nouns and those that are not are separated. For example, a speech recognition system may have a dictionary of names as a database. Speech recognition is performed by matching a dictionary with an extracted proper noun in morphological analysis.

<Outline of configuration>
(Configuration 1) A speech recognition apparatus according to an aspect includes a microphone for receiving an utterance including information for identifying a speaker and an utterance not including information for identifying a speaker, and a processor for performing speech recognition processing And a speaker for outputting voice and a processor for controlling the voice recognition device based on the result of the voice recognition processing. The processor generates a speaker identification model for identifying a speaker by associating information identifying the speaker with an utterance that does not include information identifying the speaker. The speaker identification model may include, for example, a speaker identification ID (Identification), the name of the speaker (user of the speech recognition apparatus), voiceprint information extracted from the speaker's utterance, and the like.

In this embodiment, the information for identifying a speaker is, for example, a name, a nickname, a resident number, an identification number given by a government agency, or other information that can be included in an utterance.

(Configuration 2) Preferably, the speaker outputs an inquiry asking for information for identifying a speaker. Associating information that identifies a speaker with an utterance that does not contain information that identifies the speaker means that information that identifies the speaker and utterance that does not contain information that identifies the speaker uttered after the inquiry. Including associating.

(Configuration 3) Preferably, the speaker outputs an inquiry asking for information for identifying the speaker after the utterance not including the information for identifying the speaker. Associating information that identifies a speaker with an utterance that does not include information that identifies the speaker is the process of associating the utterance before the inquiry with the information identifying the speaker included in the utterance that responds to the inquiry. Including associating.

(Configuration 4) The processor is configured to determine the content of the next utterance output from the speaker based on the content of the response to the utterance output from the speaker. For example, the speech recognition apparatus holds a plurality of inquiries in advance. The difficulty level of each inquiry is hierarchically different. In a certain situation, if a response is not returned within a predetermined time for an utterance of a query with a medium difficulty level, or if the response is not correct, the processor issues a query with a low difficulty level ( Talk about the problem of shiritori). In another aspect, when a response is returned early within a predetermined time, the processor utters a query with a high degree of difficulty (a problem of shiritori) as the next query.

(Configuration 5) The speech recognition apparatus further includes a memory for storing the generated speaker identification model. The processor is configured to update the generated speaker identification model based on the response to the query.

(Configuration 6) According to another aspect, a speech recognition system is provided. The voice recognition system includes a terminal and a device capable of communicating with the terminal. The terminal is electrically connected to a microphone for receiving an utterance including information for identifying a speaker and an utterance not including information for identifying a speaker, a speaker for outputting sound, and the microphone and the speaker. And a communication interface for communicating with the device. The apparatus includes a communication interface for communicating with a terminal, a processor for performing voice recognition processing, and a processor for controlling the apparatus based on the result of the voice recognition processing. The processor generates a speaker identification model for identifying a speaker by associating information identifying the speaker with an utterance that does not include information identifying the speaker.

<Background of technical thought>
The background of the technical idea according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an exchange between the user 1 and the terminal 2 when a shiritori game is performed. User 1 issues message 10 to terminal 2. When the terminal 2 recognizes the message 10, the terminal 2 issues a message 11 as a response.

User 1 issues a message 12 to terminal 2. When the terminal 2 recognizes the message 12, the terminal 2 issues a message 13 synthesized using the name included in the message 12 and a predefined message.

When a predetermined time has elapsed, the terminal 2 issues a message 14. When the user 1 recognizes the message 14, the user 1 considers a word following the message 14 as a response within a predetermined time. User 1 issues message 15 to terminal 2. When the terminal 2 recognizes the message 15, the terminal 2 refers to a language dictionary prepared in advance and considers words following the message 15. The terminal 2 issues a message 16 as a word for the message 15 within a predetermined time. In this way, the user 1 and the terminal 2 continue the shiritori game.

In the case where the user 1 can return the next word within a predetermined time in response to the utterance of the terminal 2, the interruption continues. For example, the user 1 issues a message 17 to the terminal 2. When the terminal 2 recognizes the message 17, it issues a message 18.

On the other hand, the user 1 may not be able to return the next word. In this case, the user 1 keeps silence or issues a message 19 indicating that he / she does not understand. When the terminal 2 determines that there is no response from the user 1 within a predetermined fixed waiting time, or when the terminal 2 recognizes the message 19, the terminal 2 issues a message 20 defined in advance for the content.

In such a case, the terminal 2 recognizes that the user 1 is “Taro” through the exchange of messages with the user 1, and associates “Taro” as user information with each data.

The configuration of the speech recognition system according to the present disclosure will be described with reference to FIGS.
[Terminal]
FIG. 2 is a diagram illustrating an outline of the configuration of the speech recognition system according to the first embodiment of the present disclosure. In the voice recognition system, one terminal 200 functions as a voice recognition system.

The terminal 200 includes a control unit 30, a voice input unit 31, a voice output unit 32, a speaker identification unit 33, a speaker identification learning unit 34, a user management unit 35, a voice recognition unit 36, and a dialog analysis. A generation unit 37 is provided. The terminal 200 may be a terminal having a voice input / output function and a voice recognition function, for example. The terminal may include, for example, a smartphone, a television, a cleaning robot that can operate standalone, and other devices.

The control unit 30 controls the operation of the terminal 200. The voice input unit 31 receives a voice input and outputs a signal to the control unit 30. The audio output unit 32 converts the signal output from the control unit 30 into audio and outputs the audio to the outside of the terminal 200. The audio output unit 32 includes, for example, a speaker, a terminal, and the like. The speaker identification unit 33 identifies a speaker who has made an utterance to the terminal 200 based on a signal sent from the control unit 30. In another aspect, the speaker identification unit 33 identifies a speaker based on the signal and data stored in the terminal 200. The data can include, for example, voiceprint information registered in advance as a user of the terminal 200.

The speaker identification learning unit 34 creates data (user profile) for each speaker using the information (user ID or the like) of the speaker identified by the speaker identifying unit 33. The user management unit 35 stores user information of the terminal 200. The user information may include a user profile and the like. The voice recognition unit 36 performs voice recognition processing using the voice signal sent from the control unit 30. For example, the voice recognition unit 36 extracts characters included in the utterance.

The dialog analysis / generation unit 37 analyzes a message to the terminal 200 based on the recognition result by the voice recognition unit 36. Furthermore, the dialog analysis / generation unit 37 generates a response according to the utterance according to the analysis result. In another aspect, the dialog analysis / generation unit 37 generates an utterance for encouraging the user of the terminal 200 based on the setting in the terminal 200. The setting may include, for example, that the terminal 200 has detected the presence of a user in the vicinity of itself, that a preset time has arrived.

[Terminal + server]
FIG. 3 is a diagram illustrating an outline of the configuration of the speech recognition system according to the second embodiment of the present disclosure. The voice recognition system includes a terminal 300 and a server 350. The terminal 300 includes an audio input unit 31 and an audio output unit 32. Terminal 300 is controlled by a processor (not shown). The server 350 includes a control unit 30, a speaker identification unit 33, a speaker identification learning unit 34, a user management unit 35, a voice recognition unit 36, and a dialog analysis / generation unit 37. The terminal 300 is realized as a terminal having a voice input / output function and a communication function, for example. Such a terminal may include, for example, a mobile phone or other information communication terminal, a cleaning robot or the like having a voice recognition function and a communication function, and the like.

When the terminal 300 accepts the user's utterance, the terminal 300 transmits an audio signal corresponding to the utterance to the server 350 via a communication interface (not shown). Upon receiving the voice signal, the server 350 executes processing such as speaker identification processing, voice recognition processing, dialog analysis, response generation, and the like. Each process is the same as the process realized by the configuration shown in FIG. 2, and thus detailed description will not be repeated.

The server 350 transmits the generated response to the terminal 300 via a communication interface (not shown). When the terminal 300 receives the response, the audio output unit 32 outputs audio corresponding to the response.

[Terminal + server + speaker identification server]
FIG. 4 is a diagram illustrating an outline of the configuration of the speech recognition system according to the third embodiment of the present disclosure. The voice recognition system includes a terminal 300, a server 400, and a speaker identification server 410. The server 400 includes a control unit 30, a user management unit 35, a voice recognition unit 36, and a dialog analysis / generation unit 37. The speaker identification server 410 includes a speaker identification unit 33 and a speaker identification learning unit 34.

The server 400 and the speaker identification server 410 are realized by a computer device having a known configuration. The computer includes, as main components, a CPU (Central Processing Unit) that executes a program, a keyboard and other input devices, a RAM (Random Access Memory), a hard disk, an optical disk drive, a monitor, a communication IF ( Interface).

Processing in the computer is realized by software executed by each hardware and CPU. In one aspect, the software is stored in advance on a hard disk. In another aspect, the software is stored in a CD-ROM or other computer-readable non-volatile data recording medium and distributed as a program product. In yet another aspect, the software may be provided as a program product that can be downloaded by an information provider connected to the Internet or other networks.

The hardware configuration of the computer is general. Therefore, the description of the hardware configuration of server 400 and speaker identification server 410 will not be repeated. It can be said that the essential part for realizing the technical idea according to the present embodiment is a program stored in the computer.

When the server 400 receives the voice signal transmitted from the terminal 300, the server 400 transmits the voice signal to the speaker identification server 410 via the communication interface.

The speaker identification server 410 recognizes the speaker and generates data for registering the speaker. The speaker identification server 410 transmits the generated data to the server 400.

[Terminal + server + speaker identification server + voice recognition server]
FIG. 5 is a diagram illustrating an outline of a configuration of a speech recognition system according to the fourth embodiment of the present disclosure. The voice recognition system includes a terminal 300, a server 500, a speaker identification server 410, and a voice recognition server 520. The server 500 includes a control unit 30, a user management unit 35, and a dialog analysis / generation unit 37. The voice recognition server 520 includes a voice recognition unit 36.

When the server 500 receives the voice signal from the terminal 300, the server 500 transmits the voice signal to the speaker identification server 410 and the voice recognition server 520. The voice recognition server 520 executes voice recognition processing using the voice signal, and transmits the recognition result to the server 500.

Other operations are the same as those in the configuration of the speech recognition system according to the other embodiments described above. Therefore, description of other operations will not be repeated.

[Function configuration]
FIG. 6 is a block diagram illustrating a configuration of functions that implement the speech recognition system according to the present disclosure. The voice recognition system includes a terminal module 600, a main module 610, a speaker identification module 620, and a voice recognition module 630.

The terminal module 600 includes an audio input unit 31 and an audio output unit 32. The terminal module 600 is in the vicinity of the user, accepts an utterance, and transmits voice data and a terminal ID to the main module 610. In another aspect, the terminal module 600 receives the synthesized voice data sent from the main module 610 and outputs voice based on the synthesized voice data from the voice output unit 32.

In the main module 610, the control unit 30 transmits the voice data and the speaker model list to the speaker identification module 620. When the speaker identification module 620 identifies a speaker, the speaker identification module 620 transmits a speaker identification result (for example, a message ID, a flag indicating that the speaker has been identified, etc.) to the main module 610.

The control unit 30 transmits the terminal ID or voice data to the user management unit 35. The user management unit 35 stores the terminal ID or voice data.

The control unit 30 reads the speaker model list from the user management unit 35.
For example, the control unit 30 exchanges text data with the dialog analysis / generation unit 37.

The control unit 30 transmits the voice data to the voice recognition module 630. When the speech recognition module 630 executes speech recognition processing using speech data, the speech recognition module 630 sends the result to the control unit 30 as text.

The function shown in FIG. 6 is realized by any of the configurations shown in FIGS.
[data structure]
With reference to FIG. 7, the data structure of the speech recognition system according to the present embodiment will be described. FIG. 7 is a diagram conceptually showing one mode of storing data held in the voice recognition system. In one aspect, the voice recognition system includes a terminal management table, a home management table, and a user management table.

(Terminal management table)
The terminal management table includes a terminal ID and a belonging user ID. The terminal ID identifies a terminal registered in the voice recognition system. In one aspect, the terminal ID is uniquely assigned by a manager of the voice recognition system (for example, a manager of a computer including the control unit 30). In another aspect, the terminal ID is configured by an arbitrary character string (for example, alphanumeric characters or symbols) desired by the user of the terminal. In this case, for example, the control unit 30 checks whether or not the ID input by the user is already used so that the terminal ID is not duplicated. If the used terminal ID is input, Notify the terminal to that effect. The affiliated user ID identifies a user registered as a user of the terminal. The number of users of the terminal is not particularly limited.

(Home management table)
The home management table includes a home ID and a terminal ID of a terminal belonging to the home. The home ID identifies the home as a group of users who use the service of the voice recognition system. The unit of the user group is not limited to the home. Any user may be used as long as a plurality of users can be associated with one group. The home ID is associated with each terminal ID of one or more terminals. The number of terminals associated with the home is not particularly limited.

(User management table)
The user management table includes a user ID, a user name, speaker model data, and a voice data list.

User ID identifies the user who uses the terminal. The user name identifies the user to whom the user ID is assigned. The speaker model data is data for identifying the user. The speaker model data may include voiceprint information, for example.

The audio data list includes audio data for identifying the user. The voice data may include an utterance from the user to the terminal, a user response to the terminal utterance, an utterance by the user of a character string displayed on the terminal, and the like.

[Generate speaker model]
The generation of a speaker model will be described with reference to FIG. FIG. 8 is a diagram illustrating a state in which the speaker model 80 is generated by the dialogue between the user 1 and the terminal 2. The description of the same state as that in FIG. 1 will not be repeated.

In the dialog between the user 1 and the terminal 2, when the user 1 is not registered, the terminal 2 first listens to the user name and then utters the user until a certain period thereafter (for example, the end of the game). To register voice data in the database. The audio data can include voiceprint information.

For each user utterance, the speaker identification learning unit learns the speaker identification using all the speech data from the target speech DB (Database) as learning data.

ID is assigned to each terminal. By managing the user by the terminal and the user name, it is possible to cope with the case where there is a user with the same name in another terminal.

When the user 1 utters his name (message 12), the terminal 2 recognizes the message 12. When the terminal 2 extracts the user name (= taro) from the message 12, the terminal 2 transmits the user name and the terminal ID of the terminal 2 to the user management unit 35. Thereafter, when the user 1 speaks, the

messages

15 and 17 are stored in the user management unit 35 through the terminal 2.

The speaker identification learning unit 34 reads the terminal ID and user name stored in the user management unit 35 and generates a speaker model 80. The speaker model 80 includes the user name and the terminal ID. Therefore, thereafter, when the user name is specified by the terminal 2 interacting with the user 1, the speaker model 80 associated with the user can be used.

[Control structure]
A control structure of the speech recognition system according to the present embodiment will be described with reference to FIGS. FIG. 9 to FIG. 11 are flowcharts showing sequences when the user is the starting point of the utterance.

In step 910, an utterance for starting a sequence for speaker identification learning by the user is performed. For example, the user issues a message 911 “Let's take a bite”. When receiving the message 911, the voice input unit 31 transmits a voice signal corresponding to the message 911 to the control unit 30.

In step 915, when the control unit 30 detects that the voice signal has been received, the control unit 30 transmits a voice recognition request to the voice recognition unit 36.

In step 920, when detecting that the voice signal has been received, the control unit 30 transmits a speaker model list acquisition request to the user management unit 35. The speaker model list acquisition request requests access to the speaker model list associated with the user who gave the utterance.

In step 925, the control unit 30 transmits a speaker model list response to the control unit 30 in response to the speaker model list acquisition request. The speaker model list response includes the acquisition result of the speaker model list associated with the user.

In step 930, the control unit 30 transmits a speaker identification request to the speaker identification unit 33. When the speaker identification unit 33 detects the reception of the speaker identification request, the speaker identification unit 33 refers to the data stored in the user management unit 35 and tries to identify the user (speaker) who made the utterance in step 910.

In step 935, the voice recognition unit 36 transmits a voice recognition response to the control unit 30 in response to the voice recognition request in step 915. The voice recognition response includes whether or not the voice recognition is successful.

In step 940, the speaker identification unit 33 transmits a speaker identification failure response to the speaker identification unit 33. That is, since the user is not registered in the voice recognition system, the speaker identifying unit 33 cannot identify the user (speaker) who gave the speech. Therefore, a speaker identification failure response notifying that speaker identification has failed is sent from the speaker identification unit 33 to the control unit 30.

In step 945, the control unit 30 transmits a dialog analysis / generation request to the dialog analysis / generation unit 37 in response to the reception of the speaker identification failure response. The dialogue analysis / generation request may include a voice identification result and a speaker identification result. Upon receiving the dialog analysis / generation request, the dialog analysis / generation unit 37 generates a message for acquiring the name of the user who gave the utterance. For example, the dialogue analysis / generation unit 37 uses a template prepared in advance in the speech recognition system and the term “shiritori” included in the message 911 to start the message 946 (let's start shiritori. Create.)

In step 950, the dialog analysis / generation unit 37 transmits the generated message 946 to the control unit 30. When detecting the reception of the message, the control unit 30 generates a voice response including the terminal ID of the terminal that has given the utterance and the message.

In step 955, the control unit 30 transmits the audio response to the audio output unit 32. When the audio output unit 32 receives the audio response signal, the audio output unit 32 outputs audio based on the signal. When the user recognizes the voice, the user speaks the voice. The utterance is received by the voice input unit 31.

In step 960, the voice input unit 31 transmits the content of the received message 961 (name registration utterance) to the control unit 30. The message 961 includes an answer (name) to the message 946, for example, “Taro would!”. When the control unit 30 detects reception of the message 961, the control unit 30 generates a voice recognition request.

In step 965, the control unit 30 transmits a voice recognition request to the voice recognition unit 36. When the voice recognition unit 36 detects the reception of the voice recognition request, the voice recognition unit 36 executes voice recognition processing of the message 961.

In step 970, the control unit 30 transmits a speaker model list acquisition request to the user management unit 35. When the user management unit 35 detects the reception of the speaker model request, the user management unit 35 tries to acquire the speaker model list. The user management unit 35 generates a result of the acquisition attempt as a speaker model list response.

In step 975, the user management unit 35 transmits a speaker model list response to the control unit 30.

In step 980, in response to receiving the speaker model list response, the control unit 30 transmits a speaker identification request to the speaker identification unit 33. When the speaker identification unit 33 detects reception of a speaker identification request, the speaker identification unit 33 starts speaker identification and generates an identification result.

Referring to FIG. 10, in step 1010, the voice recognition unit 36 transmits a voice recognition response to the control unit 30 as a response to the speaker identification request. The voice recognition response may include information indicating that the content of the message 961 has been recognized.

In step 1015, the speaker identification unit 33 transmits a speaker identification failure response to the control unit 30. That is, the speaker is not registered in the speech recognition system. Therefore, the speaker identification unit 33 generates a response indicating that the attempt to identify the speaker has failed.

In step 1020, the control unit 30 transmits a dialog analysis / generation request to the dialog analysis / generation unit 37. In response to receiving the dialog analysis / generation request, the dialog analysis / generation unit 37 generates a message 1031 for dialog. The message 1031 is generated as a message including the content of the utterance and information for identifying the speaker, such as “Taro-san. Let's start.

In step 1030, the dialog analysis / generation unit 37 transmits the message 1031 to the control unit 30. When detecting the reception of the message 1031, the control unit 30 generates an audio response including the message 1031 and the terminal ID in order to respond to the utterance to the terminal.

In step 1035, the control unit 30 transmits the voice response to the terminal. When receiving the audio response signal, the audio output unit 32 of the terminal outputs audio based on the signal. When the user recognizes the voice, the user speaks to the terminal in consideration of the next response. The voice input unit 31 receives the utterance, for example, “gorilla”.

Thereafter, several exchanges for shiritori are performed (step 1040 and subsequent steps).
In step 1040, the voice input unit 31 transmits the received message 1041 to the control unit 30. When the control unit 30 detects the reception of the message 1041, the control unit 30 generates a voice recognition request.

In step 1045, the control unit 30 transmits a voice recognition request to the voice recognition unit 36. When the voice recognition unit 36 receives the request, the voice recognition unit 36 starts a voice recognition process.

In step 1050, the control unit 30 transmits a speaker voice storage / list acquisition request to the user management unit 35. When detecting the reception of the request, the user management unit 35 saves the identification ID of the speaker (taro) and the name of the speaker (taro) by associating each other. Further, the user management unit 35 generates a response indicating that the speaker voice has been successfully stored.

In step 1055, the user management unit 35 transmits a speaker voice storage / list acquisition response to the control unit 30 as the response.

In step 1060, the control unit 30 transmits a speaker identification model learning request to the speaker identification learning unit 34. When the speaker identification learning unit 34 detects the reception of the request, the speaker identification learning unit 34 generates a model by associating a voice with the user who gave the utterance as a speaker identification model, and updates it appropriately.

In step 1065, the voice recognition unit 36 transmits the processing result based on the voice recognition request to the control unit 30 as a voice recognition response.

In step 1070, the speaker identification learning unit 34 transmits a speaker identification learning response to the control unit 30 in response to the speaker identification model learning request.

In step 1075, the control unit 30 generates a dialog analysis / generation request, and transmits the generated request to the dialog analysis / generation unit 37. For example, if the control unit 30 determines that there is not enough data for the speaker's learning and the learning has failed, the control unit 30 generates the request. When the dialog analysis / generation unit 37 detects the reception of the request, the dialog analysis / generation unit 37 generates a message 1081 (for example, “gorilla... Then,“ camel ”) for further learning.

In step 1080, the dialog analysis / generation unit 37 transmits the generated message 1081 to the control unit 30. When the control unit 30 receives the message 1081, the control unit 30 generates a voice response including the terminal ID and the message 1081.

In step 1085, the control unit 30 transmits the generated voice response to the terminal. When the terminal receives the voice response, the voice output unit 32 outputs voice based on the voice response. When the user recognizes the voice emitted from the voice output unit 32 of the terminal, the user considers the next response. When the user issues the next response within a predetermined time, the voice input unit 31 accepts the user's utterance and generates a voice response corresponding to the utterance.

Referring to FIG. 11, in step 1110, voice input unit 31 transmits message 1111 (for example, “diamond”) to control unit 30. When detecting the reception of the message 1111, the control unit 30 generates a speech recognition request and a speaker speech storage / list acquisition request.

In step 1115, the control unit 30 transmits a voice recognition request to the voice recognition unit 36. When the voice recognition unit 36 detects reception of the request, the voice recognition unit 36 starts voice recognition processing of the message 1111.

In step 1120, the control unit 30 transmits a message 1111 and a speaker voice storage / list acquisition request to the user management unit 35. When detecting the reception of the request, the user management unit 35 stores the content (voice data) of the message 1111 in association with the identification ID of the user (speaker).

In step 1130, the control unit 30 transmits a speaker identification model learning request to the speaker identification learning unit 34. When the speaker identification learning unit 34 detects reception of the request, the speaker identification learning unit 34 learns a speaker identification model. More specifically, the speaker identification learning unit 34 stores the user identification ID and voice information (for example, voiceprint information) included in the message 1111 in association with each other. When the learning is completed, the speaker identification learning unit 34 generates a response indicating that the learning of the speaker identification model is completed.

In step 1135, in response to the completion of the voice recognition process, the voice recognition unit 36 generates a voice recognition response notifying the result of the voice recognition process, and transmits the response to the control unit 30.

In step 1140, the speaker identification learning unit 34 transmits the generated response to the control unit 30. When the control unit 30 receives the response from the voice recognition unit 36 and the response from the speaker identification learning unit 34, the control unit 30 determines whether data sufficient for learning is prepared and learning is completed. For example, when more than a predetermined number of audio data is associated with the user identification ID, the control unit 30 determines that sufficient data for learning is available and learning is completed.

The control unit 30 generates a dialog analysis / generation request based on the content of the response received from the speech recognition unit 36 and the response received from the speaker identification learning unit 34. For example, based on the result of each response, the control unit 30 generates the request when it is determined that the speech recognition is successful, and that sufficient data for learning is available and learning is completed. Data sufficient for learning is, for example, information that is defined as the amount of information extracted from speech data within a predetermined time (such as the number of voiceprint information having a fixed data size) required for learning. The one that exceeds the amount.

In step 1145, the control unit 30 transmits the generated request to the dialog analysis / generation unit 37. When the dialog analysis / generation unit 37 detects reception of the request, the dialog analysis / generation unit 37 generates a message 1151 for the message 1111.

In step 1150, the dialog analysis / generation unit 37 transmits the generated message 1151 to the control unit 30. When detecting the reception of the message 1151, the control unit 30 generates a voice response including the terminal ID and the message 1151.

In step 1155, the control unit 30 generates a voice response to the terminal. When the terminal receives the voice response, the terminal outputs the voice from the voice output unit 32.

<Speaking sequence from user>
Another aspect will be described with reference to FIG. FIG. 12 is a diagram illustrating an exchange sequence between the user 1 and the terminal 2 when the user is known to the voice recognition system. The same operations as those described above are denoted by the same numbers. Therefore, description of the same operation will not be repeated.

If the user has already been registered, the speaker model is updated accordingly. Therefore, speaker identification based on the latest user's voice data is always possible.

User 1 issues message 10 to terminal 2. When the terminal 2 accepts the message 10, the terminal 2 executes voice recognition processing and speaker identification processing. If the terminal 2 determines that the speaker of the message 10 has been identified based on the result of the speaker identification process, the terminal 2 issues a message 1210 according to the determination result. Message 1210 includes a response to message 10 and an inquiry to identify the speaker of message 10. When the user 1 issues a message 1220 for the message 1210, the terminal 2 performs voice recognition processing and speaker identification processing on the message 1220.

If the terminal 2 determines from the content of the message 1220 that an answer to the inquiry has been obtained, the terminal 2 transmits data including the terminal ID of the terminal 2 and the user name (taro) to the user management unit 35. The user management unit 35 accumulates the data. Further, the terminal 2 issues a message 1230 for the message 1220.

Thereafter, whenever the terminal 2 recognizes an utterance from the user 1, the terminal 2 transmits data including the terminal ID and the user name to the user management unit 35. The user management unit 35 stores each data.

The speaker identification learning unit 34 refers to the terminal ID and the user name from the user management unit 35, reads data associated with the user from the accumulated data, and creates a speaker model 80.

With reference to FIG. 13 and FIG. 14, the sequence in the speech recognition system according to a certain aspect will be described. 13 and 14 are sequence charts showing the flow of processing performed when the user is known. The same steps as those described above are denoted by the same step numbers. Therefore, the description of the same process will not be repeated.

In step 1340, the speaker identification unit 33 transmits a speaker identification response to the control unit 30 in order to notify that the speaker identification has been successful. When the control unit 30 detects the reception of the response and the response from the voice recognition unit 36, the control unit 30 generates a dialog analysis / generation request. The request includes a voice identification result and a speaker identification result.

In step 1345, the control unit 30 transmits a dialog analysis / generation request to the dialog analysis / generation unit 37. When the dialog analysis / generation unit 37 detects reception of the request, the dialog analysis / generation unit 37 generates a message 1351 for responding to the message 911. At this time, the message 1351 includes a response to the message 911 and an inquiry for confirming the speaker of the message 911.

In step 1350, the dialog analysis / generation unit 37 transmits the generated message 1351 to the control unit 30. When the control unit 30 transmits a voice response including the message 1351 and the terminal ID to the terminal, the voice output unit 32 of the terminal utters voice. When the user recognizes the voice and determines that the voice is correct, for example, the user issues a message 1361 “Yes” (name registration utterance).

In step 1360, when the voice input unit 31 receives an input of the message 1361, the voice input unit 31 transmits a voice signal corresponding to the input to the control unit 30. Thereafter, the control unit 30 transmits a voice recognition request to the voice recognition unit 36 (step 965).

Referring to FIG. 14, in step 1410, speaker identification unit 33 transmits a response to the speaker identification request (step 980) to speaker identification unit 33 as a speaker recognition response. If the user is known to the speech recognition system, the speaker recognition response indicates that the speaker has been identified. When detecting the reception of the response, the control unit 30 generates a dialog analysis / generation request.

In step 1420, the control unit 30 transmits the generated dialog analysis / generation request to the dialog analysis / generation unit 37. When the dialog analysis / generation unit 37 detects reception of the request, the dialog analysis / generation unit 37 generates a message 1431. The message 1431 is a question {Taro-san? Included in the message 1351 based on the result of the exchange so far. ) Is included based on the correctness (after all!).

In step 1430, the dialog analysis / generation unit 37 transmits a message 1431 to the control unit 30. When the control unit 30 detects reception of the message 1431, the control unit 30 generates an audio response including the terminal ID and the message 1431.

In step 1440, control unit 30 transmits a voice response to the terminal. The voice output unit 32 outputs the message 1431 by voice based on the voice response.

Thereafter, the processing after step 1040 is performed in the same manner as described above. The voice data is stored, and the learning data (for example, voiceprint information) is constantly updated with new voice data of the target user. When the user is known, even when learning is completed, the terminal does not make an utterance for confirming the user's name.

<When the terminal is the starting point of utterance>
Still another aspect will be described with reference to FIGS. FIG. 15 is a diagram illustrating a case in which talking to the user 1 from the terminal 2 triggers a conversation.

Talking to the user from the terminal 2 and learning the voice data by associating the voice data obtained by listening to the user utterance and the user name with the user name and the terminal ID.

When the terminal 2 detects the presence of the user 1, the terminal 2 talks to the user 1. The presence of the user 1 is detected based on, for example, an output from an infrared sensor, a human sensor, or the like. The terminal 2 issues a message 1510, for example. User 1 recognizes message 1510.

User 1 issues message 1520 in response to message 1510. When the terminal 2 recognizes the message 1510, the terminal 2 executes voice recognition processing and speaker identification processing. The terminal 2 switches the utterance to the user 1 based on the result of each process. For example, if it is determined that the speaker is not known, the terminal 2 generates a message 1530 and outputs the message 1530 by voice.

User 1 issues message 1540 to terminal 2 in response to message 1530. The terminal 2 performs voice recognition processing and speaker identification processing on the message 1540. Further, the terminal 2 associates the speaker “Taro” recognized as the user name of the terminal 2 with the terminal ID, and sends the

messages

1520 and 1540 of the user 1 received so far to the user management unit 35 as voice data of the speaker. accumulate.

Further, the terminal 2 generates a message 1550 as a response to the message 1540 and outputs the message 1550 by voice.

The user management unit 35 stores voice data associated with the user “Taro” and identification information (for example, voiceprint information) acquired from the voice data.

Referring to FIG. 16 and FIG. 17, the operation of the voice recognition system in a certain aspect will be described. 16 and 17 are sequence charts showing a part of processing performed in the voice recognition system.

In step 1610, when detecting that a predetermined condition is satisfied, the control unit 30 transmits a dialog generation request to the dialog analysis / generation unit 37. The condition is, for example, that the presence of the user is detected within the range of the voice recognition system, that a predetermined time has arrived. The dialog generation request includes, for example, a request to generate a message 1510 for speaking to the detected user. When the dialog analysis / generation unit 37 detects reception of the request, the dialog analysis / generation unit 37 generates a message 1510 based on a template prepared in advance.

In step 1615, the dialog analysis / generation unit 37 transmits the message 1510 generated in response to the request to the control unit 30. When the control unit 30 detects reception of the message 1510, the control unit 30 transmits a voice utterance request including the message 1510 and the terminal ID to the terminal. When receiving the request, the voice input unit 31 of the terminal outputs the message 1510 by voice. When the user recognizes the message 1510, the user issues a message 1520 as a response to the message 1510.

In step 1625, the voice input unit 31 transmits the message 1520 as a voice signal to the control unit 30. Thereafter, from step 915 to step 1345, processing similar to that described above is executed.

In step 1350, the dialog analysis / generation unit 37 transmits a message 1530 to the control unit 30. When sound based on message 1530 is output, the user issues message 1540. The message 1540 is sent from the control unit 30 to the voice recognition unit 36, and voice recognition processing is executed (step 1045).

Referring to FIG. 17, the processing from step 1050 to step 1070 is similarly executed. Thereafter, when the control unit 30 determines that there is not enough data for learning and learning has failed, the process of step 1740 is executed. More specifically, in step 1741, the control unit 30 transmits a dialog analysis / generation request to the dialog analysis / generation unit 37. When the dialog analysis / generation unit 37 detects reception of the request, the dialog analysis / generation unit 37 generates a message 1550 corresponding to the request.

In step 1742, the dialog analysis / generation unit 37 transmits a message 1550 to the control unit 30. When the control unit 30 detects reception of the message 1550, the control unit 30 generates an audio response including the terminal ID and the message 1550.

On the other hand, if the control unit 30 determines that the data sufficient for learning is available and the user has completed, the process of step 1750 is executed. More specifically, in step 1751, the control unit 30 transmits a dialog analysis / generation request to the dialog analysis / generation unit 37. When the dialog analysis / generation unit 37 detects reception of the request, the dialog analysis / generation unit 37 generates a message 1560 for responding to the request.

In step 1752, the dialog analysis / generation unit 37 transmits a message 1560 to the control unit 30. When the control unit 30 detects reception of the message 1560, the control unit 30 generates an audio response including the terminal ID and the message 1560.

In step 1760, the control unit 30 transmits the voice response to the terminal. When receiving the voice response, the voice output unit 32 outputs the message 1560 by voice.

<Other aspects>
Still another aspect will be described. In other aspects, the following configurations may be used.

(1) Voice recognition and voice authentication are performed in parallel. Therefore, recognition of the user's utterance content and authentication of the user are performed simultaneously.

(2) For each user, a topic of interest of each user is estimated based on the log of the dialog content, and a dialog based on the estimated topic is generated.

(3) Based on the number of conversations and their frequency, the utterance content of the robot (spoken dialogue device or spoken dialogue system) changes.

As a result of these elements, the user can become familiar with the robot (voice dialogue system).

For example, with the configuration (1), the voice interactive system to which the technical idea is applied specifies a user (voice authentication) without using information from a device such as a camera or a wireless tag, and the user's Acquisition of speech contents (voice recognition) becomes possible.

Next, according to the configuration (2), the daily conversation of the user is stored in the voice dialogue system and analyzed as necessary. Based on the analysis result, the voice dialogue system acquires a topic (sports, entertainment news, etc.) that each user is interested in from another information providing device, and provides the user with a topic according to the user who is interacting. Can do.

Furthermore, with the configuration (3), the dialogue between the voice dialogue system and the user is performed for a long time and periodically, so that the expression of the utterance from the voice dialogue system (wording, tone, etc.) changes according to the dialogue contents. Can do. As a result, the user can be familiar with the voice interaction system (or a voice input / output terminal such as a robot included in the voice interaction system). Each of these configurations can be combined as appropriate.

<Summary>
As described above, according to the voice recognition system according to the present embodiment, the user performs normal voice conversation without being aware of the preprocessing for learning, thereby obtaining voice data necessary for learning. Can be given to. Therefore, the function provided by the system can be easily used.

In yet another aspect, user authentication is performed without the user's awareness, and a topic corresponding to the user is output. Therefore, the user can be familiar with the services and functions provided by the voice recognition system.

The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

30 control unit, 31 voice input unit, 32 voice output unit, 33 speaker identification unit, 34 speaker identification learning unit, 35 user management unit, 36 voice recognition unit, 37 generation unit, 80 speaker model, 350, 400, 500 server, 410 speaker identification server, 520 speech recognition server, 600 terminal module, 610 main module, 620 speaker identification module, 630 speech recognition module.

Claims

A speech recognition device,
An audio input unit for receiving an utterance including information for identifying a speaker and an utterance not including information for identifying a speaker;
A speech recognition unit for performing speech recognition processing;
An audio output unit for outputting audio;
A control unit for controlling the voice recognition device based on the result of the voice recognition processing,
The said control part is a speech recognition apparatus which produces | generates the speaker identification model for identifying a speaker by associating the information which identifies the said speaker, and the speech which does not contain the information which identifies the said speaker.
The voice output unit outputs an inquiry asking for information for identifying a speaker,
Associating the information for identifying the speaker with the utterance not including the information for identifying the speaker includes information for identifying the speaker and information for identifying the speaker issued after the inquiry. The speech recognition apparatus according to claim 1, comprising associating an utterance not included.
The voice output unit outputs an inquiry asking for information for identifying a speaker after an utterance that does not include information for identifying the speaker.
Associating the information for identifying the speaker with the utterance not including the information for identifying the speaker identifies the utterance issued before the inquiry and the speaker included in the utterance responding to the inquiry. The speech recognition apparatus according to claim 1, further comprising associating with information to be performed.
The control unit according to any one of claims 1 to 3, wherein the control unit is configured to determine a content of an utterance to be output next from the speech output unit based on a content of a response to the utterance output from the speech output unit. The speech recognition device according to any one of the above.
A storage unit for storing the generated speaker identification model;
The controller is
The speech recognition device according to any one of claims 1 to 4, wherein the speech recognition device is configured to update the generated speaker identification model based on a response to the inquiry.
A speech recognition system,
A terminal,
A device capable of communicating with the terminal,
The terminal
A voice input unit for receiving an utterance including information for identifying a speaker and an utterance not including information for identifying a speaker;
An audio output unit for outputting audio;
A communication unit that is electrically connected to the audio input unit and the audio output unit and communicates with the device;
The device is
A communication unit for communicating with the terminal;
A speech recognition processing unit for performing speech recognition processing;
A control unit for controlling the device based on the result of the voice recognition processing,
The said control part is a speech recognition system which produces | generates the speaker identification model for identifying a speaker by associating the information which identifies the said speaker, and the speech which does not contain the information which identifies the said speaker.
A terminal used in the voice recognition system according to claim 6.
A method for generating a speaker identification model, comprising:
Accepting utterances with information identifying the speaker and utterances without information identifying the speaker;
Performing speech recognition processing,
Outputting audio,
Based on the result of the speech recognition process, a speaker identification model for identifying a speaker is generated by associating information for identifying the speaker with an utterance not including information for identifying the speaker Including the method.