JP6363478B2 - Speech recognition apparatus, speech recognition method, and speech recognition program - Google Patents

Description

  The present invention relates to speech recognition technology.

  In recent years, voice recognition services for mobile terminals such as smartphones and tablets are becoming widespread, and it is expected that services that provide voice recognition functions will expand in the future.

  Conventional speech recognition technology converts input speech into text in four steps (see Non-Patent Document 1). In the first step, input speech is analyzed, noise is removed from the speech signal, and acoustic features that are clues for speech recognition are extracted. In the second step, the input speech is converted into a symbol representing a phoneme, which is the minimum unit of words, using an acoustic model in which the features of each phoneme are accumulated. In the third step, the phoneme string is converted into a word using a recognition dictionary in which the correspondence between the phoneme string and the word is accumulated. In the fourth step, a score serving as a validity index is calculated for each conversion candidate using a language model in which wording and phrases are accumulated. The language model holds a word connection rule as a statistical value. The conversion candidates output as the speech recognition result may output only the most appropriate conversion candidates, or may output N conversion candidates in order of high validity called N-Best solutions. The number of candidate outputs depends on the speech recognition service.

"Voice recognition solution: Voice recognition technology that has entered the practical stage with improved recognition accuracy", [online], October 30, 2012, Impress Inc., [searched September 26, 2014], Internet <URL: http: //it.impressbm.co.jp/articles /-/ 10240 />

  The validity of the conversion candidate depends on the situation and background (context) of the speaker at the time of utterance, but there is a problem that an inappropriate speech recognition result is presented if the context of the speaker is not considered. . For example, when a voice saying “I want to eat delicious oysters” is input, it is difficult to determine which of the words “oysters” is appropriate for the fruit “柿” or the shellfish “oysters”.

  The present invention has been made in view of the above, and an object thereof is to present a more appropriate speech recognition result.

The speech recognition apparatus according to the first aspect of the present invention includes a user information storage unit that stores user information in which a date or date and date and information indicating a user's situation on the date or date and time are associated , a plurality of words, and between the words Co-occurrence frequency storage means storing co-occurrence frequency information including the co-occurrence frequency of the user, voice of the user is input to perform speech recognition, conversion candidates are obtained , and voice input content input by the user is indicated. Voice recognition means for determining tense, and the user information corresponding to the tense indicated by the voice input content is acquired from the user information storage means, and a word included in the user information is extracted, and is included in the conversion candidate Conversion candidate alignment means for extracting words and rearranging the conversion candidates based on the co-occurrence frequency of the co-occurrence frequency information including the word extracted from each And features.

In the above speech recognition apparatus, the user information is one that excludes a speech recognition result .

In the voice recognition apparatus, the user information is information registered or updated by the user in a service .

A speech recognition method according to a second aspect of the present invention is a speech recognition method executed by a computer, which performs speech recognition by inputting a user's speech, obtains conversion candidates, and inputs the speech input by the user. Corresponding to the tense indicated by the voice input content from the step of determining the tense indicated by the content and the user information storage means storing the user information associating the date or date and the information indicating the user status on the date or the date and time Acquiring the user information, extracting a word included in the user information, extracting a word included in the conversion candidate, and a co-occurrence frequency including a plurality of words and a co-occurrence frequency between the words The co-occurrence frequency information including words extracted from each of the user information and the conversion candidates is acquired from the co-occurrence frequency storage unit storing the information. And having the steps of: rearranging the conversion candidates based on the occurrence frequency of the co-occurrence frequency information.

  A speech recognition program according to a third aspect of the present invention is characterized by operating a computer as each means of the speech recognition apparatus.

  According to the present invention, a more appropriate speech recognition result can be presented.

It is a functional block diagram which shows the structure of the speech recognition system in this Embodiment. It is a figure which shows the example of the data which a user information database hold | maintains. It is a figure which shows the example of the data which a context information database hold | maintains. It is a flowchart which shows the flow of a process of the speech recognition system in this Embodiment. It is a flowchart which shows the flow of a process of the re-ranking execution part in this Embodiment. It is a figure explaining the process of a reranking execution part concretely.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a functional block diagram showing the configuration of the speech recognition system in the present embodiment. The voice recognition system shown in the figure includes a client terminal 1 and a server 3. This speech recognition system acquires and stores user information related to a user for estimating the user's context from the service used by the user, and recognizes the speech when the user inputs speech, and converts it into a conversion candidate. After obtaining, the obtained conversion candidates are rearranged in an appropriate order suitable for the user's context based on the user information and presented. Hereinafter, the client terminal 1 and the server 3 will be described.

  The client terminal 1 includes a user information storage unit 11, a user information database (DB) 12, a voice input unit 13, an information transmission unit 14, a recognition result reception unit 15, and a display unit 16.

  The user information storage unit 11 acquires user information from the service used by the user and stores it in the user information DB 12. Examples of user information include, for example, information related to a user's schedule that can be acquired from the schedule management service and information related to a user's posted comment that can be acquired from the comment posting service. The service for which user information is acquired is registered in advance and linked to the voice recognition system. The user information storage unit 11 executes processing at the timing when the user information is updated by the linked service, and updates the user information stored in the user information DB 12 as needed. For example, when a new schedule is added by the schedule management service, a record is added to newly register user information, and when the schedule is updated, information stored in the user information DB 12 is rewritten.

  The user information DB 12 holds user information acquired from each service. FIG. 2 shows an example of data held in the user information DB 12. In the example of FIG. 2, the user information DB 12 holds records composed of item columns, date columns, and time columns. The item column stores information indicating the user status. For example, when the schedule management service is set as the service to be linked, the user information storage unit 11 acquires the scheduled item and date and time registered in the schedule management service, and the item column of the user information DB 12 Store in date column and time column. When the comment posting service is set as a service to be linked, the user information storage unit 11 acquires the posting content and date / time posted by the user to the comment posting service, and the item column of the user information DB 12 Store in date column and time column.

  For example, OAuth can be used as a method for acquiring user information from a linked service (reference URL http://oauth.net/). OAuth is a mechanism for another web service B that has received permission from the user to access various resources that the user has in the web service A and various functions to which the user has access authority. When the user grants the web service B permission to access the web service A, the web service B can access an API (Application Programming Interface) provided by the web service A within a given range.

  The voice input unit 13 inputs a user's voice to be voice-recognized and transmits the input voice information to the information transmission unit 14.

The information transmission unit 14 receives the voice information from the voice input unit 13, acquires user information corresponding to the voice input time that is the time when the voice is input to the voice input unit 13 from the user information DB 12, and acquires the acquired user Information and voice information are transmitted to the server 3. When acquiring the user information, the date column and the time column of the user information DB 12 acquire the item column of the record including the voice input time as the user information. For example, when the data shown in FIG. 2 is stored in the user information DB 12 and the voice input time is “2014/9/27 18:40”, the information transmission unit 14 includes the item column of the record including the voice input time. The user information I _u = {Family and Hiroshima trip} in FIG. At this time, the information stored in the item column of the record whose date column is the same as the date of the voice input time (“Itsukushima Shrine is amazing” in FIG. 2) is obtained, and the user information I _u = {Family and Hiroshima trip, Itsukushima The shrine is amazing. }. The rule of the record acquired from the user information DB 12 with respect to the voice input time is set in advance.

  The recognition result receiving unit 15 receives a conversion candidate that is a voice recognition result of the voice information from the server 3 and causes the display unit 16 to display the conversion candidate.

  The display unit 16 displays input speech conversion candidates at predetermined positions.

  The server 3 includes an information receiving unit 31, a voice recognition unit 32, a reranking execution unit 33, and a context information DB 34.

  The information receiving unit 31 receives voice information and user information from the client terminal 1 and transmits them to the voice recognition unit 32.

  The voice recognition unit 32 performs voice recognition on the received voice information to obtain conversion candidates, and transmits the obtained conversion candidates and user information to the reranking execution unit 33. Each conversion candidate is given a score indicating the validity of the candidate. A well-known speech recognition technique is used to execute speech recognition.

  In addition, the voice recognition unit 32 determines the tense of the voice information. If the determined tense is a specific time of “future” or a specific time of “past”, the time is read from the user information DB 12 of the client terminal 1. Is acquired and transmitted to the reranking execution unit 33. For example, when the voice information includes an expression indicating a specific future or the past such as “Tomorrow” or “Yesterday”, the user information corresponding to the date and time indicated by the expression is acquired from the user information DB 12 and acquired. The user information is transmitted to the reranking execution unit 33.

  The reranking execution unit 33 refers to the information on the co-occurrence relationship of the word combinations stored in the context information DB 34, and determines the conversion candidate based on the co-occurrence relationship between the word included in the conversion candidate and the word included in the user information. Recalculate the score. Specific processing of the reranking execution unit 33 will be described later.

  The context information DB 34 holds information regarding the co-occurrence relationship between two word combinations. FIG. 3 shows an example of data held in the context information DB 34. In the example of the figure, a record composed of a word 1 column, a word 2 column, and a co-occurrence frequency column is held. For example, the known co-occurrence frequency calculation program N-gram (reference URL http://oscar.gsid.nagoya-u.ac.jp/project/elc/genkou/ngrmapper2/node8.html), word co-occurrence frequency database ( The co-occurrence frequency between two words is calculated using the reference URL https://alaginrc.nict.go.jp), and the value is stored in the co-occurrence frequency column.

  Each unit included in the client terminal 1 and the server 3 may be configured by a computer including an arithmetic processing device, a storage device, and the like, and the processing of each unit may be executed by a program. This program is stored in a storage device included in the client terminal 1 and the server 3, and can be recorded on a recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, or provided through a network. Here, although each function is divided into the client terminal 1 and the server 3, this system may be implement | achieved by one apparatus.

  Next, the operation of the speech recognition system in this embodiment will be described.

  FIG. 4 is a flowchart showing the flow of processing of the speech recognition system in the present embodiment. It is assumed that the user information storage unit 11 collects user information from linked services as needed and stores it in the user information DB 12.

  When the voice input unit 13 inputs a voice, the information transmission unit 14 acquires user information corresponding to the voice input time from the user information DB 12, and transmits the voice information and the user information to the server 3 (step S11).

  The information receiving unit 31 transmits the voice information and user information received from the client terminal 1 to the voice recognition unit 32, and the voice recognition unit 32 performs voice recognition on the voice information to obtain conversion candidates (step S12). ).

  The voice recognition unit 32 determines the tense from the result of executing the voice recognition (step S13). If the determined tense is a specific time of “future” or “past” (YES in step S14), the time Is acquired from the user information DB 12 (step S15).

  The reranking execution unit 33 searches the context information DB 34 for a co-occurrence relationship between the word included in the user information and the word included in the conversion candidate, recalculates the conversion candidate score based on the co-occurrence relationship, The conversion candidates are reranked based on the calculated score (step S16). The client terminal 1 presents the conversion candidates reranked in descending order of score to the user.

  Next, the flow of the conversion candidate reranking process will be described.

FIG. 5 is a flowchart showing the flow of processing of the reranking execution unit 33 in the present embodiment. FIG. 6 is a diagram for specifically explaining the processing of FIG. 5. Reranking execution unit 33 executes the following process upon reception of conversion candidates R ₁ and the user information I _u from the voice recognition unit 32. As shown in FIG. 6, a score is assigned to the conversion candidate R ₁ that is the result of the voice recognition unit 32 executing voice recognition.

Reranking execution unit 33 extracts a noun N user information _{I u} to morphological analysis (step S21). For example, when user information I _u = {family and Hiroshima trip, Itsukushima Shrine is amazing}, N = {family, Hiroshima, trip, Itsukushima, shrine} is extracted. For example, MeCab, which is a known morphological analysis engine, can be used for the morphological analysis (reference URL https://code.google.com/p/mecab/).

  For each of the nouns N extracted in step S21, the reranking execution unit 33 searches for records in which the nouns are stored in the word 1 column or the word 2 column of the context information DB 34, and sets the word co-occurrence frequencies. S is acquired (step S22). For example, when N = {family, Hiroshima, travel, Itsukushima, shrine} and the data shown in FIG. 3 is stored in the context information DB 34, as shown in FIG. A set of frequencies S = {[Hiroshima, 柿, 2], [Hiroshima, oyster, 5]} is acquired.

Reranking execution unit 33 recalculates the conversion candidate _{R 1} each score (step S23). Specifically, for example, the co-occurrence frequency of words including nouns included in the n-th conversion candidate R ₁ (n) of the conversion candidate R ₁ from the set S of word co-occurrence frequencies acquired in step S22. A set S ′ is extracted. Then, the score rescore (R ₁ (n)) is calculated by the following equation (1).

ここで、Ｓ’（ｉ）は、ｉ番目の単語の共起頻度の組Ｓ’の共起頻度カラムに格納されている数値を表し、ｍは単語の共起頻度の組Ｓ’の要素の数を表す。また、α＞０である。

Here, S ′ (i) represents a numerical value stored in the co-occurrence frequency column of the co-occurrence frequency set S ′ of the i-th word, and m is an element of the co-occurrence frequency set S ′ of the word Represents a number. Also, α> 0.

The reranking execution unit 33 rearranges the conversion candidates R _{1 in} descending order of the score rescore (R ₁ (n)) calculated in step 23 to generate a conversion candidate R ₂ (step S 24). In the example illustrated in FIG. 6, the conversion candidate R ₁ is rearranged like the conversion candidate R ₂ and transmitted to the client terminal 1.

  As described above, according to the present embodiment, the user information storage unit 11 acquires user information indicating the user status from the service used by the user, stores the user information in the user information DB 12, and stores the user information in the voice input unit. When 13 inputs a user's voice, the information transmission unit 14 acquires user information corresponding to the voice input time, the reranking execution unit 33 extracts a word included in the user information, and the combination of two words A set S of word co-occurrence frequencies including the word extracted by the reranking execution unit 33 is retrieved from the context information DB 34 storing information related to the co-occurrence relationship, and the speech recognition unit 32 is searched from the set S of word co-occurrence frequencies. By extracting a set S ′ of co-occurrence frequencies of words including words included in each of the conversion candidates that are the speech recognition results, and rearranging the conversion candidates based on the co-occurrence frequencies , Considering the context of the user, it is possible to present more appropriate speech recognition result.

  According to the present embodiment, the voice recognition unit 32 determines a specific time indicated by the user's voice, and the reranking execution unit 33 acquires user information corresponding to the specific time, thereby indicating the user's voice. Based on the context corresponding to time, it is possible to present a more appropriate speech recognition result.

DESCRIPTION OF SYMBOLS 1 ... Client terminal 11 ... User information storage part 12 ... User information DB
DESCRIPTION OF SYMBOLS 13 ... Voice input part 14 ... Information transmission part 15 ... Recognition result receiving part 16 ... Display part 3 ... Server 31 ... Information receiving part 32 ... Voice recognition part 33 ... Reranking execution part 34 ... Context information DB

Claims (7)

User information accumulating means for storing user information in which date or date and date and information indicating the status of the user at the date or date and time are associated ;
Co-occurrence frequency accumulating means storing co-occurrence frequency information including a plurality of words and the co-occurrence frequency between the words;
Voice recognition means for performing voice recognition by inputting the user's voice, obtaining conversion candidates, and determining the tense indicated by the voice input content input by the user ;
The user information corresponding to the tense indicated by the voice input content is acquired from the user information storage means, and the words included in the user information are extracted, and the words included in the conversion candidates are extracted and extracted from each. Conversion candidate alignment means for rearranging the conversion candidates based on the co-occurrence frequency of the co-occurrence frequency information including a word;
A speech recognition apparatus comprising: The voice recognition apparatus according to claim 1, wherein the user information excludes a voice recognition result. The voice recognition apparatus according to claim 1, wherein the user information is information registered or updated by the user in a service. A speech recognition method executed by a computer,
Inputting a user's voice to perform voice recognition, obtaining conversion candidates, and determining a tense indicated by the voice input content input by the user ;
The user information corresponding to the tense indicated by the voice input content is acquired from user information storage means storing user information in which date or date and information indicating the user status on the date or date are stored, and the user Extracting words contained in the information;
Extracting words included in the conversion candidates;
The co-occurrence frequency information including words extracted from each of the user information and the conversion candidate is acquired from a co-occurrence frequency storage unit storing co-occurrence frequency information including a plurality of words and co-occurrence frequencies between the words. Rearranging the conversion candidates based on the co-occurrence frequency of the co-occurrence frequency information;
A speech recognition method comprising: The voice recognition method according to claim 4, wherein the user information excludes a voice recognition result. 6. The voice recognition method according to claim 4, wherein the user information is information registered or updated by the user in a service.   A speech recognition program for operating a computer as each means of the speech recognition apparatus according to claim 1.

Images