JP2014119536A - Sound recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve accuracy to recognize sound data.SOLUTION: A sound recognition device comprises: a voice processing unit 12 outputting a candidate of a type of voice included in input sound data when there is sound data input; a non-voice processing unit 13 determining whether the input sound data is non-voice sound which does not have a feature of voice when there is sound data input, and making an identifier correspond to the non-voice sound when the input signal is non-voice sound; and an operation unit 143 weighing for the candidate based on a history database D3 and determining the type of the voice based on the candidate of the type of the weighted voice when the identifier is made to correspond to the non-voice sound in the non-voice processing unit 13 and a candidate of the type of the voice from the voice processing unit 12 within an appointed time after the identifier is made to correspond to the non-voice sound.

Description

  The present invention relates to a sound recognition device that recognizes a voice or a non-voice sound included in sound data.

  Voice recognition for converting voice input through a microphone or the like into characters without using an input device such as a keyboard or a touch panel is actively performed. In addition, a technique for identifying non-speech sounds other than speech and converting it into visual data is also being studied.

  Since there are many vocabulary with different characters even in the same speech, it is difficult to convert speech to character data. In the field of speech recognition, improvement of the recognition rate is an issue.

  In order to improve the recognition rate, for example, by selectively using a dictionary (for example, for medical terms, for conference terms) used for speech recognition according to the scene, it is converted into characters that are frequently used in each scene There is also.

JP 2012-128411 A

  As described above, it has been a problem to improve the accuracy of speech recognition. Further, it has been a problem to improve the recognition accuracy of non-speech sounds as well.

  In view of the above problems, an object of the present invention is to provide a sound recognition device with improved accuracy for recognizing sound data.

  In order to achieve the above object, the sound processing unit outputs a sound type candidate included in the input sound data when the sound data is input. The non-speech processing unit determines whether or not the input sound data is a non-speech sound having no sound feature when sound data is input, and the input sound data is a non-speech sound. In this case, an identifier is associated with the non-voice sound. The calculation unit is configured so that, in the non-voice processing unit, an identifier is associated with the non-speech sound, and the identifier is associated with the non-speech sound, and the speech processing unit transmits the identifier within a predetermined time. When a voice type candidate is input, the candidate is weighted based on the history database, and the voice type is determined based on the weighted voice type candidate.

  According to the present invention, the accuracy of sound recognition can be improved.

It is a block diagram explaining the sound recognition apparatus which concerns on 1st Embodiment. It is explanatory drawing of an example of the data utilized with the sound recognition apparatus of FIG. It is a flowchart explaining an example of the process in the non-voice sound process part of the sound recognition apparatus of FIG. It is a flowchart explaining an example of the process in the weighting process part of the sound recognition apparatus of FIG. It is a block diagram explaining the sound recognition apparatus which concerns on 2nd Embodiment. It is explanatory drawing of an example of the data utilized with the sound recognition apparatus of FIG. It is a flowchart explaining an example of the process in the non-voice sound process part of the sound recognition apparatus of FIG. It is a flowchart explaining an example of the process in the weighting process part of the sound recognition apparatus of FIG. It is a block diagram explaining the sound recognition apparatus which concerns on 3rd Embodiment. It is explanatory drawing of an example of the data utilized with the sound recognition apparatus of FIG. It is a flowchart explaining an example of the process in the non-speech sound processing part of the sound recognition apparatus of FIG. It is a flowchart explaining an example of the process in the weighting process part of the sound recognition apparatus of FIG.

  The sound recognition apparatus according to each embodiment of the present invention will be described with reference to the drawings. In the following drawings, the same components are denoted by the same reference numerals and the description thereof is omitted.

<First Embodiment>
The sound recognition apparatus according to the first embodiment is an apparatus that converts input speech into character data and outputs the speech using speech recognition. When the sound recognition apparatus according to the first embodiment determines character data corresponding to speech from a plurality of character data candidates, it is input within a predetermined time in the past (for example, up to 3 minutes before) from the speech input timing. Use non-speech sound. Hereinafter, a sound produced by a human voice is referred to as “speech”. Further, a sound other than voice (for example, mechanical sound) is referred to as “non-voice sound”. In the sound recognition device, the accuracy of speech recognition can be improved by using non-speech sounds for speech recognition.

  As shown in FIG. 1, the sound recognition apparatus 1a according to the first embodiment includes an input unit 11 such as a microphone to which sound data is input, a sound processing unit 12 that analyzes sound included in the sound data, and sound data. A non-speech sound processing unit 13 for analyzing non-speech sounds other than the speech included in the sound, and a weighting process for determining character data corresponding to the speech using the processing results of the speech processing unit 12 and the non-speech sound processing unit 13 Unit 14, an output unit 15 such as a display for outputting character data, and a storage device 20.

  The non-speech sound processing unit 13 includes an analysis unit 131, a search unit 132, and an addition unit 133 in order to analyze non-speech sounds. In addition, the weighting processing unit 14 includes a determination unit 141, a selection unit 142, a calculation unit 143, and an update unit 144 in order to determine character data.

  The storage device 20 stores a sound recognition program P1, a sound database D1, a non-sound sound database D2, and a history database D3.

  Specifically, the sound recognition device 1a is an information processing device including a CPU (Central Processing Unit) 10, an input unit 11, an output unit 15, and a storage device 20, and a sound recognition program P1 stored in the storage device 20 Is executed, the CPU 10 executes processing as the voice processing unit 12, the non-voice sound processing unit 13, and the weighting processing unit 14.

  The voice database D1 is data used when the voice processing unit 12 converts voice contained in the voice data into corresponding character data, and is data used in general voice recognition. For example, in the speech database D1, “acoustic model data” in which speech waveform data, frequency data, and the like are associated with phonetic symbols, “dictionary data” in which phonetic symbols are associated with word character data, and word characters "Language model data" used to convert text data into text data. “Language model data” is, for example, data in which words are linked by probability. In the voice processing unit 12, the input voice may be converted into word character data or sentence character data.

  The non-speech sound database D2 is a database in which feature data that can specify a sound, such as waveform data and frequency data of a non-speech sound, and an identifier of the non-speech sound are associated with each other. For example, the non-speech sound database D2 associates chime sounds that are non-speech sounds with identifiers attached to chime sounds, telephone sounds that are non-speech sounds, identifiers attached to phone sounds, and the like.

  The history database D3 is data in which a weight value used for determining character data is associated with an identifier of non-voice sound and character data. Here, the frequency of input of a sound within a predetermined time (for example, within 3 minutes) after input of a certain non-speech sound is used as a weight value.

  In the history database D3 shown in FIG. 2 (a), after inputting a chime sound, which is a non-speech sound with the identifier “A sound”, the frequency of input of the voice “Thank you” is 40%. "Good morning" indicates that the frequency of input is 35%. That is, the storage device 20 stores the frequency in association with, for example, (non-speech sound with an identifier) and (voice) as the history database D3. The frequency stored in the history database D3 is, for example, “the number of times that (speech) is detected within a predetermined time after (non-speech sound with an identifier attached) / (non-speech sound with an identifier attached) is detected. It is calculated as “the number of times performed”. A specific example is 40% of the frequency calculated as <number of times “Thank you” is recognized within 3 minutes after chime is detected> / <number of times chime is detected>. In addition, since the sound uttered within 3 minutes with respect to one non-speech sound is not necessarily one, each sound can independently take a value of 0 to 100%. For example, if both “goodbye” and “Thank you” are recognized within 3 minutes after recognizing the chime sound for the first time, the history database at that time is both 100 percent. In addition, after inputting the phone sound, which is a non-speech sound with the identifier “B sound”, the frequency of input of the sound “Yes” is 75%, and the sound “Co., Ltd. ××” is input. The frequency is 72%, and the frequency at which the voice “Hello” is input is 48%. Further, it is indicated that the frequency of input of the voice “progress” after the input of the non-voice sound with the identifier “C sound” is 85%.

  When sound data is input from the input unit 11, the sound processing unit 12 analyzes the input sound data and identifies one or more character data candidates corresponding to the sound included in the sound data. Specifically, the voice processing unit 12 reads the voice database D1 stored in the storage device 20, and the voice waveform data and frequency data included in the voice data are included in the “acoustic model data” of the voice database D1. A phonetic symbol having a high degree of similarity is derived as compared with the waveform data and frequency data of each voice. Next, apply “dictionary data” or “language model data” to the derived phonetic symbols to convert them to word character data or sentence character data, and to verify the word character data or sentence character data. Is output with a score indicating. For example, the voice processing unit 12 derives a phonetic symbol together with the similarity. Then, the speech processing unit 12 determines whether or not the phonetic symbol is included in the “dictionary data” with respect to the similarity, and if it is included, whether the word included in the “acoustic model” is likely to be connected. Is derived as a score of character data indicated by a phonetic symbol. In addition, for example, as shown in FIG. 2B, the voice processing unit 12 extracts all voice character data whose score satisfies a predetermined condition, and associates the extracted character data with the score. It outputs to the weighting process part 14 as a process result (voice process result) of a voice process.

  Here, as conditions for the voice processing unit 12 to extract character data, (1) character data with a score equal to or higher than a specific value (for example, 50 or more) is extracted, and (2) the score is an upper predetermined number (for example, 3). (3) Extraction of character data having a score within a predetermined range from the first-ranked score (for example, a score within minus 5 from the first-ranked score) is conceivable.

  When sound data is input from the input unit 11, the analysis unit 131 analyzes the input sound data and stores a non-speech sound in a memory (not shown). Specifically, the analysis unit 131 determines whether or not the volume of the input sound data is greater than or equal to a predetermined threshold value. The analysis unit 131 determines whether the sound data having a volume equal to or higher than the threshold is sound. The analysis unit 131 determines sound data that is not voice and is equal to or higher than a threshold value as non-speech sound. An example of a method for determining whether or not the sound is a sound by the analysis unit 131 is to cut input sound data in units of frames of a certain time width, perform frequency conversion for each frame, and perform time average in each frequency energy and a predetermined bandwidth. Judgment is made based on whether or not the ratio to the energy exceeds the threshold value. Another example of a method for determining whether or not the sound is sound by the analysis unit 131 is to cut the input sound data in units of frames of a certain time width, perform frequency conversion, and convert the peak spectrum to a spectrum energy with a predetermined value. When it is detected by comparison or the like, and it is detected that the peak spectra have a relationship between the fundamental tone and the harmonic, it is determined that the sound is a voice. For example, the technique described in Japanese Patent Application No. 2011-254578 and Japanese Patent Application No. 2011-260036 can be used as the voice determination method by the analysis unit 131.

  Further, when the non-speech sound included in the sound data having a volume equal to or higher than the threshold value is extracted, the analysis unit 131 starts accumulation of the non-speech sound data in the memory. Thereafter, when the volume of the non-speech sound becomes less than the threshold, the analysis unit 131 ends the accumulation of the non-speech sound in the memory. The threshold used here is determined in advance by the sound recognition apparatus 1a as being used for determining the volume of sound data to be processed. As a result, it is not necessary to process a sound whose volume is less than the threshold, such as noise, and the processing burden on the sound recognition device 1a can be reduced.

  Further, when the analysis unit 131 starts accumulation of the non-speech sound in the memory, the analysis unit 131 starts timing based on the timing of the accumulation start, and a predetermined time (for example, 5 seconds) from the start of accumulation of the non-speech sound in the memory. After elapse of time, accumulation of non-speech sounds in the memory may be terminated. As a result, for long non-speech sounds, only the beginning part of the sound can be detected and compared, and the memory capacity for the database can be reduced and the detection response can be improved.

  When the non-sound sound is accumulated in the memory by the analysis unit 131, the search unit 132 searches the non-sound sound identifier stored in the memory from the non-sound sound database D2.

  Specifically, the search unit 132 reads the non-speech sound database D2 from the storage device 20, and each non-speech sound included in the non-speech sound database D2 is similar to the non-speech sound stored in the memory. Find the degree. For example, the non-speech sound waveform data and frequency data included in the non-speech sound database are compared with the non-speech sound waveform data and frequency data stored in the memory to obtain the similarity.

  Further, when the search unit 132 obtains the similarity between all the non-speech sounds included in the non-speech sound database D2 and the non-speech sounds stored in the memory, the highest similarity among the obtained similarities is predetermined. It is determined whether or not the threshold value is exceeded. The threshold used here is determined in advance by the sound recognition apparatus 1a in order to determine that the non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory represent the same content. It is a defined value.

  When the highest similarity is greater than or equal to the threshold, the search unit 132 outputs the identifier of the non-speech sound with the highest similarity to the weighting processing unit 14 as a processing result (non-speech sound processing result). Here, when the highest similarity is less than the threshold value, the search unit 132 determines that the non-speech sound stored in the memory is not included in the non-speech sound database D2, and the search unit 132 The non-speech sound is output to the adding unit 133.

  When a non-speech sound is input from the search unit 132, the adding unit 133 adds an identifier to the input non-speech sound and adds it to the non-speech sound database D2. Further, the adding unit 133 outputs the non-speech sound identifier to the weighting processing unit 14 as a processing result (non-sound sound processing result).

  The determination unit 141 determines the timing at which processing results are input from the sound processing unit 12 and the non-sound sound processing unit 13. Specifically, when the non-sound sound processing result is input from the non-sound sound processing unit 13, the determination unit 141 stores a non-sound sound identifier that is the non-sound sound processing result in a memory (not shown). At the same time, timing is started based on the input timing. Thereafter, when a new non-speech sound identifier is input from the non-speech sound processing unit 13, the determination unit 141 updates the identifier stored in the memory to a new non-speech sound identifier and measures the previous time count. End and start timing again.

  On the other hand, when the sound processing result is input from the sound processing unit 12, the determination unit 141 is within a predetermined time (for example, within 3 minutes) after the non-sound sound processing result is input from the non-sound sound processing unit 13. Determine whether or not. When it is within the predetermined time, the determination unit 141 outputs the sound processing result input from the sound processing unit 12 and the identifier of the non-speech sound stored in the memory to the selection unit 142. If the predetermined time has elapsed, the determination unit 141 outputs only the sound processing result input from the sound processing unit 12 to the output unit 15.

  The selection unit 142 reads the history database D <b> 3 from the storage device 20 when the voice processing result and the identifier of the non-voice sound are input from the determination unit 141. Further, the selection unit 142 selects a frequency corresponding to the input non-speech sound identifier and each character data included in the sound processing result from the read history database D3, and selects the selection result as the non-speech sound. Along with the identifier and the voice processing result, the result is output to the calculation unit 143.

  On the other hand, the new non-sound sound is not included in the history database D3 like the non-sound sound added to the non-sound sound database D2 by the adding unit 133 of the non-sound sound processing unit 13. As described above, when the history database D3 does not include the frequency associated with the combination of the non-speech sound identifier and the character data included in the speech processing result, the selection unit 142 cannot select the frequency for this combination. Therefore, the selection unit 142 is determined to output the character data having the highest score in the voice processing result to the output unit 15 as the character data corresponding to the voice, and to correspond to the identifier of the non-voice sound, the voice processing result, and the voice. The character data is output to the updating unit 144.

  Further, when only a sound processing result is input from the determination unit 141 because the processing result input from the sound processing unit 12 is after a predetermined time has elapsed from the input of the processing result from the non-sound sound processing unit 13. The selection unit 142 cannot select the frequency from the history database D3. Therefore, in this case, the selection unit 142 outputs the character data having the highest score in the speech processing result to the output unit 15 as the character data corresponding to the speech, and does not output the data to the update unit 144.

  When the identifier of the non-speech sound, the voice processing result, and the selection result are input from the selection unit 142, the calculation unit 143 performs a calculation for weighting the score included in the voice processing result and determines character data corresponding to the voice. . Specifically, the calculation unit 143 calculates a new score by weighting the score for each character data candidate, and outputs the character data having the highest new score to the output unit 15 as character data corresponding to speech. . In addition, the calculation unit 143 outputs the character data corresponding to the determined voice, the identifier of the non-speech sound, and the voice processing result to the update unit 144.

  For example, the calculation unit 143 obtains a new score weighted by a mathematical expression such as Expression (1) for the combination of the non-speech sound identifier and each character data. In the formula (1), r1 is a predetermined coefficient.

New score = score + frequency × r1 (1)
The update unit 144 updates the frequency of the history database D3 according to the input data. Specifically, when the character data, the non-speech sound identifier and the sound processing result determined by the calculation unit 143 are input, the update unit 144 receives the non-sound sound identifier and the character data corresponding to the selected sound. The frequency associated with the non-speech sound identifier and the character data other than the character data corresponding to the speech included in the selected speech processing result is updated to a lower frequency. The history database D3 is updated. Here, when there is no frequency associated with the non-speech sound identifier and the character data included in the speech processing result in the history database D3, the update unit 144 associates the frequency with the non-speech sound identifier and the character data and records the history database D3. Add to

  In addition, when the character data determined from the selection unit 142, the non-speech sound identifier, and the sound processing result are input, the update unit 144 is associated with the non-sound sound identifier and the character data corresponding to the determined sound. Update the frequency to a higher frequency and update the frequency associated with the character data other than the character data determined to correspond to the voice included in the input non-speech sound identifier and the voice processing result to a lower frequency Update the database D3.

(Processing in the non-voice sound processing unit)
Processing in the non-speech sound processing unit 13 will be described using the flowchart shown in FIG.

The process of FIG. 3 is started when the apparatus is turned on. The non-speech sound processing unit 13 analyzes the sound data input via the input unit 11 in the analysis unit 131, and determines whether or not the volume of the input sound data is greater than or equal to a threshold (S10). When the volume of the sound data is greater than or equal to the threshold (YES in S10), the analysis unit 131 determines whether the sound data is a non-speech sound (S11). Here, when there is no input of sound data having a volume equal to or higher than the threshold value (NO in S10) and when the non-voice sound is not included in the sound data (NO in S11), the process returns to step S10 and the process is repeated.

  When the non-speech sound is included in the sound data having a volume equal to or higher than the threshold (YES in S11), the analysis unit 131 starts accumulation of the non-speech sound in the memory. Further, the search unit 132 obtains and compares the similarity between each non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory (S12).

  When the non-speech sound similar to the non-speech sound input to the non-speech sound database D2 is included (YES in S13), the analysis unit 131 ends the storage of the input non-speech sound in the memory, The search unit 132 ends the comparison between each non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory (S14). When the storage and comparison of the data is completed and the identifier of the non-speech sound is specified, the search unit 132 outputs the non-speech sound identifier to the weighting processing unit 14 as a processing result, and the non-sound sound identifier stored in the memory is output. The data is discarded (S18).

  On the other hand, when the non-speech sound similar to the non-speech sound input to the non-speech sound database D2 is not included (NO in S13), the analysis unit 131 determines that the volume of the sound data subsequently input is a threshold value. It is determined whether or not the storage time of the non-voice sound in the memory is equal to or longer than a predetermined time (S15). When the volume of the new sound data is equal to or higher than the threshold value and when the storage time of the non-speech sound in the memory is within the predetermined time, the process returns to step S12 to repeat the data storage and comparison processing.

  When the volume of newly input sound data is less than the threshold value or when the accumulation time is equal to or longer than the predetermined time (YES in S15), the analysis unit 131 ends the accumulation of the input non-speech sound in the memory. Then, the search unit 132 ends the comparison between each non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory (S16). Thereafter, the adding unit 133 adds a new identifier to the non-speech sound and adds it to the non-speech sound database D2 (S17). When the non-speech sound and the non-speech sound identifier are added to the non-speech sound database D2, the adding unit 133 outputs the newly added non-sound sound identifier to the weighting processing unit 14 as a processing result, and the memory The data of the non-speech sound stored in (1) is discarded (S18).

(Processing in the weighting processing unit)
The process in the weighting process part 14 is demonstrated using the flowchart shown in FIG. The flowchart shown in FIG. 4 is started when the apparatus is turned on. The determination unit 141 of the weighting processing unit 14 waits for input of processing results from the audio processing unit 12 and the non-speech sound processing unit 13, and the identifier of the non-speech sound that is the processing result is received from the non-sound sound processing unit 13. When input is made (YES in S20), time measurement is started (S21).

  Thereafter, when the processing result is input from the sound processing unit 12 (YES in S22), the determination unit 141 determines whether or not it is within a predetermined time after inputting the non-sound sound identifier from the non-sound sound processing unit 13. Determine (S23).

  When it is within the predetermined time (YES in S23), the selection unit 142 determines whether or not the history database D3 includes the frequency of the input non-speech sound identifier and the character data included in the sound processing result. Determine (S24).

  When the history database D3 includes the identifier of the input non-speech sound and the frequency associated with each character data included in the speech processing result (YES in S24), the calculation unit 143 performs a weighting process. In other words, the calculation unit 143 obtains a new score for the candidate score of each character data included in the speech processing result using the frequency that is the weight value (S25). The details are the calculations using the above-described equation (1). Thereafter, the arithmetic unit 143 determines that the character data having the highest score corresponds to the voice as character data, and outputs it as a result (S26).

  On the other hand, when it is not within the predetermined time (NO in S23), or when the frequency database associated with each character data included in the input non-speech sound identifier and the voice processing result is not included in the history database D3 (NO in S24) The selection unit 142 determines the character data having the highest score included in the speech processing result as the character data corresponding to the speech, and outputs it as a result (S26).

  Further, when the character data corresponding to the voice is determined, the update unit 144 updates the frequency included in the history database D3 (S27).

  When no processing result is input from the non-sound sound processing unit 13 and the sound processing unit 12 (NO in S20 and NO in S22), the process returns to Step S20, and the process from the non-sound sound processing unit 13 or the sound processing unit 12 is performed. Wait for the result.

  In the above-described embodiment, the voice processing unit 12 outputs the character data corresponding to the voice, but may be an identifier or an icon corresponding to the voice, for example, as long as it is a form indicating the voice type instead of the character data. Not limited to data.

  As described above, the sound recognition device 1a according to the first embodiment uses a non-speech sound acquired within a predetermined time in the past at the time of voice recognition. Therefore, it is possible to predict the type corresponding to the voice acquired within a certain time from the acquisition of the non-speech sound, and the accuracy of voice recognition can be improved.

  In addition, since the non-speech sound database D2 and the history database D3 used in the sound recognition device 1a are automatically updated according to use, the accuracy of speech recognition is improved without processing such as user registration work. be able to.

Second Embodiment
In the sound recognition device according to the second embodiment, when converting non-sound sound included in sound data into visual data, the sound recognition device is input within a predetermined time in the past (for example, up to 3 minutes before) from the input timing of the non-sound sound. Use your voice. Here again, a sound produced by a human voice is referred to as “speech”, and a sound other than the sound is referred to as “non-speech sound”. By using the voice for specifying the non-voice sound, the accuracy of conversion to the non-voice sound can be improved.

  As shown in FIG. 5, the sound recognition device 1b according to the second embodiment is replaced with a sound processing unit 12 as compared with the sound recognition device 1a according to the first embodiment described above with reference to FIG. A non-speech sound processing unit 13b instead of the non-speech sound processing unit 13, and a weighting processing unit 14b instead of the weighting processing unit 14. Also, the sound recognition device 1b stores a sound recognition program P2 in place of the sound recognition program P1 and a history database D4 in place of the history database D3 in the storage device 20, as compared with the sound recognition device 1a. Yes. That is, in the sound recognition device 1b, the sound recognition program P2 stored in the storage device 20 is executed, whereby the CPU 10 performs processing as the sound processing unit 12b, the non-sound sound processing unit 13b, and the weighting processing unit 14b. .

  The history database D4 is a database in which weight values used for determining non-speech sounds are associated with character data and non-speech sound identifiers. Here, the frequency at which the non-speech sound is input within a predetermined time (for example, within 3 minutes) after inputting the sound of certain character data is used as a weight value.

  In the history database D4 shown in FIG. 6A, after inputting the voice “Thank you”, the frequency of input of the chime sound, which is a non-speech sound with the identifier “A sound”, is 40%, and the identifier “B sound” This indicates that the frequency of inputting the sound of a telephone with “” is 35%. In addition, after inputting the voice “Good morning”, the frequency with which the non-speech sound with the identifier “C sound” is input is 30%, and the frequency with which the non-speech sound with the identifier “D sound” is input. Is 25%, and the frequency of input of the non-speech sound with the identifier “E sound” is 10%.

  When sound data is input from the input unit 11, the sound processing unit 12b outputs character data corresponding to the sound included in the sound data. At this time, the voice processing unit 12b uses the voice database D1 to derive the character data corresponding to the voice from the voice data together with a score representing the probability of the character data, and only the character data having the first score is obtained. Output as audio processing result. Even when the score is ranked first, if the score is less than a predetermined value, the voice processing unit 12b may not output character data. That is, an unreliable result need not be reflected in subsequent processing.

  When the non-speech sound extracted from the sound data having a volume equal to or higher than the threshold by the analysis unit 131 is accumulated in the memory, the search unit 132b searches the non-speech sound database D2 for the identifier of the non-speech sound stored in the memory. The identifier of the searched non-speech sound is output to the weighting processing unit 14b as the non-speech sound processing result. Specifically, the search unit 132b calculates the similarity between each non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory, and uses this similarity as a score. Further, the search unit 132b extracts all the identifiers of the non-speech sounds that meet the predetermined extraction condition as candidates for the identifiers of the non-speech sounds. For example, as illustrated in FIG. The identifier and the score are associated with each other to obtain a non-voice sound processing result.

  Here, as conditions for the search unit 132b to extract the identifier of the non-speech sound, (1) extract the identifier of the non-speech sound whose score is equal to or higher than a specific value (for example, 50 or higher), Extract non-speech sound identifiers (for example, 3rd or higher), (3) character data with a score within a predetermined range from the 1st score (for example, a score within minus 5 from the 1st score) Extraction etc. can be considered.

  On the other hand, if the identifier of the non-speech sound cannot be retrieved from the non-speech sound database D2, the search unit 132b waits until a new non-speech sound is stored in the memory without outputting the non-speech sound processing result. That is, even when a non-sound sound not newly included in the non-speech sound database D2 is newly input, the non-speech sound processing unit 13b is similar to the non-sound sound processing unit 13 described above with reference to FIG. It is not added to the database D2.

  The determination unit 141b waits for the processing results input from the sound processing unit 12b and the non-speech sound processing unit 13b. Unlike the determination unit 141, the determination unit 141b is based on the timing at which the processing results are input from the sound processing unit 12b. Start timing. Further, when the processing result is input from the non-speech sound processing unit 13b, the determination unit 141b determines whether it is within a predetermined time after inputting the processing result from the sound processing unit 12b, and is within the predetermined time. At this time, the character data which is the processing result of the voice processing unit 12b and the processing result of the non-voice sound processing unit 13b are output to the selection unit 142b.

  When the character data and the non-speech sound processing result are input from the determination unit 141b, the selection unit 142b reads the history database D4 from the storage device 20, and the input character data and each non-speech sound processing result included in the non-speech sound processing result. The frequency corresponding to the voice sound identifier is selected, and the selection result is output to the calculation unit 143b together with the character data and the non-voice sound processing result.

  When the character data, the non-speech sound processing result, and the selection result are input from the selection unit 142b, the calculation unit 143b performs a calculation to weight the score included in the non-speech sound processing result and determines the input non-speech sound. . Specifically, the calculation unit 143b calculates a new score by weighting the score for each non-speech sound identifier candidate, and calculates the non-speech sound identifier having the highest new score as the input non-speech sound. And output to the output unit 15. The computing unit 143b outputs the determined non-speech sound identifier, character data, and non-speech sound processing result to the update unit 144b.

  For example, the calculation unit 143b obtains a new score weighted by a mathematical expression such as Expression (2) for the combination of character data and each non-speech sound identifier. In equation (2), r2 is a predetermined coefficient.

New score = score + frequency × r2 (2)
Here, when character data corresponding to the identifier of the non-speech sound is associated, this character data may be output. For example, when the character data “chime” is associated with the identifier “A sound”, the calculation unit 143b outputs the character data “chime” to the output unit 15, and the identifier “B sound” has the character data “phone”. When they are associated, the calculation unit 143b outputs “telephone”, which is character data, to the output unit.

  The data to be output to the output unit 15 is not limited to character data as long as it is visual data. When the output unit 15 is a plurality of lamps, the non-sound sound is associated with each non-sound sound. When the identifier of the voice sound is determined, the recognition result may be output by turning on a lamp corresponding to the input non-voice sound.

  The update unit 144b updates the frequency of the history database D4 when the non-speech sound identifier, the character data, and the non-speech sound processing result determined from the calculation unit 143b are input. Specifically, the update unit 144b increases the frequency associated with the character data and the identifier of the non-speech sound determined by the calculation unit 143b, and determines the non-speech sound included in the character data and the non-speech sound processing result. The history database D4 is updated so as to reduce the frequency associated with non-speech sound identifiers other than voice sound identifiers.

(Processing in the non-voice sound processing unit)
Processing in the non-speech sound processing unit 13b will be described using the flowchart shown in FIG. The non-speech sound processing unit 13b analyzes sound data input by the analysis unit 131 via the input unit 11, and determines whether or not the volume of the input sound data is greater than or equal to a threshold value (S30). When the volume of the sound data is equal to or higher than the threshold (YES in S30), the analysis unit 131 determines whether the sound data is a non-speech sound (S31). If there is no input of sound data having a volume equal to or higher than the threshold value (NO in S30) and no non-sound sound is included in the audio data (NO in S31), the process returns to step S30 and the process is repeated.

  When non-speech sounds are included in the sound data having a volume equal to or higher than the threshold (YES in S31), the analysis unit 131 starts accumulation of non-speech sounds in the memory. Further, the search unit 132b obtains the similarity between each non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory, and compares the similarity as a score (S32).

  When the non-speech sound similar to the non-speech sound input to the non-speech sound database D2 is included (YES in S33), the analysis unit 131 ends the storage of the input non-speech sound in the memory, The search unit 132b ends the comparison between each non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory (S34). When the non-speech sound identifiers satisfying the conditions after the storage and comparison of data are extracted, the search unit 132b weights the non-speech sound processing results including all the extracted non-speech sound identifiers and scores. The data is output to the unit 14b (S35), the data stored in the memory is discarded (S38), the process returns to step S30, and the process is repeated.

  On the other hand, when the non-speech sound similar to the non-speech sound input to the non-speech sound database D2 is not included (NO in S33), the analysis unit 131 determines that the volume of the subsequently input sound data is a threshold value. It is determined whether or not the storage time of the non-voice sound in the memory is equal to or longer than a predetermined time (S36). If the volume of the new sound data is equal to or higher than the threshold value, or if the storage time of the input non-speech sound in the memory is within the predetermined time, the process returns to step S32 to repeat the data storage and comparison processing.

  When the volume of newly input sound data is less than the threshold value or when the accumulation time is equal to or longer than the predetermined time (YES in S36), the analysis unit 131 ends the accumulation of the input non-speech sound in the memory. Then, the search unit 132b ends the comparison between each non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory (S37). Thereafter, the search unit 132b discards the data stored in the memory (S38).

(Processing in the weighting processing unit)
The process in the weighting process part 14b is demonstrated using the flowchart shown in FIG. The determination unit 141 of the weighting processing unit 14b waits for input of processing results from the speech processing unit 12b and the non-speech sound processing unit 13b, and when character data as a processing result is input from the speech processing unit 12b ( Time measurement is started (S41).

  Thereafter, when a processing result is input from the non-speech sound processing unit 13b (YES in S41), the determination unit 141b determines whether or not it is within a predetermined time after inputting character data from the sound processing unit 12b ( S43).

  When it is within the predetermined time (YES in S43), the selection unit 142b determines whether the history database D4 includes the frequency associated with the input character data and the non-speech sound identifier included in the non-speech sound processing result. Is determined (S44).

  When the frequency associated with the input character data and the identifier of the non-speech sound included in the non-speech sound processing result is included in the history database D4 (YES in S44), the calculation unit 143b executes a weighting process. That is, the computing unit 143b obtains a new score using the frequency that is the weight value for the score of each non-speech sound identifier candidate included in the non-speech sound processing result (S45). Thereafter, the calculation unit 143c determines the identifier of the non-speech sound with the highest score as the input non-speech sound identifier, and outputs the character data corresponding to the non-speech sound identifier as a result (S46). .

  On the other hand, when it is not within the predetermined time (NO in S43) or when the frequency associated with the identifier of each non-speech sound included in the input character data and the non-speech sound processing result is not included in the history database D4 (in S44) NO), the selection unit 142b outputs character data corresponding to the identifier of the non-speech sound having the highest score included in the non-speech sound processing result as result data (S44).

  Further, when the character data corresponding to the voice is determined, the updating unit 144b updates the frequency included in the history database D4 (S47).

  As described above, the sound recognition device 1b according to the second embodiment uses sound acquired within a certain time when converting non-sound sound into visual data. Therefore, it is possible to predict a non-speech sound that is acquired within a predetermined time from the acquisition of the sound, and the accuracy of conversion of the non-speech sound can be improved. Further, since the history database D4 used in the sound recognition device 1b is automatically updated according to use, the accuracy of voice recognition is improved even without processing such as user registration work.

<Third Embodiment>
In the sound recognition apparatus according to the third embodiment, when converting non-sound sound included in sound data into visual data, the sound recognition apparatus is input within a predetermined time in the past (for example, up to 3 minutes before) from the input timing of the non-sound sound. Use non-speech sound recognition results. Here again, a sound other than a voice uttered by a human voice is referred to as a “non-voice sound”. By using the recognition result of the non-speech sound input in the past for the conversion of the non-speech sound into the visual data, the accuracy of the conversion of the non-sound sound into the visual data can be improved.

  As shown in FIG. 9, the sound recognition device 1c according to the third embodiment does not have the voice processing unit 12 as compared with the sound recognition device 1a according to the first embodiment described above with reference to FIG. The difference is that a non-speech sound processing unit 13c is provided instead of the non-sound sound processing unit 13, and a weighting processing unit 14c is provided instead of the weighting processing unit 14. Also, the sound recognition device 1c stores a sound recognition program P3 in place of the sound recognition program P1 and a history database D5 in place of the history database D3 in the storage device 20, as compared with the sound recognition device 1a. Yes. That is, in the sound recognition device 1c, the sound recognition program P3 stored in the storage device 20 is executed, whereby the CPU 10 performs processing as the non-speech sound processing unit 13b and the weighting processing unit 14b.

  The history database D5 is data that associates the first non-speech sound identifier and the second non-speech sound identifier with a weight value used to determine the non-speech sound. Here, the frequency at which the second non-voice sound is input within a predetermined time (for example, within 3 minutes) after the input of the first non-voice sound is used as a weight value.

  In the history database D5 shown in FIG. 10A, the frequency at which a chime sound, which is a non-speech sound with the identifier “A sound”, is input after the non-speech sound with the identifier “F sound” is input. Indicates that the frequency of input of the sound of the telephone with the identifier “B sound” is 35%. In addition, after inputting a non-sound sound with the identifier “D sound”, the frequency of inputting the non-sound sound with the identifier “C sound” is 30%, and the non-sound with the identifier “D sound” is added. This indicates that the frequency at which a voice sound is input is 25%, and the frequency at which a non-speech sound with an identifier “E sound” is input is 10%.

  When the non-speech sound extracted from the sound data having a volume equal to or higher than the threshold by the analysis unit 131 is stored in the memory, the search unit 132c searches the non-speech sound database D2 for the identifier of the non-speech sound stored in the memory. The identifier of the searched non-speech sound is output to the weighting processing unit 14c as a non-speech sound processing result. Specifically, the search unit 132c calculates the similarity between each non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory, and uses this similarity as a score. Further, the search unit 132c extracts all non-speech sound identifiers that meet the predetermined extraction condition as candidates for non-speech sound identifiers. For example, as illustrated in FIG. The identifier and the score are associated with each other to obtain a non-voice sound processing result.

  Here, as conditions for the search unit 132c to extract the identifier of the non-speech sound, (1) extract the identifier of the non-speech sound whose score is equal to or higher than a specific value (for example, 50 or higher), and (2) the score is a predetermined higher rank. Extract non-speech sound identifiers (for example, 3rd or higher), (3) character data with a score within a predetermined range from the 1st score (for example, a score within minus 5 from the 1st score) Extraction etc. can be considered.

  On the other hand, if the identifier of the non-speech sound cannot be retrieved from the non-speech sound database D2, the search unit 132c waits until a new non-sound sound is stored in the memory without outputting the non-speech sound processing result. That is, even when a non-sound sound not included in the non-sound sound database D2 is newly input, the non-sound sound processing unit 13c is similar to the non-sound sound processing unit 13 described above with reference to FIG. It is not added to the database D2.

  The determination unit 141c waits for the non-speech sound processing result input from the non-speech sound processing unit 13c and stores the identifier of the non-speech sound input in the past in the memory. Is input, it is determined whether or not it is within a predetermined time after inputting the result of the non-audio sound processing from the non-audio sound processing unit 13c last time, and the non-audio stored in the memory is stored within the predetermined time. The voice sound identifier (first non-speech sound identifier) and the newly input non-speech sound processing result are output to the selection unit 142c. In addition, the determination unit 141c newly starts timing based on the timing at which a new non-sound sound processing result is input from the non-sound sound processing unit 13c.

  When the selector 142c receives the non-speech sound identifier and the non-speech sound processing result previously input from the determination unit 141b, the selection unit 142c reads the history database D4 from the storage device 20 and reads the non-speech sound identifier and non-sound input last time. The frequency corresponding to each non-speech sound identifier (second non-speech sound identifier) included in the voice sound processing unit is selected, and this selection result is combined with the previously input non-sound sound identifier and non-speech sound processing result. And output to the calculation unit 143c.

  When the identifier of the non-speech sound input last time, the non-speech sound processing unit, and the selection result are input from the selection unit 142c, the calculation unit 143c performs a calculation to weight the score included in the non-speech sound processing result, Determine the input non-speech sound. Specifically, the calculation unit 143c calculates a new score by weighting the score for the candidate for the non-speech sound identifier, and calculates the non-speech sound identifier with the highest new score as the non-speech sound identifier. The identifier is determined and output to the output unit 15. In addition, the calculation unit 143c outputs the determined non-sound sound identifier, the previously input non-sound sound identifier, and the non-sound sound processing result to the update unit 144c. In addition, the calculation unit 143c rewrites the identifier of the first non-speech sound stored in the memory with the identifier of the non-speech sound that has been input last time.

  For example, the computing unit 143c obtains a new score weighted by a mathematical expression such as Expression (3) for the identifier of the non-speech sound input last time and the identifier of each non-speech sound included in the non-speech sound processing result. In Expression (3), r3 is a predetermined coefficient.

New score = score + frequency × r3 (3)
Here, when character data corresponding to the identifier of the non-speech sound is associated, this character data may be output. For example, when the character data “chime” is associated with the identifier “A sound”, the calculation unit 143 c outputs the character data “chime” to the output unit 15, and the character data “phone” is included in the identifier “B sound”. When they are associated, the calculation unit 143c outputs “telephone”, which is character data, to the output unit.

  The data to be output to the output unit 15 is not limited to character data as long as it is visual data. When the output unit 15 is a plurality of lamps, the non-sound sound is associated with each non-sound sound. When the identifier of the voice sound is determined, the recognition result may be output by turning on a lamp corresponding to the input non-voice sound.

  The update unit 144c receives the non-speech sound identifier determined for the non-speech sound input this time, the identifier of the non-speech sound input last time, and the non-speech sound processing result from the calculation unit 143c. Update the frequency of D5. Specifically, the update unit 144c increases the frequency associated with the identifier of the non-speech sound input last time and the identifier of the non-speech sound determined this time, and the identifier and non-speech sound of the non-speech sound input last time The history database D5 is updated so as to reduce the frequency associated with the identifier of the non-speech sound other than the non-speech sound determined this time included in the processing result.

(Processing in the non-voice sound processing unit)
Processing in the non-speech sound processing unit 13c will be described using the flowchart shown in FIG. The non-speech sound processing unit 13c analyzes the sound data input by the analysis unit 131 via the input unit 11, and determines whether or not the volume of the input sound data is greater than or equal to a threshold value (S50). When the volume of the sound data is equal to or higher than the threshold (YES in S50), the analysis unit 131 determines whether the sound data is a non-speech sound (S51). Here, when there is no input of sound data having a volume equal to or higher than the threshold value (NO in S50) and when the sound data does not include a non-speech sound (NO in S51), the process returns to step S50 and the process is repeated.

  When non-speech sounds are included in sound data having a volume equal to or higher than the threshold (YES in S51), the analysis unit 131 starts accumulation of non-speech sounds in the memory. Further, the search unit 132c obtains the similarity between each non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory, and compares the similarity as a score (S52).

  When the non-speech sound similar to the non-speech sound input to the non-speech sound database D2 is included (YES in S53), the analysis unit 131 ends the storage of the input non-speech sound in the memory, The search unit 132c ends the comparison between each non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory (S54). When the non-speech sound identifiers satisfying the conditions after the storage and comparison of data are extracted, the search unit 132b weights the non-speech sound processing results including all the extracted non-speech sound identifiers and scores. The data is output to the unit 14b (S55), the data stored in the memory is discarded (S58), the process returns to step S50, and the process is repeated.

  On the other hand, when the non-speech sound similar to the non-speech sound input to the non-speech sound database D2 is not included (NO in S53), the analysis unit 131 determines that the volume of the sound data subsequently input is a threshold value. It is determined whether or not the storage time of the non-voice sound in the memory is equal to or longer than a predetermined time (S56). If the volume of the new sound data is equal to or greater than the threshold value, or if the storage time of the input non-speech sound in the memory is within the predetermined time, the process returns to step S52, and the data storage and comparison processing is repeated.

  When the volume of newly input sound data is less than the threshold value or when the accumulation time is equal to or longer than the predetermined time (YES in S56), the analysis unit 131 ends the accumulation of the input non-speech sound in the memory. Then, the search unit 132c ends the comparison between each non-speech sound included in the non-speech sound database D2 and the non-speech sound stored in the memory (S57). Thereafter, the search unit 132c discards the data stored in the memory (S58).

(Processing in the weighting processing unit)
The processing in the weighting processing unit 14c will be described using the flowchart shown in FIG. The determination unit 141c of the weighting processing unit 14c waits for input of the non-sound sound processing result from the non-sound sound processing unit 13c, and when the non-sound sound processing result is input from the non-sound sound processing unit 13c (S60). YES), the timing is started (S61). Moreover, the determination part 141c determines whether it is within predetermined time after inputting the last non-voice sound processing result (S62).

  When it is within the predetermined time (YES in S62), the selection unit 142c is associated with the identifier of the non-speech sound input last time and the identifier of each non-speech sound included in the newly input non-speech sound processing result. Is included in the history database D5 (S63). When the history database D5 includes the frequency associated with the identifier of the non-speech sound input last time and the identifier of the non-speech sound included in the newly input non-speech sound processing result (YES in S63), the calculation unit 143c The weighting process is executed. That is, the calculation unit 143c obtains a new score using the frequency that is the weight value for the score of the candidate identifier of each non-speech sound included in the newly input non-speech sound processing unit (S64). Thereafter, the calculation unit 143c determines the identifier of the non-speech sound with the highest score as the input non-speech sound identifier, and outputs the character data corresponding to the non-speech sound as a result (S65).

  On the other hand, when it is not within the predetermined time (NO in S62) or the frequency associated with the identifier of the non-speech sound input last time and the non-speech sound identifier included in the newly input non-speech sound processing result is stored in the history database D5. When not included (NO in S63), the selection unit 142b outputs character data corresponding to the identifier of the non-speech sound having the highest score included in the non-speech sound processing result as result data (S65).

  In addition, when the non-speech sound is determined, the update unit 144c updates the frequency included in the history database D5 (S66).

  As described above, the sound recognition device 1c according to the third embodiment uses the non-sound sound acquired in the past when the non-sound sound is converted into the visual data. Therefore, it is possible to predict a non-speech sound acquired within a certain time from the acquisition of the non-speech sound, and the accuracy of conversion of the non-speech sound can be improved. In addition, since the history database D5 used in the sound recognition device 1c is automatically updated according to use, the accuracy of voice recognition is improved without processing such as user registration work.

  As mentioned above, although this invention was demonstrated in detail using embodiment, this invention is not limited to embodiment described in this specification. The scope of the present invention is determined by the description of the claims and the scope equivalent to the description of the claims.

1a, 1b, 1c ... sound recognition device 10 ... CPU
DESCRIPTION OF SYMBOLS 11 ... Input part 12, 12b ... Voice processing part 13, 13b, 13c ... Non-voice sound processing part 131 ... Analysis part 132, 132b, 132c ... Search part 133 ... Addition part 14, 14b, 14c ... Weighting process part 141, 141b , 141c... Determination unit 142, 142b, 142c... Selection unit 143, 143b, 143c. , D5 ... History database

Claims (7)

A sound recognition apparatus using a history database in which an identifier for identifying a non-sound sound is associated with a frequency at which a predetermined type of sound sound has appeared within a predetermined time from the appearance of the non-sound sound,
When sound data is input, it is determined whether or not the input sound data is a non-speech sound having no voice characteristics, and when the input signal is a non-speech sound, the non-sound sound A non-speech processing unit that associates the identifier with a voice sound;
A voice processing unit that outputs a candidate of a type of voice included in the input sound data when the sound data is input;
In the non-speech processing unit, an identifier is associated with the non-speech sound, and the type of sound is transmitted from the speech processing unit within a predetermined time after the identifier is associated with the non-speech sound. A calculation unit that weights the candidates based on the history database and determines the type of the voice based on the weighted voice type candidates;
A sound recognition apparatus comprising:   The sound recognition apparatus according to claim 1, further comprising an update unit that updates the history database based on the type of the voice determined by the calculation unit. The history database is
The sound recognition apparatus according to claim 1, wherein the voice type is stored as character data. The non-speech sound processing unit is a non-speech sound database that stores the identifier of the non-speech sound and feature data that can identify the sound;
A search unit that searches the feature amount data matching the determined non-speech sound from the feature amount data of the non-speech sound;
4. The sound recognition apparatus according to claim 1, wherein an identifier is associated with the non-speech sound based on a search result of the search unit. When the input signal is a non-speech sound, the non-speech sound processing unit starts storing the non-speech sound in a memory, and the signal input by the search unit and the sound data If the search unit cannot retrieve the feature amount data that matches the determined non-speech sound within a predetermined period, the stored non-speech sound is associated with a new identifier of the non-speech sound. And having an additional part to add to the non-voice sound database;
The sound recognition apparatus according to claim 3. Sound using a history database associating the type of voice sound with the frequency of occurrence of a predetermined non-voice sound within a predetermined time from the voice sound, and a non-voice sound database associating a non-voice identifier with non-sound sound data A recognition device,
When sound data is input, it is determined whether or not the input sound data is a sound sound having a sound characteristic, and when the input signal is a sound sound, A voice processing unit for associating types,
When sound data is input, it is determined whether a non-speech sound having no sound characteristics is input. If the determination result is positive, the identifier of the non-speech sound is determined based on the non-speech sound database. The non-speech sound processing unit for outputting the candidates of
In the speech processing unit, a candidate for the identifier of the speech from the non-speech processing unit is within a predetermined time after the type is associated with the speech sound and the type is associated with the speech sound. A calculation unit that weights the candidates based on the history database and identifies the weighted non-speech sounds based on the candidates when input,
A sound recognition apparatus comprising: A history database associating a first non-speech sound identifier with a frequency at which a second non-speech sound identifier appears within a predetermined time from the first non-speech sound;
When there is an input, it is determined whether or not the input signal is a non-speech sound having no voice characteristics, and when the input signal is a non-speech sound, A non-speech sound processing unit for associating the identifier,
When an identifier is associated with the first non-speech sound, it is determined whether it is within a predetermined time since the identifier is associated with the second non-speech sound. A calculation unit for determining an identifier of the first non-speech sound based on the history database;
A sound recognition apparatus comprising:

Images