JP2012215673A - Speech processing device and speech processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce an operation burden.SOLUTION: A speech processing device includes speech conversion means, utterance determination means, processing determination means, and execution means. The speech conversion means converts an input speech to character string information indicating contents represented by the speech. The utterance determination means analyzes the input speech to determine a mode of user's utterance at the time of utterance of the speech. The processing determination means determines processing to be executed, on the basis of the mode of the utterance. The execution means uses the character string information to execute the processing determined by the processing determination means.

Description

  Embodiments described herein relate generally to an audio processing apparatus and an audio processing method.

  2. Description of the Related Art Conventionally, computers such as mobile devices carried by users tend to have higher performance. An interface is important for the user to use the computer comfortably. Therefore, in recent years, a technique has been proposed in which various sensors are built in a computer and the detection results of these sensors are used as user operations or the like.

  For example, a technique has been proposed in which a computer performs processing in accordance with a generated voice command by performing voice recognition processing on a user's voice.

Japanese National Patent Publication No. 11-506845

  However, when a user utters, there are various types of utterances. In the prior art, techniques such as improving speech recognition by restricting the utterance style have been proposed, but use of the difference in the utterance style as a user interface is not considered.

  This invention is made in view of the above, Comprising: It aims at providing the audio processing apparatus and audio | voice processing method which change the process performed based on the difference in the style of a user's utterance.

The speech processing apparatus according to the embodiment includes speech conversion means, speech determination means, process determination means, and execution means. The voice conversion means converts the input voice into character string information indicating the content uttered by the voice. The utterance determination means analyzes the input voice and determines the user's utterance style when the voice is uttered. The process determining means determines a process to be executed based on the utterance style. The execution means executes the process determined by the process determination means using the character string information.
Speech processing apparatus comprising

FIG. 1 is a diagram schematically illustrating an appearance of an information processing apparatus according to the embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of the information processing apparatus according to the embodiment. FIG. 3 is a diagram illustrating a functional configuration realized by a program executed by the information processing apparatus according to the embodiment. FIG. 4 is a diagram illustrating an example of processing performed by the information processing apparatus according to the embodiment when the recognized speech style is normal inflection. FIG. 5 is a diagram illustrating an example of processing performed when the utterance style recognized by the information processing apparatus according to the embodiment has no inflection. FIG. 6 is a flowchart illustrating an overall processing procedure in the information processing apparatus according to the embodiment.

  As an embodiment, an example in which a voice processing device is applied to an information processing device will be described. FIG. 1 is a diagram schematically illustrating the appearance of the information processing apparatus according to the present embodiment. The information processing apparatus 100 is an information processing apparatus having a display screen, and is realized as, for example, a slate terminal (tablet terminal), an electronic book reader, a digital photo frame, or the like.

  The information processing apparatus 100 includes a thin box-shaped casing B, and a display unit 111 is disposed on the upper surface of the casing B. The display unit 111 includes a touch sensor 112 for detecting the position on the display screen touched by the user.

  In addition, the information processing apparatus 100 includes a microphone 113 on the upper surface of the housing B for collecting sound of the external environment. The analog voice input from the microphone 113 is converted into a voice (audio) signal by internal processing. Further, in the information processing apparatus 100, various button switches 114 are arranged on the upper surface of the housing B. Various operations can be performed by pressing these button switches 114.

  FIG. 2 is a block diagram illustrating an example of the configuration of the information processing apparatus 100 according to the embodiment. As shown in FIG. 2, the information processing apparatus 100 includes a CPU 210 (Central Processing Unit), a ROM 211 (Read Only Memory), a RAM 212 (Random Access) in addition to the display unit 111, the touch sensor 112, the microphone 113, and the button switch 114 described above. Memory), storage unit 213, timing unit 214, gyro sensor 215, and communication I / F 216.

  The CPU 210 centrally controls the operation of the information processing apparatus 100. Specifically, the CPU 210 reads various programs stored in the ROM 211 and the storage unit 213, expands them in the work area of the RAM 212, and executes them sequentially, thereby allowing each unit of the information processing apparatus 100 connected via the bus line to Output a control signal. The ROM 211 stores various programs and setting data. The RAM 212 provides a work area for the CPU 210. The storage unit 213 is a large-capacity storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and reads and writes various data such as application programs executed by the CPU 210, characters, moving images, still images, and voices. Remember as much as possible. The clock unit 214 has an RTC (Real Time Clock) function and a function of synchronizing time via a network, and performs time synchronization and clocking. The time when the clock unit 214 synchronizes and clocks is notified to the CPU 10.

  The gyro sensor 215 (gyroscope) is a vibration type angular velocity sensor using MEMS technology, detects the attitude of the information processing apparatus 100 in the XYZ triaxial directions, and outputs the detection result to the CPU 210. The communication I / F 216 is an interface that performs wired or wireless communication according to a predetermined communication protocol under the control of the CPU 210. For example, the communication I / F 216 performs wireless LAN communication via a router or the like under the control of the CPU 210.

  Here, a functional configuration realized by the CPU 210 executing the voice processing program sequentially will be described. FIG. 3 is a diagram illustrating a functional configuration realized by a program executed by the information processing apparatus 100. As illustrated in FIG. 3, the information processing apparatus 100 includes a voice recognition unit 301, an utterance style determination unit 302, an operation target extraction unit 303, a process determination unit 304, a process execution unit 305, an input unit 306, An output unit 307. These functional configurations use an operation target keyword list storage unit 308 and a correspondence list storage unit 309. The operation target keyword list storage unit 308 and the correspondence list storage unit 309 are provided in the storage unit 213 in FIG.

  The voice recognition unit 301 converts the content uttered by the voice from the voice input to the microphone 113 into text data (character string information) represented in a natural language.

  The utterance style determination unit 302 performs signal processing analysis (analysis) on the voice input to the microphone 113 and determines the user's utterance style when the voice is uttered. The utterance style determination unit 302 uses, as a reference, a predetermined sound as a user's utterance style, and the input voice is a voice without inflection, a voice with severe inflection, a high voice, a low voice, a faint voice, and a whispering voice. Of which is determined. Note that one or more utterance modes may be determined.

  The operation target keyword list storage unit 308 stores keywords used as processing targets in the process execution unit 305. As a keyword, for example, a song name of music data stored in the storage unit 213, a player's name, or the like may be used.

  The operation target extraction unit 303 extracts keywords (identification information) to be processed by the process execution unit 305 from the text data converted by the voice recognition unit 301. As this keyword, for example, a search keyword such as a song name of music data or a player's name can be considered. Further, it may be a file name or metadata stored in the information processing apparatus 100. Keywords such as these search keywords, file names, or metadata are determined as objects of processing by the process execution unit 305. It should be noted that when extracting a keyword from text data converted by the speech recognition unit 301, it is not necessary to be the same as a stored file name or the like, as long as it is more than a predetermined standard.

  Further, the operation target extraction unit 303 becomes a processing target from the converted text data, a text that is not determined as a processing target, or a text obtained by removing the first and last particles from a portion that is not determined as a processing target. It may be extracted as a keyword.

  The correspondence list storage unit 309 stores the user's utterance style and the process to be executed in association with each other. Note that the correspondence list storage unit 309 does not limit the target to be associated with the process to be executed to only the user's utterance style. For example, in the correspondence list storage unit 309, the search keyword extracted by the operation target extraction unit 303 may be associated with the process to be executed. As another example, a search keyword, a user's utterance style, and a process to be executed may be associated with each other. Note that the correspondence relationship stored in the correspondence list storage unit 309 may be registered in advance at the time of shipment of the information processing apparatus 100, or may be freely registered afterwards by the user.

  The process determination unit 304 determines a process that uses the identification information extracted from the converted text data, based on the utterance style determined by the utterance style determination unit 302. The process determination unit 304 according to the present embodiment determines based on the correspondence relationship stored in the correspondence list storage unit 309. For example, the process determination unit 304 determines the process associated with the user's utterance style and the correspondence list storage unit 309 as the process executed by the process execution unit 305.

  In addition, the process determination unit 304 executes the process associated with the keyword extracted by the operation target extraction unit 303 and the correspondence list storage unit 309 instead of using only the utterance style as a determination criterion. You may determine as a process.

  Furthermore, the processing determination unit 304 according to the present embodiment associates the utterance style determined by the utterance style determination unit 302 and the keywords extracted by the action target extraction unit 303 with the correspondence list storage unit 309. The process may be determined as a process executed by the process execution unit 305.

  The process determination unit 304 according to the present embodiment may specify an application that performs the process as the determined process. In this case, the process execution unit 305 performs activation control of the identified application and requests the application to execute the process.

  The process execution unit 305 executes the process determined by the process determination unit 304 using the keyword extracted by the operation target extraction unit 303. The information used as the processing target is not limited to the keyword extracted by the operation target extraction unit 303, and may be information based on text data converted by the voice recognition unit 301, for example.

  The process execution unit 305 according to the present embodiment activates the application specified by the process determination unit 304, requests the application to execute the process, and delivers a keyword to the application.

  Note that the process execution unit 305 according to the present embodiment may simply display the process determined by the process determination unit 304 and the keyword extracted by the operation target extraction unit 303 on the display unit 111. Furthermore, such a display may be performed to prompt the user for permission to execute the process.

  The input unit 306 outputs input information input from the communication I / F 216 or the like or the gyro sensor 215 or the like to the process execution unit 305. The output unit 307 may control display of information output from the process execution unit 305 on the display unit 111 or may transmit the information to an external device via the communication I / F 216.

  Processing performed by the information processing apparatus 100 realized with the above-described configuration will be described. For example, it is assumed that the user speaks “XX (singer), Kyoku 1 (song name)”. In this case, the information processing apparatus 100 performs input processing of the utterance using the microphone 113 and performs processing based on the utterance. As a process based on this utterance, for example, music reproduction can be considered.

  By the way, conventionally, there has been an information processing apparatus that automatically performs a database search using text obtained by speech recognition. That is, when the user speaks several words, there is a technique for recognizing the contents of these utterances, converting them into text data, and performing a search using the text data as a search keyword. In such a conventional technique, since the processing to be performed is uniquely set in the Web search, the processing has no degree of freedom. In the above-described example, the search results using “XXXXX (singer), Kyoku 1 (song name)” as search keywords have been displayed.

  Therefore, in the information processing apparatus 100 according to the present embodiment, when generating text from speech recognition, the utterance style is recognized, and the process is switched based on the recognized utterance style.

  FIG. 4 is a diagram illustrating an example of processing performed by the information processing apparatus 100 when the recognized utterance style is normal inflection. As shown in FIG. 4, based on the voice input by the microphone 113 of the information processing apparatus 100, the voice recognition unit 301 converts the text data “XXXXX (singer) Kyoku 1 (song name)” and the utterance style. Assume that the determination unit 302 determines from speech the normal inflection as the utterance style (rich in ups and downs as indicated by arrows). In this case, the process determination unit 304 determines the Web search associated with the normal intonation in the correspondence list storage unit 309 as a process to be executed. As a result, the execution processing unit 305 activates the Web browser. The Web browser displays a search screen 401 in which “XXXXX (singer)” and “Kyoku 1 (song name)” extracted by the operation target extraction unit 303 are set as search keywords. Thereafter, the information processing apparatus 100 starts the search by the Web browser when the touch sensor 112 accepts the pressing of the start button 402.

  FIG. 5 is a diagram illustrating an example of processing performed by the information processing apparatus 100 when the recognized utterance style is not inflected. As shown in FIG. 5, based on the voice input by the microphone 113 of the information processing apparatus 100, the voice recognition unit 301 converts the text data “XXXXX (singer) Kyoku 2 (song name)” and the utterance style. Assume that the determination unit 302 determines from speech that there is no inflection as a speech pattern (flat as indicated by an arrow). In this case, the process determination unit 304 determines that there is no inflection and that the music reproduction (music player) associated with the correspondence list storage unit 309 is performed as a process to be executed. As a result, the execution processing unit 305 activates the music application 501. On the display screen 501 of the music application, music data based on “XXXXX (singer)” and “Kyoku 1 (song name)” extracted by the operation target extraction unit 303 is displayed in a selected state. . Thereafter, the music application automatically reproduces the selected music data.

  As another example, based on the voice input by the microphone 113 of the information processing apparatus 100, the voice recognition unit 301 converts the text data “XXXXX (singer) Kyoku 3 (song name)” and determines the speech style. Assume that the unit 302 determines that a whispering voice is present. The determination of whether or not this whisper is made possible by using, for example, NAM (Non-Audible Murmur) technology.

  Then, the process determination unit 304 determines the search and playback of the video posting site associated with the whisper and the correspondence list storage unit 309 as the process to be executed. As a result, the execution processing unit 305 activates the Web browser. Then, when starting the Web browser, the URL of the video posting site is set as the connection destination. Thereafter, the execution processing unit 305 searches the moving image posting site displayed on the Web browser for “XXXXX (singer)” and “Kyoku 3 (song name)” extracted by the operation target extraction unit 303 as a search keyword. Pass as. As a result, it is possible to search and reproduce on the video posting site using the utterance content input with a whisper as a search keyword.

  As described above, the information processing apparatus 100 according to the present embodiment is not limited to changing the process according to the utterance style, and the process execution unit 305 performs the process according to the keyword extracted by the operation target extraction unit 303. It may be different. For example, when “upper” is recognized as a keyword, the processing execution unit 305 starts to scroll up using “upper” as a trigger and continues to extend the ending “e”. The process 305 continues to scroll up.

  As another example, when “do” is recognized, the process execution unit 305 starts scrolling down with “do” as a trigger and continues to extend the ending “ta” while the process execution unit continues. 305 performs a process of continuing to scroll down. Note that the detection of the utterance extension is possible by using an HMM (Hidden Markov Model), and the description thereof is omitted. Note that these processes may be combined with the above-described utterance style.

  Command input in this utterance style can be applied to any language. For example, in the case of English, when “upperrrrrrr” is recognized, the process execution unit 305 starts up scrolling with “up” as a trigger and continues to extend the ending “r” while the process execution unit continues. The process 305 continues to scroll up. Similarly, when scrolling down, it is assumed that the bottom scrolling continues while “lowerrrr” and ending “r” are extended.

  Next, overall processing in the information processing apparatus 100 according to the present embodiment will be described. FIG. 6 is a flowchart showing the above-described processing procedure in the information processing apparatus 100 according to the present embodiment. Note that it is assumed that preparation for speech recognition by the information processing apparatus 100 is completed when performing the processing illustrated in FIG. 6.

  First, the microphone 113 of the information processing apparatus 100 performs input processing on the user's speech as an audio signal (step S601). Next, the voice recognition unit 301 recognizes the input voice signal and generates text data including a recognition result (hereinafter, recognition result text data) (step S602).

  On the other hand, the speech style determination unit 302 determines the user's speech style from the voice signal (step S603).

  Then, the operation target extraction unit 303 extracts a keyword or the like to be processed from the recognition result text data generated in step S602 (step S604).

  In addition, the process determination unit 304 identifies a process to be executed from the utterance style (step S605). At that time, an application for performing the process to be executed is specified. If the application to be started is a Web browser, the connection destination URL may also be specified.

  Thereafter, the process execution unit 305 activates an application corresponding to the identified process (step S606). Then, the process execution unit 305 executes a process using the extracted keyword on the activated application (step S607). Thereafter, the process execution unit 305 displays the process result on the display unit 111 (step S608).

  With the processing described above, processing according to the utterance style is performed, and the operation burden when the user executes the processing can be reduced. In addition, it does not restrict | limit to the process sequence mentioned above, You may replace the order of each step. For example, step S603 may be executed prior to step S602, or step S602 and step S603 may be executed simultaneously.

  Further, the information processing apparatus 100 according to the present embodiment is not limited to the above-described processing, and various modes can be considered. For example, when the utterance style is a predetermined style, processing for making a note of the uttered content as text may be performed.

  Further, in the information processing apparatus 100, after the application is started, the processing may be switched based on the utterance mode. For example, when inputting a blog sentence by voice, it is conceivable to change the size, color and font of the input character according to the utterance style.

  In addition, the processing executed by the information processing apparatus 100 according to the user's utterance style may be any processing, but Web search, music playback, music search, favorite browsing, blog writing, mail browsing, video posting Site browsing, electronic book browsing, application search, camera activation, camera shooting, telephone calls via the Internet line, etc. may be executed.

(Modification of the first embodiment)
Note that a voice command may be included in a keyword extracted from the voice recognition result text data. Therefore, the operation target extraction unit 303 of the information processing apparatus 100 according to the present modification is an example of extracting a voice command from text data. For example, when the user wants to reproduce the song “Song A” with a voice command, the user may utter “Play Song A” with an unintentional voice. In this case, it is assumed that the keyword “play” and the voice without inflection are associated with the music application (music player) in the correspondence list storage unit 309. Then, if there is a song “Song A” in the storage unit 213, the song A can be reproduced by the song player.

  On the other hand, utterances with intonation are preset in the Web search and correspondence list storage unit 309. As a result, when the “song A playback” is performed with an utterance with an inflection, the information processing apparatus 100 according to the present embodiment performs Web search with priority even if a voice command is included.

  The speech style to be determined by the speech style determination unit 302 according to the present embodiment is not limited to the voice described above, and other speech modes may be used as the determination criterion. For example, an utterance in which the voice pitch continues to rise or an utterance in which the voice pitch continues to fall may be used as the criterion.

  Thus, since the voice command intention is determined not only from the utterance content but also from the utterance method, the command intended by the user can be executed more appropriately.

  In this modification, the speech mode is combined with the voice command based on the user's speech. As a result, in the conventional information processing apparatus, in order to specify the processing, the processing to be executed by tracing the hierarchy from the top is selected (for example, [Menu]-> [Player]-> [Playback selection]- > [Playback]), the process to be executed can be specified by the combination of the voice command and the utterance mode, so that the accuracy of the process to be executed can be improved as compared with the prior art.

  In the present embodiment and the modification, the user can specify processing to be executed by the information processing apparatus 100 only by changing the manner of utterance. Thereby, in the information processing apparatus 100, it can reduce that the process which a user does not intend is performed. Thereby, convenient voice command processing can be performed.

  Furthermore, when the user places the information processing apparatus 100 in a wet place (for example, a kitchen) and specifies the process to be executed on the information processing apparatus 100, the process to be executed in the utterance mode is switched. Therefore, operability can be improved.

  The audio processing program executed in the information processing apparatus according to the present embodiment and the modification is a file in an installable format or an executable format, and is a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk). And the like recorded on a computer-readable recording medium.

  The voice processing program executed by the information processing apparatus of the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The voice processing program program executed by the information processing apparatus according to the present embodiment may be provided or distributed via a network such as the Internet.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, the constituent elements over different embodiments may be appropriately combined.

  DESCRIPTION OF SYMBOLS 100 ... Information processing apparatus, 111 ... Display part, 112 ... Touch sensor, 113 ... Button switch, 113 ... Microphone, 114 ... Button switch, 211 ... ROM, 212 ... RAM, 213 ... Memory | storage part, 214 ... Time measuring part, 215 ... Gyro sensor, 216 ... Communication I / F, 301 ... Voice recognition unit, 302 ... Utterance style determination unit, 303 ... Operation target extraction unit, 304 ... Process determination unit, 305 ... Execution processing unit, 305 ... Process execution unit, 306 ... Input unit, 307 ... output unit, 308 ... operation target keyword list storage unit, 309 ... correspondence list storage unit

The speech processing apparatus according to the embodiment includes speech conversion means, speech determination means, process determination means, and execution means. The voice conversion means converts the input voice into character string information indicating the content uttered by the voice. The utterance determination means analyzes the input voice and determines the user's utterance style when the voice is uttered. The process determining means determines a process to be executed based on the utterance style. Executing means, a process determined by the processing determination unit, and the processing recognized by the character string information, executed according to the period in which the voice is continuing before conversion to the character string information.
The speech processing apparatus according to the embodiment includes speech conversion means and execution means. The voice conversion means converts the input voice into character string information indicating the content uttered by the voice. The execution means executes the process recognized by the character string information according to a period during which the voice before being converted into the character string information is continued.

Claims (6)

Voice conversion means for converting the input voice into character string information indicating the content uttered by the voice;
An utterance determination unit that analyzes the input voice and determines a user's utterance style when the voice is uttered;
Processing determination means for determining processing to be executed based on the utterance style;
Execution means for executing the processing determined by the processing determination means using the character string information;
A speech processing apparatus comprising: The apparatus further comprises storage means for storing the user's utterance style and the process to be executed in association with each other.
The speech processing apparatus according to claim 1. A target extraction unit that extracts identification information that is a target of the process executed by the execution unit from the character string information converted by the voice conversion unit;
The execution means uses the identification information extracted by the target extraction means for the processing.
The speech processing apparatus according to claim 1 or 2. The process determination means determines an application that executes the process,
As the execution means, start the application determined by the processing determination means, and pass the character string information to the application,
The speech processing apparatus according to any one of claims 1 to 3. The utterance determination means determines any one of a voice without inflection, a voice with intense inflection, a high voice, a low voice, a faint voice, and a whisper as a user's utterance style.
The speech processing apparatus according to claim 1. A speech processing method executed by a speech processing apparatus,
A voice conversion step in which the voice conversion means converts the input voice into character string information indicating the content uttered by the voice;
An utterance determination unit analyzes the input voice and determines an utterance mode of the user when the voice is uttered;
A process determining step for determining a process to be executed based on the utterance style;
An execution step in which execution means executes the process determined in the process determination step using the character string information;
An audio processing method including:

Images