JP2009210942A - Voice reproduction system, voice reproduction method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform suitable voice reproduction processing depending on a content which is uttered, such as difficulty of included words, and the quality degree of vocalization, such as ambiguous vocalization. <P>SOLUTION: According to voice reproduction processing (for example, speech speed conversion and repeated reproduction of a specified section) corresponding to a predetermined recognition feature amount, and referring to a voice reproduction processing estimation model, voice recognition of an input voice is performed by a voice recognition means 101, and further featured values (for example, a recognition result character string regarding utterance content, and reliability on the quality degree of the vocalization) are extracted by a recognition featured value extracting means 102. On the basis of the recognition feature amount obtained as a result, voice reproduction processing to be executed is determined by a voice reproduction processing estimation means 103, and the voice reproduction processing is performed by a voice reproduction processing means 104. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an audio reproduction system, a method thereof, and an audio reproduction program. More specifically, the present invention relates to an audio reproduction system, an audio reproduction capable of changing an audio reproduction speed and setting a section for repeating audio reproduction. Method and program.

  Conventional audio reproduction systems are generally used universally in consumer audio reproduction equipment and the like. For example, in recent years, in the so-called "transcription work" of listening to the speech uttered by humans and writing the contents as text sentences manually in the minutes of public meetings such as local governments and the creation of broadcast subtitles, It is used.

  In particular, it is conceivable to make the voice easier to hear by changing the utterance reproduction speed or repeatedly reproducing difficult parts and supporting the transcription work. An example of a conventional audio reproduction system proposed for such a purpose is described in Patent Document 1 and Patent Document 2 below.

  In Patent Document 1, a phrase section sandwiched between silence sections is detected, and for each detected phrase, a predetermined time is determined based on a predetermined expansion function at a predetermined expansion ratio from the start point of the first voiced section in the phrase section. Transform speech speed over. In addition, a technique is described in which after a certain period of time has elapsed, a voice is easily reproduced by providing a voiced segment expansion unit that has a speech speed based on a decreasing function when a certain period of time has elapsed. This is based on an empirical rule that the beginning of the speech is usually fast or it is impossible to predict what word will appear first, and the beginning of the speech is relatively difficult to hear. By reducing the speed, the voice can be easily heard.

Further, in Patent Document 2, a speech speed calculation unit that calculates a speech speed for each partial data for partial speech data that is part of the speech data is provided, and the speech speed of the partial speech data calculated by the speech speed calculation unit is provided. To a speech speed conversion unit for converting the partial voice data to a predetermined desired speech speed. Thus, a technique for reproducing sound so that it can be easily heard is described. This is to make it easy to hear the voice by reproducing the voice whose utterance speed varies depending on the part at a uniform speaking speed as a whole. Furthermore, in Patent Document 3, the pitch frequency information of the whisper is extracted and converted into normal sound. In Patent Document 4, the speech speed is changed based on the intonation of the original sound. .
JP 2003-223200 A JP 2006-317768 A JP 2006-119647 A Japanese Patent Application Laid-Open No. 06-289895

  However, in the conventional audio playback system, it is fixed regardless of the content of the utterance and the quality of the utterance, such as using a simple feature such as whether it is the start part of the utterance or making the speech speed of the entire utterance uniform. Because it only performs audio playback processing based on the heuristics that have been made, it cannot process multiple feature quantities in combination, and the content of spoken words, such as difficulty of words included, or ambiguous utterances Thus, it is not possible to perform an appropriate sound reproduction process according to whether the utterance is good or bad.

  Accordingly, an object of the present invention is to provide an audio reproduction system, an audio reproduction method, and an audio reproduction program that can perform appropriate audio reproduction processing based on the content of an utterance and the quality of the utterance.

  In order to solve the above problem, a speech reproduction system according to the present invention includes a speech recognition unit that recognizes input speech, a recognition feature amount extraction unit that extracts a recognition feature amount from speech recognized by the speech recognition unit, and a recognition feature amount. According to the recognition feature amount extracted by the extracting means, there are provided an audio reproduction process estimating means for selecting a predetermined audio reproduction process, and an audio reproduction processing means for performing the audio reproduction process selected by the audio reproduction process estimating means. It is characterized by that.

  The recognition feature amount extraction means is characterized by extracting linguistic information about the utterance content as a feature amount.

  The recognition feature quantity extraction means is characterized by extracting acoustic information relating to pronunciation ambiguity as a feature quantity.

  The recognition feature quantity extraction means is characterized by extracting the posterior probability as a feature quantity.

  The recognition feature amount estimating means is characterized by combining a plurality of feature amounts and estimating a sound reproduction process.

  Furthermore, a user operation acquisition means for acquiring a user operation, a user operation history storage means for storing a user operation history obtained by the user operation acquisition means, and a recognition feature quantity extraction history extracted by the recognition feature quantity extraction means A speech reproduction process estimation model is generated from the relationship between the recognition feature quantity history storage means for storing the user, the user operation history stored in the user operation history storage means, and the recognition feature quantity stored in the recognition feature history storage means An audio reproduction process estimation model learning means and an audio reproduction process estimation model storage means for storing the audio reproduction process estimation model generated by the audio reproduction process estimation model learning means are provided, and the audio reproduction process estimation means is an audio reproduction process estimation The sound reproduction process is estimated with reference to the sound reproduction process estimation model stored in the model storage means.

  The speech reproduction method according to the present invention is extracted by a speech recognition step for recognizing an input speech, a recognition feature amount extraction step for extracting a recognition feature amount from the speech recognized by the speech recognition step, and a recognition feature amount extraction step. An audio reproduction process estimation step for selecting a predetermined audio reproduction process according to the recognized feature amount; and an audio reproduction process step for performing the audio reproduction process selected by the audio reproduction process estimation step. .

  The recognition feature amount extraction step is characterized in that linguistic information about the utterance content is extracted as a feature amount.

  The recognition feature amount extraction step is characterized in that acoustic information relating to pronunciation ambiguity is extracted as a feature amount.

  The recognition feature quantity extraction step is characterized by extracting the posterior probability as a feature quantity.

  The recognition feature amount estimating step is characterized by estimating a sound reproduction process by combining a plurality of feature amounts.

  Further, a user operation acquisition step for acquiring a user operation, a user operation history storage step for storing a user operation history obtained by the user operation acquisition step, and a recognition feature amount extraction history extracted by the recognition feature amount extraction step A speech reproduction processing estimation model based on the relationship between the recognition feature value history accumulation step for accumulating the user, the user operation history stored in the user operation history accumulation step, and the recognition feature value stored in the recognition feature value history accumulation step. A voice reproduction process estimation model learning step to be generated; and a voice reproduction process estimation model storage step for storing the voice reproduction process estimation model generated in the voice reproduction process estimation model learning step. Refer to the speech reproduction processing estimation model stored in the processing estimation model storage step and And estimating a process.

  The program according to the present invention corresponds to a process for recognizing an input voice, a process for extracting a recognition feature quantity from a voice recognized by the recognition process, and a recognition feature quantity extracted by a process for extracting a recognition feature quantity. Thus, the computer executes a process for selecting a predetermined audio reproduction process and a process for performing the audio reproduction process selected by the process for selecting the audio reproduction process.

  A process for extracting a recognition feature value, and causing the computer to execute a process for extracting linguistic information about the utterance content as a feature value.

  A process of extracting a recognition feature value, and causing a computer to execute a process of extracting acoustic information related to pronunciation ambiguity as a feature value.

  A process for extracting a recognition feature value, which extracts a posterior probability as a feature value, is executed by a computer.

  A process for selecting a sound reproduction process, which is a combination of a plurality of feature amounts and for causing the computer to execute a process for estimating the sound reproduction process.

  Furthermore, the process of acquiring user operations, the process of accumulating user operation history obtained by the process of acquiring user operations, and the recognition feature quantity extraction history extracted by the process of extracting recognition feature quantities are accumulated. A speech reproduction process estimation model from the relationship between the user operation history stored in the process of storing the user operation history and the recognition feature quantity stored in the process of accumulating the recognition feature quantity extraction history And a process for storing the sound reproduction process estimation model generated in the process for generating the sound reproduction process estimation model, and the process for selecting the sound reproduction process is a process for storing the sound reproduction process estimation model. The computer is caused to execute a process for estimating the sound reproduction process with reference to the stored sound reproduction process estimation model.

  According to the present invention, it is possible to perform appropriate audio reproduction processing based on the content of an utterance and the quality of the utterance, and to an audio reproduction apparatus for supporting transcription, and to an audio device such as a television or a radio. Applicable.

  As shown in FIG. 1, the embodiment of the present invention includes a voice recognition unit 101, a recognition feature amount extraction unit 102, a voice reproduction processing estimation unit 103, and a voice reproduction processing unit 104.

Each of these means operates as follows.
The voice recognition unit 101 recognizes a voice to be reproduced.
The recognition feature quantity extraction unit 102 extracts a feature quantity useful for estimating reproduction processing from the voice recognition result obtained from the voice recognition unit 101. Such feature amounts include notation information representing spoken content, reading information, part-of-speech information, language likelihood, other parts of recognition results, tf-idf values calculated using external resources, A posteriori probability (reliability) for each predetermined unit (such as a word, a phrase, or a sentence) indicating whether the utterance is good or bad, or an acoustic likelihood can be used. These can be used not only independently but also as a feature amount in combination.

  Moreover, it can also be used in combination with feature amounts obtained without using speech recognition, such as the speech rate per predetermined unit (number of mora per unit time), S / N ratio, and the like. The audio reproduction process estimation means 103 determines a reproduction process that is predetermined according to the feature quantity for the input feature quantity. For example, feature values (and their combinations) such as slow playback of a specific word (proprietary noun, etc.) or reading part, slow playback or repeated playback when the reliability is below a certain threshold ) And a preferred audio reproduction process corresponding thereto is selected. In the audio reproduction processing means 104, the audio reproduction processing determined by the audio reproduction process estimation means 103 is performed on the audio to be reproduced.

Examples of the audio reproduction process include the following.
1) Speak speed conversion (play slowly or fast),
2) Repeat section playback (specify a section that is difficult to hear and repeat the section playback),
3) Noise suppression processing (non-speech noise suppression processing is performed. The voice part may be distorted),
4) Volume conversion (to increase or decrease the sound),
5) Frequency conversion (to increase or decrease the sound), etc.
It is possible to perform output using visual means, such as displaying the sound-processed sound as it is, as well as displaying the part subjected to the sound-reproduction process in an emphasized manner. .

Next, an overall operation example of the present embodiment will be described in detail with reference to the flowchart of FIG.
First, speech recognition processing is performed on the input speech using the speech recognition means 101 (step A1). As a result, for example, for input speech, the distinction between speech and non-speech segments, recognition result word sequence for speech segments, part of speech / reading information for each word, acoustic likelihood, language likelihood and posterior probability, subordinate candidate word sequence Including word lattices can be obtained. Next, using the recognition feature quantity extraction means 102, a recognition feature quantity for determining an appropriate reproduction process is extracted based on the obtained speech recognition result (step A2). The recognition feature amount includes a feature related to the content of the utterance, for example, the word itself of the recognition result, part of speech information (whether it is a proper noun, etc.), a language likelihood, a feature related to the ease of hearing of the speech, such as reading information and Features related to the quality of speech, such as acoustic likelihood and posterior probability, can be used. These combinations and other characteristics obtained from other than the recognition result, for example, characteristics such as speech speed and S / N ratio can also be combined.

  Subsequently, the audio reproduction process estimation means 103 is used to determine an audio reproduction process corresponding to the recognized feature amount (step A3). Here, based on a predetermined rule, an appropriate audio reproduction process to be applied is determined based on the obtained recognition feature amount. As a predetermined rule, for example, a rule in which a portion with a low posterior probability performs a reproduction process that slows down the speech speed, or a specific part-of-speech part such as a proper noun performs a reproduction process that slows down the speech speed. Can be considered. Finally, the determined audio reproduction process is applied to the input audio using the audio reproduction processing means 104 (step A4).

Next, the effect of this embodiment will be described.
In the present embodiment, feature quantities corresponding to the contents and properties of voice to be reproduced obtained as a result of voice recognition are used, and these feature quantities and a speech reproduction processing estimation model in which processing to be performed on the voice is described are used. Thus, it is possible to perform an appropriate sound reproduction process based on the content and characteristics of the utterance.

Next, another embodiment of the present invention will be described in detail with reference to the drawings.
As shown in FIG. 3, in the present embodiment, in the above-described configuration, the recognized feature amount history storage unit 206, the user operation acquisition unit 207, the user operation history storage unit 208, the voice reproduction process estimation model learning unit 209, and the voice reproduction process The estimated model storage unit 210 is added.

  The recognition feature amount history storage unit 206 recognizes the speech input by the speech recognition unit 101 and the recognition feature amount extraction unit 102, and associates the recognition feature amount obtained as a result with the input speech in time series. To accumulate. The user operation acquisition unit 207 determines what part of the input voice the voice playback processing (for example, whether the playback was performed slowly or fast, whether the volume was increased or decreased, and repeated playback). Get what you gave. The user operation history accumulation unit 208 accumulates the user operations acquired by the user operation acquisition unit 207 in time series in association with the input voice.

  The voice reproduction process estimation model learning unit 209 performs a voice reproduction process performed by the user on the recognition feature quantity for the input voice stored in the recognition feature quantity history storage unit 206 and the input voice stored in the user operation history storage unit 208. Using both the content and the content, a speech reproduction processing estimation model is generated that associates the content of the speech reproduction processing performed by the user with the recognized feature value.

  Specifically, for example, as shown in FIG. 4, based on the frequency of the combination of the voice reproduction processing operation and the recognition feature amount when the operation is performed, the operation of the combination having the frequency equal to or higher than a certain value Associate the feature quantity. This makes it possible to generate an audio reproduction process estimation model. In this process, for example, a machine learning method using an identification model or a generation model can be applied. When a user performs an audio reproduction processing operation, the user may perform an operation by slightly changing the start / end of the audio portion to be processed. On the other hand, when associating the audio playback processing operation with the audio part in which it is performed, it is also possible to perform a process that allows the start and end of the audio part to be shifted and is regarded as the same audio part. it can.

  The sound reproduction processing estimation model generated here can be set for each user, or a plurality of users' models can be integrated and used as a common model. This can be realized, for example, depending on whether the correspondence shown in FIG. 4 is that of a specific user or is counted from a plurality of users. The voice reproduction process estimation unit 103 selects a voice reproduction process by referring to the voice reproduction process estimation model stored in the voice reproduction process estimation model storage unit 210 instead of or simultaneously with a predetermined rule.

  According to the present embodiment, the user operation history and the recognized feature amount at that time can be associated, and a model reflecting the operator's preference and habit can be created and applied, reflecting the operator's preference and habit, Appropriate sound reproduction processing can be performed.

  The present invention has been described above by the preferred embodiments of the present invention. While the invention has been described with reference to specific embodiments, various modifications and changes may be made to the embodiments without departing from the broad spirit and scope of the invention as defined in the claims. Obviously you can.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing a configuration example of the best mode for carrying out a first invention of the present invention. It is a flowchart which shows the operation example of the best form for implementing 1st invention. It is a block diagram which shows the structure of the best form for implementing 2nd invention of this invention. It is a figure which shows the example of matching with user operation and recognition feature-value.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Voice recognition means 102 Recognition feature quantity extraction means 103 Voice reproduction processing estimation means 104 Voice reproduction processing means 206 Recognition feature quantity history accumulation means 207 User operation acquisition means 208 User operation history accumulation means 209 Voice reproduction processing estimation model learning means 210 Voice reproduction Processing estimation model storage means

Claims (18)

Speech recognition means for recognizing input speech;
Recognition feature quantity extraction means for extracting a recognition feature quantity from the voice recognized by the voice recognition means;
A voice reproduction processing estimation unit that selects a predetermined voice reproduction process according to the recognition feature amount extracted by the recognition feature amount extraction unit;
An audio reproduction system comprising audio reproduction processing means for performing an audio reproduction process selected by the audio reproduction process estimation means.   The speech reproduction system according to claim 1, wherein the recognition feature amount extraction unit extracts linguistic information about the utterance content as a feature amount.   3. The voice reproduction system according to claim 1, wherein the recognition feature amount extraction unit extracts acoustic information about pronunciation ambiguity as a feature amount.   The voice reproduction system according to any one of claims 1 to 3, wherein the recognition feature amount extraction unit extracts a posterior probability as a feature amount.   The voice reproduction system according to any one of claims 1 to 4, wherein the recognition feature quantity estimation unit estimates a voice reproduction process by combining a plurality of feature quantities. Furthermore, user operation acquisition means for acquiring a user operation;
User operation history storage means for storing a history of user operations obtained by the user operation acquisition means;
Recognition feature value history storage means for storing the recognition feature value extraction history extracted by the recognition feature value extraction means;
An audio reproduction process estimation model learning unit that generates an audio reproduction process estimation model from a relationship between a user operation history stored in the user operation history accumulation unit and a recognition feature amount stored in the recognition feature amount history accumulation unit;
Voice reproduction processing estimation model storage means for storing the voice reproduction processing estimation model generated by the voice reproduction processing estimation model learning means;
6. The voice reproduction process estimation unit estimates a voice reproduction process with reference to a voice reproduction process estimation model stored in the voice reproduction process estimation model storage unit. The audio playback system described in 1. A speech recognition step for recognizing input speech;
A recognition feature amount extraction step for extracting a recognition feature amount from the speech recognized in the speech recognition step;
A voice reproduction process estimation step for selecting a predetermined voice reproduction process according to the recognition feature quantity extracted in the recognition feature quantity extraction step;
An audio reproduction method comprising: an audio reproduction process step for performing an audio reproduction process selected in the audio reproduction process estimation step.   8. The voice reproduction method according to claim 7, wherein the recognition feature amount extraction step extracts linguistic information about the utterance content as a feature amount.   9. The voice reproduction method according to claim 7, wherein the recognition feature amount extraction step extracts acoustic information relating to pronunciation ambiguity as a feature amount.   10. The audio reproduction method according to claim 7, wherein the recognition feature amount extraction step extracts a posterior probability as a feature amount.   The voice reproduction method according to any one of claims 7 to 10, wherein the recognition feature quantity estimation step estimates a voice reproduction process by combining a plurality of feature quantities. Furthermore, a user operation acquisition step for acquiring a user operation;
A user operation history storage step for storing a history of user operations obtained by the user operation acquisition step;
A recognition feature value history storage step for storing the recognition feature value extraction history extracted in the recognition feature value extraction step;
A voice reproduction process estimation model learning step for generating a voice reproduction process estimation model from the relationship between the user operation history stored in the user operation history accumulation step and the recognition feature quantity stored in the recognition feature quantity history accumulation step. When,
A voice reproduction process estimation model storing step for storing the voice reproduction process estimation model generated in the voice reproduction process estimation model learning step;
12. The sound reproduction process estimating step estimates the sound reproduction process with reference to the sound reproduction process estimation model stored in the sound reproduction process estimation model storing step. The audio reproduction method according to item. Processing to recognize input speech,
A process of extracting a recognition feature amount from the speech recognized by the recognition process;
A process of selecting a predetermined audio reproduction process according to the recognition feature quantity extracted by the process of extracting the recognition feature quantity;
The program which makes a computer perform the process which performs the audio | voice reproduction | regeneration process selected by the process which selects the said audio | voice reproduction | regeneration process.   The program according to claim 13, wherein the computer executes a process of extracting the recognized feature quantity, and extracting linguistic information about the utterance content as a feature quantity.   The program according to claim 13 or 14, wherein the computer executes a process of extracting the recognition feature value, and extracting a piece of acoustic information related to pronunciation ambiguity as a feature value.   The program according to any one of claims 13 to 15, which is a process of extracting the recognized feature quantity, and causing a computer to execute a process of extracting a posterior probability as a feature quantity.   The program according to any one of claims 13 to 16, which is a process of selecting the sound reproduction process, and causing a computer to execute a process of combining a plurality of feature amounts and estimating the sound reproduction process. Furthermore, a process for acquiring a user operation,
A process of accumulating a history of user operations obtained by the process of acquiring the user operations;
Processing for accumulating the recognition feature value extraction history extracted in the processing for extracting the recognition feature value;
A process of generating a speech reproduction process estimation model from the relationship between the user operation history stored in the process of accumulating the user operation history and the recognition feature quantity stored in the process of accumulating the recognition feature quantity extraction history When,
A process of storing the sound reproduction process estimation model generated in the process of generating the sound reproduction process estimation model;
The process for selecting the sound reproduction process causes the computer to execute a process for estimating the sound reproduction process with reference to the sound reproduction process estimation model stored in the process for storing the sound reproduction process estimation model. 18. The program according to any one of items 17.

Images