JP2003066991A - Method and apparatus for outputting voice recognition result and recording medium with program for outputting and processing voice recognition result recorded thereon - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a character string as a voice recognition result to express the tone and feeling of a speaker. SOLUTION: An apparatus for outputting voice recognition result has a character parameter outputting part 4 for associating time information, volume information and pitch information obtained from analyzed results obtained by a voice analyzing part 2 with the character string as the recognition result in accordance with each time and outputting the associated information as character parameters, and a character generating part 5 for finding speech duration time based on the time information outputted from the character parameter outputting part 4, generating parameters for expressing a speech speed, voice volume and a voice pitch by using the speech duration time, the volume information and the pitch information and generating the character string for expressing the speech speed, the voice volume and the voice pitch based on the parameters.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice information output method for recognizing voice and outputting the recognition result as a character string, a voice information output device, and a recording medium recording a voice recognition result output processing program.

[0002]

2. Description of the Related Art The recognition result recognized by a voice recognition unit is output as text. In order to utilize this recognition result, it is common to display it on a display or print it out from a printer. Here, the text is the character information of only the character code handled by the computer, and refers to the character information to which no character attribute is added. Therefore, when the text is displayed as it is, the speaker's tone (voice pitch, voice volume, voice speed, etc.) is not expressed when it is displayed on the display or printed out from the printer. It's hard to say at all whether he is speaking with such feelings (joy, surprise, discouragement, etc.).

In the case of speech recognition of a plurality of people's utterances and outputting the recognition result like the minutes of a meeting, which person uttered in what situation from the obtained text. I don't know.

Therefore, conventionally, the tone of the speaker is expressed and output in the text output as the voice recognition result,
Some technologies have been proposed that enable the speaker to be identified.

For example, Japanese Patent Laid-Open No. 2001-13993 “Voice Input Information Processing System” (hereinafter, referred to as the first conventional technique) is known as a technique for adding some character attribute to text so as to be able to express the tone of a speaker and outputting the text. Technology) and Japanese Patent Laid-Open No. 11-35290 “Voice output device and control method therefor, computer readable memory” (hereinafter referred to as the second conventional technology). An example is 294798 "Multimedia minutes preparation method and system" (hereinafter referred to as the third prior art). Here, the recognition result itself is referred to as text, and a character string added to this text is referred to as a character string.

[0006]

The above-mentioned first prior art is such that the sound pressure of the input voice is identified to make the character size according to the magnitude of the sound pressure, or the speech speed of the input voice is identified. The character font is set according to the speed. For example, in the case of outputting a character string for that voice by voice input with a word processor etc., the speaker can intentionally change the sound pressure or the voice speed to change the character size and character font accordingly. It's something you want to control.

However, in the first prior art, the character size and font are only changed according to the magnitude of sound pressure and the speech speed, and this alone does not properly express the tone of the speaker. It is not possible to infer what emotion a person speaks with. Further, as described above, when outputting the recognition result of the utterance contents of multiple speakers, it is not possible to specify the speaker, and what speaker uttered in what kind of tone. Is difficult to guess.

In the second prior art, there is a description that determines the attribute (font or color) of a character by using the voice frequency information (pitch information) and the volume information of the input voice. In the related art, when the voice frequency of the input voice is higher than the reference voice frequency, for example, it is regarded as a female voice, the color of the output character is output as pink, and When the volume of the voice is higher than the reference volume, for example, the size of the character to be output is set to 20 points. There is only a description such as determining the size.

Therefore, according to the second prior art, when the contents of a meeting held by a plurality of speakers are output as a minutes, the speakers can be specified from the minutes, and each speaker can be identified. It is thought that it is impossible to express in detail the utterance in such a tone.

The third conventional technique is to specify a speaker in a remote video conference and create a minutes in which a font is changed for each speaker. From the minutes, simply However, it is only possible to know what the speaker said and what the speaker said, and it is not possible to know the tone of each speaker. Further, the speaker identification in the third conventional technique is performed using a terminal or ID card unique to each speaker, and it is necessary to input information for identifying these speakers.

The present invention makes it possible to properly express the tone of the utterance of the speaker with respect to the text output as the voice recognition result, and the tone of the speaker can be seen only by looking at the character string as the output. It is possible to guess the emotions of the speaker due to it and the speaker recognition function, and it has a speaker identification function, which is suitable for creating minutes of meetings by multiple speakers. An object of the present invention is to provide an output device and a recording medium recording a voice recognition result output processing program.

[0012]

In order to achieve the above object, a speech recognition result output method of the present invention analyzes an input speech, performs speech recognition using the speech analysis result, and recognizes the speech. In a voice recognition result output method for outputting a result, at least one information of utterance duration information, volume information, and pitch information is acquired from the voice analysis result, and a parameter expressing a speech speed from the utterance duration information, a volume. At least one of a parameter expressing the loudness of the voice from the information and a parameter expressing the pitch of the voice from the pitch information is generated, and the speech speed and the loudness of the voice are generated by at least one of the parameters. ,
A character string in which at least one of the pitches of voices is expressed is generated.

Further, the speech recognition result output device of the present invention is
In the voice recognition result output device that analyzes the input voice, performs voice recognition using the voice analysis result, and outputs the recognition result, volume information associated with each time information obtained from the voice analysis result. And at least one of pitch information and a text obtained as a recognition result are associated with each other by the time information, and the associated information is output as a character parameter, and the character parameter output means. Obtain the utterance duration information from the time information obtained from, the parameter expressing the speech speed from the utterance duration information, the parameter expressing the volume of the voice from the volume information, the pitch of the voice from the pitch information Generate at least one parameter of the parameters to represent and at least one of those parameters The parameter, speech rate, loudness, at least one of the voice pitch has a configuration having a character generator means for generating a character string expressed.

Further, the recording medium recording the speech recognition result output processing program of the present invention analyzes the inputted speech, performs speech recognition using the speech analysis result, and outputs the recognition result. A recording medium in which an output processing program is recorded, the processing program acquiring at least one information of utterance duration information, voice volume information, and voice pitch information from the voice analysis result; A procedure for generating at least one of a parameter expressing a speech speed from duration information, a parameter expressing a voice volume from the volume information, and a parameter expressing a voice pitch from the pitch information, and those parameters. A character string in which at least one of speech speed, voice volume, and pitch is expressed by at least one parameter in And a procedure for generating.

In each of these inventions, when a character string expressing the speech speed is generated, each character is assigned to that character based on the utterance duration information obtained for each character constituting the character string. On the other hand, the parameter expressing the speech speed is reflected.

Further, when the character string representing the speech speed is generated, the language constituent unit is defined for each language constituent unit on the basis of the utterance duration information obtained in the language constituent unit constituting the character string. On the other hand, the parameter expressing the speech speed is reflected.

Further, the parameter expressing the speech speed generated from the utterance duration information sets the information for setting the character interval of the characters forming the character string and the individual character length in the arrangement direction of the character string. At least one of the information to be set.

Further, when the character string expressing the loudness of the voice is generated, based on the volume information obtained in units of individual characters forming the character string, the character of each character is described above. The parameter expressing the loudness of the voice is reflected.

Further, when the character string representing the loudness of the voice is generated, the language constituent unit is selected for each language constituent unit based on the volume information obtained in the language constituent unit that constitutes the character string. The parameter expressing the loudness of the voice is reflected.

The parameter expressing the loudness of the voice generated from the volume information is information for setting the thickness of the characters forming the character string.

Further, the parameter expressing the loudness of the voice generated from the volume information may be information for setting the density of the characters forming the character string.

Further, when a character string expressing the pitch of the voice is generated, based on pitch information obtained in units of individual characters forming the character string, the above-mentioned The parameter expressing the pitch of the voice is reflected.

Further, when the character string expressing the pitch of the voice is generated, based on the pitch information obtained in the language constituent unit forming the character string, the language constituent unit is changed to the language constituent unit for each language constituent unit. On the other hand, the parameter expressing the pitch of the voice is reflected.

The parameter expressing the pitch of the voice generated from the pitch information is information for setting the density of the characters forming the character string.

Further, the parameter expressing the pitch of the voice generated from the pitch information may be information for setting the thickness of the characters forming the character string.

Further, speaker identification information for identifying each speaker is obtained from the data obtained as a result of the analysis of the voice, and the speaker identification information is used for each speaker. A character string in which at least one of the speech speed, the volume of the voice, and the pitch of the voice is expressed is generated.

In addition to the speaker identification information for specifying the speaker, at least one of the speaker-specific voice speed information, volume information, and pitch information held by each speaker is used.
The possible range of the parameter expressing the speech speed, the possible range of the parameter expressing the loudness of the voice, and the possible range of the parameter expressing the pitch of the voice are normalized.

As described above, according to the present invention, at least one of the parameter for expressing the speech speed from the utterance duration information, the parameter for expressing the loudness of the voice from the volume information, and the parameter for expressing the pitch of the voice from the pitch information. Is generated, and a character string in which at least one of the speech speed, the loudness of the voice, and the pitch of the voice is expressed by at least one of the parameters is generated.

With this, just by looking at the character string,
You can see how the speaker speaks. In particular, if these parameters are used in combination, it is possible to read the speaking rate of the speaker, the loudness and the pitch of the voice from the output character string, and to know the tone of the speaker at that time. Can also infer the speaker's emotions.

Further, by reflecting the parameter expressing the speech speed for each character, for example, when a certain word is uttered, the utterance speed is increased for each character constituting the word. It is possible to read the change in the length, and thereby read the subtle changes in the emotion of the speaker when the word is uttered from the output character string.

Further, as described above, by reflecting the parameter expressing the speech speed for each language constituent unit such as a word, a bunsetsu, a consecutive bunsetsu, a sentence, and a plurality of sentences, for example, a utterance of a certain set of contents is uttered. At times, it is possible to read the change in the utterance speed for each language constituent unit included in the content of the unit, so that when the content of the unit is uttered, the speed of utterance of each language constituent unit can be read. Therefore, subtle changes in the emotion of the speaker can be read from the output character string.

The parameter expressing this speech speed is
Information indicating the character spacing between the characters that make up the character string and the individual character length (called the character width) in the character arrangement direction of the character string
Which is at least one of the information indicating that the character interval is smaller and the character width is smaller as the speech speed is faster.

Since the speech speed is represented by the character spacing and the character width in this way, it is possible to intuitively know whether the speech speed is fast or slow by just looking at the character string. For example, uttering in a delayed tone. It is possible to intuitively read from a character string whether the user is speaking or speaking quickly.

Further, by reflecting a parameter expressing the loudness of the voice for each character,
For example, when a certain word is uttered, it is possible to read the change in the loudness of each character forming the word, and thereby the subtle change in the emotion of the speaker when the word is uttered. It is also possible to read such as from the output character string.

Further, by reflecting the parameter expressing the loudness of the voice for each linguistic constituent unit, for example, when a utterance of a certain unit of content is uttered, the unit of the unit of utterance is uttered. It is possible to read the change in the loudness of each of the language constituent units included in the content, which makes it possible to detect the subtleties of the speaker from the loudness of each of the language constituent units when uttering the content of the unit. It is also possible to read changes in emotions from the output character string.

Further, the parameter expressing the volume of the voice generated from the volume information is information for setting the thickness or the thickness of the characters forming the character string. In this way, the loudness of the voice is expressed by the thickness of the character or the darkness of the character, so it is possible to intuitively see whether the voice is loud or soft by just looking at the character string. When the user speaks, the judgment of the emphasized part can be intuitively read from the character string.

Further, by reflecting the parameter expressing the pitch of the voice for each character, for example, when a certain word is uttered, the voice for each character constituting the word is uttered. It is possible to read the change in the height of the word, and thereby the subtle changes in the emotion of the speaker when the word is uttered can also be read from the output character string.

Further, by reflecting the parameter expressing the pitch of the voice for each linguistic constituent unit, for example, when uttering the content of a certain unit, the unit of the unit It is possible to read changes in voice pitch for each language unit included in the content,
Thereby, when the unit of content is uttered, it is possible to read the subtle changes in the emotion of the speaker from the output character string from the pitch of each language constituent unit.

The parameter expressing the pitch of the voice generated from the volume information is information for setting the darkness or the thickness of the characters forming the character string. In this way, the pitch of a voice is expressed by the density and thickness of the character, so you can intuitively know whether your voice is high or low just by looking at the character string. Can be expressed in columns.

Further, speaker identification information for identifying each speaker is obtained from the analysis result obtained by the voice analysis,
The speaker identification information is used to generate a character string in which at least one of the speech speed, the volume of the voice, and the pitch of the voice is expressed for each speaker.

This is because, for example, the character string for each speaker is color-coded and output, or the character style is changed so that the speaker can see at a glance what character the speaker is speaking. Moreover, for each speaker, at least one of the speech speed, the volume of the voice, and the pitch of the voice is output as a character string with respect to the utterance content of the speaker, With this, it is possible to know at a glance which speaker is speaking in what kind of tone just by looking at the output character string, and it is also possible to infer the emotion of each speaker.

In addition to the speaker identification information for specifying the speaker, the talk speed information peculiar to each speaker,
Using at least one of the volume information and the pitch information, a range of parameters that represent the speech speed of the character string for each speaker, a range of parameters that represent the loudness of the voice, and a parameter that represents the pitch of the voice. It is also possible to normalize the possible range of.

As a result, the range of parameters for expressing the voice speed and the parameters for expressing the voice volume are irrelevant regardless of the voice speed, the volume of the voice, and the pitch of the voice originally possessed by each speaker. The range of parameters that can be taken by the speaker and the range of parameters that express the pitch of the voice can be set in common to all speakers. It is possible to determine the character width, the character thickness corresponding to the loudness of the voice, and the density of the character corresponding to the pitch of the voice.

Therefore, when a plurality of speakers output the minutes obtained by the conference, it is possible to compare the speed of each speaker, the volume of the voice, the pitch of the voice, etc. by the same standard. It will be possible and it will be easier to guess the situation.

[0045]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below. The contents explained in this embodiment are
This is a description of the voice recognition result output method and the voice recognition result output device of the present invention, and also includes the specific processing contents of the processing program in the recording medium recording the voice recognition result output processing program of the present invention.

[First Embodiment] FIG. 1 is a block diagram for explaining a first embodiment of a voice information output device of the present invention. A voice input unit 1, a voice analysis unit 2, and a voice recognition unit 3 are shown. The character parameter output unit 4, the character generation unit 5, and the character output unit 6 are provided.

The voice input unit 1 transfers the voice uttered by the speaker to the voice analysis unit 2. As the voice input to the voice input unit 1, the speaker directs to a microphone or the like on the spot. It may be a spoken voice or a voice recorded in a voice recorder or the like.

The voice analysis unit 2 analyzes the voice passed from the voice input unit 1 and extracts its features. For example, a voice as shown in FIG. 2A input to the voice input unit 1 is analyzed for each frame of a short time (for example, about 20 msec), and the result is analyzed as frame numbers t0, t1,
Output as t2, ..., Tn.

The feature vector, volume information (hereinafter, simply referred to as volume), and pitch information (hereinafter, simply referred to as pitch) as the analysis result extracted here are, as shown in FIG. 2B, a frame number t0 ( Hereinafter, it is referred to as a frame t0, which is the same for the other frame numbers t1, t2, ...). The feature vector c0, the volume v0, the pitch p0, and the feature vector c1, the volume v1 for the frame t1. , Pitch p1 can be obtained as information for each frame t0, t1, t2, ..., Tn.

Of the data shown in FIG. 2B, the feature vector is passed to the voice recognition section 3 together with the frame number, and the volume and pitch are passed to the character parameter output section 4 together with the frame number.

The voice recognition unit 3 performs a voice recognition process using the feature vector sequence for each time passed from the voice analysis unit 2. For example, as shown in FIG. 3A, the frames t0, t1, ...
Based on the feature vectors c0, c1, ..., Cn for each n, as shown in FIG.
Phoneme every 0, t1, ..., Tn, for example, frame t0 is phoneme “s”, frames t1 to t5 is phoneme “o”, frame t6 is phoneme “d”, frame t7 is phoneme “e”. In addition, the phoneme for each frame number is output as a phoneme string, and a text is generated from the phoneme string. In this case, the generated text is "Is it so?", The frames t0 and t1 are "so", the frames t2 to t5 are "u", the frame t3.
By associating t4 with "de", the text "Is that so?" is generated.

Frame t0 shown in FIG. 3 (b),
t1, ..., Tn and their respective frames t0,
The phonemes corresponding to t1, ..., Tn and the text created by these phonemes are passed to the character parameter output unit 4 as a recognition result.

The character parameter output unit 4 is the voice recognition unit 3
Frame number, phoneme, and text passed from (Fig. 3 (b)
4) and the frame number, the volume, and the pitch (see FIG. 2B) passed from the voice analysis unit 2 are integrated using the frame number, and frames t0, t1, ..., T as shown in FIG.
The phoneme, text, volume, and pitch information corresponding to n are passed to the character generation unit 5 as character parameters.

As shown in FIG. 5, the character generation unit 5 uses the various character parameters (see FIG. 4) passed from the character parameter output unit 4 to determine the attributes of the characters (character spacing, character width, character weight, etc.). , A character density, etc.), and a character based on the parameter is generated. The character generated by the character generation unit 5 is the character output unit 6
Is passed to. Hereinafter, the processing of the character generator 5 will be described.

The character generation unit 5 uses the character parameter (see FIG. 4) passed from the character parameter output unit 4 to generate a parameter for determining the character attribute as shown in FIG.

That is, of the character parameters passed from the character parameter output unit 4 as shown in FIG. 4, the frame number and the text (including phoneme information) are used to obtain the utterance duration of each character forming the text. As a parameter for determining a character attribute from the utterance duration, a parameter for determining at least one of a character interval and a character width expressing a speech speed is generated.

Of the character parameters passed from the character parameter output unit 4, the frame number, the text (including the phoneme information), and the volume are used, and the character is extracted from the frame number, the text (including the phoneme information), and the volume. As a parameter for determining the attribute, a parameter for determining the character thickness corresponding to the volume of the voice is generated.

Further, of the character parameters passed from the character parameter output unit 4, the frame number, the text (including the phoneme information) and the pitch information are used, and from the frame number, the text (including the phoneme information) and the pitch information, As a parameter for determining the character attribute, a parameter for determining the density (hereinafter referred to as density) of the character corresponding to the voice pitch is generated.

Then, using the parameters for determining the respective character attributes, a character having the respective character attributes is generated, and this is passed to the character output unit 6, and the character output unit 6
Is output as a character string. Hereinafter, embodiments of the present invention will be described in detail using specific examples.

First, the process of determining at least one of the character interval and the character width from the utterance duration will be described.
First, the case of determining the character spacing will be described. Here, the utterance duration is the difference between the start and end times of the phonemes that form each character, but since this is almost proportional to the number of frames that make up the phoneme, in the following description, The duration is represented by the number of frames. Also, the number of frames corresponding to each phoneme is actually composed of a large number of frames, but here, in order to simplify the explanation,
In particular, short consonants and the like are described as being composed of one frame.

In FIG. 6 (a), looking at "yes" of "yes?", The frame t0 corresponds to the phoneme "s", and the frames t1 to t5 correspond to the phoneme "o". The phoneme "o" of the frames t1 to t5 corresponds to "o" that constitutes "so" and "o" that follows "u".
But also. On the speech recognition side, the frames up to frames t1 to t5 corresponding to the phoneme “o” are the phonemes “o” that make up “so”, and the frames from which correspond to the subsequent “u”. Since it is not possible to delimit whether it is a frame, it is necessary to set the delimitation by some rule.

Here, the frames t1 and t2 are defined as the frames corresponding to the "o" forming "the", and the frame t2
~ T5 is a frame corresponding to "o" forming "u". Therefore, in this case, the utterance duration of "so" corresponds to the frames t0 to t2, and the utterance duration of "u" corresponds to the frames t3 to t5.

In this way, the utterance duration for each character is obtained. This utterance duration indicates the speech speed because the speech speed is high when the speech duration time is short, and the speech speed is slow when the speech duration time is long.

The utterance duration for each character can be used as an utterance duration independent for each character. For example, a certain linguistic constituent unit (this linguistic constituent unit has been described above). The average value of the utterance durations obtained for the individual characters that make up these language building units is calculated for each word, phrase, consecutive phrase, sentence, multiple sentences, etc. It can also be the utterance duration of a language constituent unit.

For example, considering "yes" used in this embodiment, when it is used as an utterance duration independent for each character, the utterance duration of "so",
"U" utterance duration, "de" utterance duration, "su"
Utterance duration, "ka" utterance duration,
The utterance duration for each character can be obtained. In this case, the character interval (character interval between characters) is set for each character, so that the speech speed is expressed for each character.

When used as the utterance duration for each language constituent unit, "so", "u", "de",
The average value of the utterance durations obtained for the individual characters "su" and "ka" can be obtained, and the average value can be regarded as the utterance duration of the language constituent unit "Yes?". In this case, since the character spacing of the characters forming the language constituent unit is set for each language constituent unit, the speech speed is expressed by one language constituent unit.

In this way, when the utterance duration is obtained for each character or for each language constituent unit, the utterance duration is divided into several stages, and the character interval is set for each stage. To decide.

For example, if it is assumed that the utterance durations are set in five stages, such as T1, T2, ..., T5, in the order of shorter utterance durations, as shown in FIG. Prepare a table that can acquire the character spacing according to.

In the example of the table of FIG. 6 (b), the case where the utterance duration time belongs to T3 is standard, and when the obtained utterance duration time belongs to this T3, the standard character spacing (this is d It represents with).

The shorter the utterance duration (the faster the speech speed), the smaller the character interval, and the longer the utterance duration (the slower the speech speed), the larger the character interval. . In the table example of FIG. 6B,
When the utterance duration time belongs to T1, the character interval is d / 4.
If the utterance duration is T2, the character spacing is d / 2, if the utterance duration is T4, the character spacing is 1.5d, and if the utterance duration is T5, the character spacing is 2d.

Therefore, when the character duration is set independently for each character of the utterance duration, that is, when the character interval is set for each character of "Yes?", The utterance continuation of each character is continued. The character interval (interval with the next succeeding character) corresponding to the stage can be obtained from the table of FIG. 6B by simply determining which stage T1 to T5 belongs to.

For example, it is assumed that the speaker utters “yes?” By extending the “yes” portion and then uttering “?” That follows. In this case, the utterance duration of "so" belongs to T4, the utterance duration of "u" also belongs to T4, the utterance duration of "de" belongs to T3, and the utterance duration of "su" and "ka". , Belongs to T1, the character strings generated for the utterance “Is that so?” At this time are character strings in which the respective character intervals are set as shown in FIG.

Further, the character spacing can be set in one language constituent unit. In this case, "Yes?" Is set as one language building block, and the entire characters that make up that language building block are set to a certain character interval. The utterance duration at that time is, as described above, The average value of the utterance durations obtained for each of the characters constituting "" is calculated, and the average value is T1, T in the table of FIG. 6B.
It is determined which utterance duration of 2, ..., T5 belongs to, the character spacing corresponding to the utterance duration is acquired, and the word “yes?” Is generated using the acquired character spacing.

For example, the average value of the utterance durations obtained for the respective characters that make up "Is it so?"
Assuming that it belongs to T2, this “is it” means that the characters “so”, “u”, “de”, “su”, and “ka” are d / 2 characters as shown in FIG. It becomes an interval.

In the above example, the character interval is changed depending on whether the utterance duration of each character is short or long, that is, whether the utterance speed is fast or slow. It is also possible to change the width of one character.

In this case as well, the utterance duration is T
If five levels such as 1, T2, ..., T5 are set, as shown in FIG. 9, a table that can acquire the character width for each utterance duration is prepared.
The character width here means the length of each character in the arrangement direction of the character string in each character.

In the table example of FIG. 8 as well, when the utterance duration time belongs to T3, the standard is used, and when the utterance duration time belongs to s3, the standard character width (this is represented by w) is used. . The shorter the utterance duration (the faster the speech speed), the smaller the character width, and the longer the utterance duration (the slower the speech speed), the larger the character width.

In the table example of FIG. 9, when the utterance duration time belongs to T1, the character width is w / 4, and when the utterance duration time belongs to T2, the character width is w / 2.
If the utterance duration belongs to T4, the character width is 1.5w
When the utterance duration time belongs to T5, the character width is 2w.

For example, using the above example as it is,
It is assumed that the speaker utters "yes?" With the "yes" portion delayed and utters "?" That follows shortly.
In this case, the utterance durations of "so" and "u" are T
4 belongs, the duration of the utterance of "de" belongs to T3, and "su"
Assuming that the utterance duration of "and" belongs to T2, the character string generated for the utterance of "yes" at this time becomes the character string as shown in FIG.

Further, the character width can be set in one language constituent unit. In this case, "Yes?" Is set as one language building block, and the entire characters that make up that language building block are set to a certain character width. The utterance duration at that time is, as described above, "Well? , The average value of the utterance durations obtained for the respective characters constituting the character string is calculated, and the average value is T1, T2, ... In the table of FIG.
., Which of the utterance durations of T5 it belongs to is determined, the character width corresponding to it is acquired, and the word "Yes?" Is output using the acquired character width.

For example, the average value of the utterance durations obtained for each of the characters that make up "Is that so?"
Assuming that it belongs to T2, this "is it?" Is, as shown in FIG. 11, "so", "uu", "de", "su", "ka".
Each character has a width of w / 2.

In the above example, the character interval or character width is determined from the utterance duration, but, for example,
If the utterance duration is short, narrow the character spacing, and
Both may be combined such that the character width is reduced.

Next, a case where the thickness of a character is determined from time, text and volume will be described. The volume used here is an average value of volume obtained for each frame corresponding to each character, and the average value is used as the volume of the character.

For example, considering the character "so", in this example, as shown in FIG. 12A, "so" is the phoneme "s" of the frame t0 and the phoneme "o" of the frame t1.
, The volume v0 of the phoneme "s" corresponding to the frame t0 and the volume v1 of the phoneme "o" corresponding to the frame t1 are calculated and used as the volume of "so". .

Similarly, considering the character "U",
The "u" is composed of phonemes from the phoneme "o" of the frame t2 to the phoneme "o" of the frame t5.
Of the volume v2 of the phoneme "o" corresponding to, the volume v3 of the phoneme "o" corresponding to the frame t3, the volume v4 of the phoneme "o" corresponding to the frame t4, and the volume v5 of the phoneme "o" corresponding to the frame t5. Obtain the average value and use it as the volume of "U".

In this way, the volume of each character is obtained. The volume of each character can be used as an independent volume for each character, or can be used as the volume of a certain language constituent unit, for example.

For example, in the case of "Is that so?" Used in this embodiment, when used as an independent volume for each character, the volume of "So", the volume of "U",
It is possible to obtain the volume for each character such as the volume of “de”, the volume of “su”, and the volume of “ka” and reflect the volume in each character. In this case, since the character thickness is set for each character, the voice volume is expressed for each character.

When used as the volume of one language constituent unit, "so", "u", "de", "su", "ka"
It is also possible to obtain the average value of the volume obtained for each of the above, and regard that average value as the volume of the entire language building block "Is it?" it can. In this case, since the thickness of the characters forming the language constituent unit is set for each language constituent unit, the volume of the voice is expressed by one language constituent unit.

In this way, when the volume is obtained for each character or for each language constituent unit, the volume is divided into several stages and the thickness of the character is determined for each stage. .

For example, the volume is set to V
If V, 1, V2, ..., V5 are set in five stages, a table is prepared which can acquire the thickness of the character corresponding to each stage, as shown in FIG. 12 (b). . Note that such a table may be prepared for each character or each word.

In the example of the table in FIG. 12B, the stage V3 is the standard, and when the obtained volume belongs to this V3, the standard character thickness (this is represented by b) is used. And the louder the volume, the thicker the letters will be,
The lower the volume, the thinner the thickness of the characters.

In the example of the table of FIG. 12B, when the obtained volume belongs to V1, the character thickness is b / 4.
If the calculated volume belongs to V2, the character thickness is b / 2, and if the calculated volume belongs to V3, the character thickness is the standard thickness b, and the calculated volume is Is V
If it belongs to No. 4, the character thickness is set to 1.5b, and if the obtained volume belongs to V5, the character thickness is set to 2b.

Therefore, when the character thickness is set for each character independently, that is, when the character thickness is set for each character of "Yes?", The volume of each character is set. The thickness of the character corresponding to that stage can be obtained from the table of FIG. 12B only by determining which of V1 to V5 it belongs to.

For example, it is assumed that the speaker utters “yes?” In a loud voice by emphasizing the “yes” part, and then utters “?” Followed by a weak tone. in this case,
If the volume of "so" belongs to V5, the volume of "u" belongs to V4, the volume of "de" belongs to V3, and the volumes of "su" and "ka" belong to V1. The character string generated for the utterance "Is it so?" Is a character string as shown in FIG.

According to FIG. 13, the character string "Yes?" Is a character string in which "Yes" is emphasized (particularly the portion "S" is emphasized).

Further, the thickness of characters can be set in one language constituent unit. In this case, "Is it so?"
As one language building unit, the entire characters that make up that language building unit are the thickness of a certain character, and the volume at that time is calculated for each character that makes up "Yes?" The average value of the sound volume is calculated, and the average value is V1, V2, ... In the table of FIG.
.., V5 to which the volume belongs, and the corresponding character thickness is acquired, and the acquired character thickness is used to generate the language structural unit "Is that so?"

For example, suppose that the average value of the volume obtained for each of the characters that make up "is it?" Is V2, this "is it?" , “U”, “de”, “su”, and “ka” are character strings represented by a thickness of b / 2.

Next, the case of determining the character density from the time, the text, and the pitch will be described. With the pitch used here, the average value of the pitches obtained at each time corresponding to each character is obtained, and the average value is taken as the pitch of that character. Note that, here, for convenience, the description is such that the pitch is obtained in all the sections, but in reality, there may be a case where a portion where the pitch cannot be obtained such as unvoiced consonants is excluded. For such a portion where the pitch cannot be obtained, for example, the pitch can be obtained by using the vowel portion before or after the vowel.

For example, considering the character "so", in this example, as shown in FIG. 15A, "so" is a phoneme "s" of frame t0 and a phoneme "o" of frame t1.
, The pitch p0 of the phoneme "s" in the frame t0 and the pitch p1 of the phoneme "o" in the frame t1 are calculated and used as the "so" pitch.

Similarly, considering the character "U",
The "u" is composed of phonemes from the phoneme "o" of the frame t2 to the phoneme "o" of the frame t5.
Pitch p2 of phoneme "o" in frame t3, pitch p3 of phoneme "o" in frame t3, pitch p4 of phoneme "o" in frame t4, phoneme "o" in frame t5
The average value of the pitches p5 of p.

In this way, the pitch for each character is obtained. The pitch for each character can be used as an independent pitch for each character, or can be used, for example, as the pitch of a certain language constituent unit.

For example, in the case of "Is that so?" Used in this embodiment, when used as an independent pitch for each character, the pitch of "So", the pitch of "U", the pitch of "de" , “Su” pitch, “ka” pitch, etc., and the pitch can be obtained for each character and reflected in each character. In this case, since the density is set for each character, the pitch of the voice is expressed for each character.

When used as the volume of one language constituent unit, "so", "u", "de", "su",
The average value of the pitches obtained for each "ka" is calculated, and the average value is regarded as the pitch of one linguistic building unit "is it?" Can be reflected in. In this case, since the density is set for each language constituent unit, the voice pitch is expressed by one language constituent unit.

For example, if it is possible to change the density of characters within the range from D0 to Dn as shown in FIG. 15B, the pitch obtained for each of the characters described above can be obtained in this figure. The position corresponding to the change in density shown in 15 (b) is determined, the density corresponding to the obtained pitch is acquired, and the character is output using the acquired density.

Here, the higher the pitch, the higher the character density (D0 side), and the lower the pitch, the lower the character density (Dn side). Alternatively, the range in which the pitch of the word or sentence can be taken is the range D0 of the change in density shown in FIG.
It is assumed that the density is obtained for each of the pitches obtained in correspondence with Dn.

For example, if the pitch obtained for the character "so" is P1, its PI is shown in FIG. 15 (b).
, If it is associated with D0, the density D
0 is output as a parameter, whereby "so" is created with the density D0 (in this example, the density is maximum).

Further, assuming that the average pitch obtained for the character "U" is P5, that P5 is shown in FIG. 15 (b).
, If it is associated with D2, the density D
5 is output as a parameter, whereby "U" is created with the density D2.

For example, it is assumed that the speaker utters “yes?” With a very high voice in the “yes” part, and utters the following “?” With a medium height.

In this case, the pitches of "so" and "u" are respectively associated with the density D0 of FIG.
If the pitches of "su" and "ka" are associated with the densities Di of FIG. 15B, the output from the character output unit for the utterance of "Yes?" Is as shown in FIG. Become.

According to this FIG. 16, the character string "is it?"
Is expressed as a character string uttered in a voice lower than that.

It is also possible to determine the character density with one language constituent unit. In this case, "Is it so?"
As one linguistic building unit, the entire characters that make up the linguistic building unit are expressed with a certain density, and the pitch at that time was calculated for each character that makes up "Is that so?" An average value obtained by further averaging the average pitches is obtained, which density in FIG. 15 (b) the average value corresponds to is checked, the density corresponding to that is acquired, and the acquired density is used. Is the language unit "?" Output.

For example, by averaging the average pitches obtained for the respective characters that make up "Is so?", The average value of the entire language constituent unit is D0 in FIG.
As shown in FIG. 17, in this case, the entire language constituent unit is represented as a character string expressed in the maximum density.

When determining the density, in the above-mentioned example, the density corresponding to the obtained value of the pitch (average pitch) is obtained, but the density shown in FIG. It is also possible to divide the range into several stages, determine which stage the obtained average pitch value belongs to, and obtain the density corresponding to that stage.

For example, although not shown, the concentration range D0 to Dn shown in FIG. 15B is divided into five stages Q1, Q2 ,.
.., Q5, if the obtained pitch is a value belonging to the stage Q1, the representative value, for example, the density D0 is output as a parameter, and if the obtained average pitch is a value belonging to the stage Q2, It is also possible to obtain the density determined for each stage such that the density D2 is output as a parameter as the representative value.

In this case, a table having the densities corresponding to the respective levels is prepared, it is possible to easily determine the pitch by determining which section the calculated average pitch is and then referring to the table. It is possible to obtain a concentration corresponding to.

When the density is determined for each character based on the pitch, the average of the pitch is calculated for each character and the density of the character is determined according to the average value, but the invention is not limited to this. , It is also possible to determine the density by using the pitch for each phoneme that constitutes each character (the pitch for each time).

Thus, in one character,
It is also possible to change the density within each character, that is, to change the density in the direction of time change (character string direction).

For example, in FIG. 18, "Is that so?" Is gradually raised to a high pitch (the pitch is gradually raised).
This is an example of a character string that is output when a utterance is made, and the density becomes smooth and dark as a whole, and a change in density can be seen in each character.

Here, considering "su",
Since the density is obtained using the value of the pitch obtained for each phoneme corresponding to each time that constitutes "su", in this case, since the pitch is gradually raised,
As shown in FIG. 18, even in one character "su", the character is output as a character whose density gradually increases. The same can be said for other characters in FIG.

As described above, in the first embodiment, the frame number, the text, the volume, and the pitch obtained by the voice analysis are used to change the utterance duration (utterance speed) based on the number of frames to the character interval or the character interval. Acquires the parameter for determining the character width, outputs the character string with character spacing and character width or combination thereof according to the speech speed using this parameter, and acquires the parameter for determining the character thickness from the volume. Then, using this parameter, a character string is output with the thickness of the character according to the volume of the voice, and the parameter that determines the density of each character is obtained from the pitch, and the pitch of the voice is calculated using this parameter. The character string is output with the density of the character according to the size.

As a result, the character string output from the character output unit 6 is not limited to a list of characters representing the recognition result, but the character spacing, the character width, the character thickness, and the character density are different. Character by character, word, phrase,
It is output in a different state for each language constituent unit such as continuous clauses, sentences, and multiple sentences, so it is possible to know whether the speaker uttered in a fast tone or a slow tone depending on the character spacing and character width. Depending on the thickness of the character, whether the speaker uttered a loud voice or a small voice, and depending on the density of the character, whether the speaker uttered a high tone or a low tone, It is possible to know the speaker's tone at that time, and it is possible to infer the speaker's emotions.

It is needless to say that the utterance duration, volume and pitch described above can be used in combination.

For example, if the user speaks with a high speed, a loud voice, and a high voice, it becomes a character string in which thick and dark characters are arranged at narrow intervals, and conversely, the speech speed is slow and the voice is small. Moreover, if you speak with a low voice, it becomes a character string in which thin and low density characters are arranged at wide intervals,
The characteristics of the speaker's tone were well expressed, such that if the user speaks at high speed, in a loud voice, and in a low voice, it will be a character string in which thick and low-density characters are arranged at narrow intervals. It can be output as a character string, and the emotion of the speaker is further expressed.

FIG. 19 is an example of a character string output by combining parameters indicating character attributes obtained by the utterance duration, volume, and pitch.
(A) is a character string of "is it?" Output without applying the present invention, and this is considered as a standard here.

FIG. 19B shows an example of a character string generated by applying the present invention when "Yes?" Is uttered in a strong tone and in a somewhat interrogative form. ”Is thick and the density is slightly dark, so it can be seen that the part was uttered with a loud voice and a high voice. Also, the“ ka ”at the end has a high density and is thick. Since it is expressed, it can be seen that the pitch of that part is high and the voice is loud.

Further, since the density of "" is low and the thickness is close to the standard form, it can be seen that the sound is a little low and has a normal voice volume. Regarding the speech speed, since there is a narrow interval between "de" and "su", the speech speed of that part is slightly faster, and the other parts are slower than the standard speed.

From these facts, it can be inferred that "Yes?" In this case was uttered in the interrogative form of a relatively strong tone, since the ending was treble with a slightly strong tone.

FIG. 19 (c) shows an example of a character string generated by applying the present invention when "Yes?" Is uttered in a slightly discouraged tone. The speed is a slow speed. Especially, the part from "u" to "de" in the "so-de" part is extended, and the "?" Part has a small character density, Since it is also slightly thin, it can be seen that the voice is low and slightly low.

From these things, it can be inferred that "Yes?" In this case was uttered in a slightly disappointed tone.

As described above, the parameters obtained by the utterance duration, the volume, and the pitch are combined and expressed in a character string. By just looking at the character string, the tone of the speaker at that time can be understood, and the speech You will be able to guess the feelings of the person.

Further, since the pitch information also indicates the pitch of the voice, and in particular, the voice quality of men and women can be remarkably expressed. Therefore, it is also possible to express the gender in a character string by using this pitch information. .

For example, if a pitch value as a reference is set in advance and the obtained pitch is equal to or more than the reference value, the speaker is judged to be a woman, and the character color is set to a reddish color, If the obtained pitch is less than the reference value,
The speaker can judge that the character is male, and the characters can be color-coded according to gender, such as black characters. With this, just by looking at the character string output as a recognition result,
You can also know if the speaker is male or female.
Further, when the color cannot be expressed, it is possible to output by changing the font and the like between the male and the female.

As described above, the utterance duration, the volume,
If the pitch is combined and the gender is expressed by the pitch, what kind of speaker, what tone, and feeling can be uttered from the character string output as the recognition result. It will be even easier to understand if there are any.

[Second Embodiment] FIG. 20 is a block diagram for explaining the second embodiment of the present invention, in which a speaker identification unit 7 is added to the block diagram shown in FIG. The difference is that The other parts are the same as those in FIG. 1, and therefore, the same reference numerals are given to the same parts.

The speaker identifying unit 7 uses the frame number, the feature vector, the volume, and the pitch obtained by the voice analyzing unit 2 to identify the speaker, and the speaker identifying information (for example, the speaker identifying information) thus obtained. Information such as speaker A, speaker B, speaker C, etc. that can identify the speaker) and the speaker-specific speed, volume, and pitch of each speaker. The average voice speed as standard voice speed information, the average volume as standard volume information for each speaker, and the average pitch as standard pitch information for each speaker Parameter output unit 4
Pass to.

The average speech speed of each individual speaker,
The average sound volume and the average pitch may be obtained in advance for each speaker, or may be obtained by learning using the result analyzed by the voice analysis unit 2. Good.

As described above, in the second embodiment,
The information input to the character parameter output unit 4 is passed from the speaker identification unit 7 in addition to the recognition result passed from the voice recognition unit 3 and the pitch, volume, and frame number passed from the voice analysis unit 2. New speaker information added.

As shown in FIG. 21A, this speaker information specifies an individual speaker obtained from the frame number, feature vector, volume, and pitch information from the voice analysis unit 2. Information (for example, speaker A, speaker B, speaker C, etc.) and, as shown in FIG. 21 (b), the average speech speed, the average volume, the average pitch, etc. for these respective speakers. Information is passed to the character parameter output unit 4.

Then, the recognition result passed from the voice recognition unit 3, the pitch, the volume, the frame number passed from the voice analysis unit 2 and the speaker information passed from the speaker identification unit 7 are integrated, The integrated information (frame number, text as a recognition result, volume, pitch, speaker information) is used by the character generator 5
Pass to.

As described above, in the second embodiment, since the speaker information is passed to the character generator 5 together with the frame number, the text as the recognition result, the volume, and the pitch, the character string output as the recognition result is It is possible to indicate which speaker the character string is for.

For example, a plurality of speakers (speaker A, speaker B,
In the case where the speaker C, ...) Has a meeting, the recognition result for each utterance content and the information specifying the speaker (speaker A, speaker B, speaker C, ...) Can be associated with.

As a result, for example, it is possible to add information for identifying the speaker to the character string generated by the character generator 5 described in the first embodiment. For example, the color of the character string is changed for each speaker (the character string for the utterance content of speaker A is red, the character string for the utterance content of speaker B is black, the character string for the utterance content of speaker C is blue, etc.). ) Can be understood at a glance which speaker's utterance content.

Further, in the case of a system capable of only displaying in black and white, the font of the character string is changed for each speaker (for example, speaker A is Mincho, speaker B is Gothic, speaker C is italic, etc.). ) Thus, the speakers may be distinguished.

As described above, when the information for identifying the speakers is added to the character string generated by the character generation unit 5, the contents of the conference by the plurality of speakers are voice-recognized, and the recognition result is recorded in the minutes. When output as, the minutes can be seen at a glance as to which speaker uttered.

The character string generated for each speaker is as described in the above-mentioned first embodiment.
It is possible to express the voice speed based on the character spacing, the voice speed based on the character thickness, and the voice speed based on the character spacing. You can see at a glance if you uttered at and can guess the atmosphere of the place at the meeting.

Further, the speaker information includes not only the information for identifying the speaker but also the average speech speed, the average volume, the average pitch, etc. of each speaker, as described above. Then, using the average voice speed, average volume, and average pitch for each individual speaker, the range of parameters that express the voice speed, the range of parameters that express the volume of the voice, the pitch of the voice, After normalizing the range of the parameters that express, the parameter that expresses the speech speed for each speaker, the character thickness that expresses the volume of voice, and the parameter that expresses the pitch of voice. You can also decide the intensity of.

This is because each speaker has different voice quality originally, and there are various people such as a person who originally has a loud voice, a person who originally speaks fast, and a person who originally has a high voice. It is also necessary to be able to express the speech speed, voice volume, and voice pitch with the same criteria, using the maximum possible range of these parameters, without being affected by the voice quality. Because it is.

For example, the information obtained by analyzing the voice of the speaker is directly used for the recognition result of the utterance content in which the speaker having a large voice originally utters a voice of a normal volume for the speaker. If a character string is generated by using a parameter that expresses the loudness of the voice, the character string will be expressed as a thick character as a whole, and the speaker will be Even if you intend to speak in a small voice, it may be expressed with the same character thickness as the loud voices of other speakers.

For example, even if this loud speaker utters a soft voice for the speaker, the above-mentioned table (see FIG. 12 (b)) allows the speaker to output a character with a character thickness of 1.5t. In some cases, this is a thickness that is generally considered to be a slightly loud voice.

On the contrary, even if a speaker who originally has a low voice utters a loud voice for the speaker, the character string is thinner than that when an ordinary person utters with a normal voice volume. Sometimes expressed. For example, even if a speaker who originally speaks in a small voice speaks loudly to the speaker, the above-mentioned table (FIG. 12 (b)) is used.
In some cases, it may be output only as a character with a character thickness t / 2. This is the thickness that is generally said to have been spoken in a slightly soft voice.

This means that, for example, if the character string is entirely thick, the speaker A, and if the character string is entirely thick, the speaker B is used.
In this way, if you look at the output character string, there is an advantage that you can identify the speaker automatically, so you can make use of it, but like the minutes of a meeting, you can output the character string for multiple speakers. In such cases, it is difficult to compare the voice speed, loudness, and pitch based on the same criteria, depending on the voice quality of each speaker, and it is difficult to read the emotion of each speaker in the conference. Becomes

Therefore, the average speech speed, the average volume, and the average pitch of each speaker are used, and the average speech speed, the average volume, and the average pitch of each of these speakers are in the range that can be taken by the parameter indicating the character spacing and the character width. (Speech speed expressible range), Character thickness parameter range (voice volume expression range), Character density parameter range (voice pitch expression range) ) Center or near the center and normalize the expressible range of each speaker's voice speed, voice volume, and pitch, and use each expressible range to the maximum extent. You can express the speaking speed, loudness, and pitch of the speaker.

By this, the range in which each parameter can take the voice speed, the voice volume, and the voice pitch on the same basis is influenced by the voice quality originally possessed by each speaker. It can be expressed with maximum effectiveness,
The tone and feelings of each speaker can be expressed more richly.

The present invention is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the present invention. For example, in each of the above-described embodiments, a parameter that acquires the utterance duration, the volume, and the pitch, and expresses the speech speed from the utterance duration,
A character string that expresses the voice volume from the volume and the voice pitch from the voice pitch, and expresses the voice speed, the voice volume, and the voice pitch with these parameters. However, it is not necessary to use all of them, for example, a character string expressing only the speech speed, a character string expressing only the loudness of the voice, and only the pitch of the voice. You can also decide what you want to express, such as a character string, as needed.

Further, a character interval or a character width is used as a parameter expressing the utterance duration (speaking speed), and a character thickness is used as a parameter expressing the loudness of the voice, and a parameter expressing the pitch of the voice. As an example, the character density is used, but these are only examples, and the present invention is not limited to this, and the utterance duration (speaking speed), the loudness, and the loudness of the voice are other parameters. Can also be expressed by. For example, the density of characters may be used as a parameter expressing the volume of a voice, and the thickness of characters may be used as a parameter expressing the pitch of a voice.

Further, according to the present invention, a processing program in which the processing procedure for implementing the present invention described above is described is created, and the processing program is recorded in a recording medium such as a floppy disk, an optical disk, a hard disk. The present invention also includes a recording medium in which the processing program is recorded. Further, the processing program may be obtained from the network.

[0157]

As described above, according to the present invention, the parameter for expressing the speech speed from the duration of the voice, the parameter for expressing the volume of the voice from the volume, and the parameter for expressing the pitch of the voice from the pitch are used. At least one parameter is generated, and a character string in which at least one of the speech speed, the loudness of the voice, and the pitch of the voice is expressed by at least one of the parameters is generated.

As a result, it is possible to know what tone the speaker is speaking just by looking at the character string output as the recognition result. In particular, if all these parameters are used, the speed of the speaker's utterance from the output character string,
The loudness and pitch of the voice can be read, and the emotion of the speaker at that time can be estimated.

Further, by reflecting the parameters expressing the speech speed, the volume of the voice, and the pitch of the voice for each character, for example, when a certain word is uttered, It is possible to read changes in the speed of speech, changes in voice volume, and changes in voice pitch for each character that makes up a word, and thus the subtle emotions of the speaker when the word is spoken. It is also possible to infer the change in the character string from the output character string.

It is also possible to reflect the character attribute parameters expressing the speech speed, the volume of the voice, and the pitch of the voice for each language constituent unit. For example, when you utter a lump of content,
It is possible to read the change in the utterance speed for each language constituent unit included in the content of the group, and thereby, when the content of the group is uttered, the utterance speed and voice of each language constituent unit. It is also possible to deduce from the output character string the subtle changes in the speaker's emotions based on the loudness of the voice and the pitch of the voice.

From the analysis result obtained by the voice analysis, speaker identification information for identifying each speaker is obtained,
The speaker identification information is used to generate a character string in which at least one of the voice speed, the voice volume, and the voice pitch is expressed for each speaker.

This is done by, for example, color-coding and outputting the character strings for the utterances of each speaker or changing the font so that each speaker can be specified.
It is possible to see at a glance which speaker's utterance content just by looking at the character string, and for each speaker, with respect to the utterance content of that speaker, the speaking speed, loudness, and pitch. At least one of the above is output as a character string that is expressed. With this, it is possible to see at a glance which speaker is speaking in what situation just by looking at the output character string. .

In addition to the speaker identification information for specifying the speaker, the talk speed information peculiar to each speaker,
Using at least one of the volume information and the pitch information, a range of parameters that represent the speech speed of the character string for each speaker, a range of parameters that represent the loudness of the voice, and a parameter that represents the pitch of the voice. Can be normalized so that the same criteria can be used, without being affected by the voice quality of the individual speaker.
It is possible to express the speech speed, the loudness of the voice, and the pitch of the voice by maximizing the range that each parameter can take, and to express the tone and emotion of each speaker more richly.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a voice recognition result output device for explaining a first embodiment of the present invention.

2A is a diagram showing an example of a voice input to a voice input unit 1 of FIG. 1, and FIG. 2B is a feature vector sequence as an analysis result obtained by voice analysis of the voice, It is a figure which matches each information of a volume and a pitch with each time.

FIG. 3 is a diagram illustrating an example of a voice recognition process in a voice recognition unit 3 using the voice analysis result shown in FIG.

FIG. 4 is a diagram showing an example in which a frame number, a volume, and a pitch obtained by the voice analysis result shown in FIG. 2 are integrated with a recognition result obtained by a voice recognition unit, and a frame number, a text, a volume, and a pitch are associated with each other. Is.

FIG. 5 is a diagram illustrating acquisition of character parameters necessary for setting character spacing, character width, character thickness, and character density performed by the character generator 5 based on time, text, volume, and pitch shown in FIG. It is a figure explaining.

6A and 6B are diagrams illustrating setting of a character interval and a character width for expressing a speech speed from an utterance duration, FIG. 6A is a diagram illustrating processing for obtaining an utterance duration, and FIG. It is a figure which shows the example of a table which can acquire a character space by making a speech continuation time into a key.

FIG. 7 is a diagram showing an example of a character string in which a character interval is set for each character according to the utterance duration obtained for each character.

FIG. 8 is a diagram showing an example of a character string in which a character interval is set for the entire word according to the utterance duration obtained for each word.

FIG. 9 is a diagram showing an example of a table in which the character width can be acquired with the utterance duration as a key.

FIG. 10 is a diagram showing an example of a character string in which a character width is set for each character by using the table of FIG.

11 is a diagram showing an example of a character string in which a character width is set for each word by using the table of FIG.

12A and 12B are diagrams illustrating setting of character thickness for expressing the volume of a voice from volume, FIG. 12A is a diagram illustrating a process of obtaining volume, and FIG. It is a figure which shows the example of a table which can acquire the character thickness.

FIG. 13 is a diagram showing an example of a character string in which the character thickness is set for each character according to the volume obtained for each character.

FIG. 14 is a diagram showing an example of a character string in which the character weight is set for the entire word according to the volume obtained for each word.

15A and 15B are diagrams illustrating setting of the density of characters for expressing the pitch of a voice from the pitch, FIG. 15A is a diagram illustrating a process of obtaining a pitch, and FIG. It is a figure explaining correspondence of a range and a pitch.

FIG. 16 is a diagram showing an example of a character string in which the character density is set for each character according to the pitch obtained for each character.

FIG. 17 is a diagram showing an example of a character string in which the density of characters is set for the entire word by the pitch obtained for each word.

FIG. 18 is a diagram showing an example of a character string in which the density is set in each character by the pitch obtained for each character at each time.

FIG. 19 is a diagram illustrating an example of a character string in which parameters are obtained by combining parameters obtained by the utterance duration, volume, and pitch.

FIG. 20 is a configuration diagram of a voice recognition result output device for explaining a second embodiment of the present invention. And

FIG. 21 is a diagram showing an example of data from the voice analysis unit 2 necessary for speaker identification, and an average voice speed, an average volume, and an average pitch corresponding to each speaker, as speaker-specific information.

[Explanation of symbols]

1 voice input unit 2 voice analysis unit 3 voice recognition unit 4 character parameter output unit 5 character generation unit 6 character output unit 7 speaker identification units t0, t1, ..., Tn frame numbers v0, v1 ,. Volume information p0, p1, ..., Pn Pitch information T1, T2, ..., T5 Obtained utterance duration stage d Character spacing w Character width V1, V2, ..., V5 Obtained volume information Stage b Character thickness P1, P2, ... Obtained pitch information D0, D1, ..., Dn Density

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Hiroshi Hasegawa             Seiko, 3-3-3 Yamato, Suwa City, Nagano Prefecture             -In Epson Corporation F-term (reference) 5D015 AA06 CC12 CC14 CC15 FF03                       HH00 LL05

Claims (42)

[Claims] 1. A voice recognition result output method for analyzing input voice, performing voice recognition using the voice analysis result, and outputting the recognition result, comprising: utterance duration information and volume based on the voice analysis result. information,
A parameter that acquires at least one piece of pitch information, and expresses a speech speed from the utterance duration information,
At least one of a parameter expressing the volume of voice from the volume information and a parameter expressing the pitch of voice from the pitch information
At least one of the speech speed, the loudness, and the pitch of the voice is generated by at least one of the parameters.
A method for outputting a speech recognition result, characterized in that a character string in which one is expressed is generated. 2. When the character string representing the speech speed is generated, the character is described for each character based on the utterance duration information obtained for each character constituting the character string. The method for outputting a voice recognition result according to claim 1, wherein a parameter expressing a speech speed is reflected. 3. When generating a character string representing the speech speed, based on utterance voice duration information obtained in a language constituent unit such as a word, a clause, a continuous clause, a sentence, or a plurality of sentences that constitute the character string. The speech recognition result output method according to claim 1, wherein a parameter expressing the speech speed is reflected in the language constituent unit for each language constituent unit. 4. The parameter expressing the speech speed generated from the utterance duration information sets information for setting the character spacing of the characters forming the character string and individual character lengths in the arrangement direction of the character string. 4. The voice recognition result output method according to claim 1, wherein the voice recognition result output method is at least one of the following information. 5. When generating a character string expressing the loudness of the voice, based on the volume information obtained in units of individual characters forming the character string, the character is described for each character. The voice recognition result output method according to claim 1, wherein a parameter expressing the volume of the voice is reflected. 6. When a character string representing the loudness of the voice is generated, the language constituent unit is selected for each language constituent unit based on the volume information obtained in the language constituent unit that constitutes the character string. The voice recognition result output method according to claim 1, wherein a parameter expressing the volume of the voice is reflected. 7. The parameter expressing the loudness of a voice generated from the volume information is information for setting the thickness of characters forming the character string.
5. The voice recognition result output method according to any one of 5 and 6. 8. The parameter expressing the volume of a voice generated from the volume information is information for setting the density of characters forming the character string.
5. The voice recognition result output method according to any one of 5 and 6. 9. When generating a character string expressing the pitch of the voice, based on pitch information obtained for each character constituting the character string, for each character, The voice recognition result output method according to claim 1, wherein a parameter expressing the pitch of the voice is reflected. 10. When a character string representing the pitch of the voice is generated, the language constituent unit is assigned to each language constituent unit on the basis of pitch information obtained in the language constituent unit that constitutes the character string. The voice recognition result output method according to claim 1, wherein a parameter expressing the pitch of the voice is reflected. 11. The parameter representing the pitch of a voice generated from the pitch information is information for setting the density of characters forming the character string. The voice recognition result output method according to any one of 1. 12. The parameter expressing the pitch of a voice generated from the pitch information is information for setting the thickness of characters forming the character string. The voice recognition result output method according to any one of 1. 13. Speaker identification information for identifying an individual speaker is obtained from the data obtained by the analysis result of the voice, and the speaker identification information is used for each speaker. 13. The voice recognition result output method according to claim 1, wherein a character string in which at least one of the speech speed, the volume of the voice, and the pitch of the voice is expressed is generated. 14. In addition to speaker identification information for specifying the speaker, at least one of speaker-specific voice speed information, volume information, and pitch information held by each speaker is used,
14. The normal range of a parameter expressing a speech rate, the range of a parameter expressing a loudness of a voice, and the range of a parameter expressing a pitch of a voice are normalized. Speech recognition result output method. 15. A voice recognition result output device which analyzes input voice, performs voice recognition using the voice analysis result, and outputs the recognition result, corresponding to each time information obtained from the voice analysis result. At least one of the attached volume information and pitch information, and a text obtained as a recognition result are associated with each other by the time information, and a character parameter output means for outputting the associated information as a character parameter, Obtaining speech duration information from the time information obtained from the character parameter output means, a parameter expressing the speech speed from the speech duration information, a parameter expressing the volume of the voice from the volume information, from the pitch information Generate at least one of the parameters expressing the pitch of the voice, and By at least one parameter, speech rate, loudness, the voice recognition result output device, characterized in that at least one of the voice pitch having a character generator means for generating a character string expressed. 16. The character generation unit, when generating a character string expressing the speech speed, for each character based on utterance duration information obtained for each character that constitutes the character string. 16. The voice recognition result output device according to claim 15, wherein a parameter expressing the speech speed is reflected on the character. 17. The character generation unit, when generating a character string expressing the speech speed, for each language constituent unit based on utterance duration information obtained in the language constituent unit that constitutes the character string. 16. The speech recognition result output device according to claim 15, wherein the parameter expressing the speech speed is reflected in the language constituent unit. 18. The parameter for expressing the speech speed generated from the utterance duration information sets information for setting the character interval between the characters forming the character string and individual character lengths in the arrangement direction of the character string. 18. The voice recognition result output device according to claim 15, wherein the voice recognition result output device is at least one of information to be output. 19. The character generation unit, when generating a character string expressing the loudness of the voice, for each character based on volume information obtained in units of individual characters forming the character string. 16. The voice recognition result output device according to claim 15, wherein a parameter expressing the loudness of the voice is reflected on the character. 20. The character generating unit, when generating a character string expressing the loudness of the voice, for each language constituent unit based on volume information obtained in the language constituent unit that constitutes the character string. 16. The speech recognition result output device according to claim 15, wherein a parameter expressing the loudness of the voice is reflected in the language constituent unit. 21. The parameter representing the loudness of a voice generated from the volume information is information for setting the thickness of a character forming the character string.
5. The voice recognition result output device according to any one of 5, 19, and 20. 22. The parameter expressing the loudness of the voice generated from the volume information is information for setting the density of the characters forming the character string.
5. The voice recognition result output device according to any one of 5, 19, and 20. 23. In the case of generating a character string expressing the pitch of the voice, the character generating unit, for each character, based on pitch information obtained in units of individual characters forming the character string. 16. The voice recognition result output device according to claim 15, wherein a parameter expressing the pitch of the voice is reflected on the character. 24. In the case of generating a character string expressing the pitch of the voice, the character generation unit, based on pitch information obtained in the language structural unit that constitutes the character string, for each language structural unit. 16. The voice recognition result output device according to claim 15, wherein the parameter expressing the pitch of the voice is reflected in the language constituent unit. 25. The parameter representing the pitch of a voice generated from the pitch information is information for setting the density of characters forming the character string.
5. The voice recognition result output device according to any one of 5, 23, and 24. 26. The parameter expressing the pitch of a voice generated from the pitch information is information for setting the thickness of a character forming the character string.
5. The voice recognition result output device according to any one of 5, 23, and 24. 27. A speaker identifying unit for identifying an individual speaker from data obtained by the analysis result of the voice is used, and the speaker identifying information identified by the speaker identifying unit is used. 27. A character string representing at least one of the speech speed, the loudness of the voice, and the pitch of the voice is generated for each speaker, and the character string is generated. Speech recognition result output device. 28. In addition to speaker identification information for specifying the speaker, at least one of speaker-specific voice speed information, volume information, and pitch information held by each speaker is used,
28. The range of possible parameters for expressing the speed of speech, the range of possible parameters for expressing the loudness of voice, and the range of possible parameters for expressing the pitch of voice are normalized. Speech recognition result output device. 29. A recording medium which records a voice recognition result output processing program for analyzing input voice, performing voice recognition using the voice analysis result, and outputting the recognition result, wherein the processing program is A procedure for obtaining at least one piece of information of speech duration information, speech volume information, and speech pitch information from the speech analysis result, and a parameter expressing a speech speed from the speech duration time information,
A procedure for generating at least one of a parameter expressing the loudness of the voice from the volume information and a parameter expressing the pitch of the voice from the pitch information, and a speech speed according to at least one of the parameters. , At least one of loudness and pitch
A recording medium on which a voice recognition result output processing program is recorded, including a procedure for generating a character string in which one is expressed. 30. In the case of generating a character string representing the speech speed, the character is described for each character based on utterance duration information obtained for each character forming the character string. 30. The recording medium recording the voice recognition result output processing program according to claim 29, wherein a parameter expressing a voice speed is reflected. 31. In the case of generating a character string representing the speech speed, the language constituent unit is set for each language constituent unit based on the utterance duration information obtained in the language constituent unit that constitutes the character string. 30. The recording medium recording the voice recognition result output processing program according to claim 29, wherein the parameter expressing the speech speed is reflected. 32. The character attribute parameter expressing the speech speed generated from the utterance duration information is information for setting a character interval between characters forming the character string and individual character lengths in a character string arrangement direction. 32. The information is at least one of information for setting
A recording medium recording the voice recognition result output processing program according to any one of 1. 33. In the case of generating a character string representing the loudness of the voice, based on volume information obtained for each character constituting the character string, for each character, 30. The recording medium recording the voice recognition result output processing program according to claim 29, wherein a parameter expressing the volume of the voice is reflected. 34. In the case of generating a character string expressing the loudness of the voice, based on the volume information obtained in the language constituent unit forming the character string, the language constituent unit is set to each language constituent unit. 30. The recording medium recording the voice recognition result output processing program according to claim 29, wherein a parameter expressing the volume of the voice is reflected. 35. The parameter expressing the loudness of the voice generated from the volume information is information for setting the thickness of the characters forming the character string.
A recording medium recording the voice recognition result output processing program according to any one of 9, 33, and 34. 36. The parameter expressing the volume of a voice generated from the volume information is information for setting the darkness of the characters forming the character string.
A recording medium recording the voice recognition result output processing program according to any one of 9, 33, and 34. 37. When a character string representing the pitch of the voice is generated, the character is described for each character based on pitch information obtained for each character that constitutes the character string. 30. The recording medium recording the voice recognition result output processing program according to claim 29, wherein a parameter expressing the pitch of a voice is reflected. 38. In the case of generating a character string representing the pitch of the voice, based on pitch information obtained in the language constituent unit forming the character string, the language constituent unit is set to each language constituent unit. 30. The recording medium recording the voice recognition result output processing program according to claim 29, wherein a parameter expressing the voice pitch is reflected. 39. The parameter expressing the pitch of a voice generated from the pitch information is information for setting the density of characters forming the character string.
A recording medium recording the voice recognition result output processing program according to any one of 9, 37, and 38. 40. The parameter expressing the pitch of a voice generated from the pitch information is information for setting the thickness of characters forming the character string.
A recording medium recording the voice recognition result output processing program according to any one of 9, 37, and 38. 41. Speaker identification information for identifying an individual speaker is obtained from the data obtained by the analysis result of the voice, and the speaker identification information is used for each speaker. 41. The voice recognition result output processing program according to claim 29, wherein a character string in which at least one of the speech speed, the volume of the voice, and the pitch of the voice is expressed is generated. Recording medium. 42. In addition to speaker identification information for identifying the speaker, at least one of speaker-specific voice speed information, volume information, and pitch information held by each speaker is used,
42. The possible range of a parameter expressing a speech speed, the possible range of a parameter expressing a loudness of a voice, and the possible range of a parameter expressing a voice pitch are normalized. A recording medium in which a voice recognition result output processing program is recorded.