JP2010224392A - Utterance support device, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To relieve a user using a speech synthesis device from the psychological sense of self-doubt by directly feeling the vibration in response to the synthesized speech in conformity with the timing of speech utterance. <P>SOLUTION: The utterance support device includes: a speech waveform generation means generating a speech waveform based on a text inputted through a text input means; a waveform power calculation means calculating a waveform power value for each prescribed time section based on the speech waveform from the speech waveform generation means; a physical vibration conversion means obtaining physical vibration quantity in response to the waveform power value of each time section from the waveform power calculation means; and a vibration conduction means imparting to the utterer the vibration based on the physical vibration quantity in response to the waveform power value of each time section. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an utterance assisting device, method, and program, and is applied to, for example, an apparatus that assists a user who has excised vocal cords so as to improve psychological loss of confidence when speaking using a speech synthesizer. To get.

  For example, there is a speech synthesizer that converts input text into speech and outputs synthesized speech, and such speech synthesizer is used in various fields of use.

  As a conventional speech synthesis method, there is a technique described in Patent Document 1, for example. As described in Patent Document 1, in the conventional speech synthesis method, the language processing unit performs morphological analysis and syntax analysis of the input Japanese text to generate a kana character string with an accent position. In the prosody pattern generation unit, prosodic information (for example, duration of individual phonemes, voice pitch in each time interval (frame), and silence) from a kana character string with accent position from the language processing unit 201 If there is a section, the duration is calculated). Then, the phoneme waveform generation unit generates and outputs a synthesized speech waveform based on the prosodic information from the prosody generation unit 20.

JP 2003-208188 A

  By the way, as a usage mode of the speech synthesizer, for example, a person who has excised the vocal cords due to illness or accident may use the speech synthesizer.

  For example, you can record your own voice and create a database of voice data before excision of vocal cords, and output synthesized voice with your own voice (voice quality) and your own tone (speaking method). is there. As a result, it is possible to reproduce the identity of the user, such as personal characteristics, emotional expression, and naturalness of speech.

  However, in such a usage mode, the following problems may occur.

  Normally, a healthy person feels not only the speaker's own ears but also vibrations transmitted directly from the throat, cheekbones, and the like, thereby listening to his / her speech.

  However, when a person who has excised the vocal cords outputs synthesized speech that reproduces his / her identity using a speech synthesizer as described above, even if he / she can hear the utterance sound from his / her ears, I can't feel the vibrations.

  The voice of one's own life is usually conducted to the bone, and the voice reaching the ear by vibrating the eardrum is prioritized by the voice from the ear. Synthetic speech, on the other hand, is a voice that is heard from the ear as the voice uttered from the mouth, like other people's voice, and there is a gap between it and the image of its own voice. It can happen. In addition, there may be a gap between the image of “your voice” due to imperfection of the synthesized speech. For example, with the deterioration of the quality of the synthesized speech, the voice quality may not be unique, and the way of speaking and taking time may differ.

  In such a case, it often happens that the speaker feels uncomfortable and cannot speak with confidence.

  Compensation of information based on human speech / cognitive mechanisms is effective as an improvement measure against such psychological loss of confidence resulting from the imperfection of synthesized speech.

  In other words, if the vibration directly transmitted from the throat and cheekbones at the time of speech utterance is artificially generated from the synthesized voice according to the synthesized voice generation timing, and this can be felt at the synthesized voice output timing, It is possible to obtain a strong feeling that the user is speaking, and to reduce the feeling of loss of confidence due to the imperfection of the synthesized speech.

  Therefore, the user who uses the speech synthesizer can directly feel the vibration corresponding to the synthesized speech in accordance with the timing at the time of speech utterance, and can reduce the feeling of loss of psychological confidence. And a program is needed.

  The speech assisting device according to the first aspect of the present invention includes: (1) a speech waveform generation unit that generates a speech waveform based on text input through a text input unit; and (2) a predetermined waveform based on a speech waveform from the speech waveform generation unit. Waveform power calculating means for calculating a waveform power value for each time interval of (3), (3) physical vibration converting means for obtaining a physical vibration amount corresponding to the waveform power value of each time interval from the waveform power calculating means, and (4) Vibration conduction means for providing a speaker with vibration based on the amount of physical vibration corresponding to the waveform power of each time section.

  The speech assisting method of the second aspect of the present invention is an speech assisting method of an speech assisting device comprising a text input means and a vibration conducting means. (1) The speech waveform generating means is based on text input through the text input means. (2) a waveform power calculation step in which the waveform power calculation means calculates a waveform power value for each predetermined time interval based on the voice waveform from the voice waveform generation means; 3) The physical vibration converting means has a physical vibration converting step of obtaining a physical vibration amount corresponding to the waveform power value of each time section from the waveform power calculating means and supplying the physical vibration amount to the vibration conducting means. Features.

  According to a third aspect of the present invention, there is provided a speech assist program comprising: (1) a speech waveform generating means for generating a speech waveform based on text input through a text input means; 2) Waveform power calculation means for calculating a waveform power value for each predetermined time interval based on the voice waveform from the voice waveform generation means; and (3) Physical corresponding to the waveform power value of each time interval from the waveform power calculation means. The vibration amount is obtained, and this physical vibration amount is made to function as a physical vibration conversion means for supplying the vibration conduction means.

  ADVANTAGE OF THE INVENTION According to this invention, the user who uses a speech synthesizer can directly feel the vibration according to a synthetic | combination voice according to the timing at the time of speech utterance, and can reduce a psychological loss of self-confidence.

It is a block diagram which shows the structure of the speech assistance apparatus of 1st Embodiment. It is a block diagram explaining the function of the speech synthesizer of 1st Embodiment. It is a block diagram explaining the function of the power calculation part of 1st Embodiment. It is explanatory drawing explaining the calculation method of the waveform power of the flame | frame by the power calculation part of 1st Embodiment. It is explanatory drawing explaining the method to produce | generate a physical vibration amount from the waveform power of each flame | frame of 1st Embodiment. It is a flowchart explaining the process by the speech assistance apparatus of 1st Embodiment.

(A) 1st Embodiment Below, the 1st Embodiment of the speech assistance apparatus, method, and program of this invention is described, referring drawings.

(A-1) Configuration of the First Embodiment FIG. 1 is a configuration diagram illustrating the configuration of the speech assisting device of the first embodiment. In FIG. 1, the first embodiment includes at least a text input unit 101, a speech synthesis unit 102, a power calculation unit 103, a vibration generation unit 104, a vibration conduction unit 105, and an output unit 106.

  The voice synthesis unit 102, the power calculation unit 103, and the vibration generation unit 104 are functions realized by an information processing apparatus such as a personal computer, for example. These functions can be realized by software processing. For example, in an information processing apparatus having a hardware configuration such as a CPU, ROM, RAM, and EEPROM, the CPU reads and executes a processing program stored in the ROM. These functions are realized.

  The text input unit 101 inputs a desired text by a user's operation, and provides the input text data to the speech synthesis unit 102. The text input unit 101 corresponds to, for example, a keyboard or a touch panel.

  The speech synthesizer 102 generates a synthesized speech waveform for each frame (predetermined processing time interval) or for each expiratory paragraph based on the text data input from the text input unit 101, and the synthesized speech waveform is converted into a power calculator 103. And the output unit 106.

  As a speech synthesis method by the speech synthesizer 102, for example, an existing method described in Patent Document 1 can be widely applied. Here, a speech synthesis method will be described with reference to FIG.

  FIG. 2 shows the internal configuration of the speech synthesizer 102. Note that the content described in JP 2007-233216 A can be applied to the internal configuration of the speech synthesizer 102. As shown in FIG. 2, the speech synthesis unit 102 includes at least a language processing unit 201, a prosody generation unit 202, a waveform generation unit 203, a temporary storage unit 204, a word dictionary 205, and a speech unit database 206.

  The language processing unit 201 performs morphological analysis and syntax analysis of the input text character string, determines accents, intonations, and the like, and provides a kana character string (intermediate language) with an accent position to the prosody generation unit 202.

  The language processing unit 201 refers to the word dictionary 205, performs text processing such as morpheme analysis and syntax analysis on the input text character string, divides the input text string into phoneme units which are units of speech synthesis, and is obtained by analysis Prosody information is added and output as a synthesis target. Here, the word dictionary 205 is a dictionary in which kana, grammatical information, accent type, accent combination rules, and the like of each word are registered.

  Further, the language processing unit 201 gives an accent position according to a grammatical or semantic group of the word series while referring to an accent combination rule of the word dictionary 205.

  Further, the language processing unit 201 determines a dependency from a grammatical or semantic group (accent phrase) to which an accent position is added, and forms a dependent accent phrase as an exhalation paragraph.

  The prosody generation unit 202 generates an acoustic feature parameter (target parameter) corresponding to the prosody of the speech to be synthesized for the target phoneme sequence constituting the synthesis target output from the language processing unit 201, and attaches it to each phoneme. To be output as a target phoneme string consisting of target phonemes.

  Here, the prosodic information is a parameter necessary for speech synthesis. For example, when there is a speech segment, a duration length of each phoneme, a pitch (a voice height in each frame), and a silent section, The duration time is applicable.

  The speech segment database 206 is a database of speech segment data obtained by analyzing and processing speech data recorded from a user who performs vocal cord excision, for example, before excision of the vocal cords.

  When the target phoneme sequence is given from the prosody generation unit 202, the waveform generation unit 203 refers to the speech unit database 206, extracts phoneme unit data, and connects the speech unit data waveforms to each other according to the synthesis target. Generates and outputs a waveform. Various methods can be applied as a waveform generation method by the waveform generation unit 203. For example, a waveform superposition method in which speech segments are shifted and superimposed for each pitch period can be applied.

  The temporary storage unit 204 temporarily stores the synthesized speech waveform generated by the waveform generation unit 203 and then outputs the stored synthesized speech waveform. This may occur when the time efficiency of the synthetic speech waveform generation process (that is, the generation bit rate of the generated waveform data) is not constant. For example, the amount of CPU processing required for generating a synthesized speech waveform may differ greatly depending on whether the input text character string is large or small. In this case, the synthesized speech waveform generated in frame units or expiratory paragraph units is temporarily stored in the temporary storage unit 204, so that subsequent processing can be performed at a constant bit rate.

  The power calculation unit 103 receives the synthesized speech waveform from the speech synthesis unit 102, calculates a waveform power value in a predetermined time interval, and obtains a time series change in the waveform power value in each time interval. The power calculation unit 103 gives the calculated waveform power of each time interval to the vibration generation unit 104.

  FIG. 3 is an internal configuration diagram showing an internal configuration of the power calculation unit 103. As shown in FIG. 3, the power calculation unit 103 includes at least a windowing unit 301 and a waveform power calculation unit 302.

  The windowing unit 301 cuts out a predetermined time section from the synthesized speech waveform from the speech synthesizing unit 102 by windowing, and performs windowing processing such as a Hanning window and a Hamming window.

  FIG. 4 is an explanatory diagram for explaining a windowing process in a predetermined time interval with respect to the synthesized speech waveform.

  Here, in order to calculate the strength of the signal in one frame section, a process of cutting out waveform data representing that frame is performed. As shown in FIG. 4, in the first embodiment, the center time of a frame is used as the center of the window, and clipping is performed with a Hanning window that is twice the frame interval. The window function may be changed as appropriate, such as a Hamming window, a Gaussian window, a tapered window, etc., and the window length is the time resolution of the moving average value of power, and is based on twice the frame period, depending on the application. You may change suitably.

  In the first embodiment, each sample value of the cut-out waveform is obtained by accumulating a square value as the waveform power value of the frame section. Here, the time series information of the waveform power value to be calculated is a moving average value of the power time series of the generated synthesized speech, and as a calculation method thereof, a method of quantifying the waveform amplitude, such as accumulating absolute values Any change may be made accordingly.

  As will be described later, the vibration generation unit 104 converts the physical vibration amount according to the waveform power, and the vibration conduction unit 105 outputs the vibration according to the physical vibration amount, but outputs the vibration according to the waveform power. In order to improve the sensitivity (responsiveness) up to, a relatively short time interval is used.

  The waveform power calculation unit 302 calculates the waveform power value of the waveform data (windowed waveform data) in a predetermined time section cut out by the windowing unit 301.

  Here, as a calculation method of the waveform power value by the waveform power calculation unit 302, for example, sampling is performed on the windowed waveform data at a predetermined sampling frequency to obtain a sample value of the windowed waveform data. A value obtained by squaring each sample value of the windowed waveform data and accumulating it in one frame section is set as a waveform power value in the frame section.

  Note that the method of calculating the waveform power value by the waveform power calculation unit 302 is not limited to the above example, and it is only necessary that each sample value of the windowed waveform data can be moving averaged. A value obtained by accumulating the absolute values of these may be used as the waveform power value of the frame section.

  The vibration generation unit 104 obtains a physical vibration amount based on the waveform power value of each frame section calculated by the power calculation unit 103, and gives the physical vibration amount to the vibration conduction unit 105.

  Here, as a method of generating a physical vibration amount from the waveform power value of each frame section by the vibration generation unit 104, for example, a method of generating a physical vibration amount proportional to the magnitude of the waveform power value can be applied. .

  FIG. 5 is a diagram illustrating an example of generating the physical vibration amount according to the magnitude of the waveform power value. The vibration generation unit 104 holds relation information indicating a proportional relation as shown in FIG. When the waveform power value of each frame is received from the power calculation unit 103, the vibration generation unit 104 refers to the relationship information as illustrated in FIG. 5 and obtains the corresponding physical vibration amount. That is, the vibration generation unit 104 obtains the physical vibration amount by multiplying the waveform power value of each frame by a predetermined constant.

  As another method, the vibration generation unit 104 may quantize the waveform power value of each frame section and obtain a physical vibration amount corresponding to the quantized value. In this case, the vibration generation unit 104 may preset a plurality of physical vibration amounts corresponding to the quantized values of the waveform power values in each frame section, and select the corresponding physical signal amount. Further, the physical vibration amount may be obtained by multiplying a quantized value obtained by quantization by a predetermined constant.

  The vibration conducting unit 105 physically vibrates according to the amount of physical vibration generated by the vibration generating unit 104, and transmits vibration to the throat, cheekbone, neck, etc. of the user (human body). The vibration conduction unit 105 corresponds to, for example, a vibrator (for example, a vibrator) or a bone conduction output device (for example, a bone conduction speaker).

  Thereby, the vibration artificially generated from the synthesized speech can be directly transmitted to the user in accordance with the generation timing of the synthesized speech. Therefore, the user himself / herself can feel that he / she has spoken, and the feeling of loss of confidence due to the imperfection of the synthesized speech can be reduced.

  In addition, the vibration conducting unit 105 is provided with a mounting member such as a soft cloth pad or a skin adhesive tape wrapped around a vibrator so that the vibration conducting unit 105 can be applied to the skin of the human body. Attach to etc.

  The output unit 106 outputs synthesized speech based on the synthesized speech waveform synthesized by the speech synthesis unit 102, and corresponds to, for example, a speaker.

(A-2) Operation of the First Embodiment Next, the operation of the speech assist method using the speech assist device 100 of the first embodiment will be described with reference to FIG.

  First, the user who uses the speech assisting apparatus 100 inputs a desired text character string using the text input unit 101 such as a keyboard (step S101).

  The text character string input to the text input unit 101 is given to the speech synthesizer 102, and the speech synthesizer 102 generates a synthesized speech waveform for the text character string (step S102).

  The synthesized speech waveform synthesized by the speech synthesis unit 102 is given to the power calculation unit 103, and the power calculation unit 103 calculates the waveform power value of the time section of the short frame for each frame (step S103). .

  The power calculation unit 103 cuts out a time period of one frame from the speech synthesis waveform and performs window processing such as a Hanning window or a Hamming window. The power calculation unit 103 performs window processing on the time interval of one frame, thereby obtaining a mean square value obtained by accumulating the squared values of each sample value of the extracted windowed waveform data. Waveform power value.

  Further, the power calculation unit 103 obtains a time series of the waveform power of each frame of the windowed waveform data by performing such waveform power calculation processing at each frame interval.

  Next, the waveform power of each frame calculated by the power calculation unit 103 is given to the vibration generation unit 104. In the vibration generation unit 104, a physical vibration amount corresponding to the waveform power value of each frame is obtained (step S104).

  The vibration generation unit 104 only needs to be able to determine the amount of physical vibration that causes the vibration conducting unit 105 to vibrate so as to be proportional to the magnitude of the waveform power value of each frame. For example, a predetermined constant is set for the waveform power value of each frame. This can be realized by multiplying to obtain the physical vibration amount, quantizing the waveform power value of each frame, and obtaining the physical vibration amount corresponding to the quantized value.

  The vibration generation unit 104 obtains the physical vibration amount corresponding to the waveform power value of each frame. However, since the power calculation unit 103 obtains the waveform power value of each frame in time series, the physical vibration obtained by the vibration generation unit 104 is obtained. The quantity is also time-series for each frame.

  The vibration conducting unit 105 conducts vibration to the user based on the amount of physical vibration generated by the vibration generating unit 104 (step S105).

(A-3) Effects of the First Embodiment As described above, according to the first embodiment, when a user inputs text, a synthesized speech is output from an output unit such as a speaker, and the user's Since vibration corresponding to the magnitude of the synthesized speech is fed back to the neck or the like, it is possible to give the user a stronger sense of security in a psychological aspect.

(B) Other Embodiments In the first embodiment, the method of feeding back the short-time power of the generated speech waveform to the neck or the like using a vibration conducting unit such as a vibrator has been described. It is also possible to substitute by using it.

  DESCRIPTION OF SYMBOLS 100 ... Vibration conduction part, 101 ... Text input part, 102 ... Speech synthesis part, 103 ... Power calculation part 103 ... Vibration generation part, 105 ... Vibration conduction part, 106 ... Output part.

Claims (6)

Speech waveform generation means for generating a speech waveform based on text input through the text input means;
Waveform power calculation means for calculating a waveform power value for each predetermined time interval based on the voice waveform from the voice waveform generation means;
Physical vibration conversion means for obtaining a physical vibration amount according to the waveform power value of each time interval from the waveform power calculation means;
An utterance assisting device comprising: vibration conduction means for providing a speaker with vibration based on a physical vibration amount corresponding to the waveform power value in each time interval. Voice output means for outputting voice corresponding to the voice waveform from the voice waveform generation means;
The vibration conducting means is capable of contacting the throat, cheekbone or neck of the speaker, and at the timing when the voice is output from the voice output means, the waveform power value of the time interval corresponding to the output voice is obtained. The utterance assisting device according to claim 1, wherein the corresponding vibration is transmitted to a speaker.   The speech waveform generation means includes a database storing a plurality of speech segment data and prosodic information, and generates synthesized speech corresponding to an input text with reference to the database. The speech assisting device according to 1 or 2.   The speech assisting device according to any one of claims 1 to 3, wherein the vibration conducting means is a vibrator or a bone conduction speaker. An utterance assisting method of an utterance assisting device comprising text input means and vibration conducting means,
A voice waveform generation step for generating a voice waveform based on the text input through the text input means;
A waveform power calculation step in which a waveform power calculation means calculates a waveform power value for each predetermined time interval based on the voice waveform from the voice waveform generation means;
A physical vibration converting means for obtaining a physical vibration amount corresponding to the waveform power value of each time interval from the waveform power calculating means and supplying the physical vibration amount to the vibration conducting means; An utterance assistance method characterized by An utterance assisting device comprising a text input means and a vibration conduction means,
Speech waveform generation means for generating a speech waveform based on the text input through the text input means;
Waveform power calculating means for calculating a waveform power value for each predetermined time interval based on the voice waveform from the voice waveform generating means;
An utterance assist characterized by obtaining a physical vibration amount corresponding to the waveform power value of each time interval from the waveform power calculation means and functioning as a physical vibration conversion means for supplying the physical vibration amount to the vibration conduction means program.

Images