JP2007018006A - Speech synthesis system, speech synthesis method, and speech synthesis program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To maintain a high recognition rate even with a small sound volume without being affected by surrounding noises in speech recognition. <P>SOLUTION: The speech recognition system includes: a sound information processing section 10 which acquires a sound signal and calculates a sound information parameter based on a change in the acquired sound signal; an electromuscular signal processing section 13 which acquires the potential change in an object surface as the electromuscular signal and calculates an electromuscular signal parameter based on the change in the acquired sound signal; an image information processing section 16 which acquires the video to be photographed object as image information and calculates an image information parameter based on the change in an object in the video; a speech recognition means 20 which recognizes the speech based on the sound information parameter, the electromuscular signal parameter and the image information parameter; and a recognition result presentation means 21 which presents the recognition result by the speech recognition means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a speech synthesis system, a speech synthesis method, and a speech synthesis program for recognizing sound such as speech and synthesizing speech based on the recognized speech.

  Ordinary speech detection devices employ speech recognition technology that recognizes and processes speech signals by treating speech in speech as acoustic signals and analyzing the frequency of the acoustic signal. Envelopes are used.

  However, in order to produce a good speech detection result using this speech recognition technology, a certain level of volume is required during speech, and speech information could not be detected unless an acoustic signal from speech was input. . Accordingly, since the voice of the speaker becomes annoying to the people around when inputting voice, such a voice detection device cannot be used in offices, libraries, and public institutions where quietness is required. In addition, there is a drawback in that a problem of crosstalk occurs in a place where the surrounding noise is large, and the voice detection function is deteriorated.

  On the other hand, research for acquiring voice information from other than acoustic signals has also been conducted. If voice information can be acquired from information other than acoustic information, it becomes possible to speak without producing sound, and the above-described problems can be solved. As a speech recognition method based on visual information on the lips, there is a method based on image processing using an image input by a video camera (see, for example, Patent Document 1 or Patent Document 2).

  Furthermore, there is a study of recognizing the types of vowels produced by processing myoelectric signals generated with the movement of muscles around the mouth (for example, see Non-Patent Document 1). Non-Patent Document 1 describes that after a myoelectric signal is passed through a band-pass filter, the number of crossing thresholds is counted to discriminate five vowels (a, i, u, e, o).

  As another method, Japanese Patent Application Laid-Open No. 7-181888 discloses a method of processing a myoelectric signal of muscles around the mouth using a neutral net and detecting not only a vowel of a vocal speaker but also a consonant. It is shown. Furthermore, not only information from one input channel but also a multimodal interface using a plurality of input channels has been proposed and realized.

On the other hand, in a conventional speech synthesis system, a method for preliminarily storing data characterizing a speaker's speech and synthesizing speech according to the speaker's speech has been developed.
JP-A-52-112205 JP-A-6-43897 Noboru Sugie et al., “A speech Employing a Speech Synthesizer Vowel Discrimination from Perioral Muscles Activities and Vowel Production,” IEEE transactions on Biomedical Engineering, Vol.32, No.7, pp485-490

  However, the speech detection method for acquiring speech information from information other than the above-described acoustic information has a problem that the recognition rate is lower than speech recognition using acoustic information. In particular, it has been difficult to recognize consonants generated by the movement of muscles in the mouth.

  In addition, in the conventional speech synthesis system, as described above, since speech is synthesized based on data characterizing the speaker's speech, the synthesized speech is mechanical and the expression becomes unnatural. There was a problem that emotions could not be expressed properly.

  The present invention has been made in view of the above problems, and a speech recognition system, method, and method capable of maintaining a high recognition rate even at a small volume without being affected by ambient noise. The purpose is to provide a program. Another invention is a speech synthesis system, method, and program capable of making a synthesized speech more natural and appropriately expressing a speaker's emotion and the like by using speech recognition for speech synthesis. The purpose is to provide.

  In order to solve the above problem, the present invention acquires an acoustic signal, calculates an acoustic information parameter based on the acquired change in the acoustic signal, acquires a potential change on the surface of the object as a myoelectric signal, and acquires EMG signal parameters are calculated based on the change in the acoustic signal, the image of the captured object is acquired as image information, the image information parameter is calculated based on the change in the object in the image, and these acoustic information parameters The voice is recognized based on the myoelectric signal parameter and the image information parameter, and the recognition result is presented.

  According to the present invention as described above, since voice recognition is performed using a plurality of parameters such as an acoustic signal, a myoelectric signal, and image information, noise resistance and the like can be greatly improved.

  Another invention recognizes speech, obtains the spectrum of the acoustic signal from the acoustic information as the first spectrum, and generates the spectrum of the acoustic signal reconstructed from the recognition result by the speech recognition means as the second spectrum. Then, the first spectrum and the second spectrum are compared, a corrected spectrum is generated according to the comparison result, and a sound synthesized from the corrected spectrum is output.

  According to such an invention, since the voice is synthesized based on the spectrum from the voice recognized using not only the spectrum from the acoustic information but also other parameters, ambient noise can be effectively removed. .

  In the voice recognition in the above two inventions, the recognition processing is performed for each of the acoustic information parameter, the myoelectric signal parameter, and the image information parameter, and then the respective recognition results are compared, and the final recognition is performed based on the comparison result. It is desirable to perform processing. Furthermore, the speech recognition may be performed using the acoustic information parameter, the myoelectric signal parameter, and the image information parameter at the same time.

  As another speech recognition, an element group that is a set of nonlinear elements having a data input unit and an output unit is arranged hierarchically from upstream to downstream, and an upstream nonlinear element is arranged between adjacent element groups. The output part of the element and the input part of the downstream non-linear element are connected to each other, and each non-linear element is given a weighting factor for each connection of the non-linear element to the input part and a combination of these connections. In accordance with the input data and the weighting factor, a hierarchical network that determines the data to be output downstream and the connection from the output unit is constructed, and the acoustic information parameter, the myoelectric signal parameter, and the image information parameter are input from the upstream side. The data output from the most downstream element group is preferably recognized voice.

  When this hierarchical network is used, a learning function for changing the weighting coefficient assigned to each nonlinear element is realized by inputting sample data from the downstream side of the hierarchical network and causing the data to flow backward to the upstream side. be able to.

  As described above, according to the speech recognition system, method and program of the present invention, a high recognition rate can be maintained even with a small volume without being affected by ambient noise. In addition, according to the speech synthesis system, method and program of another invention, by using speech recognition for speech synthesis, the synthesized speech can be made more natural and the emotion of the speaker can be appropriately expressed. Can do.

[First Embodiment]
(Basic configuration)
Hereinafter, a voice recognition system according to an embodiment of the present invention will be described in detail. FIG. 1 is a block diagram showing a basic configuration of a speech recognition system according to the present embodiment.

  As shown in the figure, the voice recognition system includes an acoustic information processing unit 10, a myoelectric signal processing unit 13, an image information processing unit 16, and an information comprehensive recognition unit 19.

  The acoustic information processing unit 10 processes acoustic information emitted at the time of utterance, and includes an acoustic signal acquisition unit 11 for acquiring an acoustic signal at the time of utterance, and a spectrum envelope of the acoustic signal obtained by the acoustic signal acquisition unit. And an acoustic signal processing means 12 for extracting acoustic information parameters by separating the spectral fine structure.

  The sound signal acquisition unit 11 is a device that acquires sound such as a microphone, detects the sound uttered during speech with the microphone, and transmits the acquired sound signal to the sound signal processing unit 12.

  The acoustic signal processing means 12 is a device for calculating acoustic information parameters that can be processed by the speech recognition means 20 for the acoustic signal acquired from the acoustic signal acquisition means 11, and the acoustic signal is cut out and cut out in a set time window. For acoustic signals, acoustic information parameters are set using analysis methods such as short-time spectrum analysis, gepstrum analysis, maximum likelihood spectrum estimation, covariance method, PARCOR analysis, and LSP analysis, which are used in general speech recognition. calculate.

  The myoelectric signal processing unit 13 performs processing by detecting the movement of the muscle around the mouth at the time of utterance, and acquires the myoelectric signal for acquiring the myoelectric signal accompanying the movement of the muscle around the mouth at the time of utterance. Means 14 and myoelectric signal processing means 15 for extracting myoelectric signal parameters by calculating the power of the myoelectric signal obtained by the myoelectric signal acquiring means or performing frequency analysis are provided.

  The myoelectric signal acquisition means 14 is a device that detects a myoelectric signal associated with the activity of muscles around the mouth during speech, and detects changes in the skin surface around the mouth of the speaker. That is, a plurality of muscles in the vicinity of the mouth act in cooperation during utterance, and in order to capture the activities of the plurality of muscles, the myoelectric signal acquisition means 14 has a plurality of muscles corresponding to the plurality of muscles active during the utterance. A plurality of myoelectric signals are derived from the skin surface electrode, amplified and transmitted to the myoelectric signal processing means 15.

  The myoelectric signal processing means 15 is a device that calculates myoelectric signal parameters from a plurality of myoelectric signals transmitted from the myoelectric signal acquisition means 14, and specifically, a set time for the myoelectric signal. Cut out in the window, and for the cut out myoelectric signal, features of the average amplitude such as spectrum analysis, root mean square (RMS), rectified average (ARV), integral electromyogram (IEMG) Calculation is performed to calculate myoelectric signal parameters.

  The image information processing unit 16 performs image processing by detecting spatial deformation around the mouth at the time of utterance, and image information acquisition means 17 for photographing the spatial deformation around the mouth at the time of utterance with a video camera. And image information processing means 18 for extracting motion parameters around the lips from the image information obtained by the image information processing means.

  The image information acquisition means 17 is a photographing machine such as a video camera that captures the movement around the mouth at the time of speech, detects the movement around the mouth as an image, and transmits it to the image information processing means 18.

  The image information processing means 18 is an apparatus for calculating image information parameters from the image information obtained by the image information acquisition means 17, and specifically extracts the feature amount of the movement around the mouth from the image information by optical flow. The image information parameter is calculated.

  The information comprehensive recognition unit 19 recognizes the various information obtained from the acoustic information processing unit, the myoelectric signal processing unit, and the image information processing unit, and presents the recognition result. Speech recognition that compares and integrates the obtained acoustic information parameters, the myoelectric signal parameters obtained by the myoelectric speech recognition unit, and the image information parameters obtained by the image information processing unit to determine the speech recognition result Means 20 and a recognition result presentation means 21 for presenting the recognition result obtained by the voice recognition means.

  The voice recognition means 20 is an arithmetic unit that performs voice recognition using the acoustic information parameter, the myoelectric signal parameter, and the image information parameter acquired from each of the units 10, 13, and 16. This voice recognition means 20 is a function for performing voice recognition only from the acoustic information parameter when the voice information can be sufficiently recognized from the acoustic information parameter, such as when there is little ambient noise or when the volume of speech is high. It has. In addition, the voice recognition unit 20 is configured to use only the acoustic information parameter when it is impossible to recognize the voice sufficiently only from the acoustic information parameter, such as when the surrounding noise is large or when the volume of speech is low. In addition, it has a function of performing speech recognition in consideration of information obtained from the myoelectric signal parameters and the image information parameters.

  Further, when the speech recognition unit 20 recognizes speech using the myoelectric signal parameter and the image information parameter and the recognition rate is low, the speech recognition unit 20 uses the acoustic information parameter for the phoneme and the like that are recognized incorrectly as a whole. Has a function to increase the recognition rate.

  The recognition result presentation unit 21 is an output device that outputs the recognition result obtained by the voice recognition unit 20. A display monitor such as a liquid crystal that displays text can be used. Further, as the recognition result presentation means 21, by providing a communication interface or the like, the speech recognition result is not only presented to the speaker, but also output as data to an application running on a terminal device such as a personal computer. You may make it do.

(basic action)
The speech recognition system according to the present embodiment having the above basic configuration operates as follows. FIG. 2 is a flowchart showing the operation of the speech recognition system according to the present embodiment.

  First, the speaker starts speaking (S101). At this time, the acoustic signal, the myoelectric signal, and the image information when the speaker is speaking are detected by the acoustic signal acquiring unit 11, the myoelectric signal acquiring unit 14, and the image information acquiring unit 17, respectively (S102 to S104). ). The detected acoustic signal, myoelectric signal, and image information are calculated as an acoustic information parameter, myoelectric signal parameter, and image information parameter by the acoustic signal processing unit 12, the myoelectric signal processing unit 15, and the image information processing unit 18, respectively. (S105 to S107).

  The calculated various parameters are recognized by the speech recognition means 20 (S108), and the recognition result presentation means 21 presents the speech recognition result (S109). As described above, the recognition result can be presented by voice or displayed on the screen.

(Operation of each means)
The operation of each means in the basic configuration will be described in detail below.

(1) Voice Recognition Unit FIG. 4 is a block diagram illustrating the voice recognition unit 20. Here, after performing the recognition process for each of the acoustic information parameter, the myoelectric signal parameter, and the image information parameter, the recognition results are compared, and the final recognition process is performed based on the comparison result.

  Specifically, as shown in the figure, the speech recognition means 20 according to the present embodiment uses only acoustic information parameters, only myoelectric signal parameters, and only image information parameters before performing final speech recognition. The speech recognition is performed, and the recognition results obtained from the respective parameters are integrated to finally perform the speech recognition. Of the recognition results obtained from each parameter, if multiple recognition results match, the matching result is the final recognition result. If all the recognition results do not match, the recognition accuracy is the highest. The recognition result that seems to be high is used as the final recognition result.

  Also, for example, in speech recognition using only myoelectric signal parameters, when it is known in advance that the recognition rate of a specific phoneme or utterance pattern is low, the result of speech recognition using other parameters, When it can be estimated that those utterances are being made, the recognition rate of the final speech recognition result can be improved by ignoring the speech recognition result using the myoelectric signal parameter.

  Furthermore, for example, if it can be determined from the acoustic information parameter that the ambient noise is large or the volume during speech is low, the effect of the speech recognition result using the acoustic information parameter on the final speech recognition result is affected. The final speech recognition is performed by reducing the size and placing importance on the result of speech recognition using the myoelectric signal parameter and the image information parameter. In addition, the speech recognition using each parameter can use the method normally used.

  Further, the voice recognition means 20 may perform voice recognition from three parameters instead of the above method. FIG. 3 is an explanatory diagram for explaining the operation of the speech recognition means 20 when performing speech recognition from three parameters.

  As a method of performing speech recognition from these three parameters, for example, there is a method using a neural network. In this neural network, as shown in the figure, a group of elements, which are a set of nonlinear elements having a parameter input part and an output part, are arranged hierarchically from upstream to downstream, and between adjacent element groups, It is constructed by mutually connecting the output part of the upstream nonlinear element and the input part of the downstream nonlinear element.

  Then, in each nonlinear element, a weighting factor is given for each connection of the nonlinear element to the input unit and a combination of these connections, and the parameter and output to be output downstream according to the parameter and the weighting factor input to the input unit Determine the connection from the department. Specifically, the voice recognition means 20 receives acoustic information parameters, myoelectric signal parameters, and image information parameters, and outputs are vowels and consonants.

  In this embodiment, a fully-coupled three-layer neural network (Nishikawa / Kitamura, “Neural Network and Measurement Control”, Asakura Shoten, pp. 18-50) is used as this neural network.

  In this neural network, it is necessary to learn weighting factors in advance. Learning in the present embodiment is performed by a back propagation method. For that purpose, utterance operation is performed according to the prepared utterance pattern, acoustic information parameters, myoelectric signal parameters, and image information parameters are acquired, and various parameters are set using the prepared utterance pattern as a teacher signal. To learn. This learning process will be described later.

  In the speech recognition unit 20 according to the present embodiment, the myoelectric signal is uttered earlier in time than the acoustic signal and the image information in the speech operation performed when the speaker speaks. It has a function of synchronizing the acoustic signal, the myoelectric signal, and the image information by delaying.

  The neural network of the speech recognition means 20 that receives various parameters as input outputs which phoneme the input parameter corresponds to. When a phoneme is uttered, the corresponding myoelectric signal is output earlier in time than the acoustic signal and image information. Therefore, the myoelectric signal is input to the neural network after being delayed in time. It is also possible to synchronize.

  As this neural network, a recurrent neural network that returns the previous recognition result to the input can also be used. In the present invention, an algorithm used for recognition is not limited to a neural network, and other recognition algorithms such as a hidden Markov model (HMM) can be used.

  According to the voice recognition means 20, when the utterance volume is low, the ambient noise is large, or the myoelectric signal cannot be detected well, the acoustic information parameter, myoelectric signal parameter, and image information parameter are set. Even if one of the parameters becomes invalid for speech recognition, the final speech recognition can be performed using meaningful parameters, and the noise resistance can be greatly improved. .

  In the speech recognition unit 20 according to the present embodiment, the speech information speech recognition using various acoustic signals currently used can be used for the acoustic information speech recognition. Also, myoelectric signal speech recognition is described in the document “Noboru Sugie et al.,“ A speech Employing a Speech Synthesizer Vowel Discrimination from Perioral Muscles Activities and Vowel Production, ”IEEE transactions on Biomedical Engineering, Vol.32, No.7, pp485-490. Or a method disclosed in JP-A-7-181888 can be used. Further, the image information voice recognition can use a method disclosed in Japanese Patent Laid-Open No. 2001-51693 or Japanese Patent Laid-Open No. 2000-206986. Furthermore, it is possible to use a speech recognition method other than the methods listed above.

  Furthermore, the speech recognition in the present invention may be performed only in one of the method shown in FIG. 3 or the method shown in FIG. In addition, when the method shown in FIG. 4 is performed and the voice cannot be recognized depending on all the parameters, the voice recognition using the neural network shown in FIG. 3 may be performed. The final speech recognition may be performed by comparing or integrating the recognition result performed by the method shown in FIG. 4 with the recognition result performed by the method shown in FIG.

  In the present embodiment, the neural network shown in FIG. 3 is described as an example of a method for performing speech recognition using three parameters. However, the present invention is not limited to this, and a method other than the neural network is used. Can be used to recognize speech from three parameters.

(2) Acoustic signal processing means and myoelectric signal processing means The operations of the above-described acoustic signal processing means 12 and myoelectric signal processing means 15 will be described in detail. FIG. 6 is a diagram for explaining an example of acoustic information parameter and myoelectric signal parameter extraction.

  The acoustic signal and myoelectric signal detected by the acoustic signal acquiring unit 11 and the myoelectric signal acquiring unit 14 are first cut out by a time window by the acoustic signal processing unit 12 and the myoelectric signal processing unit 15 ((a) in the figure). ) Next, a spectrum is extracted from the extracted signal using FFT ((b) in the figure). Then, 1/3 octave analysis is performed on the extracted spectrum ((c) in the figure), and the power of each band is calculated and used as the acoustic information parameter and the myoelectric signal parameter ((d) in the figure). The acoustic information parameter and the myoelectric signal parameter are sent to the voice recognition means 20 for voice recognition.

  Note that the method for extracting the acoustic information parameter and the myoelectric signal parameter in the present invention can be performed by a method other than that shown in FIG.

(3) Image Information Processing Unit The operation of the image information processing unit 18 will be described in detail. FIG. 7 is a diagram for explaining a method of extracting image information parameters.

  First, the position of the feature point around the mouth is extracted from the image around the mouth at time t0 ((a) in the figure, S501). The feature points around the mouth are extracted by placing a marker on the feature point around the mouth and setting the position of the marker as the feature point position or by searching for the feature point from the captured image. It is also possible to extract. The position may be a two-dimensional position on the image, or a three-dimensional position may be extracted using a plurality of cameras.

  Next, similarly to the time t0, the position of the feature point around the mouth at time t1 when Δt has elapsed from time t0 is extracted ((b) in the figure, S502). Then, the amount of movement of each feature point is calculated by calculating the difference from the position of the feature point around the mouth at time t0 and time t1 ((c) in the figure, S503). A parameter is generated from this calculation result ((d) in the figure, S504).

  Note that the image information parameter extraction method can be performed by a method other than that shown in FIG.

(Learning process)
Next, the learning process described above will be described. FIG. 8 is a flowchart for explaining the learning process in the present embodiment. In order to improve the voice recognition accuracy in the present embodiment, it is important to learn the characteristics of the speaker's individual speech. Note that the learning method described individually is based on the assumption that speech recognition is performed using the above-described neural network. When speech recognition is performed using another method, a learning method suitable for the method is appropriately employed.

  In this embodiment, as shown in the figure, first, the speaker starts an utterance operation (S301, S302). At the same time as the utterance, the speaker inputs the content uttered by the keyboard or the like, that is, teacher data (sample data) in learning (S305). In parallel with this, acoustic signals, myoelectric signals, and image information are detected by the voice recognition system (S303), and parameters are extracted from the respective signals (S304).

  The extracted parameters are learned based on the teacher signal input from the keyboard (S306). That is, the weighting factor given to each nonlinear element is changed by inputting teacher data from the downstream side of the hierarchical network described above and causing the data to flow backward to the upstream side.

  Thereafter, when the recognition error due to learning becomes equal to or smaller than a certain value, it is determined that the learning is finished (S307), and the learning is finished (S308). On the other hand, if it is determined in step S307 that the learning has not ended, the learning is repeated again in steps S302 to S306.

(effect)
According to the speech recognition system according to the present embodiment described above, since speech recognition is performed using a plurality of parameters obtained from acoustic information, myoelectric signals, and image information, noise resistance is greatly improved. To improve. In other words, having three types of input interfaces makes it difficult to be affected by noise, etc. Even if there are interfaces that cannot be used among the three types, it is possible to perform voice recognition by using the remaining interfaces. The recognition rate can be improved. As a result, it is possible to provide a speech recognition system that can sufficiently recognize a speech even when a speaker speaks at a low volume or speaks in a place where the surrounding noise is large.

[Second Embodiment]
A speech synthesis system can be configured by applying the speech recognition system described above. FIG. 9 is a flowchart showing an operation when speech synthesis is performed using the speech recognition system described above.

  In the speech synthesis system according to the present embodiment, as shown in the figure, after performing the operation steps S202 to S208 in the speech recognition system described above, in step S209, other than the acoustic signal emitted by the speaker from the detected acoustic signal In step S20, a clear synthesized speech is output.

  This speech synthesis will be described in detail. As shown in FIG. 10, in the present embodiment, the spectrum of the uttered phoneme is reconstructed from the feature amount such as the formant frequency of the uttered phoneme using the recognition result by the speech recognition system. To do. Then, by multiplying this reconstructed spectrum ((a) in the figure) with the spectrum of the detected acoustic signal including the noise component ((c) in the figure), the spectrum of the speech from which noise has been removed is obtained ( (D) in the figure). The spectrum of the speech from which noise has been removed is output as an acoustic signal ((b) in the figure) from which noise has been removed by inverse Fourier transform. That is, an acoustic signal including a noise component is output through a filter having a frequency characteristic represented by a reconstructed spectrum.

  According to the voice recognition system according to the present embodiment, voice recognition is performed by various methods, a signal reconstructed from the recognition result, an acoustic signal uttered by the speaker from the detected acoustic signal, and surroundings It is possible to separate the noise from the noise, so that even if the surrounding noise is large, it is possible to output a clear synthesized voice utilizing the voice of the speaker. As a result, according to the present embodiment, even if the speaker speaks at a low volume, or the speaker speaks in a noisy place, the other party normally speaks in an environment with no noise. It is possible to output a synthesized speech as if it were.

  In the present embodiment, the method according to the first embodiment described above is used for the speech recognition process. However, the present invention is not limited to this, and performs speech recognition using parameters other than acoustic information. You may make it synthesize | combine a sound by this and acoustic information.

[Third Embodiment]
The speech recognition system and speech synthesis system described above can be implemented in the following forms. FIG. 11 is a diagram for explaining a third embodiment of the speech recognition and synthesis system according to the present embodiment.

  As shown in the figure, the speech recognition / synthesis system according to the present embodiment includes a mobile phone body 30 and a wristwatch-type terminal 31 that is separated from the mobile phone body 30.

  The mobile phone main body 30 is obtained by adding the above-described acoustic information processing unit 10, myoelectric signal processing unit 13, voice recognition means 20 and voice synthesis means to a known mobile phone, A myoelectric signal acquisition unit 14 and an acoustic signal acquisition unit 11 are provided. In the present embodiment, the myoelectric signal acquisition means 14 is composed of a plurality of skin surface electrodes provided so as to be able to contact the skin of the speaker 32, and the acoustic signal acquisition means 11 is located near the mouth of the speaker 32. It is comprised with the provided microphone.

  In addition, the mobile phone body 30 has a function of transmitting a synthesized voice synthesized based on a recognition result of the voice recognition means 20 as a call voice of the speaker 32 because the communication means is built in.

  The wristwatch type terminal 31 includes the image information processing unit 16 and the recognition result presentation unit 21 described above, a video camera as the image information acquisition unit 17 provided on the surface of the wristwatch type terminal 31, a recognition result, and the like. A screen display device as the presenting means 21 is provided.

  The voice recognition / synthesis system having such a configuration acquires the myoelectric signal and the acoustic signal from the speaker 32 by the myoelectric signal acquisition unit 14 and the acoustic signal acquisition unit 11 of the mobile phone body 30, and The image information acquisition means 17 acquires the image information of the speaker 32. Then, the mobile phone body 30 and the wristwatch-type terminal 31 communicate with each other by wire or wirelessly, collect each signal in the voice recognition means 20 built in the mobile phone body 30, perform voice recognition, and perform wired or wireless communication. Thus, the recognition result is displayed on the recognition result presentation means 21 of the wristwatch type terminal 31. Further, the mobile phone body 30 synthesizes a clear voice from which ambient noise has been removed based on the recognition result and transmits it to the other party.

  In the present embodiment, the voice recognition means is built in the mobile phone body 30 and the recognition result is displayed on the recognition result presentation means 21 of the wristwatch type terminal 31. For example, the voice recognition means is the wristwatch type terminal 31. Alternatively, the voice recognition and the voice synthesis may be performed on the other terminal side capable of communicating with each of the devices 30 and 31. In addition, the recognition result when voice recognition is performed can be output by voice from the mobile phone body 30, displayed on the screen of the wristwatch-type terminal 31 (or the mobile phone body 30), and communicated with them. It is also possible to output to other terminals.

[Fourth Embodiment]
Furthermore, the above-described speech recognition system and speech synthesis system can be implemented in the following forms. FIG. 12 is a diagram for explaining a fourth embodiment of the present invention.

  As shown in the figure, the speech recognition and synthesis system according to the present embodiment includes a holding device 41 in the shape of glasses that can be worn on the head of the speaker 32, and a speaker 32 that is a sound source in the holding device 41. It is composed of a video camera as the image information acquisition means 17 fixed so that the periphery of the mouth can be photographed, a fixing unit 42, a see-through HMD as the recognition result acquisition means 21, and a voice recognition means built in the holding device 41. The A skin surface electrode as the myoelectric signal acquisition unit 14 and a microphone as the acoustic signal acquisition unit 11 are attached to the fixing unit 42.

  By mounting such a speech recognition and synthesis system, the speaker 32 can perform speech recognition and speech synthesis in a freehand state.

  Note that the voice recognition means can be stored in the holding device 41 or in an external terminal capable of communicating with the holding device 41. In addition, the recognition result of the voice recognition can be displayed on a see-through HMD (transparent display unit), or can be output by voice from an output device such as a speaker provided in the holding device 41, and an external terminal. Can also be output. Furthermore, when a voice output device such as a speaker is provided in the holding device 41, a voice synthesized based on voice recognition may be output.

[Fifth Embodiment]
The speech recognition system / speech synthesis system and method according to the first to fourth embodiments described above are described in a predetermined computer language on a general-purpose computer such as a personal computer or an IC chip provided in a mobile phone or the like. This can be realized by executing the programmed program.

  Such a communication control program is recorded on a recording medium (floppy (registered trademark) disk 116, CD-ROM 117, RAM 118, cassette tape 119) readable by a computer 115 as shown in FIG. The voice recognition system and the voice synthesis system described in the above-described embodiment can be realized through the computer 115 or directly installed in the memory of the mobile phone body 30 or the like.

It is a block diagram for demonstrating the basic composition of the speech recognition system concerning 1st Embodiment. It is a flowchart for demonstrating operation | movement of the speech recognition system concerning 1st Embodiment. It is explanatory drawing for demonstrating operation | movement of the speech recognition means concerning 1st Embodiment. It is explanatory drawing for demonstrating operation | movement of the speech recognition means concerning 1st Embodiment. It is explanatory drawing for demonstrating operation | movement of the hierarchical network in the speech recognition means concerning 1st Embodiment. It is explanatory drawing for demonstrating the extraction process of the parameter in 1st Embodiment. It is explanatory drawing for demonstrating the extraction process of the parameter in 1st Embodiment. It is a flowchart for demonstrating the learning process in 1st Embodiment. It is a flowchart for demonstrating operation | movement of the speech synthesis system concerning 2nd Embodiment. It is explanatory drawing for demonstrating operation | movement of the speech synthesis system concerning 2nd Embodiment. It is explanatory drawing of the speech recognition synthesis system concerning 3rd Embodiment. It is explanatory drawing of the speech recognition synthesis system concerning 4th Embodiment. It is a perspective view of the computer-readable recording medium which recorded the speech recognition program and speech synthesis program concerning 5th Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Acoustic information processing part 11 ... Acoustic signal acquisition means 12 ... Acoustic signal processing means 13 ... Myoelectric signal processing part 14 ... Myoelectric signal acquisition means 15 ... Myoelectric signal processing means 16 ... Image information processing part 17 ... Image information acquisition Means 18 ... Image information processing means 19 ... Information comprehensive recognition unit 20 ... Speech recognition means 21 ... Recognition result presentation means 30 ... Mobile phone body 31 ... Wristwatch type terminal 32 ... Speaker 41 ... Holding device 42 ... Fixing part

Claims (6)

Speech recognition means for recognizing speech;
Sound acquisition means for acquiring an acoustic signal;
Means for acquiring a spectrum of an acoustic signal as a first spectrum from the acquired acoustic information;
Means for generating a spectrum of an acoustic signal reconstructed from a recognition result by the voice recognition means as a second spectrum;
Means for comparing the first spectrum with the second spectrum and generating a modified spectrum according to the comparison result;
And a speech synthesis system comprising: output means for outputting speech synthesized from the modified spectrum. The speech synthesis system according to claim 1, wherein the output unit includes a communication unit that transmits the synthesized speech as data. Step (1) for recognizing speech;
Obtaining an acoustic signal (2);
(3) acquiring a spectrum of an acoustic signal as a first spectrum from the acquired acoustic information;
Generating a spectrum of the acoustic signal reconstructed from the recognition result by the voice recognition means as a second spectrum;
Comparing the first spectrum with the second spectrum, and generating a modified spectrum according to the comparison result;
And (6) outputting a voice synthesized from the modified spectrum. The speech synthesis method according to claim 3, wherein the step (6) includes a step of transmitting the synthesized speech as data. On the computer,
Step (1) for recognizing speech;
Obtaining an acoustic signal (2);
(3) acquiring a spectrum of an acoustic signal as a first spectrum from the acquired acoustic information;
Generating a spectrum of the acoustic signal reconstructed from the recognition result by the voice recognition means as a second spectrum;
Comparing the first spectrum with the second spectrum, and generating a modified spectrum according to the comparison result;
A speech synthesis program for executing a process including a step (6) of outputting speech synthesized from the modified spectrum. The speech synthesis program according to claim 5, wherein the step (6) includes a step of transmitting the synthesized speech as data.

Images