JP2007240654A - In-body conduction ordinary voice conversion learning device, in-body conduction ordinary voice conversion device, mobile phone, in-body conduction ordinary voice conversion learning method and in-body conduction ordinary voice conversion method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an in-body conduction ordinary voice conversion learning device, an in-body conduction ordinary voice conversion device, a mobile phone, an in-body conduction ordinary voice conversion learning method, and an in-body conduction ordinary voice conversion method, capable of improving voice quality of in-body conduction ordinary voice. <P>SOLUTION: A linear prediction analysis is performed on the in-body conduction ordinary voice which conducts through in-body soft tissue, which is taken from a skin surface on sternocleidal papillary muscle directly under mastoid bones of a skull under ear lobes. Moreover, the linear prediction analysis is performed on air conduction sound. Based on the analysis results, a mixture normal distribution model is learned by a GMM (Gaussian Mixture Model) learning means 13, and stored in a GMM storing memory 14. The in-body conduction ordinary voice is converted into quasi air conduction sound by using the learning results stored in the GMM storing memory 14. With this configuration, voice communication is comfortably performed with the voice quality close to the air conduction sound even in very noisy environment. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  TECHNICAL FIELD The present invention relates to a body conduction normal speech conversion learning device, a body conduction normal speech conversion device, a mobile phone, a body conduction normal speech conversion learning method, and a body conduction normal speech conversion learning method, and in particular, a microphone mounted on the skin surface immediately below a mastoid process. (Body conduction normal speech conversion learning device, body conduction normal speech conversion learning device, mobile phone, which improves the sound quality of speech (internal conduction normal speech) collected by (meat conduction (registered trademark) microphone) and conducted through the soft tissue in the body, The present invention relates to a normal conduction speech conversion learning method and a normal conduction speech conversion method.

Meat conduction (registered trademark) that is attached to the skin surface on the thoracic papillary muscle, directly below the mastoid process of the cranial bone, and collects the vibration sound that conducts the soft tissue in the body, as described in Patent Document 1. ) Microphone is structurally resistant to external noise. Taking advantage of this feature, studies are underway to use normal body-conducted speech as an input to speech recognition and speech communication in a noisy environment.
WO2004 / 021738

  In general, the sound is output through the articulating organ from the throat to the palate using a regular wave (pitch) generated by regular vibration of the vocal cords or turbulent noise generated by narrowing the airway as a sound source. Intracorporeal normal speech is collected from the skin surface above the thoracic papillary muscle, just below the mastoid of the skull, just below the pinna, slightly away from the articulatory organ. The problem is that the high frequency components are attenuated, resulting in a muffled sound that is slightly less clear than the sound collected by a normal air conduction microphone (hereinafter referred to as air conduction sound). was there.

  The present invention has been made in order to solve the above-described problems of the prior art, and the object thereof is a body conduction normal speech conversion learning device and a body conduction normal speech conversion device capable of improving the sound quality of body conduction normal speech. It is intended to provide a mobile phone, a body conduction normal speech conversion learning method, and a body conduction normal speech conversion learning method.

The internal conduction normal speech conversion learning device according to claim 1 of the present invention,
First linear predictive analysis that performs linear predictive analysis on normal body conduction speech conducted through the soft tissue of the body, taken from the surface of the skin on the thoracic papillary muscle, just below the mastoid process of the skull, below the auricle. Means, second linear prediction analysis means for performing linear prediction analysis on the air conduction sound,
GMM learning means for learning a mixed normal distribution model based on the analysis result of the first linear prediction analysis means and the analysis result of the second linear prediction analysis means. According to such a configuration, it is possible to obtain a mixed normal distribution model for enabling a voice call with a sound quality close to air conduction sound even in a noisy environment.

The internal conduction normal speech conversion learning device according to claim 2 of the present invention is the body conduction normal speech conversion learning device according to claim 1, wherein the analysis result of the first linear prediction analysis unit and the analysis result of the second linear prediction analysis unit are LSFs,
Pre-processing means for performing pre-processing so that the variance value of the LSF calculated by the second linear prediction analysis means becomes large;
The GMM learning unit learns a mixed normal distribution model based on the LSF preprocessed by the preprocessing unit and the LSF calculated by the first linear prediction analysis unit. According to such a configuration, the sound quality can be improved.

The internal conduction normal speech conversion learning apparatus according to claim 3 of the present invention is the preconducted normal speech conversion learning device according to claim 2, wherein the preprocessing by the preprocessing means expands the amplitude at a predetermined ratio centered on the average value with respect to each next LSF. It is a process. According to such a configuration, the sound quality can be improved.
The internal conduction normal speech conversion learning device according to claim 4 of the present invention, in any one of claims 1 to 3, with respect to a frame that is an analysis result of the first linear prediction analysis means in terms of time. First frame connecting means for connecting a plurality of consecutive frames;
A second frame concatenation unit that concatenates a plurality of temporally continuous frames with respect to a frame that is an analysis result of the second linear prediction analysis unit;
The GMM learning means learns a mixed normal distribution model for a plurality of frames respectively connected by the first frame connecting means and the second frame connecting means. According to such a configuration, the sound quality can be further improved.

The internal conduction normal speech converter according to claim 5 of the present invention is provided.
A GMM storage memory for storing a mixed normal distribution model created by the body conduction normal speech conversion learning device according to any one of claims 1 to 3;
Third linear predictive analysis that performs a linear predictive analysis of normal speech conducted in the soft tissue of the body taken from the surface of the skin on the papillary muscle of the thoracic papillary muscle, just below the mastoid of the skull, below the pinna Means,
Analysis filter means for obtaining a prediction residual based on the analysis result of the third linear prediction analysis means and the body conduction normal speech;
Conversion means for converting the analysis result of the third linear prediction analysis means into the analysis result of the pseudo air conduction sound using the mixed normal distribution model stored in the memory for GMM storage;
Synthetic filter means for generating a simulated air conduction sound based on the prediction residual and the analysis result of the simulated air conduction sound. According to such a configuration, it is possible to make a voice call comfortably with a sound quality close to air conduction sound even in an environment where noise is high.

The internal-conduction normal speech conversion device according to claim 6 of the present invention is as follows.
The GMM storage memory stores a mixed normal distribution model created by the body conduction normal speech conversion learning device according to claim 4,
A third frame connecting means for connecting a plurality of temporally continuous frames with respect to the analysis result of the body conduction normal speech before conversion by the converting means;
It further includes frame dividing means for dividing the conversion result by the converting means into a plurality of frames. According to such a configuration, the sound quality can be further improved.

A mobile phone according to claim 7 of the present invention is
A body conduction normal speech conversion learning device according to any one of claims 1 to 4 and a body conduction normal speech conversion device according to claim 5 or 6, wherein the body conduction normal speech conversion device. The pseudo air conduction sound generated by the conversion device is used for a call. According to such a configuration, it is possible to make a voice call comfortably with a sound quality close to air conduction sound even in an environment where noise is high.

The method for learning normal speech conversion in body conduction according to claim 8 of the present invention conducts soft tissue in the body, which is collected from the skin surface on the thoracic papillary muscle in the lower part of the pinna, just below the mastoid process of the skull. A first linear predictive analysis step for performing a linear predictive analysis for the body conduction normal speech; a second linear predictive analysis step for performing a linear predictive analysis for the air conduction sound;
And a GMM learning step of learning a mixed normal distribution model based on the analysis result of the first linear prediction analysis step and the analysis result of the second linear prediction analysis step. According to such a method, it is possible to obtain a mixed normal distribution model for enabling a voice call with a sound quality close to air conduction sound comfortably even in a noisy environment.

The internal conduction normal speech conversion learning method according to claim 9 of the present invention is as follows.
The analysis result of the first linear prediction analysis step and the analysis result of the second linear prediction analysis step are LSFs,
A pre-processing step of performing pre-processing so that the variance value of the LSF calculated in the second linear prediction analysis step increases;
In the GMM learning step, a mixed normal distribution model is learned based on the LSF preprocessed in the preprocessing step and the LSF calculated in the first linear prediction analysis step. According to such a method, sound quality can be improved.

The internal conduction normal speech conversion learning method according to claim 10 of the present invention is as follows.
The pre-processing in the pre-processing step is a process for expanding the amplitude at a predetermined ratio centering on the average value for each next LSF. According to such a method, sound quality can be improved.
The internal conduction normal speech conversion learning method according to claim 11 of the present invention is any one of claims 8 to 10,
A first frame concatenation step of concatenating a plurality of temporally continuous frames with respect to a frame that is an analysis result of the first linear prediction analysis step;
A second frame concatenation step of concatenating a plurality of temporally continuous frames with respect to a frame that is an analysis result of the second linear prediction analysis step;
In the GMM learning step, a mixed normal distribution model is learned for a plurality of frames connected in the first frame connecting step and the second frame connecting step, respectively. According to such a method, the sound quality can be further improved.

The internal conduction normal speech conversion method according to claim 12 of the present invention comprises:
Third linear predictive analysis that performs a linear predictive analysis of normal speech conducted in the soft tissue of the body taken from the surface of the skin on the papillary muscle of the thoracic papillary muscle, just below the mastoid of the skull, below the pinna Steps,
An analysis filter step for obtaining a prediction residual based on the analysis result of the third linear prediction analysis step and the body conduction normal speech;
Using the mixed normal distribution model created by the body conduction normal speech conversion learning method according to any one of claims 8 to 10, the analysis result of the third linear prediction analysis step is simulated. A conversion step for converting into the analysis result of the conduction sound;
And a synthesis filter step of generating a pseudo air conduction sound based on the prediction residual and the analysis result of the pseudo air conduction sound. According to such a method, it is possible to make a voice call comfortably with a sound quality close to air conduction sound even in an environment where noise is high.

The internal conduction normal speech conversion method according to claim 13 of the present invention is as follows.
A third frame linking step for linking a plurality of temporally continuous frames with respect to the analysis result of the body conduction normal speech before conversion by the conversion step;
A frame dividing step of dividing the conversion result of the conversion step into a plurality of frames,
In the conversion step, a mixed normal distribution model created by the body conduction normal speech conversion learning method according to claim 11 is used. According to such a method, the sound quality can be further improved.

  Air conduction sound and body conduction normal voice are voices normally uttered, and as shown in FIGS. 17 (a) and 17 (b), regular waves (pitch) generated by regular vibration of the vocal cords or airway sounds, respectively. After the excitation source (sound source) signal, which is turbulent noise generated by narrowing, is given resonance and semi-resonance characteristics in the articulating organ from the throat to the palate, it propagates through air and soft tissue in the body with different propagation characteristics, This sound is collected by an air conduction microphone or a meat conduction (registered trademark) microphone. Here, the combination of resonance / semi-resonance characteristics and air propagation characteristics by articulatory organs is defined as the air conduction spectrum, and the combination of resonance / semi-resonance characteristics and arterial soft tissue propagation characteristics by the articulatory organs is defined as normal conduction spectrum in the body. To do. Then, as shown in FIGS. 18 (a) and 18 (b), the excitation source (sound source), the air conduction spectrum, and the excitation source (excitation source) are obtained from the air conduction sound and the body conduction normal voice, respectively, as shown in FIGS. It is possible to extract the sound source) and the internal conduction normal speech spectrum, and further synthesize the original speech from the two pieces of information by the analysis filter.

  In the present invention, when the same utterance content is sampled by the air conduction microphone and the meat conduction microphone, and the excitation source (sound source) and the air conduction spectrum and the excitation source (sound source) and the body conduction normal speech spectrum are respectively obtained and compared, Note that the difference between the air conduction spectrum and the body conduction normal speech spectrum occurs, as shown in FIG. 19, first, the body conduction normal speech spectrum extracted from the body conduction normal speech is converted by the conversion process. By approximating the air conduction spectrum and combining it with the excitation source (sound source) information extracted from the body conduction normal speech, a pseudo air conduction sound is synthesized (hereinafter referred to as a pseudo air conduction sound), thereby The sound quality is improved.

  According to the present invention, the sound quality of the body-conducted normal voice is improved to a sound quality close to that of the air-conducted sound, compared to the case where the meat conduction (registered trademark) microphone having the feature that the external noise is structurally difficult to enter. Therefore, it is possible to make a voice call comfortably even in a noisy environment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings referred to in the following description, the same parts as those in the other drawings are denoted by the same reference numerals.
(Configuration example of normal speech exchange device for internal conduction)
1 (a) and 1 (b) are block diagrams showing one embodiment of the internal conduction normal speech conversion device according to the present invention, and FIG. 1 (a) is a processing block diagram during learning. (B) is a process block diagram at the time of conversion. Japanese Patent Application Laid-Open No. 10-254473 uses a primary conversion function or a codebook conversion function for an LPC cepstrum representing spectral characteristics as a method for converting a whispering voice into a normal voice uttered by a normal utterance method. Then, a conversion method has been proposed. In this application, features representing the spectral characteristics of normal body conduction speech and air conduction sound can be expressed as statistical models. As a function for accurately converting these statistical models, a mixed normal distribution model (Gaussian Mixture Model (GMM) is used. In addition, as a parameter representing a spectrum characteristic, stability verification is easy, and in recent years, a line spectrum frequency (LSF) or a line spectrum pair (Line Spectrum Pair) widely used in an audio codec such as a mobile phone is used. : LSP).

  First, FIG. 1A, which is a processing block diagram during learning, will be described. The linear prediction analysis unit 11 cuts out the body-conducted normal speech in units called frames of several tens of ms, and calculates the LSF by linear prediction analysis for each frame. The linear prediction analysis unit 12 cuts out the air conduction sound in units called frames of several tens of ms, and calculates LSF by linear prediction analysis for each frame. Regarding the calculation method of LSF in these linear predictive analysis means 11 and 12, see, for example, Furui, “Speech Information Processing”, Morikita Publishing, March 15, 2002, p. 26-38. The GMM learning means 13 uses the LSF extracted from the body conduction normal speech and air conduction sound of the same utterance content by the linear prediction analysis means as learning data, and converts the LSF of the body conduction normal speech into the LSF of the air conduction sound. Learn the parameters. Specifically, x is the LSF for normal conduction sound, y is the LSF for air conduction sound,

をｘとｙとの結合特徴ベクトルとして、式（１）、式（２）のように結合特徴ベクトルｚの確率分布をＧＭＭで表し、そのパラメータ

Is a combined feature vector of x and y, and the probability distribution of the combined feature vector z is expressed by GMM as shown in equations (1) and (2), and its parameters

を、例えば中川著、「確率モデルによる音声認識」、電子情報通信学会、１９８８年７月１日、Ｐ．５１−５５に記載のＥＭアルゴリズムにより推定する。

, For example, by Nakagawa, “Voice Recognition by Probabilistic Model”, IEICE, July 1, 1988, P.A. Estimated by the EM algorithm described in 51-55.

  Here, m is the number of GMM mixtures, and N (x; μ, Σ) is a normal distribution of the covariance matrix Σ with an average vector μ expressed by Equation (3).

Further, the covariance matrix Σ _i ^(z) and the average vector μ _i ^(z) at the mixture number i are expressed as shown in Expression (4).

The GMM storage memory 14 stores the GMM parameters after learning.
Next, FIG. 1B, which is a processing block diagram at the time of conversion, will be described. The linear prediction analysis means 15 cuts out the body-conducted normal speech in units called frames of several tens of ms, performs linear prediction analysis for each frame in the same way as during learning, and calculates the LSF. The analysis filter means 16 constitutes an LPC analysis filter based on the LSF obtained by the linear prediction analysis means, and calculates a prediction residual (excitation source) from the body conduction normal speech. The conversion means 17 converts the LSF of the body conduction normal speech using the GMM parameters stored in the GMM storage memory 14 according to the equations (5) and (6), and generates the pseudo air conduction sound LSF. .

Here, x is the LSF of the body conduction normal voice, y is the LSF of the air conduction sound, p is the LSF order, and m is the GMM mixture number.
The synthesis filter means 18 constitutes an LPC synthesis filter based on the LSF of the pseudo air conduction sound, and generates the pseudo air conduction sound from the prediction residual (excitation source) of the normal body conduction speech.
FIG. 2 is a block diagram showing a method for evaluating the sound quality of pseudo-air conduction sound obtained by GMM conversion of LSF of body conduction normal speech. As shown in the figure, the LMM extracted from the in-body conduction normal speech and the air conduction sound for learning is obtained by the GMM learning means 13 and stored in the GMM storage memory 14. In addition, for the purpose of evaluation, body conduction normal speech and air conduction sound with the same utterance content are recorded, and the distortion amount and pseudo air between the LSF of the body conduction normal speech before conversion and the LSF of the air conduction sound with the same utterance are recorded. The amount of distortion between the LSF of the conduction sound and the LSF of the air conduction sound of the same utterance is calculated by Expression (7), and it is determined that the synthesized sound is closer to the air conduction sound, that is, the sound quality is better as the distortion of the LSF is smaller.

Here, l _i ^x is the LSF of the target air conduction sound, l _i ^y is the LSF of the normal body conduction voice or the pseudo air conduction sound after conversion, and K is the number of conversion frames.
Figure 3 shows about 700 words of ATR balance words (for learning) uttered by 15 men and women, and about 10 sentences for evaluation of JEITA speech synthesis comprehensive evaluation sentence set using flesh conduction microphone and air conduction microphone. Using simultaneous recording and internal conduction normal speech and air conduction sound, GMM learning is performed from the LSF of learning internal conduction speech and LSF of learning air conduction sound by the method described in Example 1, LSF distortion of LSF of body conduction normal speech before conversion and LSF distortion after GMM conversion in LSF results of conversion from LSF of body conduction normal speech for evaluation using GMM after learning to LSF of pseudo air conduction sound It is the graph which compared the LSF distortion of the LSF of sound. The number of GMM mixtures was 256, and the linear prediction analysis was performed under the conditions shown in FIG. Looking at the average of all 30 persons, it can be seen that the LSF distortion is reduced by 57.7% compared to the case where the conversion is not performed, and the sound quality is greatly improved.

5 and 6 show the LSF of the body conduction normal speech and the LSF of the air conduction sound of the 10 sentences for evaluation uttered by the above 15 women, and the pseudo air conduction converted from the body conduction normal speech by the method of the first embodiment. It is the graph which showed the average value and the dispersion | distribution value for every order in the 1st-10th coefficient which comprises LSF of a sound, respectively (15 persons average).
Referring to FIG. 5, the value of the average LSF of air conduction sound and the average LSF of normal body conduction sound differs in many orders. However, the average LSF of the air conduction sound and the average LSF of the pseudo air conduction sound are almost the same in all orders, and it can be seen that the GMM conversion functions well.

  On the other hand, in FIG. 6, when comparing the graphs with respect to the LSF dispersion value, the peak position and shape are different between the air conduction sound and the body conduction normal sound. However, between the air conduction sound and the pseudo air conduction sound, the peak position and shape are similar, and it can be seen that the GMM conversion functions well from this. However, the LSF dispersion value of the pseudo air conduction sound is only about ½ of the LSF dispersion value of the air conduction sound, and the average value is substantially the same, but the amplitude around the average value is about the pseudo air conduction sound. Is uniformly smaller than the air conduction sound. This is considered to be one of the causes of LSF distortion.

  Therefore, as shown in FIG. 7, the LSF dispersion value of the pseudo air conduction sound is increased in the learning data before the GMM learning by the GMM learning unit 13 so that the LSF dispersion value of the converted pseudo air conduction sound becomes larger. It is considered that such preprocessing is performed by the preprocessing means 19. For example, as shown in FIG. 20, the preprocessing by the preprocessing unit 19 is a process of expanding the amplitude at a constant rate around the average value for each LSF of the air conduction sound.

  FIG. 8 is a block diagram showing another embodiment of the internal conduction normal speech conversion device according to the present invention. FIG. 8 (a) is a processing block diagram during learning, and FIG. 8 (b) is a processing block diagram during conversion. It is. The difference from the configuration of the first embodiment described with reference to FIG. 1 is that in FIG. 8A, after the linear predictive analysis means 12 for calculating the LSF from the air conduction sound, each of the LSFs of the air conduction sound. Thus, preprocessing means 19 is included for performing processing to increase the amplitude at a constant rate with the average value as the center. Since the other block functions are the same as those in the first embodiment, description thereof is omitted.

  FIG. 9 shows a speech sound of about 700 ATR balance words (for learning) uttered by 15 men and women, and a sentence voice for evaluation of about 10 JEITA speech synthesis comprehensive evaluation sentence sets using a meat conduction microphone and an air conduction microphone. Simultaneously recording and using the internal conduction normal speech and air conduction sound, GMM learning is performed from the LSF of the learning internal conduction normal speech and the learning air conduction sound LSF by the apparatus of Example 2, LSF distortion of LSF of body conduction normal speech before conversion and LSF distortion after GMM conversion in LSF results of conversion from LSF of body conduction normal speech for evaluation using GMM after learning to LSF of pseudo air conduction sound It is the graph which also compared the result of Example 1 with the LSF distortion of the LSF of sound. The number of GMM mixtures and the linear prediction analysis conditions were the same as in Example 1. As preprocessing for the air conduction sound LSF before GMM learning, the amplitude was expanded by 10% centering on the average value for each order. Looking at the average of all 30 people, the LSF distortion is 62.2% lower than that without conversion, and 10.7% lower than that in Example 1. Further improvement in sound quality is expected compared to Example 1. You can see that With respect to the processing amount, the pretreatment for the LSF of the air conduction sound is increased from that in the first embodiment. However, since this increase in processing occurs only during learning, the load during conversion does not increase.

  The following points are raised as problems in using the GMM for the conversion function. That is, since the GMM is a statistical model, a conversion function is generated with high accuracy for a combination of a conversion source and a conversion destination feature quantity pattern that appears stably during learning. On the other hand, there is a problem in that a combination of feature quantity patterns with a small number of stable appearances cannot be sufficiently learned, and as a result, there is a tendency to convert to a feature quantity pattern at a conversion destination close to the average value. .

  Considering the case of the first embodiment, it follows from phonemes (minimum basic unit of speech such as “a”, “i”, “u”, “e”, “o”, “k”, “s”). In the transition part to the phoneme, the LSF pattern is greatly affected by the preceding and following phonemes and is not stable. For example, the LSF pattern in the transition section from “a” to “i” is also the transition section from “a” to “u”. This LSF pattern is also converted to a pattern close to the average value of “a”. As a result, it is considered that the LSF dispersion value of the pseudo air conduction sound is smaller than the LSF dispersion value of the air conduction sound as shown in FIG. As one solution, it is conceivable to extend the linear prediction analysis interval to such an extent that it includes a transition part between phonemes and a steady part of phonemes in which an LSF pattern appears stably. However, from the viewpoint of time resolution and frequency resolution, it is not preferable to extend the analysis interval.

  Therefore, as shown in FIG. 10, it is considered that a plurality of LSF patterns of continuous frames on the time axis are connected to form a new feature pattern for the audio signal. In this example, LSF (i−1), LSF (i), LSF (i + 1), which are LSF patterns corresponding to three consecutive frames of (i−1) frame, (i) frame, and (i + 1) frame. To form a new feature pattern. Since the feature pattern of each frame is 10-dimensional, the combined feature pattern of LSF (i) is 30-dimensional. The number of frames to be combined is not limited to “3”, and two or more frames may be combined.

FIG. 11 is a block diagram showing still another embodiment of the internal conduction normal speech conversion device according to the present invention. FIG. 11 (a) is a processing block diagram during learning, and FIG. 11 (b) is a processing block during conversion. FIG.
First, the learning process will be described with reference to FIG.
In the same figure, the linear prediction analysis means 11 cuts out the body-conducting normal speech in units of several tens of frames and calculates the LSF by linear prediction analysis for each frame. Similarly, the linear prediction analysis means 12 cuts out the air conduction sound in units of frames of several tens of ms, and calculates LSF by linear prediction analysis for each frame. Regarding the LSF calculation method in the linear predictive analysis means 11 and 12, see, for example, Furui, “Speech Information Processing”, Morikita Publishing, March 15, 2002, p. 24-26, P.I. 106-111.

The frame connecting means 21 and 22 store and connect the LSF of the frame unit calculated by the linear prediction analyzing means 11 and 12 for a predetermined number of frames, and output as a new LSF pattern. The frame connecting means 21 and 22 are configured using, for example, a buffer that can store frames for a predetermined number of frames.
The GMM learning means 13 uses the LSF of the normal body conduction speech connected for the predetermined number of frames and the LSF of the air conduction sound connected for the predetermined number of frames as learning data, and the combined internal conduction. GMM parameters for converting a normal speech LSF into a combined air conduction sound LSF are learned. Since a specific learning method is the same as that in the first embodiment, description thereof is omitted. The GMM storage memory 14 stores the GMM parameters after learning.

Next, with reference to FIG. 11B, a process at the time of conversion will be described.
In the figure, the linear prediction analysis means 15 cuts out the body-conducting normal speech in units of frames of several tens of ms, performs linear prediction analysis for each frame as in the learning, and calculates the LSF. The analysis filter means 16 constitutes an LPC analysis filter based on the LSF obtained by the linear prediction analysis means 15, and calculates a prediction residual (excitation source) from the normal body conduction speech. The converting means converts the LSF of the body conduction normal speech using the GMM parameters stored in the GMM storage memory according to the equations (4) and (5), and generates the LSF of the pseudo air conduction sound.

The frame connecting means 23 stores and connects the LSFs in units of frames calculated by the linear prediction analyzing means 15 for the predetermined number of frames, which is the same as during learning, and outputs the result as a new LSF pattern. The frame connecting means 23 is configured using, for example, a buffer capable of storing frames for a predetermined number of frames.
The conversion means 17 converts the LSF connected for a predetermined number of frames of the normal body conduction speech using the GMM parameters stored in the GMM storage memory 14 according to the expressions (4) and (5). Then, LSFs of pseudo air conduction sounds connected for a predetermined number of frames are generated.

The frame dividing unit 24 divides the LSFs of the pseudo air conduction sound connected by a predetermined number of frames into LSFs in units of frames. For example, the frame dividing unit 24 is configured to accumulate frames for a predetermined number of frames in a buffer, and to divide and read the frames accumulated in the buffer.
The synthesis filter means 18 constructs an LPC synthesis filter based on the LSF of the pseudo air conduction sound for each frame, and generates the pseudo air conduction sound from the prediction residual (excitation source) of the normal body conduction speech.

FIG. 12 shows a word speech of about 700 ATR balance words (for learning) uttered by 15 men and women, and an evaluation sentence speech of about 10 sentences of the JEITA speech synthesis comprehensive evaluation sentence set as a meat conduction microphone and an air conduction microphone. Are simultaneously recorded and used as the body conduction normal voice and the air conduction sound, and the device of Example 3 performs GMM learning from the LSF of the learning body conduction normal voice and the LSF of the learning air conduction sound. LSF distortion of LSF of body conduction normal speech before conversion and LSF distortion after GMM conversion in LSF results of conversion from LSF of body conduction normal speech for evaluation using GMM after learning to LSF of pseudo air conduction sound It is the graph which also compared the result of Example 1 with the LSF distortion of the LSF of sound. Note that the number of GMM mixtures and the linear prediction analysis conditions are the same as in the first embodiment, and the number of connected frames is three.
In the figure, looking at the average of all 30 persons, the LSF strain was reduced by 62.5% compared to the case without conversion, and by 11.4% compared to Example 1, and from Examples 1 and 2 It can be seen that the sound quality is further improved.

Further, FIG. 13 shows the LSF of the air conduction sound of the 10 sentences for evaluation uttered by the above 15 women, the LSF of the pseudo air conduction sound converted from the normal body conduction sound by the method of the first embodiment, and the third embodiment. It is the graph which showed the dispersion | distribution value (15 persons average) for every order in the 1st-10th coefficient which comprises LSF of the pseudo air conduction sound converted from the body conduction normal sound by the system. Referring to FIG. 13, between the air conduction sound, the simulated air conduction sound by the apparatus of Example 1, and the simulated air conduction sound by the apparatus of Example 3, the position and shape of the peak are similar. Further, the waveform of the pseudo air conduction sound by the apparatus of the third embodiment is closer to the waveform of the air conduction sound than the waveform of the pseudo air conduction sound by the apparatus of the first embodiment. This also confirms the effectiveness of frame connection.
In the third embodiment, as in the case of the second embodiment, pre-processing may be performed on each order LSF of air conduction sound so that the amplitude is increased at a constant rate around the average value.

(Mobile phone system)
FIG. 21 is a schematic configuration diagram of a communication interface system using a mobile phone including a body conduction normal speech conversion learning device and a body conduction normal speech conversion learning device.
The meat conduction microphone 1-1 is attached by being attached to 1-2 directly below the milky protrusion, and the earphone 1-3 or the speaker is attached to the ear hole. The meat conduction microphone 1-1 and the earphone 1-3 are connected to the cellular phone 1-4 by wired or wireless communication means. A speaker may be used instead of the earphone 1-3.

The radio network 1-5 includes, for example, radio base stations 51a and 51b, base station controllers 52a and 52b, exchanges 53a and 53b, and a communication network 50. In this example, the cellular phone 1-4 communicates wirelessly with the radio base station 51a, and the cellular phone 1-6 communicates wirelessly with the radio base station 51b. Calls can be made between the two.
The vibration sound of the normal utterance sound 1-7 that has reached 1-2 immediately below the mastoid process is collected by the meat conduction microphone 1-1 attached thereto, and becomes an electric signal by the condenser microphone in the meat conduction microphone. This signal is transmitted to the cellular phone 1-4 by wired or wireless communication means.

The vibration sound of the normal utterance sound transmitted to the cellular phone 1-4 is transmitted to the cellular phone 1-6 possessed by the other party via the wireless network 1-5.
On the other hand, the voice of the other party is transmitted to the earphone 1-3 or the speaker via the mobile phone 1-6, the wireless network 1-5, and the mobile phone 1-4 by wired or wireless communication means. Note that the earphone 1-3 is not necessary when listening directly from the mobile phone 1-4.
As a result, it is possible to talk with the other party even in a noisy environment.
In short, in this example, a communication interface system is configured by combining a meat conduction microphone and a mobile phone as a signal processing device.

(Body conduction normal speech conversion method)
In the above-described body conduction normal speech conversion device, the body conduction normal speech conversion method shown in FIGS. 14, 15, and 16 is realized.
Referring to FIG. 14 (a) showing the processing flow at the time of learning, conduction is performed through the soft tissue in the body, which is collected by the skin surface on the thoracic papillary muscle in the lower part of the pinna, directly below the mastoid process of the skull. A first linear prediction analysis step (S101) for performing a linear prediction analysis on the body conduction normal speech to be performed, a second linear prediction analysis step (S102) for performing a linear prediction analysis on the air conduction sound, and the first linear prediction. A GMM learning step (S103) for learning and storing a mixed normal distribution model based on the analysis result of the analysis step and the analysis result of the second linear prediction analysis step is sequentially executed. In the first linear prediction analysis step (S101) and the second linear prediction analysis step (S102), body conduction normal speech and air conduction sound are cut out in units called frames of several tens of ms, and linear prediction analysis is performed for each frame. To calculate the LSF. Regarding these linear prediction analysis methods, see, for example, Furui, “Speech Information Processing”, Morikita Publishing, Mar. 15, 2002, P.A. 26-38. As the GMM learning method, the method described in the configuration example of the internal conduction normal speech conversion device can be used. Next, referring to FIG. 14 (b) showing the processing flow at the time of conversion, the body-conducted normal speech is cut out in units called frames of several tens of ms, and linear prediction analysis is performed for each frame in the same way as during learning, and LSF The linear prediction analysis step (S104) for calculating the LPC, and the analysis filter processing for constructing the LPC analysis filter based on the LSF obtained by the linear prediction analysis means and calculating the prediction residual (excitation source) from the body conduction normal speech Using the GMM parameters learned and stored in the GMM learning step (S103), the LSF of step (S105) is converted to LSF of the pseudo air conduction sound using the equations (5) and (6) A step (S106) and an LPC synthesis filter are configured based on the LSF of the pseudo air conduction sound, and the pseudo air conduction sound is determined from the prediction residual (excitation source) of the body conduction normal speech. Generating synthesized filtering step (S107) are sequentially executed. According to this body conduction normal voice conversion method, it is possible to make a voice call with a sound quality close to air conduction sound even in a noisy environment.

  Further, as shown in FIG. 15 (a), a preprocessing step (S108) for performing preprocessing for increasing the LSF dispersion value of the pseudo air conduction sound is further performed on the analysis result in the second linear prediction analysis step. In the GMM learning step (S103), a mixed normal distribution model is generated based on the analysis result preprocessed in the preprocessing step (S108) and the analysis result in the first linear prediction analysis step (S101). You may make it learn. According to this body conduction normal speech conversion method, sound quality can be improved. In addition, the preprocessing by the said preprocessing step is a process which expands an amplitude with a fixed ratio centering on an average value with respect to each order LSF of an air conduction sound, for example.

  Furthermore, as shown in FIG. 16 (a), a first frame concatenation step (S109) for concatenating a plurality of temporally continuous frames with respect to a frame that is an analysis result of the first linear prediction analysis step; A second frame concatenation step (S110) for concatenating a plurality of frames that are temporally continuous with respect to a frame that is an analysis result of the second linear prediction analysis step, and the GMM learning step (S103) includes the second frame concatenation step. A mixed normal distribution model is learned for a plurality of frames connected in the first frame connection step (S109) and the second frame connection step (S110), and the conversion step in the processing flow at the time of conversion in FIG. With respect to the normal body-conducted speech before conversion in (S106), a plurality of temporally continuous files A third frame connecting step of connecting the over arm (S 111), the frame division step (S112) of dividing the conversion result by the conversion step into a plurality of frames and may further include a. According to such a configuration, the sound quality can be further improved.

  INDUSTRIAL APPLICABILITY The present invention can be used for a comfortable voice call with sound quality close to air conduction sound even in a noisy environment.

It is a block diagram which shows one Embodiment of the body conduction normal speech converter by this invention, (a) is a processing block diagram at the time of learning, (b) is a processing block diagram at the time of conversion. It is the block diagram which showed the evaluation method of the sound quality of the pseudo air conduction sound obtained by carrying out GMM conversion of LSF of the body conduction normal sound. It is the graph which compared the LSF distortion of LSF of the body conduction normal sound before conversion, and the LSF distortion of LSF of the pseudo air conduction sound after GMM conversion. It is a figure which shows the conditions of a linear prediction analysis. LSF of body conduction normal speech and LSF of air conduction sound of 10 sentences for evaluation uttered by 15 women, LSF of pseudo air conduction sound converted from body conduction normal speech by the method of Example 1 It is the graph which showed the average value for every order in the 10th-order coefficient. LSF of body conduction normal speech and LSF of air conduction sound of 10 sentences for evaluation uttered by 15 women, LSF of pseudo air conduction sound converted from body conduction normal speech by the method of Example 1 It is the graph which showed the dispersion value for every order in the 10th-order coefficient. It is a figure which shows the pre-processing given to the learning data before GMM learning so that the LSF dispersion value of a pseudo air conduction sound may become large. It is a block diagram which shows the other form of implementation of the body conduction normal speech converter by this invention, (a) is a processing block diagram at the time of learning, (b) is a processing block diagram at the time of conversion. It is the graph which compared the LSF distortion of the LSF of the body conduction normal sound before conversion, and the LSF distortion of the LSF of the pseudo air conduction sound after the GMM conversion. It is a figure which shows the view which connects the LSF pattern of a continuous flame | frame and makes it a new feature pattern. FIG. 11 is a block diagram showing still another embodiment of the internal conduction normal speech conversion method according to the present invention, where (a) is a processing block diagram during learning and (b) is a processing block diagram during conversion. It is the graph which compared the LSF distortion of the LSF of the body conduction normal sound before conversion, and the LSF distortion of the LSF of the pseudo air conduction sound after the GMM conversion. It is the graph which showed the dispersion | variation value for every order in the 1st to 10th coefficient which comprises LSF of the pseudo air conduction sound and LSF of the pseudo air conduction sound converted from the body conduction normal voice. It is a flowchart which shows the body conduction normal speech conversion method by this invention, (a) is a flowchart at the time of learning, (b) is a flowchart at the time of conversion. It is a flowchart which shows the body conduction normal speech conversion method by this invention, (a) is a flowchart at the time of learning, (b) is a flowchart at the time of conversion. It is a flowchart which shows the body conduction normal speech conversion method by this invention, (a) is a flowchart at the time of learning, (b) is a flowchart at the time of conversion. (A) is a figure which shows the example of the sampling method of an air conduction sound, (b) is a figure which shows the example of the sampling method of a body conduction normal sound. (A) is a figure which shows the example of the synthetic | combination method of an air conduction sound, (b) is a figure which shows the example of the synthesis | combination method of a body conduction normal sound. It is a figure which shows the outline | summary of the internal-conduction normal speech converter by this invention. It is a figure which shows the view which expands the amplitude of LSF of each order centering on an average value. It is a schematic block diagram of the communication interface system using the mobile telephone containing a body conduction normal speech conversion learning apparatus and a body conduction normal speech conversion apparatus.

Explanation of symbols

1-1 Meat conduction microphone 1-2 Directly below milky process 1-3 Earphone 1-4, 1-6 Mobile phone 1-5 Wireless network 11, 12 Linear prediction analysis means 13 GMM learning means 14 GMM storage memory 15 Linear prediction Analysis means 16 Analysis filter means 17 Conversion means 18 Synthesis filter means 19 Preprocessing means 21, 22, 23 Frame connection means 24 Frame division means 50 Communication networks 51a, 51b Radio base stations 52a, 52b Base station controllers 53a, 53b Exchange

Claims (13)

First linear predictive analysis that performs linear predictive analysis on normal body conduction speech conducted through the soft tissue of the body, taken from the surface of the skin on the thoracic papillary muscle, just below the mastoid process of the skull, below the auricle. Means, second linear prediction analysis means for performing linear prediction analysis on the air conduction sound,
GMM learning means for learning a mixed normal distribution model based on the analysis result of the first linear prediction analysis means and the analysis result of the second linear prediction analysis means; Conversion learning device. The analysis result of the first linear prediction analysis means and the analysis result of the second linear prediction analysis means are LSF,
Pre-processing means for performing pre-processing so that the variance value of the LSF calculated by the second linear prediction analysis means becomes large;
The GMM learning unit learns a mixed normal distribution model based on the LSF preprocessed by the preprocessing unit and the LSF calculated by the first linear prediction analysis unit. Body conduction normal speech conversion learning device.   3. The internal conduction normal speech conversion learning device according to claim 2, wherein the preprocessing by the preprocessing means is a process of expanding the amplitude at a predetermined ratio centering on an average value with respect to each next LSF. First frame concatenation means for concatenating a plurality of temporally continuous frames with respect to a frame that is an analysis result of the first linear prediction analysis means;
A second frame concatenation unit that concatenates a plurality of temporally continuous frames with respect to a frame that is an analysis result of the second linear prediction analysis unit;
4. The GMM learning unit learns a mixed normal distribution model for a plurality of frames connected by the first frame connecting unit and the second frame connecting unit, respectively. The internal conduction normal speech conversion learning device according to claim 1. A GMM storage memory for storing a mixed normal distribution model created by the body conduction normal speech conversion learning device according to any one of claims 1 to 3;
Third linear predictive analysis that performs a linear predictive analysis of normal speech conducted in the soft tissue of the body taken from the surface of the skin on the papillary muscle of the thoracic papillary muscle, just below the mastoid of the skull, below the pinna Means,
Analysis filter means for obtaining a prediction residual based on the analysis result of the third linear prediction analysis means and the body conduction normal speech;
Conversion means for converting the analysis result of the third linear prediction analysis means into the analysis result of the pseudo air conduction sound using the mixed normal distribution model stored in the memory for GMM storage;
And a synthetic filter means for generating a simulated air conduction sound based on the prediction residual and the analysis result of the simulated air conduction sound. The GMM storage memory stores a mixed normal distribution model created by the body conduction normal speech conversion learning device according to claim 4,
A third frame connecting means for connecting a plurality of temporally continuous frames with respect to the analysis result of the body conduction normal speech before conversion by the converting means;
6. The body conduction normal sound converting apparatus according to claim 5, further comprising a frame dividing unit that divides a conversion result obtained by the converting unit into a plurality of frames.   A body conduction normal speech conversion learning device according to any one of claims 1 to 4 and a body conduction normal speech conversion device according to claim 5 or 6, wherein the body conduction normal speech conversion device. A cellular phone characterized in that a pseudo air conduction sound generated by a conversion device is used for a telephone call. First linear predictive analysis that performs linear predictive analysis on normal body conduction speech conducted through the soft tissue of the body, taken from the surface of the skin on the thoracic papillary muscle, just below the mastoid process of the skull, below the auricle. A second linear prediction analysis step for performing linear prediction analysis on the air conduction sound;
And a GMM learning step of learning a mixed normal distribution model based on the analysis result of the first linear prediction analysis step and the analysis result of the second linear prediction analysis step. Conversion learning method. The analysis result of the first linear prediction analysis step and the analysis result of the second linear prediction analysis step are LSFs,
A pre-processing step of performing pre-processing so that the variance value of the LSF calculated in the second linear prediction analysis step increases;
9. The mixed normal distribution model is learned in the GMM learning step based on the LSF preprocessed in the preprocessing step and the LSF calculated in the first linear prediction analysis step. Body conduction normal speech conversion learning method.   10. The in-body conduction normal speech conversion learning method according to claim 9, wherein the preprocessing in the preprocessing step is a process of expanding the amplitude at a predetermined ratio centering on an average value for each next LSF. A first frame concatenation step of concatenating a plurality of temporally continuous frames with respect to a frame that is an analysis result of the first linear prediction analysis step;
A second frame concatenation step of concatenating a plurality of temporally continuous frames with respect to a frame that is an analysis result of the second linear prediction analysis step;
11. The mixed normal distribution model is learned for a plurality of frames connected in the first frame connecting step and the second frame connecting step in the GMM learning step, respectively. The body conduction normal speech conversion learning method according to any one of the above items. Third linear predictive analysis that performs a linear predictive analysis of normal speech conducted in the soft tissue of the body taken from the surface of the skin on the papillary muscle of the thoracic papillary muscle, just below the mastoid of the skull, below the pinna Steps,
An analysis filter step for obtaining a prediction residual based on the analysis result of the third linear prediction analysis step and the body conduction normal speech;
Using the mixed normal distribution model created by the body conduction normal speech conversion learning method according to any one of claims 8 to 10, the analysis result of the third linear prediction analysis step is simulated. A conversion step for converting into the analysis result of the conduction sound;
And a synthesis filter step for generating a simulated air conduction sound based on the prediction residual and the analysis result of the simulated air conduction sound. A third frame linking step for linking a plurality of temporally continuous frames with respect to the analysis result of the body conduction normal speech before conversion by the conversion step;
A frame dividing step of dividing the conversion result of the conversion step into a plurality of frames,
13. The internal conduction normal speech conversion method according to claim 12, wherein a mixed normal distribution model created by the internal conduction normal speech conversion learning method according to claim 11 is used in the conversion step.

Images