JP2011205389A - Acoustic reproducing device having sound quality automatic adjusting function and hands-free telephone incorporated with the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make the sound reproduction of the most suitable sound quality to a receiver to be automatically executed even though an environment of a circumference is varied in each use in an acoustic reproducing device such as a hands-free telephone etc.SOLUTION: Acoustic characteristics of a transmission voice which is easy to be heard by the receiver are previously stored in a transmission voice feature storing unit 212, then a filter is formed by a filter formation unit 250 based on acoustic characteristics of a selected transmission voice and an acoustic transfer function from a speaker selected by a vehicle transfer function storing unit 222 to the receiver when the voice is really reproduced, and the sound quality adjustment of transmission voice is performed so as to make the voice to be heard easily by the receiver. Furthermore, the acoustic characteristics of the transmission voice which is easy for hearing by a user are automatically learned from the transmission voice at a time when the user usually performs conversation by a telephone, and then the information is utilized for adjusting the sound quality of the acoustic reproducing device.

Description

  The present invention relates to a sound reproduction device such as a hands-free telephone device, and more particularly to a sound reproduction device having an automatic sound quality adjustment function.

  There is a commercially available hands-free telephone device that can make a phone call without holding a mobile phone by hand. The sound quality of hands-free telephone devices is mostly designed and shipped by development manufacturers before shipment. As for the sound quality of the telephone, the sound quality that is easy to hear varies depending on the user. Therefore, it is desirable to be able to adjust the sound quality that the user feels optimal. This can be realized by designing the user so that the user can set the equalizer or the like for the output sound of the hands-free telephone device.

However, even when the same user uses the same hands-free telephone device, the sound quality in the vehicle changes depending on the scene and environment. Therefore, in order to always maintain the optimum sound quality, it is necessary for the user himself to change the setting each time, which takes time and effort. In addition, during operation, the acoustic environment frequently changes due to frictional noise with the road surface, engine driving sound, air conditioner sound, etc., and it is difficult for the user to perform adjustments by operating each time.
In addition, it is considered that there will be more opportunities for the same user to ride various cars using services such as car sharing and car rental without own cars. In that case, it takes time and effort to set the sound quality each time the vehicle type or hands-free telephone device changes.

  Patent Document 1 discloses a technique related to a hearing aid used in a mobile phone. According to this document, there is disclosed a method for determining a user's mobile phone device, determining a noise environment in which the user has a conversation, and determining a filter for hearing aid optimal for the user.

JP 2002-223500 A

  In the technology disclosed in Patent Document 1, a hand-held phone such as a mobile phone is assumed, and a case where there is a distance between a speaker and a microphone and a user, such as a hands-free phone, has not been considered. . Moreover, the method of investigating a user's favorite acoustic environment is not disclosed.

  According to the present invention, in an audio reproduction device such as a hands-free telephone device, sound reproduction with an optimum sound quality for a user (listener) can be automatically performed even if the surrounding environment changes every time it is used. The purpose is to.

  In the present invention, in a sound reproduction device such as a hands-free telephone device, the acoustic characteristics (speech speech characteristics) of the transmitted speech from the talker that is easy for the user (listener) to hear are stored, and the actual speech is stored. At the time of reproduction, sound quality adjustment is performed based on the acoustic characteristics of the selected transmitted voice and the acoustic transfer function from the speaker to the listener so that the user can easily hear the voice. Furthermore, the user automatically learns the acoustic characteristics of the transmitted voice that is easy to hear from the transmitted voice from the talker when the user usually talks on the phone, and uses this information to adjust the sound quality of the sound playback device. .

  The acoustic reproduction device of the present invention includes a transmission voice feature storage unit that stores the acoustic characteristics of one or a plurality of transmission voices, a speaker identification unit that identifies a speaker or a speaker attribute, and the transmitter. Spoken voice that selects and acquires the acoustic characteristics of the transmitted voice from the acoustic characteristics of the transmitted voice stored in the spoken voice feature storage unit based on the information of the speaker specified by the speaker specifying unit The feature selection unit, a transfer function storage unit that stores one or more transfer functions from the speaker that reproduces the transmitted voice to the position of the listener, and a transfer function that is stored in the transfer function storage unit A transfer function selection unit that selects and acquires a transfer function based on information including the position information of the person, the acoustic characteristics of the transmitted voice selected by the transmission voice feature selection unit, and the transfer selected by the transfer function selection unit Based on the function, the playback sound at the listener's location A filter creation unit that creates a filter whose echo characteristics are close to the acoustic characteristics of the transmitted voice selected by the transmitted voice feature selection unit, and a filter process based on the filter created by the filter creation unit is performed on the reproduced sound And a filter processing unit.

  The sound reproducing apparatus of the present invention further includes a learning unit, and the learning unit adjusts the sound quality of the transmitted voice, and the transmitted voice whose sound quality is adjusted by the transmitted voice adjusting unit. Based on the output of the transmitted voice analysis unit, the voice characteristic storage unit that stores the acoustic characteristics of the transmitted voice output by the transmitted voice analysis unit, and the output of the transmitted voice analysis unit A speech feature learning unit that learns the acoustic characteristics of speech and stores the learning results in the speech feature storage unit, and stores the transmitted speech acoustic characteristics of the transmitted speech in the speech feature storage unit of the learning means. It is used as an acoustic characteristic of the transmitted voice of a part.

  According to the present invention, it is possible to perform sound reproduction with optimum sound quality for a user who uses a sound reproduction device such as a hands-free telephone with little effort.

It is a figure which shows the whole structure of the hands-free telephone apparatus of Example 1 of this invention. It is a figure which shows the structure of the filter design part of Example 1 of this invention. It is a flowchart of the process of the transmission voice characteristic selection part of Example 1 of this invention. It is a flowchart of a process of the vehicle transfer function selection part of Example 1 of this invention. It is a flowchart of a process of the vehicle interior acoustic environment estimation part of Example 1 of this invention. It is a flowchart of a process of the filter preparation part of Example 1 of this invention. It is a figure explaining the data of Example 1 of this invention. It is a figure which shows the whole structure of the learning means of Example 2 of this invention. It is a figure which shows the structure of the transmission voice characteristic learning part of Example 2 of this invention. It is a flowchart of the process of the determination part of Example 2 of this invention. It is a flowchart of a process of the audio | voice feature learning part of Example 2 of this invention.

  Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings, taking a hands-free telephone device in a vehicle as an example. The terms “receiver”, “speaker”, “transmitted voice”, and “received voice” in the present invention will be described by taking a hands-free telephone device as an example. The “speaker” refers to a person who is the other party of the telephone and who transmits to the receiver. "Speech" means the voice that the talker speaks and receives by the user's receiver, and "received voice" means the voice that the user's talker speaks and sends to the talker. .

  As a first embodiment, a hands-free telephone device having an automatic sound quality adjustment function will be described.

[overall structure]
FIG. 1 is a diagram showing an overall configuration of a hands-free telephone device having a sound quality adjusting function according to the present invention. The telephone line network 110 is a general line network in which general landline phones and mobile phones can be connected to make a call. The mobile phone 120 is a mobile phone used for a hands-free phone.

The hands-free telephone device 125 is a system for making a hands-free telephone having a sound quality adjustment function disclosed in the present invention.
The control unit 130 performs control generally provided in the hands-free telephone device. Specifically, it is processing of communication with the mobile phone 120, control of transmitted voice / received voice, and control of various modules. Further, it may have a connection device between a mobile phone and the Internet so that information can be exchanged with a server. For communication between the mobile phone 120 and the hands-free telephone device 125 performed in the control unit 130, a device provided in a commercially available device can be used. That is, it may be connected by wire or may communicate wirelessly by a method defined by the Bluetooth standard.
The speaker 190 reproduces sound in order to let the user hear the transmitted sound.
The microphone 180 collects the voice uttered by the user.

The transmitted voice adjustment unit 134 adjusts the sound quality of the sound (transmitted sound) to be heard by the user (receiver) of the hands-free phone. Specifically, an equalizing process for changing the sound pressure level for each frequency band is performed.
The received voice adjustment unit 132 adjusts the sound quality of the voice (received sound) uttered by the user of the hands-free phone. This includes noise canceling processing for reducing running noise in addition to equalizing for each frequency band.
As equalizing methods, methods using digital filters such as FIR filters and IIR filters and methods based on FFT analysis are known. As a noise canceling method, a spectral subtraction method and an MMSE STSA method are known. Since these methods are well-known, description is abbreviate | omitted.

The echo cancellation unit 140 performs a process of erasing an echo that occurs when the microphone 180 collects sound reproduced from the speaker 190. For this method, the following methods in known literature can be used.
FK Soong, AM Peterson: `` Fast least-squares (LS) in the voice echo cancellation application, '' Proc. ICASSP, pp.1398-1403, 1982
The filter design unit 160 determines an amplification factor for each frequency band for performing sound quality adjustment on the voice to be heard by the user. This process will be described later.
Based on the amplification factor of each frequency band designed by the filter design unit 160, the filter processing unit 170 performs signal processing on the transmission sound output from the transmission voice adjustment unit 134. This process will be described later.
The vehicle information acquisition unit 150 acquires information on the vehicle. This will be described later.

[Filter design department]
FIG. 2 is a diagram showing the details of the filter design unit 160.

The transmitted voice analysis unit 240 calculates a frequency-power characteristic for the voice output from the transmitted voice adjustment unit 134.
In this embodiment, the frequency-power characteristics are calculated by several methods. Several methods are known for calculating the power in the frequency domain, and such known methods can be used.
In this embodiment, a method using FFT is used. First, a certain length of speech is cut out from the input speech waveform. This time length is called a frame length, and a value of about 10 ms is often used. Secondly, a time window for suppressing side lobes such as Hanning window or Hamming window is applied to the extracted voice. Third, the FFT operation is performed on the waveform multiplied by the time window. As a result, the waveform in the time domain is converted into values of the real part and the imaginary part in the frequency domain. Finally, a square value of the magnitude is obtained for each value obtained by the FFT calculation. Thereby, the frequency-power characteristic at each time can be obtained. This process is sequentially repeated at a prescribed frame interval (here, ¼ of the frame length).
As an alternative to this FFT method, a method of obtaining a frequency spectrum envelope by performing LPC analysis on a speech waveform for each frame can be used.

  Further, the frequency axis interval in the frequency-power characteristics will be described. When FFT is simply performed, frequency power characteristics can be obtained with a resolution of the frequency that is the reciprocal of the sampling period. However, in this embodiment, power is obtained for each frequency band in consideration of hearing in order to convert it into sound that is optimally heard on human hearing. Known definitions of such frequency bands include critical band filter and bark band filter. In accordance with these known frequency band definitions, the power obtained by the FFT operation is used as the power in each frequency band by taking the sum after a predetermined weighting.

[Acquisition of transmitted voice characteristics]
The speaker identification unit 210 and the transmission voice feature selection unit 211 specify who the speaker is, and further acquire a voice feature corresponding to the speaker. This flow will be described with reference to the flowcharts of FIGS.
The flowchart of FIG. 3 is started when an incoming call is received. In addition, the process is terminated by performing it once for each incoming call.

In step 310, the speaker identification unit 210 identifies who is making the call. As this method, a method in which a telephone directory provided in a mobile phone is referred to based on a caller's telephone number notification function generally used in a telephone can be used. In addition, various voice features are stored in a transmitted voice feature storage unit 212 described later. Therefore, speaker recognition using the information may be performed. As a method for speaker recognition, a recognition method based on a long-time spectrum of speech and a recognition method based on dynamic features of a cepstrum are known, and are described in detail in the following documents.
Sadaaki Furui: “Speech Information Processing”, Morikita Publishing, 1998
In these methods, the transmitted voice output from the transmitted voice adjusting unit 134 is analyzed in the frequency domain by the transmitted voice analyzing unit 240 to calculate the feature amount, and the transmitted voice feature storing unit 212 It can be implemented by calculating the distance from each speaker's voice feature value stored in.
Further, the speaker identification unit 210 may output only information related to the attributes of the speaker. For example, it is possible to output information indicating any attribute of the speaker, such as whether the person is male or female, and how old the age is. In the following, even when only the attributes of such a speaker are specified, it can be performed in the same manner as in the case of specifying the speaker, so only the case where the speaker is specified will be described.

In step 320, the process branches depending on whether or not the speaker (or the attribute of the speaker) has been identified in step 310. If specified, go to step 330.
In step 330, the transmitted voice feature selection unit 211 searches for the voice feature of the sender according to the sender (or the attribute of the sender) specified in step 320. The search target is data stored in the transmitted voice feature storage unit 212.

The transmitted voice feature storage unit 212 will be described. The transmitted voice feature storage unit 212 stores transmitted voice features that are easy to hear for the user for each speaker. The feature of the transmitted voice that is easy to hear for the user for each speaker is specified separately, and any method may be used. In the present embodiment, a method for learning the features of transmitted voice that is easy to hear for each user by a user from a usual telephone conversation will be described later.
The form of storage in the transmitted voice feature storage unit 212 will be described with reference to FIG.

As the characteristics of the transmitted voice, one or more of the lower limit sound pressure level, the upper limit sound pressure level, and the average sound pressure level in the conversation voice in each frequency band of the specified sender are stored. To do. The lower limit sound pressure level, upper limit sound pressure level, and average sound pressure level written here are used later in the filter creation unit 250. The sound pressure level written here may be in any form as long as it is a value converted into sound pressure, and may be intensity or power. The value holding form may be a logarithmic value or a linear value, but will be described as a logarithmic value in units of decibels for convenience.
The lower limit sound pressure level is the lowest sound pressure level that can occur when the user listens to the conversational voice of the talker over the phone, and the lowest sound pressure level that must be compensated for in order to make the sound easier for the user to hear Referred to as
The upper limit sound pressure level is the maximum sound pressure level that can occur when the user listens to the conversational voice of the talker by telephone, and is referred to as the upper limit value of the sound pressure level that is not uncomfortable when listening to the user.
The average sound pressure level is an average sound pressure level that can occur when the user listens to the conversational voice of the talker by telephone, and is referred to as the average sound pressure level when the voice is easy to hear for the user.

  The transmitted voice feature is not limited to the format of the upper limit sound pressure level, the lower limit sound pressure level, and the average sound pressure level, but may be held in another format. For example, the variation of the sound pressure level in each frequency band may be applied to an arbitrary probability distribution function (for example, normal distribution, beta distribution, etc.) and recorded as a parameter of the distribution function. If a normal distribution is assumed, the average value and variance are stored as parameters. Then, as an alternative to the upper limit sound pressure level and the lower limit sound pressure level memorized directly earlier, values separated from the average value by unit standard deviation (eg, values of -3σ, + 3σ) from the distribution function, respectively Used as the upper and lower sound pressure levels. In addition, the average value or expected value of the distribution function can be used as it is for the average sound pressure level.

In the processing of the transmitted voice feature selection unit 211 in step 330, a suitable transmitted voice feature is searched from the transmitted voice feature storage unit 212 based on the speaker information specified by the speaker specifying unit 210.
In addition, even if the information of the transmitted voice feature storage unit 212 is not used, the same type of information is used through the control unit 130, the transmitted voice feature stored in the mobile phone is used, and the transmitted voice downloaded from the server is used. You may do it. In this case, in step 330, the transmitted voice feature selection unit 211 requests information on the transmitter through the control unit 130.

  In step 340, it is determined whether or not there is a transmitted voice feature to be searched. As a case where it is determined that the call does not exist, a case where a call is made to the transmission voice characteristic unit 212 but a transmission voice characteristic for the user is not stored can be cited. Further, when used in combination with a learning means to be described later, there is a case where the voice feature of the sender is stored in the sent voice feature unit 212 but the amount is small. In other words, if the amount of data stored as a transmitted voice feature is small, it is determined that the reliability of the transmitted voice feature calculated from the amount is low, and there is no transmitted voice feature that is the search target to decide. If it exists, go to step 350. If it is determined that it does not exist, the process proceeds to step 355.

In step 350, the transmission voice feature selection unit 211 acquires the transmission voice feature that is the search target. Here, if the transmission voice feature storage unit 212 can be searched in step 330, the transmission voice feature searched by the transmission voice feature storage unit 212 is acquired. Further, if a search request is made to the server or mobile phone through the control unit 130, the transmitted voice feature to which the transmitted voice feature is transferred from the server or the mobile phone is acquired. The data transferred from the server or the mobile phone may be stored in the transmission voice feature storage unit 212 as needed, and the data may be obtained from the transmission voice feature storage unit 212 when used again.
Further, the transmitted voice feature acquired in step 350 is sent to the filter creation unit 250 in step 357.

If the sender cannot be identified in step 320, or if the transmitted voice to be searched cannot be searched in step 340, the process proceeds to step 355. In step 355, a default transmitted voice feature that can be selected in any case is obtained. It is assumed that this default transmission voice feature is also stored in the transmission voice feature storage unit 212 or a server or a mobile phone.
It is desirable that the default transmitted voice feature is a relationship between the frequency and the sound pressure level that the user feels easy to hear no matter what person speaks. As such, a method using a value based on an audiogram measured by an audiometry is conceivable. That is, the auditory threshold value for each frequency sound in the hearing test is made to correspond to the lower limit sound pressure level, and the unpleasant threshold value is made to correspond to the upper limit sound pressure level. Note that this default characteristic does not take into account the voice quality of the sender, so it is slightly different from the true easy-to-hear sound, but by having a characteristic that considers the frequency characteristic of the user's easy-to-hear sound, This is to compensate for the minimum sound quality.
The transmitted voice feature acquired in step 355 is sent to the filter creation unit 250 in step 357.
Thus, the processing of the transmitter identification unit 210 and the transmitted voice feature selection unit 211 ends.

[Obtain vehicle information]
The vehicle information identification unit 220 and the vehicle transfer function selection unit 221 obtain a transfer function from the speaker to the user's (listener's) head in an environment where the user listens to a hands-free telephone voice in the vehicle. This process will be described with reference to the flowchart of FIG.
The flowchart of FIG. 4 is started when an incoming call is received. In principle, the process is terminated by performing it once for each incoming call. However, if there is an excessive change in vehicle information during a call, it may be updated each time.

In step 460, vehicle information is specified by the processing of the vehicle information specifying unit 220. In order to describe this process, the vehicle information acquisition unit 150 and the vehicle transfer function storage unit 222 will be described.
The vehicle information acquisition unit 150 senses various information in the vehicle. The type of this information is not limited to a specific type, but includes the following information. First, temperature and humidity are sensed. In addition, in order to specify the position where the passenger is riding, information on the pressure of the seat of each seat and information on the camera installed in the vehicle are acquired. In addition, in order to specify the user's head position, the height information of the headrest of the seat, the front / rear position information of the seat, and the angle information are acquired.

The vehicle transfer function storage unit 222 stores an acoustic transfer function from the speaker at various positions in the vehicle under various conditions (this is called a vehicle transfer function). The vehicle transfer function can be measured using known impulse response measurement methods. In addition, as positions for which the vehicle transfer function is obtained, measurements are made at a plurality of assumed positions with respect to the position of the head when the user uses the hands-free telephone. As other conditions, the vehicle transfer function is measured when conditions such as temperature, humidity, and passenger position change.
FIG. 7B shows an example of the vehicle transfer function stored in the vehicle transfer function storage unit 222. The vehicle transfer function is stored in relation to the frequency and the transmission rate (gain) of sound energy in each frequency band. Since the vehicle transfer function fluctuates due to various external factors even under the same conditions, one or more of the maximum, minimum, and average gains are stored. Alternatively, the gain fluctuation range of each frequency band may be stored in the form of a probability distribution function parameter.

  In the process of the vehicle information specifying unit 220 in step 460, a condition for selecting a vehicle transfer function is specified according to the vehicle acquisition information acquired from the vehicle information acquisition unit 150. Specifically, temperature, humidity, passenger position, user head position, and the like. In order to estimate the user's head position, not only the information on the above-mentioned sheet, but also the method of calculating from the sheet position and the user's body dimension information using information on the user's body shape stored in advance. good. As a method for preserving the user's body shape, it can be stored in an external storage device in the driver's license, or it can be stored in a server or mobile phone and the body shape can be specified as soon as the user is driving. The method of obtaining the information from these devices can be used.

In step 470, the vehicle transfer function selection unit 221 searches the vehicle transfer function storage unit 222 for a vehicle transfer function that best matches the conditions based on the conditions specified in step 460.
The vehicle transfer function to be searched may be a method of issuing a search request to the server or mobile phone via the control unit 130 each time, even if the vehicle transfer function is not stored in the vehicle transfer function storage unit 222. Alternatively, a search request may be issued to the server or mobile phone via the control unit 130 only when the vehicle transfer function storage unit 222 does not exist. If the vehicle transfer function is stored in the server, it may be added to the server as needed according to the user's demand and the number of units sold, without taking all the pattern transfer characteristics before shipping the car. It becomes possible.

  In step 475, it is determined whether or not the vehicle transfer characteristic that is the search target has been searched. Cases that cannot be searched include when there is no vehicle transfer function under conditions that match the specified vehicle information, or when some conditions match but the distance between the conditions is calculated by a predetermined method Is a large case. If the search is successful, the process proceeds to step 480. If the search has failed, the process proceeds to step 485.

In step 480, the vehicle transfer function searched in step 470 is acquired by the processing of the vehicle transfer function selection unit 221. In this acquisition, when the vehicle transfer function storage unit 222 is set as a search target in step 470, the acquisition is performed from the vehicle transfer function storage unit 222. When a search request is issued to the server or mobile phone, the vehicle transfer function transferred from the server or mobile phone is used. Further, when transferring from the server or the mobile phone, this information is accumulated in the vehicle transfer function storage unit 222. When the same condition is requested again, the vehicle transfer function storage unit 222 It may be made readable.
Further, the vehicle transfer function acquired in step 480 is sent to the filter creation unit by the processing in step 490.

In step 485, if the vehicle transfer function cannot be searched in step 475, a vehicle transfer function that is the default is acquired. It is assumed that this default vehicle transfer function is also stored in the vehicle transfer function storage unit 222 or a server or a mobile phone. This default vehicle transfer function is the acoustic transfer function from the speaker for the user's average head position. Therefore, it is possible to use a transfer function measured at one place or a transfer function that simply describes the volume attenuation.
Further, the vehicle transfer function acquired in step 485 is sent to the filter creation unit by the processing in step 490.
Thus, the processes of the vehicle information specifying unit 220 and the vehicle transfer function selecting unit 221 are finished.

[Acquisition of interior acoustic environment]
The in-vehicle acoustic environment estimation unit 230 estimates noise (engine noise, road noise, music sound, passenger's conversation sound, etc.) that is different from the telephone conversation sound that the user is listening to while driving. This process will be described with reference to the flowchart of FIG.
The flowchart of FIG. 5 is started when an incoming call is received. It is repeated during a telephone call. Thereby, the noise is sequentially estimated. It is assumed that the repetition time interval is 1/4 of the frame length used in the FFT operation as described above.

  In step 510, it is determined whether the telephone call is continued. This processing is performed by the mobile phone 120 and the control unit 130. If the call is not continued, the process is terminated.

In step 515, it is determined whether or not the user is making any sound. This is determined by performing signal processing on the sound captured from the microphone 180. The reason for performing this process is that, when estimating the noise in the vehicle, if the noise is estimated based on the voice during the conversation, the estimation accuracy is adversely affected. As the detection of the voice utterance, a method based on a publicly known voice level or a method for discriminating between voice and non-voice based on GMM (Gaussian mixture model) can be used. In the method based on GMM, it is possible to always record the noise when not making a normal telephone call and learn the noise at this time.
If it is determined by this determination that the user is not speaking, the process returns to step 510. If it is determined that the user is speaking, the process proceeds to step 520.

  In step 520, frequency analysis is performed on the sound from the microphone to calculate a frequency-power characteristic. This method can be performed by the FFT method described above. Further, when the method using GMM is used in the speech determination in step 515, the frequency-power characteristics and similar characteristics have already been calculated, so that information may be used.

  In step 525, a predetermined calculation is performed on the frequency-power characteristic obtained in step 520, and the frequency-power characteristic is updated. The purpose of this predetermined calculation is to estimate the current noise situation during operation. There are several possible calculation methods for this purpose. First, it is conceivable to calculate an average value of the frequency-power characteristics collected in the past during operation and the frequency-power characteristics collected at the present time. By doing so, it is possible to collect a reliable value in consideration of the past noise situation. Further, when the road surface condition suddenly changes, the instantaneous value of the frequency-power characteristic collected at the present time may be used.

  In step 530, the head position of the user stored in the noise data storage unit 235 based on the frequency characteristics of the microphone sound obtained in step 525 and one or more information output from the vehicle information acquisition unit 150 Search the frequency characteristics of noise at.

The noise data storage unit 235 will be described. This preserves the frequency characteristics of noise at the user's head position under various driving conditions and various vehicle conditions. As conditions, as in the vehicle transfer function storage unit 222, the temperature, humidity, passenger position, etc. in the vehicle can be used. Also, vehicle speed information and engine speed information can be used.
Further, in the search of the noise data, it is assumed that the frequency-power characteristic of the noise stopped at step 525 is included in the search condition. The frequency-power characteristics of the noise stopped at step 525 already represent the characteristics of the noise. However, it is considered that the frequency characteristics of noise collected by the microphone and the frequency characteristics actually heard by the user are different. Therefore, it is desirable to be able to estimate a noise characteristic close to the sound heard by the user. Therefore, it is possible to obtain a highly accurate noise characteristic at the user's head position by preliminarily searching for a relationship between the noise taken from the microphone and the noise at the user's head position.

In step 540, it is determined whether the noise characteristics have been searched as a result of the search in step 530. A case that cannot be searched here is a case in which the noise data storage unit 235 does not have a noise characteristic under a condition that matches the specified vehicle information and the noise characteristic of the microphone.
If a search result exists, the process proceeds to step 550. If not, the process proceeds to step 560.

  In step 550, the noise characteristic retrieved in step 530 is acquired. In step 570, the acquired noise characteristics are sent to the filter creation unit.

  In step 560, a default noise characteristic is acquired when the noise characteristic cannot be retrieved in step 530. As the default noise characteristic, the frequency-power characteristic of the noise collected by the microphone 180 stopped at step 525 may be used as it is. Alternatively, it may be used after correcting the noise level by multiplying by a constant. In step 570, the obtained noise characteristics are sent to the filter creation unit.

  The operation of the in-vehicle acoustic environment estimation unit 230 is repeated during a call. Therefore, the noise characteristics output to the filter creation unit are also updated each time.

Create filter
Based on the information calculated by each of the above units, the filter creation unit 250 creates a filter that processes the transmitted sound into a sound that is easy for the user to hear. The flow of processing in the filter creation unit 250 will be described with reference to the flowchart of FIG.
Note that the processing of the filter creation unit 250 is repeatedly performed while a call is continued. It is assumed that the repetition time interval is 1/4 of the frame length used in the FFT operation as described above.

  In step 610, it is determined whether a telephone call is ongoing. When the call is finished, the process is finished. If the call continues, the process moves to step 620.

In step 620, it is determined whether the sender is speaking. This is done to process the sound quality adjustment filter only when the talker is speaking. As a method for determining whether or not this utterance is used, the method based on the known voice level described above or a method based on GMM can be used.
If it is determined that it is not an utterance, the process returns to step 610. If it is determined that it is an utterance, the process proceeds to step 630.

  In step 630, the transmission voice feature output by the transmission voice feature selection unit 211 is read. This is used as a target value by the filter creation unit 250 as the frequency characteristic of the transmitted sound heard by the user.

In step 635, the product of the frequency characteristic that is the transmitted voice characteristic and the inverse function of the vehicle transfer function output by the vehicle transfer function selection unit 221 is calculated.
An example of the inverse function of the vehicle transfer characteristic is shown in FIG. This inverse function is calculated by calculating the reciprocal of the gain with respect to the gain of the vehicle transfer function shown in FIG. When the gain is expressed in dB, it is calculated as a negative value with respect to the original logarithmic gain. When a plurality of values such as maximum gain, minimum gain, and average gain are defined, inverse functions are obtained for all of these values. When the vehicle transfer function is recorded as a probability distribution, a distribution parameter obtained by taking the reciprocal of the gain is obtained in such a manner that the distribution is stored. This inverse function is used as a frequency characteristic for reproducing sound with a flat frequency characteristic in which the influence of the acoustic characteristic in the passenger compartment is canceled in the user's head when reproducing the sound from the speaker.

  FIG. 7D shows a frequency characteristic obtained by multiplying an example of the frequency characteristic which is a transmitted voice characteristic, FIG. 7A, and an inverse function diagram 7C of the vehicle transfer function. The product of frequency characteristics can be calculated as the sum of these values on the axis in decibels. In FIG. 7D, a characteristic obtained by multiplying the lower limit sound pressure level in the transmitted voice characteristic FIG. 7A by the characteristic corresponding to the maximum gain in FIG. 7C and the characteristic for the minimum gain is obtained. (Indicated by a and b in FIG. 7D, respectively). In addition, a characteristic obtained by multiplying the upper limit sound pressure level in the transmitted voice characteristic FIG. 7A by the characteristic corresponding to the maximum gain in FIG. 7 (D), indicated by c and d).

  In step 640, the frequency characteristic of noise at the user's head position is acquired from the in-vehicle acoustic environment estimation unit 230.

In step 645, the frequency characteristic of the noise acquired in step 640 is multiplied by the inverse function of the vehicle transfer function. An example of this processing is shown in FIG. In the example of this figure, the product of the inverse function of the vehicle transfer function and the inverse function corresponding to the average gain in the inverse function of the vehicle transfer function with respect to the frequency characteristic of the noise obtained in step 640 is shown. ing. In FIG. 7 (E), e represents a noise characteristic obtained by multiplying the noise characteristic output from the vehicle interior acoustic environment estimation unit by the inverse function of the vehicle transfer characteristic, and f represents the output from the vehicle interior acoustic environment estimation unit. Represents the noise characteristics.
By calculating the product with the inverse function, it is possible to determine what frequency characteristic is to be used to reproduce the noise corresponding to the noise characteristic heard by the user on the speaker. This data is used to evaluate the influence of noise in the transmitted sound that the user actually hears.

In step 650, the range of the sound pressure level for each target frequency band is determined from the voice characteristics calculated in step 635 and the noise characteristics calculated in step 645.
A specific example of this processing is shown in FIG. In FIG. 7D, the voice feature (a, b, c, d) calculated in step 635 and the noise characteristic (e) calculated in step 645 are described.
First, the sound pressure level that is easy for the user to hear is that the frequency characteristic of the transmitted voice characteristic FIG. 7A is reproduced in the sound at the head position of the user. In order to achieve this, the speaker may be reproduced in the sound pressure range from the lower limit sound pressure level to the upper limit sound pressure level in FIG. This is because the lower limit sound pressure level and upper limit sound pressure level in FIG. 7 (D) are reproduced from the speaker in this frequency characteristic because the transmitted voice feature is multiplied by the inverse function of the frequency transfer characteristic in the passenger compartment. This is because the sound is reproduced at the user's head through the acoustic transfer characteristic of the passenger compartment, and thus reproduced at the frequency characteristic of the transmitted voice characteristic FIG. 7 (A).
However, the vehicle transfer function varies for various reasons. Therefore, for example, when sound is reproduced along the curve d in FIG. 7D, if the gain of the vehicle transfer function has a large gain within the fluctuation range, the voice heard by the user is , It becomes larger than the sound pressure that I usually hear. This may be uncomfortable for the user. On the other hand, when the sound is reproduced along the curve a in FIG. 7D, if the gain of the vehicle transfer function is a small gain within the fluctuation range, the sound heard by the user becomes excessively small. There is a fear.
In order to suppress such side effects, here, the range of sound pressures that fall within the transmitted voice features that are easy to hear for the user within the predicted fluctuation range of the vehicle transfer function is considered. That is, it is within the range from b to c in FIG.
When the vehicle transfer function and the frequency-power characteristic of the transmitted voice feature are defined in the form of a probability distribution function, the distribution in FIG. 7D can also be calculated by the probability distribution function. In this calculation, when a random variable is defined in decibels, it corresponds to taking the sum of two probability distributions. Therefore, the probability representing the fluctuation range of the sound pressure in each frequency band in FIG. The parameter of the distribution function can be obtained by a known calculation method. From the probability distribution obtained in this way, when the sound pressure decreases as it is difficult for the user to hear, the fluctuation range of the sound pressure within which the probability that the sound pressure increases as the user becomes uncomfortable falls below a predetermined threshold can be calculated.

  Next, consider the effects of noise. Looking at the noise characteristics indicated by e in FIG. 7D, the sound pressure is lower in the high frequency band (for example, frequency f2) than the reproduction sound pressure range of the speaker's conversational sound, but the low frequency band (for example, frequency In f1), there is a band where the sound pressure of noise is partly within the sound pressure range of conversational sound reproduction. Therefore, in this band, when the reproduced sound is reproduced with a sound pressure in the range of b to c, the noise heard by the user may be louder than the conversation sound. In this case, it becomes difficult for the user to hear the conversation sound. In auditory psychology, this phenomenon is called a masking phenomenon. The masking phenomenon is a phenomenon in which the loudness of a certain sound (subjective feeling loudness) is felt smaller when noise in the same band is present than when no noise is present.

The magnitude of the decrease in loudness in the masking phenomenon due to noise is described in the following document.
JPA Lochner, JF Burger: `` Form of the loudness function in the presence of masking noise, '' Journal of the Acoustical Society of America, vol.33, no.12, pp.1705-1707, 1961
According to this document, the loudness ψ of the target sound in an environment where noise exists is expressed by the following equation.
ψ = k (I ⁿ -I ₀ ⁿ )
However, k and n are constants. I is the sound intensity of the target sound, and I ₀ is the sound intensity of the noise.
When calculating the loudness of the conversational sound according to this equation, if the sound pressure level of the conversational sound is sufficiently larger than the sound pressure level of the noise, the decrease in the loudness is negligibly small. On the other hand, when the sound pressure level of the conversation voice is a value close to the sound pressure level of the noise, the loudness of the conversation sound is significantly reduced. Therefore, when the sound pressure level of the conversation voice is close to the sound pressure level of the noise, the conversation sound can be heard by performing compensation that amplifies the conversation voice more greatly. Therefore, the range of the reproduction sound pressure is determined based on this criterion. Specifically, a range is adopted in which the sound pressure that is so large that the loudness does not decrease from e in FIG.
As for the noise characteristics, the case where only the average value has been used has been described so far, but the maximum and minimum noise values and the fluctuation range based on the probability distribution function can be used as with the frequency-power characteristics of the transmitted voice characteristics. There is a case. In such a case, the masking influence can be calculated in the same manner as the above-described coping methods in various forms of the transmitted voice characteristics.

In step 655, the frequency-power characteristic of the transmitted sound subjected to frequency analysis by the transmitted speech analysis unit 240 is acquired. This frequency-power characteristic is used in the following two ways.
(1) Instantaneous characteristics of conversational sound
(2) Statistical characteristics of the talker's conversational sound from the start of the current call to the present
For (1), the frequency-power characteristics obtained from the result of the FFT calculation in units of frames described above are directly used. (2) is a statistic obtained in order to determine the upper limit sound pressure level and the lower limit sound pressure level in each frequency band in the currently spoken voice. This method may be a method in which the maximum sound pressure level is used as the upper limit value and the minimum sound pressure level is used as the lower limit value from the value at each time of the frequency-power characteristics. Alternatively, a method may be used in which instantaneous values of sound pressure levels of past conversational sounds are accumulated, a parameter in the probability distribution function is calculated, and a lower limit and an upper limit of the sound pressure are obtained from this parameter.

These upper limit value and lower limit value of the sound pressure are used for grasping the dynamic range of the input voice when determining the amplification factor of the filter in step 660.
Note that the information stored in the transmitted voice feature storage unit 212 stored in the past conversation can be used as the voice when the talker is talking. If the voice currently being spoken reproduces the transmitted voice feature storage unit 212, the analysis by the transmitted voice analysis unit 240 is unnecessary, and is stored in the transmitted voice feature storage unit 212. Information can be used as it is. However, the voice characteristics of the currently speaking talker and the voice characteristics of the sender accumulated in the past do not always match. Reasons for this include differences in the physical condition of the speaker, differences in the environment in which the speaker is currently present, and changes in the utterance mode due to the conversation partner being driving. In addition, the voice feature stored in the transmitted voice feature unit 212 may be learned from the conversation of a normal fixed phone or mobile phone by the receiver in a learning device described later. Therefore, the sound includes the equalization unique to the device normally used by the user and the influence of the sound quality setting that the user normally sets on the device. However, in the hands-free call system, even if the user uses a mobile phone that is normally used, the equalization set in the mobile phone may not be used in the hands-free phone system. In hands-free telephone systems, it is common to perform unique equalization and sound quality adjustment. In addition, the user may use a mobile phone that is not normally used and have a hands-free phone conversation. In many cases, the sound reproduced by such a hands-free phone is different from the feature included in the transmitted voice feature 212. Therefore, the transmitted voice analysis unit 240 is used to analyze the voice that is currently being spoken.
An example of the frequency characteristic of the voice analyzed by the transmitted voice analysis unit 240 is shown in FIG.

In step 660, the amplification factor in each frequency band is calculated so that the sound pressure of the transmitted voice falls within the target sound pressure.
The first part of this process is the fluctuation range of the sound pressure (referred to as input sound pressure) of the transmitted voice in the current conversation and the fluctuation of the target sound pressure (referred to as output sound pressure) calculated in 650. Calculate the conversion formula between the ranges.
(G) and (H) in FIG. 7 show the relationship between the input sound pressure and the output sound pressure at frequencies f2 and f1, respectively.

  First, the relationship at the frequency f2 will be described. The lower limit sound pressure i2min and the upper limit sound pressure i2max in the voice currently being talked at the frequency f2 are obtained from FIG. 7 (F). Next, the target lower limit sound pressure o2min and upper limit sound pressure o2max are obtained from FIG. 7 (D). Next, a conversion formula for converting the input sound pressure into the output sound pressure within these ranges is obtained by a linear function of sound pressure in dB (this is called a conversion function). The conversion function is drawn by a thick line in FIG. In auditory psychology, it is known that the magnitude of loudness can be approximated by a power function of sound pressure intensity or sound pressure energy. Therefore, by converting a linear function on the decibel axis into an output sound pressure as a conversion function, it is possible to output an amplified sound while maintaining a loudness magnitude relationship. Next, an instantaneous value of the sound pressure that is output is obtained from the conversion function with respect to the instantaneous value of the input sound pressure obtained from the transmitted voice analysis unit. Finally, a value obtained by dividing the instantaneous value of the output sound pressure by the instantaneous value of the input sound pressure is obtained, and this value is calculated as an amplification factor in the frequency band f2. When both the input sound pressure and the output sound pressure are expressed in decibels, it can be calculated as a value obtained by subtracting the output sound pressure level from the input sound pressure level.

Next, the relationship at the frequency f1 will be described. The lower limit sound pressure i1min and the upper limit sound pressure i1max in the voice currently being spoken at the frequency f1 are obtained from FIG. 7 (F). Next, the target lower limit sound pressure o1min and upper limit sound pressure o1max are obtained from FIG. 7 (D). Next, a conversion formula for converting the input sound pressure into the output sound pressure within these ranges is obtained. At the frequency f1, masking occurs due to the influence of noise. Therefore, the conversion formula is not a linear function but a curved conversion function as shown in FIG. This conversion function makes the output sound pressure somewhat large even when the input sound pressure is small, and avoids the influence of masking. Based on this conversion formula, the instantaneous value of the output sound pressure with respect to the instantaneous value of the input sound pressure is obtained. Finally, the amplification factor in the frequency band f1 is calculated from the input sound pressure and the output sound pressure.
The above processing is performed in each frequency band as a unit of processing, and the amplification factor in each frequency band is calculated.
Here, the method for calculating the amplification factor using the correspondence between the upper limit value and the lower limit value of the input sound pressure and the output sound pressure has been described. However, other methods can be used. For example, the conversion function may be obtained from the correspondence between the average value of the input sound pressure and the output sound pressure and the upper limit value, and the amplification factor may be calculated.

In step 665, smoothing processing is performed on the gain of each frequency band obtained in step 660, and the gain is recalculated. The purposes of the smoothing process are two purposes: (1) suppressing the difference in amplification factor between bands, and (2) suppressing the difference in amplification factor between times.
The processing performed for the purpose of suppressing the difference in amplification factor between bands in (1) is to obtain a spline function from the amplification factor of each frequency band obtained in step 660, and to perform amplification that continuously changes with respect to frequency change. There are ways to adopt rates. As described at the beginning, it is assumed that the frequency band for calculating the frequency-power characteristics is divided into several frequency bands such as a critical band and obtained for each band. Therefore, if the amplification factor obtained for each band is applied as it is, the amplification factor changes abruptly when the band changes. Therefore, a spline function is obtained from the gain of each band, and the gain other than the center frequency of each frequency band is calculated from the spline function. Thereby, the change of the amplification factor according to the frequency is smoothed.
The purpose of suppressing the amplification factor between times in (2) is a method of using the average value of the amplification factors in the past several frames without using the instantaneous value of the amplification factor obtained in step 660. The method described in the following literature can be used for these smoothing methods.
F. Asano, Y. Suzuki, T. Sone, S. Kakehata, M. Satake, K. Ohyama, T. Kobayashi, T. Takasaka: `` A digital hearing aid that compensates loudness for sensorineural impaired listeners, '' Proc. Of ICASSP91, pp.3625-3628, 1991
In step 670, the amplification factor of each frequency band calculated in step 665 is sent to the filter processing unit 170.
The above processing of the filter creation unit 250 is repeated during a call.

[Filter processing section]
Next, the processing of the filter processing unit 170 will be described. The filter processing unit 170 adjusts the reproduction sound of the transmitted voice based on the amplification factor of each frequency band output from the filter creation unit 250.

In the present invention, this processing will be described on the premise of a known FFT calculation method.
First, the transmitted sound is cut out by the frame length. It is called the frame length, and a value of about 10 ms is often used. Windowing is performed on the extracted audio. It is assumed that the audio is cut out at a time interval of 1/4 frame length.
Second, perform FFT on the analysis target speech. As a result, the waveform in the time domain is converted into values of the real part and the imaginary part in the frequency domain.
Third, the value of each frequency region output by the FFT is multiplied by the amplification factor for each band output by the filter design unit 160.
Fourth, inverse Fourier transform is performed on each value in the frequency domain obtained by the third process, and the result is returned to the time domain.
Fifth, the waveform obtained by the fourth process is output. Since the cut out of the transmitted sound in the first processing is performed at a time interval of 1/4 frame length, since there are sample values at the same time even in different frames, such values are added and output. . This method is known as an overlap addition method.
Through the above processing, the speaker 190 outputs the sound whose amplitude is increased or decreased based on the amplification factor in each frequency band output from the filter design unit 160 for the transmitted sound.

[Summary]
From the above processing, the sound reproduced from the speaker 190 is heard by being converted into frequency characteristics corresponding to the speaker that the user normally listens to when listening at the user's head position. In addition, when there is noise, sound quality adjustment is performed so as to suppress an adverse effect on listening due to the noise. Therefore, it is possible to reproduce the hands-free telephone voice that is easy to hear for the user.

[Use outside the driver's seat]
In the present embodiment, the description has been given on the assumption that the user makes a phone call at the driver's seat. However, the user may use it in other seats such as a rear seat and a passenger seat. In general, a hands-free telephone is developed as a device for a driver to make a telephone call. However, as a possible situation, it is also conceivable that a plurality of people in the vehicle have conversations alternately with the people behind the phone.
Even in such a case, the sound quality can be adjusted for the person who is currently talking. In order to implement this method, it is necessary to know in advance where the person who is having a conversation is in the vehicle before the processing by the in-vehicle acoustic environment estimation unit 230 and the vehicle transfer function selection unit 221. For this purpose, a method of explicitly designating the position of the talker on the device by operating a switch or the like, a method of identifying the position of the talker from the image information from a camera installed in the car, one or more installed in the car For example, a method of identifying a talker from a microphone by predetermined signal processing can be used.
Moreover, if the person who has a conversation differs, the transmitted audio | voice characteristic to like differs. Therefore, the transmission voice feature storage unit stores a plurality of transmission voice features in different regions for each listener. When actually correcting the sound quality, it is detected who the conversation person (that is, the listener) is, and then the transmitted sound quality characteristic corresponding to the conversation person is read to perform the sound quality correction. To identify a conversation person, a method of explicitly instructing the device who is the conversation person by operating a switch or the like, a method of recognizing a conversation person's face image from image information obtained by a camera installed in the car, From the sound of the installed microphone, it is possible to implement a method of identifying who is currently talking by speaker recognition.

  Next, as a second embodiment, a configuration for learning transmitted speech will be described with respect to a method for constructing transmitted speech features stored in the transmitted speech feature storing unit 212. FIG.

[overall structure]
FIG. 8 shows a system configuration when learning transmitted speech. The learning of the transmitted voice is performed by learning the characteristics of the voice of the talker (acoustic characteristics of the transmitted voice) when the user is usually talking on a mobile phone, a fixed phone, or the like generally used by the user. Also, learning can be performed in a call using a hands-free telephone device.
The telephone line network 810 is a general line network in which general landline phones and mobile phones can be connected to make a call with each other.
The mobile phone 820 communicates with the telephone line network 810 and the learning means 830. This mobile phone is a component that is necessary only when the learning means is provided in the hands-free telephone device. When learning is performed on a fixed telephone or a mobile phone, the mobile phone 820 is not necessary because the telephone line network 810 and the control unit 840 in the learning means 830 can communicate directly. When learning is performed in the hands-free telephone device, communication between the mobile phone 820 and the control unit 840 is performed by wire or wireless (Bluetooth standard).

  The learning means 830 is a device that contains components for learning the transmitted voice. This learning means is provided in any of a mobile phone, a fixed phone, and a hands-free phone device. Also, in the embodiment, a description will be given as a method that can be implemented in any of these cases.

  The control unit 840 controls the learning unit 830. In addition, the device is controlled according to the type of device equipped with the learning means 830 (fixed phone, mobile phone, hands-free phone device). Therefore, it controls transmission voice / reception voice and various modules necessary for the telephone function. In the present embodiment, only operations necessary for learning transmitted speech will be described.

  The operation input unit 860 inputs a user operation. As this operation, when the learning means 830 is provided in a fixed telephone or a mobile phone, it is conceivable to accept an operation of pressing a telephone button. In addition, when the learning means 830 is provided in the hands-free telephone device, the buttons provided in the hands-free telephone device, the car navigation provided with the hands-free telephone device, the car audio, the car cockpit button, the dial, the remote control Accepting operations such as.

  The vehicle information acquisition unit 870 acquires information about the vehicle when the user is in the vehicle.

The transmitted voice adjusting unit 842 adjusts the sound quality of the sound (transmitted sound) to be heard by the user. Specifically, an equalizing process for changing the sound pressure level for each frequency band is performed.
In addition, in the case of a fixed phone, a mobile phone, and a hands-free phone, even when the user makes a special sound quality adjustment, this transmission voice adjustment unit 842 can perform the adjustment. For example, when the user likes a sound with a small high frequency band, equalization with a lower amplification factor of the high frequency band can be set in the transmission voice adjustment unit 842.
In addition, when the equalizer setting that is easy for the user to hear is different for each speaker (the person who is speaking the transmission sound), the transmission voice adjusting unit 842 performs a different equalization setting for each speaker. Specifically, the separately set equalizing setting for each transmitter is held in the transmission voice adjusting unit 842. In an actual call, the sender is identified from information such as the telephone number of the sender, and the equalizing setting associated with the identified sender is used in the transmitted voice adjustment unit 842.
The sound quality adjustment in the transmitted voice adjustment unit 842 can be used as a hearing aid device by amplifying sound in each frequency band even when the user's hearing loss is lost.

  To summarize the role of the transmitted voice adjustment unit 842, sound quality adjustment is performed so that the original transmitted sound becomes a transmitted sound that is easy to hear for the user in a scene where the user normally uses the telephone. As described above, the sound quality adjustment of the transmitted sound performed by the transmitted voice adjustment unit 842 is performed by adjusting the sound quality specialized to the user's individual hearing ability (that is, the auditory characteristic) or the transmitted sound characteristic of the transmitter. Includes sound quality adjustments specifically for. That is, the sound quality adjustment performed by the transmitted voice adjustment unit 842 includes detailed sound quality adjustment customized by the user.

The output voice of the transmission voice adjustment unit 842 is sent to the transmission voice feature learning unit 850. The operation of the transmitted voice feature learning unit 850 will be described later. In brief, the voice feature for each transmitter is learned based on the voice output from the transmitted voice adjustment unit 842. Therefore, the transmitted voice feature learning unit 850 has
Voice features that are easy to hear for the user adjusted by the transmission voice adjustment unit 842 are stored for each speaker.
The learned voice feature is stored in the transmitted voice feature storage unit 212 of the hands-free telephone device 125. In the automatic adjustment of the sound quality of the hands-free telephone device 125 described above, the acoustic characteristics at the user's pinna position are described. Is used as the target value.
Therefore, the hands-free telephone device 125 can perform sound quality adjustment so that the transmitted sound at the user's pinna position has an easy-to-listen sound quality specified by the user information or the speaker information.

  The received voice adjustment unit 844 adjusts the sound quality of the voice (received sound) uttered by the user. This includes equalization for each frequency band. In the case of a hands-free telephone device, a noise canceling process for reducing driving noise is included.

  The microphone 882 and the speaker 880 are respectively a microphone and a speaker used for conversation in a telephone device (fixed phone, mobile phone, hands-free telephone device) used by the user.

  The echo cancel unit 884 performs a process of erasing an echo that occurs when the microphone 882 collects sound reproduced from the speaker 880. For this method, the above-mentioned known literature methods can be used.

[Configuration of transmitted voice feature learning unit]
The transmitted voice feature learning unit 850 learns the features of the voice of the sender who has made a call to the user's phone. FIG. 9 is a diagram illustrating the configuration of the transmitted voice feature learning unit 850 in more detail.

  The transmission voice analysis unit 910 performs frequency analysis on the transmission sound transmitted from the transmission voice adjustment unit 842. Here, it is assumed that the frequency-power characteristic is obtained by the FFT analysis for the waveform having the frame length described above. Further, this process is sequentially repeated at a prescribed frame interval (here, 1/4 of the frame length).

  The received voice analysis unit 930 performs frequency analysis on the received sound sent from the echo cancellation unit 884. This method is the same as that of the transmitted voice analysis unit 910.

  The determination unit 920 determines at each time whether or not the transmission sound subjected to frequency analysis by the transmission voice analysis unit 910 should be learned. The flow of this process will be described with reference to the flowchart of FIG.

[Processing flow of judgment unit]
In step 1010, it is determined whether the call is continued. If the call is ended, the process is ended.
In step 1020, branching according to the learning mode is performed. Here, branching is performed according to the input of the operation input unit 860 obtained via the control unit 840. First, when the user explicitly specifies that learning of the transmitted voice is not performed, the process proceeds to step 1070, and a learning stop is output to the voice feature learning unit 940. If the user instructs to forcibly learn, the process proceeds to step 1090 and is output to the speech feature learning unit 940 so as to execute learning. Further, in the case of the automatic learning mode, it is determined whether or not learning is performed based on information on the transmission sound and the reception sound. Therefore, the process proceeds to step 1030.

In step 1030, the frequency-power characteristic of the transmission sound output from the transmission voice analysis unit is acquired.
In step 1040, the noise level of the transmitted sound is determined based on the frequency-power characteristic of the transmitted sound acquired in step 1030. The noise level is a loudness level different from the telephone conversation sound included in the transmitted sound. In an environment where the noise level is high, it can be inferred that the speaker is calling from a noisy place. The transmitted sound obtained from such an environment is considered to be difficult for the user to hear. Therefore, such a sound is determined to be excluded from the learning target. This method can be implemented by various methods. For example, if the frequency-power characteristics of the transmitted sound are averaged over a long period of time, and the overall power level exceeds a certain threshold, the noise level is high. And a speech non-speech discrimination method using a GMM model.
As a result of the determination, if the noise level is equal to or higher than the threshold value, the process proceeds to step 1070 to instruct the speech feature learning unit 940 to stop learning. If the noise level is less than the threshold, the process proceeds to step 1050.

In step 1050, the frequency-power characteristic of the received sound output from the received sound analysis unit is acquired.
In step 1060, the noise level of the received sound is determined based on the frequency-power characteristic of the received sound acquired in step 1050. This is intended to be excluded from the learning target because it is determined that it is difficult for the user to talk when the user is talking in an environment where the noise is loud.
As this determination method, a method similar to the noise determination of the transmitted voice in step 1040 can be used. As a result of the determination, if the noise level is equal to or higher than the threshold value, the process proceeds to step 1070 and a learning stop is output to the speech feature learning unit 940. If the noise level is less than the threshold, the process proceeds to step 1080.
In step 1080, the voice feature learning unit 940 is instructed to execute learning.

  After steps 1070, 1080, and 1090 are completed, the process returns to step 1010 and the process is repeated. Note that this processing interval is performed at a frame interval in the transmitted voice analysis unit and the received voice analysis unit.

  Further, although not shown in the flowchart of FIG. 10, the determination as to whether or not to instruct the speech feature learning unit 940 to perform learning can also be performed using other information. Specifically, when the user is in the vehicle and uses the learning device 830 provided in the hands-free telephone device, the vehicle is stopped or running from the information obtained from the vehicle information acquisition unit 870. Judgment is made on whether or not there is a learning execution instruction only when the vehicle is stopped. By doing this, it is possible to target conversational sounds in an environment in which the user is more likely to concentrate on the conversation and has less noise.

[Processing flow of voice feature learning unit]
Next, the operation of the speech feature learning unit 940 in the transmitted speech feature learning unit 850 will be described with reference to the flowchart of FIG.

  In step 1102, a speaker is specified. There are various methods for this specific method. First, the information of the incoming telephone number based on the information of the control unit 840 can be used. Further, even if it is not known who the speaker is, it is possible to determine only the attribute and use the attribute as the speaker information. For example, it is possible to determine whether the voice is male or female from the information of the transmitted voice analysis unit 910. Further, speaker recognition may be performed from the already collected transmitter and voice feature data stored in the voice feature storage unit 950.

  In step 1104, it is searched whether or not the transmitted voice feature corresponding to the sender is stored in the voice feature storage unit 950. As described in step 1102, when the speaker specifies only the attributes such as male / female, the voice feature storage unit 950 stores the data corresponding to the transmitter attribute determined in step 1102. It is determined whether it is done. If there is data, go to step 1110. If no data exists, the process proceeds to step 1106.

  In step 1106, the voice feature storage unit 950 creates voice feature information for a new talker. It should be noted that at the stage where information is created here, the voice characteristics of the new sender have not been accumulated yet.

The processing from step 1110 to step 1160 is repeatedly performed during a call. This repeated time interval is assumed to be the same as the frame interval, which is the period in the speech analysis of this apparatus.
In step 1110, it is determined whether the telephone call is continued. If the call is not continued, the process is terminated.

  In step 1120, it is determined whether learning execution is instructed by referring to the information output by the determination unit. If learning execution is instructed, the process proceeds to step 1130. If not, the process returns to step 1110 to repeat the process.

  In step 1130, the frequency-power characteristic output from the transmitted voice analysis unit is obtained.

  In step 1140, it is determined whether the voice of the sender is speaking. As the detection of the voice utterance, a method based on a known voice level or a method of voice non-voice discrimination based on GMM can be used.

  In step 1150, the frequency-power characteristic output from the transmission analysis unit is stored in the speech feature storage unit 950. By this processing, the value of each time is sequentially added to the voice feature storage unit 950 as the voice-frequency characteristics of the transmitted sound.

In step 1160, the distribution of the sound pressure level in each frequency band is obtained from the frequency-power characteristics at each time of the transmitted voice stored in the voice feature storage unit 950. This distribution is used as information for grasping the fluctuation range of the sound pressure within the range heard by the user in the transmitted voice. There are several possible formats for this distribution. One is a method of calculating an average value, a maximum value, and a minimum value from the sound pressure at each time. Second, there is a method of estimating each parameter of the probability distribution function by performing a calculation for fitting to a probability distribution function (for example, normal distribution, beta distribution, etc.) from past sound pressure time-series data. Either method may be implemented.
The above processing is repeated until the call is finished.

  In the above-described embodiment, the method for specifying the sender at the beginning of the process is used. However, when the speaker is specified by speaker recognition, it may be difficult to recognize without a sufficient amount of speech. In such a case, the voice characteristics of the talker during the call are recorded as a temporary memory, and if the talker is identified during the call, it is stored in the temporary memory during or after the call. A method of storing the voice feature in the voice feature storage unit 950 may be used.

[How to use the transmitted voice features]
For the transmitted voice features stored in the voice feature storage unit 950, the transmission voice adjustment unit 842 performs a correction that takes into account the user's ease of listening and a correction that takes into account the ease of hearing of each speaker. The sound pressure fluctuation range of the transmitted voice after being applied is stored. Therefore, the sound pressure fluctuation range of the transmitted voice stored in the voice feature storage unit 950 can be referred to as a voice characteristic that is easy to hear for the user. Here, the sound pressure fluctuation range of the transmission voice stored in the voice feature storage unit 950 is used in the transmission voice feature storage unit 212 described in the automatic adjustment of the sound quality of the hands-free telephone device 125.

A method of transferring the transmission voice feature in the voice feature storage unit 950 in the learning unit 830 to the transmission voice feature storage unit 212 of the hands-free telephone device 125 will be described.
The first method is via a mobile phone. In this method, the information of the voice feature storage unit 950 is transferred to a memory provided in the mobile phone. Next, a call is made by the hands-free telephone device 125 via this mobile phone. At this time, it is transferred to the transmitted voice feature storage unit 212.
As a second method, a method via a network such as the Internet can be considered. Here, it is assumed that the learning unit 830 or the fixed telephone / mobile phone / hands-free telephone device provided with the learning unit 830 has a data communication function. Thereby, the transmitted voice feature in the voice feature storage unit 950 is uploaded to a predetermined server. The hands-free telephone device 125 issues a request for the transmitted voice feature to the server and downloads it.
As a third method, there is transfer by an external storage device such as a flash memory or an IC chip. In this method, it is assumed that the learning unit 830 has a data communication function with an external storage device. As a result, the transmitted voice feature of the voice feature storage unit 950 is written into a flash memory or an IC chip. Further, assuming that the hands-free telephone device 125 has a data communication function with the external storage device, the transmitted voice feature is transferred. Alternatively, a method may be used in which an IC embedded in a driver's license or a storage area embedded in a car key is written, and this is transferred to the hands-free telephone device 125 through a car device. .

[Selecting voice features to use]
Although described in the explanation of the operation of the hands-free telephone device 125, the transmitted voice feature is generally considered to be a voice feature with higher reliability when learned from a long-time voice. Therefore, in the hands-free telephone device 125, the embodiment has been described in which it is determined whether or not to use the transmitted voice feature according to the learning time of the transmitted voice feature. In order to enable this implementation, the speech feature learning unit 940 of the learning means 830 not only stores the speech feature but also records the time length (number of frames) of the speech used to calculate the feature. And

[Learning on hands-free phone]
The learning means can learn in different environments such as a fixed phone, a mobile phone, and a hands-free phone. Therefore, when adjusting the sound quality in the hands-free telephone device 125 using the learned voice feature, it is considered that the closer the learned scene is, the better the sound quality is. Therefore, when there is a sufficient amount of learning data when using a hands-free phone, the sent voice feature learned with a hands-free phone is used instead of the sent voice feature learned with a fixed-line or mobile phone. It is preferable to do.

  On the other hand, the transmitted voice feature learned in the hands-free telephone is learned under the condition that the user is listening to the voice affected by the transfer function of the vehicle. Therefore, when using this transmitted voice feature for sound quality adjustment, it is necessary to consider the influence of vehicle transfer characteristics during learning. There are two methods for this. The first is a method in which learning is performed after a vehicle transfer function is calculated in advance for a vehicle in which the user is riding during learning, and this characteristic is canceled. The second is a method of skipping the process of step 635 in FIG. 6 because the voice is already taken into account when the voice is reproduced by the hands-free telephone device 125, and the vehicle transfer function is taken into account. This method considers that the transfer function of the vehicle on which the user was riding at the time of learning and the transfer characteristics of the vehicle on which the user makes a phone call while correcting the sound quality are the same. In practice, the transfer characteristics are different, but the general characteristics of the vehicle cabin are reflected.

[Summary]
As described above, in a situation where the user is usually talking on the phone, by learning the transmitted voice feature, it is possible to automatically learn the voice feature that is usually easy to hear. Also, by using this in a hands-free telephone device, it is possible to provide sound quality that is good for conversations during driving of a car.

  Although the embodiment of the hands-free telephone has been described above, the technology disclosed in the present invention is not limited to the hands-free telephone apparatus, and can be used in general sound reproduction apparatuses such as audio reproduction. Moreover, it can utilize not only in vehicles, such as a motor vehicle, but in a general room.

  If the technology disclosed in the present invention is applied to the voice adjustment unit, the sound quality heard by the listener can be made comfortable.

DESCRIPTION OF SYMBOLS 110 ... Telephone line network, 120 ... Mobile phone, 125 ... Hands free telephone apparatus, 130 ... Control part, 132 ... Received voice adjustment part, 134 ... Transmitted voice adjustment part, 140 ... Echo cancellation part, 150 ... Vehicle information acquisition part , 160 ... Filter design section, 170 ... Filter processing section, 180 ... Microphone, 190 ... Speaker,
210: Speaker identification unit 211: Transmission voice feature selection unit 212 ... Transmission voice feature storage unit 220 ... Vehicle information identification unit 221 ... Vehicle transfer function selection unit 222: Vehicle transfer function storage unit 230 ... In-car acoustic environment estimation unit, 235 ... Noise data storage unit, 240 ... Send speech analysis unit, 250 ... Filter creation unit,
810 ... Telephone line network, 820 ... Mobile phone, 830 ... Learning means, 840 ... Control unit, 842 ... Transmitted voice adjustment unit, 844 ... Received voice adjustment unit, 850 ... Transmitted voice feature learning unit, 860 ... Operation input unit 870 ... Vehicle information acquisition unit, 880 ... Speaker, 882 ... Microphone, 884 ... Echo cancellation unit,
910 ... Transmitted voice analysis unit, 920 ... Determination unit, 930 ... Received voice analysis unit, 940 ... Voice feature learning unit, 950 ... Voice feature storage unit.

Claims (20)

A transmission voice feature storage unit that stores the acoustic characteristics of one or a plurality of transmission voices;
A speaker identification unit for identifying a speaker or a speaker attribute;
Based on the information of the speaker specified by the speaker identification unit, the acoustic characteristic of the transmission voice is selected and acquired from the acoustic characteristic of the transmission voice stored in the transmission voice feature storage unit. A speech feature selection unit;
A transfer function storage for storing one or more transfer functions from the speaker that reproduces the transmitted voice to the position of the listener,
From the transfer function stored in the transfer function storage unit, based on information including at least the location information of the listener, a transfer function selection unit that selects and acquires a transfer function;
Based on the acoustic characteristics of the transmitted voice selected by the transmitted voice feature selection section and the transfer function selected by the transfer function selection section, the acoustic characteristics of the reproduced sound at the position of the receiver is the transmitted voice feature selection section. A filter creation unit for creating a filter close to the acoustic characteristics of the transmitted voice selected by
A sound reproduction apparatus comprising: a filter processing unit that performs a filter process based on the filter created by the filter creation unit on a reproduced sound. In the sound reproduction device according to claim 1,
The transmission voice feature storage unit has acoustic characteristics of a plurality of transmission voices stored for each attribute of the speaker,
The transmission voice feature selection unit is configured to transmit a transmission voice from an acoustic characteristic of the transmission voice stored in the transmission voice feature storage unit based on an output of the speaker specification unit that specifies a speaker attribute. An acoustic reproduction apparatus characterized by selecting and acquiring the acoustic characteristics of speech. The sound reproducing device according to claim 1,
The acoustic reproduction device characterized in that the acoustic characteristics of the transmitted voice stored in the transmitted voice characteristic storage unit are stored by a value based on the sound pressure of the voice for each frequency band. The sound reproduction device according to claim 3,
The value based on the sound pressure of the sound for each frequency band in the acoustic characteristics of the transmitted sound stored in the transmitted sound feature storage unit is a probability distribution in which the relationship between the sound pressure and the occurrence probability is recorded. A sound reproducing device characterized by the above. The sound reproduction device according to claim 3,
In the acoustic characteristics of the transmitted voice stored in the transmitted voice feature storage unit, the values based on the sound pressure of the sound for each frequency band are the average value of the sound pressure, the lower limit value of the sound pressure, and the upper limit of the sound pressure. A sound reproducing device characterized by being one or more of the values. The sound reproducing device according to claim 1,
Furthermore, an acoustic environment estimation unit that estimates acoustic characteristics at the position of the listener is provided,
The said filter preparation part changes the filter to produce based on the acoustic characteristic output from an acoustic environment estimation part, The acoustic reproduction apparatus characterized by the above-mentioned. In the sound reproduction device according to claim 6,
The acoustic environment estimation unit predicts acoustic characteristics of noise at the position of the listener,
The filter creation unit is configured such that the frequency characteristic of the reproduced sound at the position of the listener is within the distribution of the acoustic characteristic of the transmitted voice selected by the transmitted voice feature selection unit, or the loudness masking by the non-reproduced sound A sound reproduction device characterized in that a filter is created based on one or a plurality of criteria among those having little influence on reproduction sound. The sound reproducing device according to claim 1,
Furthermore, a vehicle information acquisition unit that acquires information on the vehicle used by the listener is provided.
The transfer function storage unit stores one or more transfer functions associated with various positions and conditions of the vehicle,
The sound reproduction apparatus, wherein the transfer function selection unit selects a transfer function based on an output of the vehicle information acquisition unit. The sound reproducing device according to claim 1,
The transfer function stored in the transfer function storage unit is stored with accompanying information on the fluctuation range thereof. The sound reproducing device according to claim 1, further comprising learning means,
The learning means is
A transmission voice adjustment unit for adjusting the sound quality of the transmission voice;
A transmission voice analysis unit for analyzing the acoustic characteristics of the transmission voice whose sound quality has been adjusted by the transmission voice adjustment unit;
A voice feature storage unit that stores the acoustic characteristics of the transmitted voice output by the transmitted voice analysis unit;
A speech feature learning unit that learns the acoustic characteristics of the transmitted speech based on the output of the transmitted speech analysis unit, and stores a learning result in the speech feature storage unit,
An acoustic reproduction apparatus characterized in that the acoustic characteristic of the transmitted voice stored in the voice feature storage unit of the learning means is used as the acoustic characteristic of the transmitted voice of the transmitted voice feature storage unit. The sound reproducing device according to claim 10, wherein
In the voice feature storage unit, acoustic characteristics of a plurality of transmission voices are stored for each attribute of the speaker,
The voice feature learning unit specifies the attribute of the speaker based on the output of the speaker specifying unit that specifies the attribute of the speaker,
An audio reproducing apparatus, wherein the audio feature storage unit stores the specified attribute information and a learning result in a correlated form. The sound reproducing device according to claim 10, wherein
The sound reproduction apparatus according to claim 1, wherein the learning unit further includes a determination unit that determines whether or not the acoustic characteristics of the transmitted sound can be learned. The sound reproducing device according to claim 12,
The determination unit determines that learning is possible when it is determined that the sound is not a non-conversation sound as a result of analyzing one or more of the transmitted sound or the received sound. The sound reproducing device according to claim 12,
The learning means further includes an operation input unit that receives a user operation,
The determination unit determines that learning is possible according to the output of the operation input unit. The sound reproducing device according to claim 12,
A vehicle information acquisition unit for acquiring information on the vehicle used by the listener;
The determination unit determines that learning is possible based on the output of the vehicle information acquisition unit. The sound reproducing device according to claim 10, wherein
The transmitted voice analysis unit calculates a value based on a sound pressure for each frequency band of the transmitted voice and outputs the calculated value as an acoustic feature amount. The sound reproducing device according to claim 16, wherein
The voice feature learning unit receives a value based on the sound pressure for each frequency band output from the transmitted voice analysis unit, calculates a sound pressure occurrence probability distribution for each frequency band, and stores this in the voice feature storage unit. A sound reproducing device for storing. The sound reproducing device according to claim 16, wherein
The voice feature learning unit receives a value based on the sound pressure for each frequency band output from the transmitted voice analysis unit, and from the sound pressure for each frequency band, the average value of the sound pressure, the lower limit value of the sound pressure, the sound pressure One or more of the upper limit values are calculated and stored in the transmitted voice feature storage unit. In the sound reproduction device according to claim 1,
In the transmission voice feature storage unit, acoustic characteristics of different transmission voices are stored depending on the listener,
The transmission voice feature selection unit is configured to determine the transmission voice from the acoustic characteristics of the transmission voice stored in the transmission voice feature storage unit based on the output of the listener specification unit that specifies the attributes of the listener. An acoustic reproduction apparatus, wherein acoustic characteristics are selected and acquired.   A hands-free telephone device incorporating the sound reproducing device according to any one of claims 1 to 19.

Images