JP2019169835A - Out-of-head localization processing apparatus, out-of-head localization processing method, and program - Google Patents

Abstract

To provide an out-of-head localization processing apparatus, a method, and a program, capable of appropriately performing out-of-head localization processing.SOLUTION: An out-of-head localization processing apparatus 100 includes: a left unit 43L having a built-in microphone 48; a reference signal acquisition unit 122 for acquiring a reference signal on the basis of a sound collection signal collected by the built-in microphone; an ECTF acquisition unit 132 for acquiring transfer characteristics on the basis of a sound collection signal collected by the left microphone 2L in a measurement state; a first storage unit 141 for storing a first reference signal acquired by the reference signal acquisition unit in the measurement state; a second storage unit 142 for storing a second reference signal acquired by the reference signal acquisition unit 122 in a listening state; a conversion function calculation unit 151 for calculating a conversion function of a frequency characteristic on the basis of the first and second reference signals; and a correction unit 152 for correcting the frequency characteristics of an ECTF using the conversion function.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an out-of-head localization processing apparatus, an out-of-head localization processing method, and a program.

  For example, as a sound image localization technique, there is an out-of-head localization technique in which a sound image is localized outside the listener's head using headphones. Out-of-head localization technology cancels the characteristics from the headphones to the ears (tympanic membrane) (the ear canal transmission characteristics), and gives four characteristics from the stereo speakers to the ears (spatial acoustic transmission characteristics), thereby moving the sound image out of the head. I have been localized.

  In the out-of-head localization reproduction, a measurement signal (impulse sound or the like) emitted from the headphones is recorded with a microphone (hereinafter referred to as a microphone) installed in the ear of the listener (Patent Document 1). Based on the collected sound signal obtained by the impulse response, the processing device measures the headphone characteristics. The processing device creates an inverse filter for the headphone characteristics. After the processing device convolves the spatial acoustic transfer characteristic, the out-of-head localization reproduction can be realized by further convolving the inverse filter.

JP 2014-38608 A

  However, the ear canal transfer characteristic (also referred to as the ear canal transfer function) from the headphone speaker unit to the eardrum may change depending on the wearing state of the headphones. For example, each time the headphones are worn, there is a possibility that the wearing position of the headphones is shifted. In this case, since the position of the speaker unit of the headphones is shifted, the ear canal transmission characteristic changes. That is, the mounting position of the headphones changes between when measuring the ear canal transmission characteristics with a microphone attached to the ear and when listening with the microphone removed from the ear and performing stereotaxic listening. In this case, the transmission characteristics of the external auditory canal are changed, and there is a possibility that the out-of-head localization process cannot be performed appropriately.

  The present embodiment has been made in view of the above points, and an object thereof is to provide an out-of-head localization processing apparatus, an out-of-head localization processing method, and a program that can appropriately perform out-of-head localization processing.

  The out-of-head localization processing apparatus according to the present embodiment includes an audio output unit having a reference microphone, a reference signal acquisition unit that acquires a reference signal based on a collected sound signal collected by the reference microphone, and the audio output unit In the measurement state in which the measurement microphone is arranged in the ear canal of the user wearing the first storage unit, the first storage unit that stores the first reference signal acquired by the reference signal acquisition unit, and the measurement microphone is removed from the ear canal A second storage unit that stores the second reference signal acquired by the reference signal acquisition unit in the received listening state, and a sound acquisition signal acquired by the measurement microphone in the measurement state. A third storage unit that stores the transfer characteristic, a conversion function calculation unit that calculates a conversion function of a frequency characteristic based on the first reference signal and the second reference signal, and the conversion function. And said A correction unit that corrects the frequency characteristic of the arrival characteristic, a filter generation unit that generates a filter based on the corrected frequency characteristic, an out-of-head localization processing unit that performs out-of-head localization processing using the filter, It is equipped with.

  In the out-of-head localization processing method according to the present embodiment, the collected sound signal collected by the measurement microphone in the measurement state in which the measurement microphone is arranged in the ear canal of the user wearing the audio output unit having the reference microphone. Obtaining a transfer characteristic based on the first step, obtaining a first reference signal based on a collected sound signal collected by the reference microphone in the measurement state, and removing the measurement microphone from the ear canal In the listening state, a frequency characteristic is obtained based on the step of obtaining a second reference signal based on the collected sound signal collected by the reference microphone, and the first reference signal and the second reference signal. A step of calculating a conversion function, a step of correcting a frequency characteristic of the transfer characteristic using the conversion function, and generating a filter based on the corrected frequency characteristic A step that, using the filter, those having a step of performing out-of-head localization processing, the.

  The program according to the present embodiment is a computer program for collecting sound signals collected by the measurement microphone in a measurement state in which the measurement microphone is arranged in a user's external auditory canal equipped with a sound output unit having a reference microphone. Obtaining a transfer characteristic based on the input signal; obtaining a first reference signal based on a collected sound signal collected by the reference microphone in the measurement state; and removing the measurement microphone from the ear canal In a listening state, a step of obtaining a second reference signal based on a collected sound signal collected by the reference microphone, and a frequency characteristic based on the first reference signal and the second reference signal A step of calculating a conversion function; a step of correcting a frequency characteristic of the transfer characteristic using the conversion function; and a step of correcting a frequency characteristic based on the corrected frequency characteristic. Generating a data, by using the filter, it is intended to execute a step of performing out-of-head localization processing, the.

  According to the present embodiment, an out-of-head localization processing apparatus, an out-of-head localization processing method, and a program that can appropriately perform out-of-head localization processing can be provided.

It is a block diagram which shows the out-of-head localization processing apparatus which concerns on this Embodiment. It is a figure which shows the measurement structure of an external auditory canal transfer characteristic. It is a figure which shows the structure for correct | amending an external auditory canal transfer characteristic. It is a flowchart which shows the process for correct | amending an external auditory canal transfer characteristic. It is a flowchart which shows the process for calculating a conversion function. It is a figure which shows the vector between the extreme values of a power spectrum. It is a figure which shows the conversion method of a vector. It is a figure which shows the logarithmic power spectrum of a reference signal and an external auditory canal transfer characteristic.

  An outline of sound image localization processing using a filter will be described. The out-of-head localization processing according to the present embodiment performs out-of-head localization processing using spatial acoustic transmission characteristics and ear canal transmission characteristics. The spatial acoustic transfer characteristic is a transfer characteristic from a sound source such as a speaker to the ear canal. The ear canal transfer characteristic is a transfer characteristic from the speaker unit of the headphones or earphones to the eardrum, and is also referred to as a headphone characteristic. The present embodiment is characterized by the process of measuring the external auditory canal transfer characteristic in a state where headphones or earphones are worn, and generating a filter using those measurement data.

  The out-of-head localization processing according to the present embodiment is executed by a user terminal such as a personal computer, a smart phone, or a tablet PC. The user terminal is an information processing apparatus having processing means such as a processor, storage means such as a memory and a hard disk, display means such as a liquid crystal monitor, and input means such as a touch panel, buttons, a keyboard, and a mouse. The user terminal may have a communication function for transmitting and receiving data. Furthermore, an audio output unit (headphone or earphone) having left and right output units is connected to the user terminal. Hereinafter, the configuration when the audio output unit is a headphone will be exemplified.

(Out-of-head localization processor)
FIG. 1 shows an out-of-head localization processing apparatus 100 that is an example of a sound field reproducing apparatus according to the present embodiment. FIG. 1 is a block diagram of the out-of-head localization processing apparatus 100. The out-of-head localization processing apparatus 100 reproduces a sound field for the user U wearing the headphones 43. Therefore, the out-of-head localization processing apparatus 100 performs sound image localization processing on the Lch and Rch stereo input signals XL and XR. The Lch and Rch stereo input signals XL and XR are analog audio playback signals output from a CD (Compact Disc) player or the like, or digital audio data such as mp3 (MPEG Audio Layer-3). Note that audio playback signals or digital audio data are collectively referred to as playback signals. That is, the Lch and Rch stereo input signals XL and XR are reproduction signals.

  The out-of-head localization processing apparatus 100 is not limited to a physically single apparatus, and some processes may be performed by different apparatuses. For example, a part of the processing may be performed by a personal computer or the like, and the remaining processing may be performed by a DSP (Digital Signal Processor) built in the headphones 43 or the like.

  The out-of-head localization processing apparatus 100 includes an out-of-head localization processing unit 10, a filter unit 41, a filter unit 42, and headphones 43. Specifically, the out-of-head localization processing unit 10, the filter unit 41, and the filter unit 42 can be realized by a processor or the like.

  The out-of-head localization processing unit 10 includes convolution operation units 11 to 12 and 21 to 22 and adders 24 and 25. The convolution operation units 11 to 12 and 21 to 22 perform convolution processing using spatial acoustic transfer characteristics. Stereo input signals XL and XR from a CD player or the like are input to the out-of-head localization processing unit 10. Spatial acoustic transfer characteristics are set in the out-of-head localization processing unit 10. The out-of-head localization processing unit 10 convolves a spatial acoustic transfer characteristic filter (hereinafter also referred to as a spatial acoustic filter) with respect to the stereo input signals XL and XR of each channel. The spatial acoustic transfer characteristic may be a head-related transfer function HRTF measured with the head or auricle of the measurement subject, or may be a dummy head or a third-party head-related transfer function.

  A set of four spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs is defined as a spatial acoustic transfer function. Data used for convolution in the convolution operation units 11, 12, 21, and 22 is a spatial acoustic filter. A spatial acoustic filter is generated by cutting out the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs with a predetermined filter length.

  Each of the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs is acquired in advance by an impulse response measurement or the like. For example, the user U attaches microphones to the left and right ears. The left and right speakers arranged in front of the user U output impulse sounds for performing impulse response measurement. A measurement signal such as an impulse sound output from the speaker is collected by a microphone. Spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs are acquired based on a sound collection signal from the microphone. Spatial acoustic transmission characteristic Hls between the left speaker and the left microphone, spatial acoustic transmission characteristic Hlo between the left speaker and the right microphone, spatial acoustic transmission characteristic Hro between the right speaker and the left microphone, right speaker and right microphone The spatial acoustic transfer characteristic Hrs between the two is measured.

  Then, the convolution unit 11 convolves a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hls with respect to the Lch stereo input signal XL. The convolution operation unit 11 outputs the convolution operation data to the adder 24. The convolution operation unit 21 convolves a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hro with respect to the Rch stereo input signal XR. The convolution operation unit 21 outputs the convolution operation data to the adder 24. The adder 24 adds the two convolution calculation data and outputs the result to the filter unit 41.

  The convolution operation unit 12 convolves a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hlo with respect to the Lch stereo input signal XL. The convolution operation unit 12 outputs the convolution operation data to the adder 25. The convolution operation unit 22 convolves a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hrs with respect to the Rch stereo input signal XR. The convolution operation unit 22 outputs the convolution operation data to the adder 25. The adder 25 adds the two convolution calculation data and outputs the result to the filter unit 42.

  In the filter units 41 and 42, an inverse filter for canceling the headphone characteristics (characteristics between the headphone speaker unit and the microphone) is set. Then, an inverse filter is convoluted with the reproduction signal (convolution operation signal) that has been processed by the out-of-head localization processing unit 10. The filter unit 41 convolves the Lch signal from the adder 24 with a headphone characteristic inverse filter on the Lch side. Similarly, the filter unit 42 convolves the Rch signal from the adder 25 with an Rch-side headphone characteristic inverse filter. The reverse filter cancels the headphone characteristics from the headphone unit to the microphone when the headphone 43 is attached. The microphone may be placed anywhere from the ear canal entrance to the eardrum. The inverse filter is calculated from the measurement result of the characteristics of the user U himself / herself, as will be described later.

  The filter unit 41 outputs the processed Lch signal YL to the left unit 43L of the headphones 43. The filter unit 42 outputs the processed Rch signal YR to the right unit 43R of the headphones 43. User U is wearing headphones 43. The headphones 43 output the Lch signal YL and the Rch signal YR (hereinafter, the Lch signal YL and the Rch signal are collectively referred to as a stereo signal) to the user U. Thereby, the sound image localized outside the user U's head can be reproduced.

  As described above, the out-of-head localization processing apparatus 100 performs the out-of-head localization processing using the spatial acoustic filter corresponding to the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs and the inverse filter of the headphone characteristics. In the following description, a spatial acoustic filter according to the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs and an inverse filter with headphone characteristics are collectively referred to as an out-of-head localization processing filter. In the case of a 2ch stereo reproduction signal, the out-of-head localization filter is composed of four spatial acoustic filters and two inverse filters. Then, the out-of-head localization processing apparatus 100 performs the out-of-head localization processing by performing convolution operation processing on the stereo reproduction signal using a total of six out-of-head localization filters. The out-of-head localization filter is preferably based on the measurement of the individual user U. For example, an out-of-head localization filter is set based on a sound collection signal collected by a microphone attached to the ear of the user U.

  Thus, the spatial acoustic filter and the headphone characteristic inverse filter are filters for audio signals. These filters are convolved with the reproduction signals (stereo input signals XL and XR), so that the out-of-head localization processing apparatus 100 performs out-of-head localization processing.

(Equipment for measuring ear canal transmission characteristics)
Next, a measurement apparatus 200 that measures the ear canal transfer characteristics in order to generate an inverse filter will be described with reference to FIG. FIG. 2 shows a configuration for measuring the external auditory canal transfer characteristic of the person under measurement 1. The measuring device 200 includes a microphone unit 2, headphones 43, and a processing device 201. Here, the person to be measured 1 is the same person as the user U in FIG.

  The microphone unit 2 and the headphones 43 are connected to the processing device 201. The microphone unit 2 may be detachably attached to the headphones 43. The microphone unit 2 includes a left microphone 2L and a right microphone 2R. The left microphone 2L is attached to the left ear 9L of the person 1 to be measured. The right microphone 2R is attached to the right ear 9R of the measurement subject 1. The processing device 201 may be the same processing device as the out-of-head localization processing device 100, or may be a different processing device. In the following description, it is assumed that the out-of-head localization processing apparatus 100 and the processing apparatus 201 are the same apparatus.

  The headphone 43 has a headphone band 43B, a left unit 43L, and a right unit 43R. The left unit 43L and the right unit 43R are output units that output sound to the left and right ears 9L and 9R, respectively. The headphone band 43B connects the left unit 43L and the right unit 43R. The left unit 43L outputs a sound toward the left ear 9L of the person 1 to be measured. The right unit 43R outputs sound toward the right ear 9R of the person 1 to be measured. The headphones 43 are a sealed type, an open type, a semi-open type, a semi-sealed type, etc., and the type of headphones is not limited. The subject 1 wears the headphones 43 in a state where the microphone unit 2 is attached to the subject 1. That is, the left unit 43L and the right unit 43R of the headphones 43 are respectively attached to the left ear 9L and the right ear 9R to which the left microphone 2L and the right microphone 2R are attached. The headphone band 43B generates a biasing force that presses the left unit 43L and the right unit 43R against the left ear 9L and the right ear 9R, respectively.

  The left microphone 2L collects sound output from the left unit 43L of the headphones 43. The right microphone 2R collects the sound output from the right unit 43R of the headphones 43. The microphone portions of the left microphone 2L and the right microphone 2R are arranged at a sound collection position near the outer ear hole. The left microphone 2L and the right microphone 2R are configured not to interfere with the headphones 43. That is, the person to be measured 1 can wear the headphones 43 in a state where the left microphone 2L and the right microphone 2R are disposed at appropriate positions of the left ear 9L and the right ear 9R.

  The processing device 201 outputs a measurement signal to the headphones 43. Thereby, the headphones 43 generate an impulse sound or the like. Specifically, the impulse sound output from the left unit 43L is measured by the left microphone 2L. The impulse sound output from the right unit 43R is measured by the right microphone 2R. When the measurement signal is output, the microphones 2L and 2R acquire the collected sound signal, thereby performing impulse response measurement.

  The processing device 201 stores a collected sound signal based on the impulse response measurement in a memory or the like. Thereby, the transfer characteristic between the left unit 43L and the left microphone 2L (that is, the external ear canal transfer characteristic of the left ear) and the transfer characteristic between the right unit 43R and the right microphone 2R (that is, the external ear canal transfer characteristic of the right ear). ) Is acquired. The left ear external ear canal transmission characteristic acquired by the left microphone 2L is Lch (left ch) external ear canal transmission characteristic, and the right ear external ear canal transmission characteristic acquired by the right microphone 2R is Rch (right ch) external ear canal transmission characteristic. . The processing device 201 cuts out the transfer characteristic measurement data with a predetermined filter length, thereby obtaining a filter coefficient. The processing device 201 calculates an inverse filter that cancels the ear canal transfer characteristic (headphone characteristic) from the filter coefficient.

  The processing device 201 includes a memory that stores measurement data of transfer characteristics. The processing device 201 generates an impulse signal, a TSP (Time Stretched Pulse) signal, or the like as a measurement signal for measuring the ear canal transfer characteristic. The measurement signal includes measurement sound such as impulse sound.

(Correction of ear canal transmission characteristics)
The out-of-head localization processing apparatus 100 according to the present embodiment corrects the ear canal transmission characteristics according to the wearing state of the headphones 43. Then, the out-of-head localization processing apparatus 100 calculates an inverse filter based on the corrected ear canal transfer characteristics. By doing in this way, the out-of-head localization processing apparatus 100 can perform out-of-head localization processing using an inverse filter adapted to the headphone wearing state.

  The correction process of the ear canal transfer characteristic will be described with reference to FIG. FIG. 3 is a diagram showing a configuration for correcting the ear canal transfer characteristics. In FIG. 3, only the left unit 43L is shown for simplification of explanation, but the right unit 43R has the same configuration. Therefore, the description regarding the right unit 43R is omitted as appropriate.

  The out-of-head localization processing apparatus 100 includes a measurement signal generation unit 111, a D / A converter 112, an A / D converter 121, a reference signal acquisition unit 122, an A / D converter 131, an ECTF acquisition unit 132, and a memory. 140, a conversion function calculation unit 151, a correction unit 152, and an inverse filter generation unit 153. The memory 140 includes a first storage unit 141, a second storage unit 142, and a third storage unit 143.

  The left unit 43L includes a housing 45, a headphone speaker 46, and a built-in microphone 48. The housing 45 includes an ear cup that covers the ear. The housing 45 is provided with a headphone speaker 46 and a built-in microphone 48. The headphone speaker 46 has a porcelain circuit, a diaphragm, etc., and outputs sound to the left ear of the user U.

  The built-in microphone 48 is built in the left unit 43L. The built-in microphone 48 is disposed in a space covered with the housing 45. The built-in microphone 48 collects sound output from the headphone speaker 46. The signal collected by the built-in microphone 48 is used as a reference sound collection signal. That is, the built-in microphone 48 is a reference microphone for collecting a reference sound collection signal. The built-in microphone 48 and the headphone speaker 46 are fixed to the housing 45. Therefore, the position of the built-in microphone 48 with respect to the headphone speaker 46 is constant.

  The headphones 43 may be a wireless type using wireless communication such as Bluetooth (registered trademark). Furthermore, the headphones 43 may include a DSP that performs a part of the processing. The DSP or the like of the headphone 43 may perform a part or all of the processing shown in FIG. For example, the D / A converter 112, the A / D converter 121, the A / D converter 131, and the like may be built in the headphones 43.

  Further, a left microphone 2L for measuring the external auditory canal transfer characteristic is disposed in the left ear 9L (not shown in FIG. 3). As shown in FIG. 2, the left microphone 2L is a measurement microphone for measuring the ear canal transfer characteristics. The left microphone 2L is detachably attached to the left ear 9L. At the time of measuring the external ear canal transfer characteristic, the left microphone 2L is attached to the left ear 9L. In addition, when listening to music out of the head with headphones 43 (hereinafter simply referred to as listening), the left microphone 2L is removed from the left ear 9L. The left microphone 2L may be detachably attached to the left unit 43L, or may be configured independently of the left unit 43L. For example, the left microphone 2L may be directly attached to the left ear 9L.

  At the time of measuring the external auditory canal transfer characteristics, the left microphone 2L is attached to the left ear 9L, and this state is set as a measurement state. At the time of listening to the headphone 43 listening to music out of the head, the state in which the left microphone 2L is removed from the left ear 9L is the non-wearing state, and this state is the listening state.

  The measurement signal generator 111 generates a measurement signal. The measurement signal generated by the measurement signal generator 111 is D / A converted by the D / A converter 112 and output to the headphone speaker 46. The headphone speaker 46 outputs a measurement signal for measuring the transfer characteristic.

  In the measurement state, the microphone 2L collects the measurement signal from the headphone speaker 46. A sound collection signal (also referred to as a characteristic sound collection signal) collected by the microphone 2L is A / D converted by the A / D converter 131. The A / D converted characteristic sound pickup signal is output to the ECTF acquisition unit 132. Note that the impulse response measurement may be performed a plurality of times, and the characteristic sound pickup signals may be synchronously added.

  The ECTF acquisition unit 132 acquires the ear canal transfer characteristic (ECTF) based on the characteristic sound pickup signal. For example, the ECTF acquisition unit 132 calculates the external auditory canal transfer characteristic in the frequency domain from the collected sound signal for the characteristic in the time domain by FFT (Fast Fourier Transform). Thereby, the power characteristic (power spectrum) of the ear canal transmission characteristic and the phase characteristic (phase spectrum) are generated. An amplitude spectrum may be generated instead of the power spectrum. The ECTF acquisition unit 132 can convert the characteristic sound pickup signal into frequency domain data (frequency characteristics) by discrete Fourier transform, discrete cosine transform, or the like.

  The built-in microphone 48 picks up the measurement signal from the headphone speaker 46 in both the measurement state and the listening state. A sound collection signal (also referred to as a reference sound collection signal) collected by the built-in microphone 48 is A / D converted by the A / D converter 121. The A / D converted reference sound pickup signal is output to the reference signal acquisition unit 122. Specifically, impulse response measurement is performed both in the measurement state in which the listener U wears the left microphone 2L and in the listening state in which the listener U is not wearing. Note that the impulse response measurement may be performed a plurality of times, and the reference collected sound signal may be synchronously added.

  The reference signal acquisition unit 122 acquires a reference signal based on the reference sound collection signal. For example, the reference signal acquisition unit 122 calculates a reference signal in the frequency domain from the reference sound pickup signal in the time domain by FFT (Fast Fourier Transform). As a result, a power characteristic (power spectrum) and a phase characteristic (phase spectrum) of the reference collected sound signal are generated. An amplitude spectrum may be generated instead of the power spectrum. Note that the reference signal acquisition unit 122 can convert the collected sound signal for reference into frequency domain data (frequency characteristics) by discrete Fourier transform, discrete cosine transform, or the like.

  Here, in the measurement state, the reference sound collection signal collected by the built-in microphone 48 is defined as a first reference sound collection signal. The reference signal acquired based on the first reference sound pickup signal is set as the first reference signal. The first reference sound pickup signal is picked up substantially simultaneously with the characteristic sound pickup signal. That is, as for the first reference sound pickup signal measured in the same impulse response measurement, the first reference signal and the ear canal transfer characteristic are acquired based on the characteristic sound pickup signal.

  In the listening state, the reference sound collection signal collected by the built-in microphone 48 is set as a second reference sound collection signal. The reference signal acquired based on the second reference sound collection signal is set as the second reference signal. The first reference signal and the second reference signal are signals that change according to the wearing state of the headphones 43. In other words, the difference between the first reference signal and the second reference signal corresponds to the difference in the wearing state of the headphones 43.

  The memory 140 includes a first storage unit 141, a second storage unit 142, and a third storage unit 143. The first storage unit 141 stores the first reference signal. The second storage unit 142 stores the second reference signal. The third storage unit 143 stores the ear canal transmission characteristics. The first storage unit 141, the second storage unit 142, and the third storage unit 143 may be statically secured in the memory 140, and arbitrary areas may be dynamically secured. .

  Specifically, the third storage unit 143 stores the power spectrum and phase spectrum of the ear canal transfer characteristic. The first storage unit 141 stores the power spectrum of the first reference signal. The second storage unit 142 stores the power spectrum of the second reference signal. Each of the first storage unit 141 and the second storage unit 142 may or may not store the phase spectrum of the reference signal.

  The first storage unit 141, the second storage unit 142, and the third storage unit 143 may be physically single memory devices or different devices. For example, one memory 140 may store the ear canal transfer characteristic, the first reference signal, and the second reference signal. Alternatively, the ear canal transfer characteristic, the first reference signal, and the second reference signal may be stored separately in two or more memories.

  The conversion function calculation unit 151 reads the first reference signal and the second reference signal from the first storage unit 141 and the second storage unit 142. Then, the conversion function calculation unit 151 calculates a frequency characteristic conversion function based on the first reference signal and the second reference signal. The conversion function calculation unit 151 calculates a conversion function based on the two power spectra. That is, the conversion function calculation unit 151 calculates the conversion function by comparing the power spectra of the two reference signals.

  The correction unit 152 reads the ear canal transfer characteristic from the third storage unit 143. Then, the correction unit 152 corrects the ear canal transfer characteristic using the conversion function. The correction unit 152 corrects the power spectrum of the ear canal transfer characteristic. Here, the correction unit 152 corrects only the power spectrum and does not correct the phase spectrum, but may correct the phase spectrum.

  The inverse filter generation unit 153 calculates an inverse filter using the corrected ear canal transfer characteristic (power spectrum). Specifically, the inverse filter generation unit 153 calculates a time signal using the corrected power spectrum (amplitude characteristic) and phase spectrum (phase characteristic) by inverse discrete Fourier transform. The inverse filter generation unit 153 calculates an inverse filter having a predetermined filter length based on the time signal.

  Hereinafter, a description will be given with reference to FIG. 4 based on processing for generating an inverse filter. FIG. 4 is a flowchart showing a process for generating an inverse filter.

  First, by measurement in the measurement state, the ECTF acquisition unit 132 and the reference signal acquisition unit 122 acquire the ear canal transfer characteristic ECTF and the first reference signal Ref1 (S11). That is, with the user U wearing the microphone 2L and the headphones 43, the out-of-head localization processing apparatus 100 performs impulse response measurement. As a result, the microphone 2L and the built-in microphone 48 each pick up a sound pickup signal. Then, based on the characteristic sound pickup signal, the ECTF acquisition unit 132 calculates the ear canal transfer characteristic. Based on the reference sound pickup signal, the reference signal acquisition unit 122 calculates the first reference signal Ref1.

  Next, the reference signal acquisition unit 122 acquires the second reference signal Ref2 by measurement in the listening state (S12). That is, the user U wears the headphones 43 with the microphone 2L removed. Impulse response measurement is performed with the user U wearing only the headphones 43. Based on the reference sound pickup signal, the reference signal acquisition unit 122 calculates the second reference signal Ref2.

  Next, the conversion function calculation unit 151 calculates a conversion function from the first reference signal Ref1 and the second reference signal Ref2 (S13). The correction unit 152 applies a conversion function to the ear canal transfer characteristic ECTF to calculate a corrected ear canal transfer characteristic (hereinafter, referred to as a correction characteristic AdECTF) (S14). Here, a conversion function is applied to the logarithmic power spectrum. That is, the correction unit 152 corrects only the logarithmic power spectrum. The inverse filter generation unit 153 calculates an inverse filter from the correction characteristic AdECTF (S15).

  Next, the process of S13-S15 is demonstrated in detail using FIG. FIG. 5 is a flowchart illustrating an example of the processing of S13 to S15. That is, FIG. 5 is a flowchart illustrating processing in the conversion function calculation unit 151, the correction unit 152, and the inverse filter generation unit 153.

  First, the conversion function calculation unit 151 calculates and normalizes the logarithmic power spectrum of the ear canal transfer characteristic ECTF, the first reference signal Ref1, and the second reference signal Ref2 (S21). The conversion function calculation unit 151, the correction unit 152, and the inverse filter generation unit 153 perform the following processing on the normalized data.

  The conversion function calculation unit 151 obtains extreme values EX1 and EX2 in the logarithmic power spectra PSD1 and PSD2 of the first reference signal Ref1 and the second reference signal (S22). The conversion function calculation unit 151 obtains the extreme value EX1 of the logarithmic power spectrum PSD1 of the first reference signal Ref1 and the extreme value EX2 of the logarithmic power spectrum PSD2 of the second reference signal Ref2. FIG. 6 shows extreme values EX1 and EX2. FIG. 6 is a diagram illustrating a part of the logarithmic power spectrum PSD1, PSD2, where the horizontal axis represents frequency (Hz) and the vertical axis represents power (dB).

  The logarithmic power spectra PSD1 and PSD2 usually have a plurality of extreme values. Here, a plurality of extreme values EX1 of the logarithmic power spectrum PSD1 are assumed to be EX1-1, EX1-2, EX1-3,... EX1-N in order from the low frequency side. Similarly, a plurality of extreme values EX2 of the logarithmic power spectrum PSD2 are set as EX2-1, EX2-2, EX2-3,... EX2-N in order from the low frequency side. In FIG. 6, N = 4, and extreme values EX1-1 to EX1-4 and extreme values EX2-1 to EX2-4 are illustrated.

  The conversion function calculation unit 151 calculates a change vector V between corresponding extreme values for the extreme values EX1 and EX2 (S23). That is, as shown in FIG. 6, the conversion function calculation unit 151 obtains a change vector between the extreme value EX1-1 and the extreme value EX2-1 as a change vector V1. Similarly, the conversion function calculation unit 151 obtains a change vector between the extreme value EX1-2 and the extreme value EX2-2 as a change vector V2. The conversion function calculation unit 151 calculates change vectors V1 to VN. In the example of FIG. 6, since N = 4, four change vectors V1 to V4 are illustrated. The change vector V is a two-dimensional vector whose elements are frequency and logarithmic power.

  If the number of extreme values EX1 of the logarithmic power spectrum PSD1 and the extreme value EX2 of the logarithmic power spectrum PSD2 are different, the change vector V between the nearest extreme values may be obtained. For example, when the number of extreme values EX1 of the logarithmic power spectrum PSD is smaller than the number of extreme values EX2 of the logarithmic power spectrum PSD2, the maximum value of the logarithmic power spectrum PSD closest to the maximum value of the logarithmic power spectrum PSD1 is taken as a pair. A change vector is determined. Similarly, a change vector is obtained by using the minimum value of the logarithmic power spectrum PSD closest to the minimum value of the logarithmic power spectrum PSD1 as a pair.

  Next, the conversion function calculation unit 151 obtains a conversion function based on the change vector V (S24). Here, the conversion function calculation unit 151 calculates a conversion function using a vector conversion method. Specifically, as shown in FIG. 7, grid-like control points are arranged in a logarithmic power spectrum graph. Then, the mesh structure of the control points is changed based on the plurality of change vectors V1 to VN.

  For example, when the control point at (5, 5) changes to (6, 6), the adjacent control point moves in conjunction with the change. The conversion function calculation unit 151 moves the control point close to the extreme value based on the change vector V. The conversion function calculation unit 151 obtains the movement destinations of all control points on the logarithmic power spectrum graph based on the plurality of change vectors V1 to VN. That is, the conversion function calculation unit 151 sets a conversion function (conversion mesh). Since this vector conversion method is a method equivalent to a mesh structure used for, for example, image shape change or morphing of three-dimensional data, detailed description thereof is omitted.

  The correcting unit 152 applies a conversion function to the logarithmic power spectrum PSD of the ear canal transfer characteristic ECTF (S25). Here, for the sake of explanation, the logarithmic power spectrum of the ear canal transfer characteristic ECTF is PSD and the phase spectrum is ASD. The correction unit 152 corrects the logarithmic power spectrum PSD using the conversion function. Let the logarithmic power spectrum after correction be AdPSD. That is, the correction unit 152 calculates a corrected logarithmic power spectrum AdPSD by applying a conversion function to the logarithmic power spectrum PSD. The correction characteristic AdECTF obtained by correcting the ear canal transmission characteristic ECTF is composed of a phase spectrum ASD and a corrected logarithmic power spectrum AdPSD.

  The inverse filter generation unit 153 generates an inverse filter based on the corrected logarithmic power spectrum AdPSD (S26). Specifically, the inverse filter generation unit 153 calculates a time signal from the corrected amplitude characteristic (logarithmic power spectrum AdPSD) and phase characteristic (phase spectrum ASD) by inverse discrete Fourier transform or inverse discrete cosine transform. The ear canal transfer characteristic AdECTF (hereinafter also referred to as a time domain correction characteristic AdECTF) in which the time signal is corrected is shown. The inverse filter generation unit 153 generates an inverse filter whose amplitude and phase are inverted from the correction characteristic AdECTF in the time domain. Since a known method can be used for the generation method of the inverse filter, description thereof is omitted.

  This inverse filter is set in the filter unit 41 shown in FIG. Further, the inverse filter obtained by performing the same process on the right unit 43 is set in the filter unit 42.

  FIG. 8 shows a logarithmic power spectrum PSD of the ear canal transfer characteristic ECTF and a logarithmic power spectrum AdPSD of the correction characteristic AdECTF. Further, FIG. 8 shows a logarithmic power spectrum PSD1 of the first reference signal Ref1 and a logarithmic power spectrum PSD2 of the second reference signal Ref2. Thus, the logarithmic power spectrum AdPSD after correction | amendment can be calculated | required by applying the conversion function based on logarithmic power spectrum PSD1, PSD2 to logarithmic power spectrum PSD.

  The difference between the logarithmic power spectrum PSD1 and the logarithmic power spectrum PSD2 corresponds to a change in the wearing state of the headphones 43. The correction unit 152 can appropriately correct the ear canal transfer characteristic by using a conversion function based on the logarithmic power spectrum PSD1 and the logarithmic power spectrum PSD2. The inverse filter generation unit 153 calculates an inverse filter from the corrected ear canal transfer characteristics. Therefore, the out-of-head localization processing apparatus 100 can perform out-of-head localization processing with high accuracy.

  Specifically, impulse response measurement in a measurement state is performed as a pre-measurement before listening to out-of-head localization. As a result, the first reference signal Ref1 and the ear canal transfer characteristic ECTF can be stored in the memory 140 in advance. When the preliminary measurement is completed, the user U removes the microphone unit 2. Then, when the user U wears the headphones 43, the impulse response measurement in the listening state is performed to obtain the second reference signal Ref2. It is preferable that the reference signal acquisition unit 122 acquires the second reference signal Ref2 every time the headphones 43 are worn. That is, the reference signal acquisition unit 122 acquires the second reference signal Ref2 before the user U wears the headphones 43 and performs out-of-head localization listening.

  Through the above processing, the out-of-head localization processing apparatus 100 calculates an inverse filter from the ear canal transfer characteristic ECTF, the first reference signal Ref1, and the second reference signal Ref2. By doing so, it is possible to perform out-of-head localization processing using an adapted inverse filter. That is, even if the wearing state of the headphones 43 changes, the out-of-head localization processing apparatus 100 can perform out-of-head localization processing using an appropriate inverse filter. Note that it is possible to automatically detect that the headphones 43 are worn at the time of listening and automatically perform the measurement in the listening state. Alternatively, the user U may instruct to perform measurement in the listening state by operating an operation unit such as a touch panel.

  As described above, the conversion function calculation unit 151 calculates a conversion function based on the power spectrum. Therefore, each of the first storage unit 141 and the second storage unit 142 may store only the power spectrum. That is, the first storage unit 141 and the second storage unit 142 may not store the phase spectrum. Or the 1st memory | storage part 141 and the 2nd memory | storage part 142 may memorize | store the sound-collection signal of a time domain as a reference signal as it is. Similarly, the third storage unit 143 may store a time domain sound pickup signal as an ear canal transfer characteristic. Then, each time the conversion function calculation unit 151 calculates the conversion function, the power spectrum of the reference signal and the ear canal transfer characteristic may be calculated by performing discrete Fourier transform or the like.

  Note that the calculation method of the conversion function is not limited to the above method. The conversion function calculation unit 151 can calculate the conversion function not only by the above-described vector conversion but also by various parameters and conversion methods. For example, the conversion function may be obtained based on the difference between the logarithmic power spectrum PSD1 and the logarithmic power spectrum PSD2.

Using the logarithmic power spectrum PSD of the ear canal transfer characteristic ECTF before correction, the logarithmic power spectrum PSD1 of the first reference signal Ref1, and the logarithmic power spectrum PSD2 of the second reference signal Ref2, the logarithmic power of the ear canal transfer characteristic ECTF after correction is used. The following formula (1) can be used as a conversion function for the spectrum AdPSD.
AdPAD = PSD + (PSD2-PSD1) (1)

  2 may be the same device as the out-of-head localization processing device 100, or may be a different device. When the out-of-head localization processing apparatus 100 and the processing apparatus 201 are different apparatuses, the out-of-head localization processing apparatus 100 may use the ear canal transfer characteristic ECTF and the first reference signal Ref1 acquired by the processing apparatus 201.

  Specifically, the processing device 201 performs an impulse response measurement in a measurement state in order to acquire the ear canal transfer characteristic ECTF and the first reference signal Ref1. Then, the processing device 201 transmits the ear canal transfer characteristic ECTF and the first reference signal Ref1 to the out-of-head localization processing device 100 wirelessly or by wire. Alternatively, part or all of the data of the ear canal transfer characteristic ECTF and the first reference signal Ref1 may be stored outside. The processing device 201 stores the ear canal transfer characteristic ECTF and the first reference signal Ref1 in an external storage device, a cloud network, or the like. The out-of-head localization processing apparatus 100 reads or receives data stored in an external storage device, a cloud network, or the like, thereby acquiring the ear canal transfer characteristic ECTF and the first reference signal Ref1. In this case, the memory 140 may be a memory that temporarily stores the ear canal transfer characteristic ECTF, the first reference signal Ref1, and the second reference signal Ref2. That is, the data may be deleted after each data is used.

  Before listening to the headphones 43, the out-of-head localization processing apparatus 100 performs impulse response measurement in the listening state to obtain the second reference signal Ref2. Using the external auditory canal transfer characteristic ECTF, the first reference signal Ref1, and the second reference signal Ref2 acquired in this way, the out-of-head localization processing apparatus 100 performs the above correction processing. Therefore, the out-of-head localization processing apparatus 100 can obtain an inverse filter adapted to the headphone wearing state. Thereby, an out-of-head localization process can be performed appropriately, and a sound image localization effect with high accuracy can be obtained.

  Part or all of the above processing may be executed by a computer program. The programs described above can be stored and provided to a computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

  As mentioned above, the invention made by the present inventor has been specifically described based on the embodiment. However, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the scope of the invention. Needless to say.

U user 1 person to be measured 10 out-of-head localization processing unit 11 convolution operation unit 12 convolution operation unit 21 convolution operation unit 22 convolution operation unit 24 adder 25 adder 41 filter unit 42 filter unit 43 headphone 45 housing 46 headphone speaker 48 built-in microphone DESCRIPTION OF SYMBOLS 100 Out-of-head localization processing apparatus 111 Measurement signal production | generation part 112 D / A converter 121 A / D converter 122 Reference signal acquisition part 131 A / D converter 132 ECTF acquisition part 140 Memory 141 1st memory | storage part 142 2nd memory | storage part 143 Third storage unit 151 Conversion function calculation unit 152 Correction unit 153 Inverse filter generation unit

Claims (4)

An audio output unit having a reference microphone;
A reference signal acquisition unit for acquiring a reference signal based on a collected sound signal collected by the reference microphone;
A first storage unit that stores the first reference signal acquired by the reference signal acquisition unit in a measurement state in which a measurement microphone is arranged in the ear canal of the user wearing the audio output unit;
A second storage unit that stores the second reference signal acquired by the reference signal acquisition unit in a listening state in which the measurement microphone is removed from the ear canal;
A third storage unit that stores a transfer characteristic acquired based on a collected sound signal collected by the measurement microphone in the measurement state;
A conversion function calculating unit that calculates a conversion function of a frequency characteristic based on the first reference signal and the second reference signal;
A correction unit that corrects the frequency characteristic of the transfer characteristic using the conversion function;
A filter generation unit that generates a filter based on the corrected frequency characteristics;
An out-of-head localization processing apparatus comprising: an out-of-head localization processing unit that performs out-of-head localization processing using the filter. The conversion function calculation unit obtains a change vector between extreme values of power spectra of the first reference signal and the second reference signal,
The out-of-head localization processing apparatus according to claim 1, wherein the conversion function is calculated based on the change vector. Acquiring a transfer characteristic based on a collected sound signal collected by the measurement microphone in a measurement state in which the measurement microphone is arranged in the ear canal of the user wearing a sound output unit having a reference microphone;
Obtaining a first reference signal based on a collected sound signal collected by the reference microphone in the measurement state;
Obtaining a second reference signal based on a collected sound signal collected by the reference microphone in a listening state in which the measurement microphone is removed from the ear canal;
Calculating a conversion function of a frequency characteristic based on the first reference signal and the second reference signal;
Correcting the frequency characteristic of the transfer characteristic using the conversion function;
Generating a filter based on the corrected frequency characteristic;
Performing an out-of-head localization process using the filter. On the computer,
Acquiring a transfer characteristic based on a collected sound signal collected by the measurement microphone in a measurement state in which the measurement microphone is arranged in the ear canal of the user wearing a sound output unit having a reference microphone;
Obtaining a first reference signal based on a collected sound signal collected by the reference microphone in the measurement state;
Obtaining a second reference signal based on a collected sound signal collected by the reference microphone in a listening state in which the measurement microphone is removed from the ear canal;
Calculating a conversion function of a frequency characteristic based on the first reference signal and the second reference signal;
Correcting the frequency characteristic of the transfer characteristic using the conversion function;
Generating a filter based on the corrected frequency characteristic;
Performing out-of-head localization using the filter;
A program that executes

Images