JP2021052315A - Out-of-head localization filter determination system, out-of-head localization processing device, out-of-head localization filter determination device, out-of-head localization filter determination method, and program - Google Patents

Abstract

To provide an out-of-head localization filter determination system that can determine a filter appropriately, an out-of-head localization processing device, an out-of-head localization filter determination device, an out-of-head localization filter determination method, and a program.SOLUTION: An out-of-head localization filter determination system according to an embodiment includes a frequency conversion portion 112 that converts frequency of a sound collection signal to obtain frequency characteristics, a smoothing portion 113 that smoothes the frequency characteristics, a feature amount extraction portion 114 that obtains a peak and a notch of the smoothed frequency characteristic and extracts the feature amount of the frequency characteristic as a user feature amount on the basis of the peak and the notch, a data extraction portion 302 that extracts second preset data from a data storage portion 303, a comparison unit 304 that compares the user feature amount with the extracted second preset data, and a selection portion 305 that selects the first preset data from the plurality of pieces of the first preset data on the basis of the comparison result.SELECTED DRAWING: Figure 4

Description

The present invention relates to an out-of-head localization filter determination system, an out-of-head localization processing device, an out-of-head localization filter determination device, an out-of-head localization filter determination method, and a program.

As a sound image localization technique, there is an out-of-head localization technique in which a sound image is localized on the outside of the listener's head using headphones. In the out-of-head localization technology, the sound image is localized out of the head by canceling the characteristics from the headphones to the ears and giving four characteristics from the stereo speakers to the ears.

In out-of-head localization playback, measurement signals (impulse sounds, etc.) emitted from two-channel (hereinafter referred to as ch) speakers are recorded with a microphone (hereinafter referred to as a microphone) installed in the listener's (user's) ear. To do. Then, the processing device creates a filter based on the sound pick-up signal obtained by the impulse response. By convolving the created filter into a 2ch audio signal, out-of-head localization reproduction can be realized.

Furthermore, in order to generate a filter for canceling the characteristics from the headphones to the ear, a microphone in which the characteristics from the headphones to the ear to the eardrum (also called the external auditory canal transfer function ECTF or external auditory canal transfer characteristic) is installed in the listener's own ear Measure with.

Patent Document 1 discloses a binaural hearing device using an extracranial sound image localization filter. In this device, a large number of human pre-measured spatial transfer functions are converted into feature parameter vectors corresponding to human auditory characteristics. Then, the apparatus uses the data aggregated in a small number by performing clustering. Further, the device clusters the spatial transfer function measured in advance and the reverse transfer function of the actual ear headphones according to the physical dimensions of a human being. Then, the human data closest to the center of gravity of each cluster is used.

Patent Document 2 discloses an out-of-head localization filter determining device including a headphone and a microphone unit. In Patent Document 1, the server device associates the first preset data regarding the spatial acoustic transmission characteristic from the sound source to the ear of the person to be measured with the second preset data regarding the external auditory canal transmission characteristic of the ear of the person to be measured. I remember. The user terminal is measuring measurement data regarding the user's ear canal transmission characteristics. The user terminal transmits user data based on the measurement data to the server device. The server device compares the user data with a plurality of second preset data. Specifically, the features based on the external auditory canal transmission characteristics of the user and the subject are compared. The server device extracts the first preset data based on the comparison result.

Japanese Unexamined Patent Publication No. 8-11189 Japanese Unexamined Patent Publication No. 2018-191208

However, in the device of Patent Document 1, since clustering is performed based on the physical dimensions, it is necessary to measure the physical dimensions of the individual user. In addition, clustering may not be performed properly. In this case, there is a problem that an out-of-head sound image localization filter suitable for the user cannot be used.

Further, in Patent Document 2, feature quantities based on the external auditory canal transmission characteristics of the user and the subject are compared. Specifically, the similarity of the feature amount is calculated, and the first preset data of the subject to be measured with high correlation is extracted. The feature amount has a frequency amplitude characteristic of 2 kHz to 20 kHz. The spatial acoustic transmission characteristics corresponding to the external auditory canal transmission characteristics of the subject with high similarity are extracted and used as the user's spatial acoustic filter.

When performing a process using a spatial acoustic transmission filter based on the external auditory canal transmission characteristics of the user as in Patent Document 2, it is desired to perform the process appropriately and quickly. However, in Patent Document 2, since the feature amount is a frequency amplitude characteristic of 2 kHz to 20 kHz, it is difficult to reduce the amount of data. If there is preset data for many subjects, there is a problem that the processing time becomes long.

This embodiment has been made in view of the above points, and can perform an appropriate and quick process of an out-of-head localization filter determination system, an out-of-head localization processing device, an out-of-head localization filter determination device, and an out-of-head localization filter determination. The purpose is to provide methods and programs.

The out-of-head localization filter determination system according to the present embodiment is attached to an output unit that is attached to the user and outputs sound toward the user's ear, and a sound that is attached to the user's ear and is output from the output unit. A microphone unit that collects sound, a measurement processing device that outputs a measurement signal to the output unit and measures the sound collection signal output from the microphone unit, and a server device that can communicate with the measurement processing device. The server device is an out-of-head localization filter determination system comprising, the first preset data regarding the spatial acoustic transmission characteristics from the sound source to the subject's ear, and the external auditory canal transmission characteristics of the subject's ear. A data storage unit that stores the second preset data related to the feature amount in association with each other, and stores a plurality of the first and second preset data acquired for a plurality of subjects. The out-of-head localization filter determination system including a storage unit includes a frequency conversion unit that frequency-converts the sound collection signal picked up by the microphone unit to obtain frequency characteristics, and a smoothing unit that smoothes the frequency characteristics. A feature amount extraction unit that obtains a smoothed peak and notch of the frequency characteristic and extracts the feature amount of the frequency characteristic as a user feature amount based on the peak and notch, and a feature amount extraction unit based on the user feature amount. A data extraction unit that extracts a part of the second preset data among the plurality of second preset data stored in the data storage unit, the user feature amount, and the extracted second preset data. It is provided with a comparison unit for comparing with and a selection unit for selecting the first preset data from the plurality of first preset data based on the comparison result.

The out-of-head localization processing device according to the present embodiment includes a frequency conversion unit that obtains frequency characteristics by frequency-converting a sound-collecting signal acquired by collecting the measurement signal output from the output unit with the microphone unit. , A smoothing unit for smoothing the frequency characteristics, and the peaks and notches of the smoothed frequency characteristics are obtained, and the feature amount of the frequency characteristics is extracted as a user feature amount based on the peaks and notches. A space acoustic filter setting unit that sets a spatial acoustic filter based on a unit, a spatial acoustic transmission characteristic associated with a feature amount similar to the user feature amount, and an output unit of the output unit based on the sound pick-up signal. It is provided with an inverse filter calculation unit that calculates an inverse filter that cancels the characteristics.

The extrahead localization filter determining device according to the present embodiment has the first preset data regarding the spatial acoustic transmission characteristics from the sound source to the subject's ear, and the first preset data regarding the characteristics of the external auditory canal transmission characteristics of the subject's ear. A data storage unit that stores two preset data in association with each other, and a data storage unit that stores a plurality of the first and second preset data acquired for a plurality of subjects, and a user. A data extraction unit that extracts a part of the second preset data among a plurality of second preset data stored in the data storage unit based on the feature amount, the user feature amount, and the second preset data. The external auditory canal transmission characteristic is provided with a comparison unit for comparing the feature amount of the preset data of the above and a selection unit for selecting the first preset data from the plurality of first preset data based on the comparison result. The frequency characteristics of the above are smoothed, and the feature amount is extracted based on the peaks and notches of the smoothed frequency characteristics.

The method for determining an out-of-head localization filter according to the present embodiment includes an output unit that is attached to the user and outputs a sound toward the user's ear, and a sound that is attached to the user's ear and is output from the output unit. This is an out-of-head localization filter determination method for determining an out-of-head localization filter for the user by using a microphone unit having a microphone for collecting sound, and frequency-converting the sound collection signal collected by the microphone unit. The step of obtaining the frequency characteristic, the step of smoothing the frequency characteristic, the peak and the notch of the smoothed frequency characteristic are obtained, and the feature amount of the frequency characteristic is calculated as the user feature amount based on the peak and the notch. A step of extracting a part of the second preset data among a plurality of second preset data stored in the data storage unit based on the user feature amount, and the user feature amount. A step of comparing the extracted second preset data with the extracted second preset data, and a step of selecting the first preset data from the plurality of first preset data based on the comparison result.

The program according to the present embodiment includes an output unit that is attached to the user and outputs sound toward the user's ear, and a microphone that is attached to the user's ear and collects the sound output from the output unit. This is a program for causing a computer to execute an out-of-head localization filter determination method for determining an out-of-head localization filter for the user by using a microphone unit having the above, and the out-of-head localization filter determination method is the microphone unit. The step of frequency-converting the sound pick-up signal picked up in (1) to obtain the frequency characteristic, the step of smoothing the frequency characteristic, and the peak and notch of the smoothed frequency characteristic are obtained, and the peak and the notch are used. Based on the step of extracting the feature amount of the frequency characteristic as the user feature amount, and the second of a part of the plurality of second preset data stored in the data storage unit based on the user feature amount. A step of extracting preset data, a step of comparing the user feature amount with the extracted second preset data, and a first preset from a plurality of first preset data based on the comparison result. Includes steps to select data.

According to the present embodiment, an out-of-head localization filter determination system, an out-of-head localization processing device, an out-of-head localization filter determination device, an out-of-head localization filter determination method, and a program capable of performing processing appropriately and quickly are provided. Can be done.

It is a block diagram which shows the out-of-head localization processing apparatus which concerns on this embodiment. It is a figure which shows typically the structure of the measuring apparatus. It is a figure which shows typically the structure of the measuring apparatus. It is a block diagram which shows the structure of the out-of-head localization filter determination system. It is a flowchart which shows the process of extracting a feature amount. It is a graph which shows the smoothing property schematically. This is a table showing preset data stored in the data storage unit. It is a table showing the data of the feature quantity. It is a flowchart which shows the process in a server device. It is a flowchart which shows the process of calculating the distance of a feature vector. It is a flowchart which shows the process of calculating the correlation between the user data and the second preset data. It is a graph which shows the interpolation characteristic schematically. It is a figure for demonstrating the cluster of preset data.

(Overview)
First, the outline of the sound image localization process will be described. Here, the out-of-head localization processing, which is an example of the sound image localization processing apparatus, will be described. The extra-head localization process according to the present embodiment is to perform the extra-head localization process using the spatial acoustic transmission characteristic and the external auditory canal transmission characteristic. The spatial acoustic transmission characteristic is a transmission characteristic from a sound source such as a speaker to the ear canal. The ear canal transmission characteristic is the transmission characteristic from the ear canal entrance to the eardrum. In the present embodiment, the external auditory canal transmission characteristic is measured while the headphones are worn, and the extra-head localization process is realized by using the measurement data.

The out-of-head localization process according to this embodiment is executed on a user terminal such as a personal computer (PC), a smart phone, or a tablet terminal. A user terminal is an information processing device having a processing means such as a processor, a storage means such as a memory or a hard disk, a display means such as a liquid crystal monitor, and an input means such as a touch panel, a button, a keyboard, and a mouse. The user terminal has a communication function for transmitting and receiving data. Further, an output means (output unit) having headphones or earphones is connected to the user terminal.

In order to obtain a high localization effect, it is necessary to measure the characteristics of the user himself / herself and generate an out-of-head localization filter. The spatial acoustic transmission characteristics of an individual user are generally performed in a listening room in which acoustic equipment such as a speaker or indoor acoustic characteristics are arranged. That is, it is necessary for the user to go to the listening room or prepare a listening room at the user's home or the like. Therefore, it may not be possible to appropriately measure the spatial acoustic transmission characteristics of the individual user.

Further, even when a speaker is installed at the user's home or the like to prepare a listening room, the speaker may be installed asymmetrically or the acoustic environment of the room may not be optimal for listening to music. In such cases, it is very difficult to measure appropriate spatial acoustic transmission characteristics at home.

On the other hand, the measurement of the external auditory canal transmission characteristic of an individual user is performed with the microphone unit and headphones attached. That is, if the user wears a microphone unit and headphones, the external auditory canal transmission characteristic can be measured. There is no need for the user to go to the listening room or set up a large listening room in the user's home. Further, the generation of the measurement signal for measuring the external auditory canal transmission characteristic, the recording of the sound collection signal, and the like can be performed by using a user terminal such as a smart phone or a PC.

As described above, it may be difficult to measure the spatial acoustic transmission characteristics for an individual user. Therefore, in the extrahead localization processing system according to the present embodiment, the filter according to the spatial acoustic transmission characteristic is determined based on the measurement result of the external auditory canal transmission characteristic. That is, an extracranial localization processing filter suitable for the user is determined based on the measurement result of the external auditory canal transmission characteristic of the individual user.

Specifically, the out-of-head localization processing system includes a user terminal and a server device. The server device stores the spatial acoustic transmission characteristics and the external auditory canal transmission characteristics measured in advance for a plurality of subjects other than the user. That is, measurement of spatial acoustic transmission characteristics using a speaker as a sound source (hereinafter, also referred to as first pre-measurement) using a measuring device different from the user terminal, and measurement of external auditory canal transmission characteristics using headphones (second). (Also referred to as pre-measurement of). The first pre-measurement and the second pre-measurement are performed on a person to be measured other than the user.

The server device stores the first preset data according to the result of the first pre-measurement and the second preset data according to the result of the second pre-measurement. By performing the first and second pre-measurements on the plurality of subjects, the plurality of first preset data and the plurality of second preset data are acquired. The server device stores the first preset data regarding the spatial acoustic transmission characteristic and the second preset data regarding the external auditory canal transmission characteristic in association with each other for each person to be measured. The server device stores a plurality of first preset data and a plurality of second preset data in the database.

Further, for the individual user who executes the out-of-head localization process, only the external auditory canal transmission characteristic is measured by using the user terminal (hereinafter referred to as user measurement). The user measurement is a measurement using headphones as a sound source, as in the second pre-measurement. The user terminal acquires measurement data regarding the external auditory canal transmission characteristic. Then, the user terminal transmits the user data based on the measurement data to the server device. The server device compares the user data with the plurality of second preset data, respectively. The server device determines the second preset data having a high correlation with the user data from the plurality of second preset data based on the comparison result.

Then, the server device reads out the first preset data associated with the second preset data having a high correlation. That is, the server device extracts the first preset data suitable for the individual user from the plurality of first preset data based on the comparison result. The server device transmits the extracted first preset data to the user terminal. Then, the user terminal performs the out-of-head localization process by using the filter based on the first preset data and the inverse filter based on the user measurement.

Embodiment 1.
(Out-of-head localization processing device)
First, FIG. 1 shows an out-of-head localization processing device 100 which is an example of the sound field reproducing device according to the present embodiment. FIG. 1 is a block diagram of the out-of-head localization processing device 100. The out-of-head localization processing device 100 reproduces the sound field for the user U who wears the headphones 43. Therefore, the out-of-head localization processing device 100 performs sound image localization processing on the stereo input signals XL and XR of Lch and Rch. The Lch and Rch stereo input signals XL and XR are analog audio reproduction signals output from a CD (Compact Disc) player or the like, or digital audio data such as mp3 (MPEG Audio Layer-3). The out-of-head localization processing device 100 is not limited to a physically single device, and some of the processing may be performed by different devices. For example, a part of the processing may be performed by a PC or the like, and the remaining processing may be performed by a DSP (Digital Signal Processor) or the like built in the headphones 43.

The out-of-head localization processing device 100 includes an out-of-head localization processing unit 10, a filter unit 41, a filter unit 42, and headphones 43. The out-of-head localization processing unit 10, the filter unit 41, and the filter unit 42 can be realized by a processor.

The out-of-head localization processing unit 10 includes convolution calculation units 11 to 12, 21 to 22, and adders 24 and 25. The convolution calculation units 11 to 12 and 21 to 22 perform a convolution process using the spatial acoustic transmission characteristic. Stereo input signals XL and XR from a CD player or the like are input to the out-of-head localization processing unit 10. Spatial acoustic transmission characteristics are set in the out-of-head localization processing unit 10. The out-of-head localization processing unit 10 convolves a filter having spatial acoustic transmission characteristics (hereinafter, also referred to as a spatial acoustic filter) with the stereo input signals XL and XR of each channel. The spatial acoustic transmission characteristic may be a head-related transfer function HRTF measured on the head or auricle of the subject, or may be a dummy head or a third-party head-related transfer function.

The spatial acoustic transfer function is a set of four spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs. The data used for convolution by the convolution calculation units 11, 12, 21, and 22 serves as a spatial acoustic filter. Each of the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs is measured using a measuring device described later.

Then, the convolution calculation unit 11 convolves the spatial acoustic filter corresponding to the spatial acoustic transmission characteristic Hls with respect to the stereo input signal XL of the Lch. The convolution calculation unit 11 outputs the convolution calculation data to the adder 24. The convolution calculation unit 21 convolves a spatial acoustic filter corresponding to the spatial acoustic transmission characteristic Hro with respect to the stereo input signal XR of Rch. The convolution calculation unit 21 outputs the convolution calculation data to the adder 24. The adder 24 adds two convolution operation data and outputs the data to the filter unit 41.

The convolution calculation unit 12 convolves a spatial acoustic filter corresponding to the spatial acoustic transmission characteristic Hlo with respect to the Lch stereo input signal XL. The convolution calculation unit 12 outputs the convolution calculation data to the adder 25. The convolution calculation unit 22 convolves a spatial acoustic filter corresponding to the spatial acoustic transmission characteristic Hrs with respect to the stereo input signal XR of Rch. The convolution calculation unit 22 outputs the convolution calculation data to the adder 25. The adder 25 adds two convolution operation data and outputs the data to the filter unit 42.

Inverse filters that cancel the headphone characteristics (characteristics between the headphone playback unit and the microphone) are set in the filter units 41 and 42. Then, the inverse filter is convoluted into the reproduced signal (convolution calculation signal) processed by the out-of-head localization processing unit 10. The filter unit 41 convolves the inverse filter with respect to the Lch signal from the adder 24. Similarly, the filter unit 42 convolves the inverse filter with respect to the Rch signal from the adder 25. The reverse filter cancels the characteristics from the headphone unit to the microphone when the headphone 43 is attached. The microphone may be placed anywhere between the ear canal entrance and the eardrum. The inverse filter is calculated from the measurement result of the characteristics of the user U himself / herself.

The filter unit 41 outputs the corrected Lch signal to the left unit 43L of the headphones 43. The filter unit 42 outputs the corrected Rch signal to the right unit 43R of the headphones 43. The user U is wearing the headphones 43. The headphone 43 outputs the Lch signal and the Rch signal toward the user U. As a result, the sound image localized outside the head of the user U can be reproduced.

As described above, the out-of-head localization processing device 100 performs the out-of-head localization processing by using the spatial acoustic filter corresponding to the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs and the inverse filter of the headphone characteristics. In the following description, the spatial acoustic filter corresponding to the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs and the inverse filter of the headphone characteristics are collectively referred to as an out-of-head localization processing filter. In the case of a 2ch stereo reproduction signal, the out-of-head localization filter is composed of four spatial acoustic filters and two inverse filters. Then, the out-of-head localization processing device 100 executes the out-of-head localization processing by performing a convolution calculation process on the stereo reproduction signal using a total of six out-of-head localization filters.

(Measuring device for spatial acoustic transmission characteristics)
The measuring device 200 for measuring the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs will be described with reference to FIG. FIG. 2 is a diagram schematically showing a measurement configuration for performing the first pre-measurement on the person to be measured 1.

As shown in FIG. 2, the measuring device 200 includes a stereo speaker 5 and a microphone unit 2. The stereo speaker 5 is installed in the measurement environment. The measurement environment may be the user U's home room, an audio system sales store, a showroom, or the like. The measurement environment is preferably a listening room with speakers and sound.

In the present embodiment, the measurement processing device 201 of the measuring device 200 performs arithmetic processing for appropriately generating the spatial acoustic filter. The measurement processing device 201 includes, for example, a music player such as a CD player. The measurement processing device 201 may be a personal computer (PC), a tablet terminal, a smart phone, or the like. Further, the measurement processing device 201 may be the server device itself.

The measurement processing device 201 includes a memory and a processor. The memory stores processing programs, various parameters, measurement data, and the like. The processor executes a processing program stored in memory. Each process is executed when the processor executes the process program. The processor may be, for example, a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit), or the like. ..

The stereo speaker 5 includes a left speaker 5L and a right speaker 5R. For example, a left speaker 5L and a right speaker 5R are installed in front of the person to be measured 1. The left speaker 5L and the right speaker 5R output an impulse sound or the like for measuring an impulse response. Hereinafter, in the present embodiment, the number of speakers serving as sound sources will be described as 2 (stereo speakers), but the number of sound sources used for measurement is not limited to 2, and may be 1 or more. That is, the present embodiment can be similarly applied to a so-called multi-channel environment such as 1ch monaural or 5.1ch, 7.1ch, etc.

The microphone unit 2 is a stereo microphone having a left microphone 2L and a right microphone 2R. The left microphone 2L is installed in the left ear 9L of the person to be measured 1, and the right microphone 2R is installed in the right ear 9R of the person to be measured 1. Specifically, it is preferable to install microphones 2L and 2R at positions from the entrance of the ear canal to the eardrum of the left ear 9L and the right ear 9R. The microphones 2L and 2R pick up the measurement signal output from the stereo speaker 5 and acquire the sound pick-up signal. The microphones 2L and 2R output the sound pick-up signal to the measurement processing device 201. The person to be measured 1 may be a person or a dummy head. That is, in the present embodiment, the person to be measured 1 is a concept including not only a person but also a dummy head.

As described above, the impulse response is measured by measuring the impulse sound output by the left speaker 5L and the right speaker 5R with the microphones 2L and 2R. The measurement processing device 201 stores the sound pick-up signal acquired by the impulse response measurement in a memory or the like. As a result, the spatial acoustic transmission characteristic Hls between the left speaker 5L and the left microphone 2L, the spatial acoustic transmission characteristic Hlo between the left speaker 5L and the right microphone 2R, and the spatial acoustic between the right speaker 5R and the left microphone 2L. The transmission characteristic Hro and the spatial acoustic transmission characteristic Hrs between the right speaker 5R and the right microphone 2R are measured. That is, the spatial acoustic transmission characteristic Hls is acquired by the left microphone 2L collecting the measurement signal output from the left speaker 5L. The spatial acoustic transmission characteristic Hlo is acquired by the right microphone 2R collecting the measurement signal output from the left speaker 5L. The spatial acoustic transmission characteristic Hiro is acquired by the left microphone 2L collecting the measurement signal output from the right speaker 5R. The spatial acoustic transmission characteristic Hrs is acquired by the right microphone 2R picking up the measurement signal output from the right speaker 5R.

Further, the measuring device 200 generates a spatial acoustic filter corresponding to the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs from the left and right speakers 5L and 5R to the left and right microphones 2L and 2R based on the sound pick-up signal. May be good. For example, the measurement processing device 201 cuts out the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs with a predetermined filter length. The measurement processing device 201 may correct the measured spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs.

By doing so, the measurement processing device 201 generates a spatial acoustic filter used for the convolution calculation of the out-of-head localization processing device 100. As shown in FIG. 1, the out-of-head localization processing device 100 uses a spatial acoustic filter according to the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs between the left and right speakers 5L and 5R and the left and right microphones 2L and 2R. Perform out-of-head localization processing using. That is, the out-of-head localization process is performed by convolving the spatial acoustic filter into the audio reproduction signal.

The measurement processing device 201 performs the same processing on the sound pick-up signals corresponding to each of the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs. That is, the same processing is performed on each of the four sound pick-up signals corresponding to the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs. As a result, it is possible to generate spatial acoustic filters corresponding to the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs, respectively.

(Measurement of ear canal transmission characteristics)
Next, the measuring device 200 for measuring the external auditory canal transmission characteristic will be described with reference to FIG. FIG. 3 shows a configuration for performing a second pre-measurement on the person to be measured 1.

The microphone unit 2 and the headphones 43 are connected to the measurement processing device 201. The microphone unit 2 includes a left microphone 2L and a right microphone 2R. The left microphone 2L is attached to the left ear 9L of the person to be measured 1. The right microphone 2R is attached to the right ear 9R of the person to be measured 1. The measurement processing device 201 and the microphone unit 2 may be the same as or different from the measurement processing device 201 and the microphone unit 2 of FIG.

The headphone 43 has a headphone band 43B, a left unit 43L, and a right unit 43R. The headphone band 43B connects the left unit 43L and the right unit 43R. The left unit 43L outputs sound toward the left ear 9L of the person to be measured 1. The right unit 43R outputs sound toward the right ear 9R of the person to be measured 1. The headphone 43 may be of any type, such as a closed type, an open type, a semi-open type, or a semi-closed type. In the headphone 43, the microphone unit 2 is attached to the person to be measured 1 with the headphone 43 attached. That is, the left unit 43L and the right unit 43R of the headphones 43 are attached to the left ear 9L and the right ear 9R to which the left microphone 2L and the right microphone 2R are attached, respectively. The headphone band 43B generates an urging force that presses the left unit 43L and the right unit 43R against the left ear 9L and the right ear 9R, respectively.

The left microphone 2L collects the sound output from the left unit 43L of the headphones 43. The right microphone 2R collects the sound output from the right unit 43R of the headphones 43. The microphone portions of the left microphone 2L and the right microphone 2R are arranged at sound collecting positions near the outer ear canal. The left microphone 2L and the right microphone 2R are configured so as not to interfere with the headphone 43. That is, the subject 1 can wear the headphones 43 in a state where the left microphone 2L and the right microphone 2R are arranged at appropriate positions of the left ear 9L and the right ear 9R. The left microphone 2L and the right microphone 2R may be built in the left unit 43L and the right unit 43R of the headphone 43, respectively, or may be provided separately from the headphone 43.

The measurement processing device 201 outputs a measurement signal to the left microphone 2L and the right microphone 2R. As a result, the left microphone 2L and the right microphone 2R generate an impulse sound or the like. Specifically, the impulse sound output from the left unit 43L is measured by the left microphone 2L. The impulse sound output from the right unit 43R is measured by the right microphone 2R. By doing so, the impulse response measurement is performed.

The measurement processing device 201 stores a sound pick-up signal based on the impulse response measurement in a memory or the like. As a result, the transmission characteristic between the left unit 43L and the left microphone 2L (that is, the ear canal transmission characteristic of the left ear) and the transmission characteristic between the right unit 43R and the right microphone 2R (that is, the ear canal transmission characteristic of the right ear). ) Is acquired. Here, the measurement data of the external auditory canal transmission characteristic of the left ear acquired by the left microphone 2L is referred to as measurement data ECTFL, and the measurement data of the external auditory canal transmission characteristic of the right ear acquired by the right microphone 2R is referred to as measurement data ECTFR.

The measurement processing device 201 has a memory for storing measurement data ECTFL and ECTFR, respectively. The measurement processing device 201 generates an impulse signal, a TSP (Time Stretched Pulse) signal, or the like as a measurement signal for measuring the external auditory canal transmission characteristic or the spatial acoustic transmission characteristic. The measurement signal includes a measurement sound such as an impulse sound.

The external auditory canal transmission characteristics and the spatial acoustic transmission characteristics of the plurality of subjects 1 are measured by the measuring device 200 shown in FIGS. 2 and 3. In the present embodiment, the first pre-measurement according to the measurement configuration of FIG. 2 is performed on a plurality of subjects 1. Similarly, the second pre-measurement according to the measurement configuration of FIG. 3 is performed on a plurality of subjects 1. As a result, the external auditory canal transmission characteristic and the spatial acoustic transmission characteristic are measured for each person to be measured 1.

(Out-of-head localization filter determination system)
Next, the out-of-head localization filter determination system 500 according to the present embodiment will be described with reference to FIG. FIG. 4 is a diagram showing the overall configuration of the out-of-head localization filter determination system 500. The out-of-head localization filter determination system 500 includes an out-of-head localization processing device 100 and a server device 300.

The out-of-head localization processing device 100 and the server device 300 are connected via a network 400. The network 400 is, for example, a public network such as the Internet or a mobile phone communication network. The out-of-head localization processing device 100 and the server device 300 can communicate with each other wirelessly or by wire. The out-of-head localization processing device 100 and the server device 300 may be integrated devices.

As shown in FIG. 1, the out-of-head localization processing device 100 is a user terminal that outputs a reproduction signal that has undergone out-of-head localization processing to the user U. Further, the out-of-head localization processing device 100 measures the external auditory canal transmission characteristic of the user U. Therefore, the microphone unit 2 and the headphones 43 are connected to the out-of-head localization processing device 100. The out-of-head localization processing device 100 performs impulse response measurement using the microphone unit 2 and the headphones 43, similarly to the measuring device 200 of FIG. The microphone unit 2 and the headphones 43 may be wirelessly connected by Bluetooth (registered trademark) or the like.

The out-of-head localization processing device 100 includes a characteristic acquisition unit 111, a frequency conversion unit 112, a smoothing unit 113, a feature amount extraction unit 114, an inverse filter calculation unit 115, a filter setting unit 116, a transmission unit 121, and a reception unit. It has 122. When the out-of-head localization processing device 100 and the server device 300 are integrated devices, the device may include an acquisition unit for acquiring user data instead of the reception unit 122.

The characteristic acquisition unit 111 outputs a measurement signal that becomes an impulse sound to the headphones 43 in order to perform user measurement. The microphone unit 2 collects the impulse sound output by the headphones 43. The microphone unit 2 outputs a sound pick-up signal to the characteristic acquisition unit 111. Since the impulse response measurement is the same as that described in FIG. 3, the description thereof will be omitted as appropriate. That is, the out-of-head localization processing device 100 has the same function as the measurement processing device 201 of FIG. The out-of-head localization processing device 100, the microphone unit 2, and the headphones 43 perform user measurement. The characteristic acquisition unit 111 may perform A / D conversion, synchronous addition processing, or the like on the sound pick-up signal.

The characteristic acquisition unit 111 acquires measurement data regarding the external auditory canal transmission characteristic. The characteristic acquisition unit 111 has the same function as the measurement processing device 201 of FIG. 3 for performing impulse response measurement and the like. The measurement data includes the measurement data regarding the external auditory canal transmission characteristic of the left ear 9L of the user U and the measurement data regarding the external auditory canal transmission characteristic of the right ear 9R. Here, let f (t) be the external auditory canal transmission characteristic in the time domain.

Hereinafter, the process of obtaining the feature amount of the external auditory canal transmission characteristic will be described with reference to FIG. FIG. 5 is a flowchart showing a process of extracting a feature amount.

The frequency conversion unit 112 frequency-converts the external auditory canal transmission characteristic f (t) (S11). For example, the frequency conversion unit 112 calculates the frequency amplitude characteristic and the frequency phase characteristic by performing a discrete Fourier transform on the external auditory canal transmission characteristic in the time domain. Further, the frequency conversion unit 112 may calculate the frequency amplitude characteristic and the frequency phase characteristic not only by the discrete Fourier transform but also by the discrete cosine transform or the like. The frequency power characteristic may be used instead of the frequency amplitude characteristic. Let F (f) be the frequency amplitude characteristic obtained by frequency conversion.

The smoothing unit 113 smoothes the frequency amplitude characteristic F (f) (S12). As the smoothing method, cepstrum analysis, simple moving average, Savitzky-Golay filter, smoothing spline, and the like can be used. The smoothed frequency amplitude characteristic is defined as the smoothing characteristic SF (f).

The feature amount extraction unit 114 detects the peak and notch of the smoothing characteristic SF (f) and extracts the feature amount (S13). Specifically, peaks and notches are detected based on the slope of the smoothing characteristic SF (f). The maximum value of the smoothing characteristic SF (f) is the peak, and the minimum value is the notch. FIG. 6 is a diagram schematically showing smoothing characteristics. Here, an example is shown in which the smoothing characteristic includes three peaks and two notches.

The peak with the lowest frequency is referred to as the first peak P [1], and the notch with the lowest frequency is referred to as the first notch N [1]. Similarly, the second peak P [2], the second notch N [2], and the third peak P [3] are set in order from the lower frequency side. Peaks and notches are generalized as P [l], N [l]. l is an integer of 1 or more and indicates a peak and notch number.

The feature amount extraction unit 114 extracts the peak frequency, the amplitude (gain) at the peak frequency, the notch frequency, and the amplitude (gain) at the notch frequency as the feature amount. The amplitude at the peak frequency and the amplitude at the notch frequency are defined as the peak value and the notch value, respectively. The peak frequencies of P [1] to P [3] are fp1 to fp3, respectively, and the peak values are gp1 to gp3. The notch frequencies of N [1] to N [2] are set to np1 to np2, respectively, and the notch values are set to np1 to np2. The peak value and the notch value are the amplitude values of the frequency amplitude characteristic F (f) before smoothing, but may be the amplitude values of the smoothing characteristic SF (f). Of course, in the case of frequency power characteristics, the peak value and the notch value are power values.

The peak P [1] is represented by a two-dimensional vector of (gp1, fp1). Similarly, peaks P [2] and P [3] are represented by two-dimensional vectors of (gp2, fp2) and (gp3, fp3), respectively. Notches N [1] and N [2] are represented by two-dimensional vectors of (gn1, fn1) and (gn2, fn2), respectively.

All peak and notch frequencies and amplitude values are extracted as features. The feature amount extracted by the feature amount extraction unit 114 is shown as a feature vector. The feature vector has a dimension corresponding to the sum of the number of peaks and the number of notches. Specifically, assuming that the number of peaks is l_max and the number of notches is m_max, the number of dimensions of the feature vector is [2 × (l_max + m_max)]. For example, when the number of peaks is 3 and the number of notches is 2, the feature amount vector has 10 dimensions (fp1, gp1, fn1, gn1, fp2, gp2, fn2, gn2, fp3, gp3). In this way, the feature amount extraction unit 114 extracts the feature amount of the external auditory canal transmission characteristic. The feature amount extraction unit 114 extracts the feature amount of the left and right ear canal transmission characteristics, respectively.

The band for which the feature amount extraction unit 114 obtains the peak and the notch may be the entire frequency band or a part of the frequency band. For example, in the smoothing characteristics, peaks and notches in a frequency band of 4 kHz or higher may be obtained.

The transmission unit 121 transmits the feature amount extracted by the feature amount extraction unit 114 to the server device 300 as user data. User data is data related to external auditory canal transmission characteristics. Specifically, the user data includes features of the user's ear canal transmission characteristics. Let the feature amounts of the user's ear canal transmission characteristics be the user feature amounts hpL_U and hpR_U. The user feature amount hpL_U is a feature amount of the external auditory canal transmission characteristic of the user's left ear, and the user feature amount hpR_U is a feature amount of the external auditory canal transmission characteristic of the user's left ear.

The inverse filter calculation unit 115 calculates the inverse filter based on the external auditory canal transmission characteristic f (t). For example, the inverse filter calculation unit 115 corrects the frequency amplitude characteristic F (f) and the frequency phase characteristic of the external auditory canal transmission characteristic f (t). The inverse filter calculation unit 115 calculates a time signal using the frequency characteristic and the phase characteristic by the inverse discrete Fourier transform. The inverse filter calculation unit 115 calculates an inverse filter by cutting out a time signal with a predetermined filter length. The out-of-head localization processing device 100 stores the inverse filter in a memory or the like.

As described above, the inverse filter is a filter that cancels the headphone characteristics (characteristics between the headphone playback unit and the microphone). The out-of-head localization processing device 100 stores the left and right inverse filters calculated by the inverse filter calculation unit 115. As for the calculation method of the inverse filter, a known method can be used, and therefore detailed description thereof will be omitted. The inverse filter calculation unit 115 generates the left inverse filter Linv based on the external auditory canal transmission characteristic of the left ear. The inverse filter calculation unit 115 generates the right inverse filter Rinv based on the external auditory canal transmission characteristic of the right ear.

Although the out-of-head localization processing device 100 performs the above processing, some or all of the processing may be performed by the server device 300. For example, the external auditory canal transmission characteristic f (t) measured by the out-of-head localization processing device 100 may be transmitted to the server device 300, and the server device 300 may perform frequency conversion, smoothing, and feature extraction processing. The frequency conversion, smoothing, and feature amount extraction processing may be performed by either the server device 300 or the out-of-head localization processing device 100. Furthermore, a device other than the out-of-head localization processing device 100 or the server device 300 may perform some processing.

The server device 300 will be described. The server device 300 includes a receiving unit 301, a data extraction unit 302, a data storage unit 303, a comparison unit 304, a selection unit 305, and a transmission unit 306.

The receiving unit 301 receives the user data transmitted from the out-of-head localization processing device 100. The user data includes user feature amounts hpL_U and hpR_U. The data extraction unit 302 extracts a part of the preset data stored in the data storage unit 303 based on the user feature amount.

The data storage unit 303 is a database that stores data related to a plurality of subjects measured in advance measurement as preset data. The database may be distributed across multiple storage devices. The data stored in the data storage unit 303 will be described with reference to FIG. 7. FIG. 7 is a table showing the data stored in the data storage unit 303.

The data storage unit 303 stores preset data for each of the left and right ears of the person to be measured. Specifically, the data storage unit 303 has a table format in which the subject ID, the left and right ears, the feature amount, the spatial acoustic transmission characteristic 1, and the spatial acoustic transmission characteristic 2 are arranged in one row. The data format shown in FIG. 7 is an example, and a data format or the like in which objects of each parameter are associated with each other by a tag or the like may be adopted instead of the table format.

Two data sets are stored in the data storage unit 303 for one person A to be measured. That is, the data storage unit 303 stores a data set relating to the left ear of the subject A and a data set relating to the right ear of the subject A.

One data set includes the subject ID, the left and right ears, the feature amount, the spatial acoustic transmission characteristic 1, and the spatial acoustic transmission characteristic 2. The feature amount is data based on the second pre-measurement by the measuring device 200 shown in FIG. The feature amount is extracted from the ear canal transmission characteristics from the first position in front of the external auditory canal to the microphones 2L and 2R. Specifically, the feature amount is extracted by subjecting the external auditory canal transmission characteristic of the person to be measured 1 to the process of FIG. The feature amount of the user and the feature amount of the person to be measured are extracted by performing the same processing on the respective external auditory canal transmission characteristics.

The feature amount of the left ear of the subject A is shown as the feature amount hpL_A, and the feature amount of the right ear of the subject A is shown as the feature amount hpR_A. The feature amount of the left ear of the subject B is shown as the feature amount hpL_B, and the feature amount of the right ear of the subject B is shown as the feature amount hpR_B.

The spatial acoustic transmission characteristic 1 and the spatial acoustic transmission characteristic 2 are data based on the first pre-measurement by the measuring device 200 shown in FIG. In the case of the left ear of the subject A, the spatial acoustic transmission characteristic 1 is Hls_A, and the spatial acoustic transmission characteristic 2 is Hro_A. In the case of the right ear of the subject A, the spatial acoustic transmission characteristic 1 is Hrs_A, and the spatial acoustic transmission characteristic 2 is Hlo_A. In this way, two spatial acoustic transmission characteristics for one ear are paired. For the left ear of the subject B, Hls_B and Hro_B are paired, and for the right ear of the subject B, Hrs_B and Hlo_B are paired. The spatial acoustic transmission characteristic 1 and the spatial acoustic transmission characteristic 2 may be data after being cut out by the filter length, or may be data before being cut out by the filter length.

For the left ear of the subject A, the feature amount hpL_A, the spatial acoustic transmission characteristic Hls_A, and the spatial acoustic transmission characteristic Hro_A are associated with each other to form one data set. Similarly, for the right ear of the subject A, the feature amount hpR_A, the spatial acoustic transmission characteristic Hrs_A, and the spatial acoustic transmission characteristic Hlo_A are associated with each other to form one data set. Similarly, for the left ear of the subject B, the feature amount hpL_B, the spatial acoustic transmission characteristic Hls_B, and the spatial acoustic transmission characteristic Hro_B are associated with each other to form one data set. Similarly, for the right ear of the subject B, the feature amount hpR_B, the spatial acoustic transmission characteristic Hrs_B, and the spatial acoustic transmission characteristic Hlo_B are associated with each other to form one data set.

The pair of spatial acoustic transmission characteristics 1 and 2 is used as the first preset data. That is, the spatial acoustic transmission characteristic 1 and the spatial acoustic transmission characteristic 2 constituting one data set are set as the first preset data. The features that make up one data set are used as the second preset data. One data set contains a first preset data and a second preset data. Then, the data storage unit 303 stores the first preset data and the second preset data in association with each of the left and right ears of the person to be measured.

Here, it is assumed that the first and second pre-measurements are performed in advance for n (n is an integer of 2 or more) person 1 to be measured. In this case, the data storage unit 303 stores 2n data sets for both ears.

The second preset data will be described in detail with reference to FIG. FIG. 8 is a table showing the second preset data. A feature amount is included for each of the left and right ears of the subject 1. As described above, the feature amount includes the peak value, the peak frequency, the notch value, and the notch frequency. GP1 and FP1 are the peak value and peak frequency of the first peak. GN1 and FG1 are notch values and notch frequencies of the first notch. In FIG. 8, the data relating to the second and subsequent peaks and the notch are omitted. Further, the second preset data may include the number of peaks and the number of notches of the smoothing characteristic.

In addition, the number of peaks or the number of notches is different for each person to be measured. Further, even in the same subject 1, the number of peaks or the number of notches may differ between the left and right ears. Therefore, the number of data included as a feature amount differs for each data set. The number of peaks and the number of notches may be the number of peaks and notches in all frequency bands, or may be the number in some frequency bands. For example, peaks and notches in a band above a predetermined frequency may be used as feature quantities. The predetermined frequency is, for example, a frequency in the range of 2 kHz to 4 kHz.

The data extraction unit 302 searches for the second preset data in which the number of peaks and the number of notches match. Then, the data extraction unit 302 extracts the second preset data in which the number of peaks and the number of notches match. The data extraction unit 302 extracts the second preset data in which the number of peaks and the number of notches match from the data set stored in the data storage unit 303.

For example, in FIG. 6, the number of peaks is 3 and the number of notches is 2. Therefore, in the table shown in FIG. 8, the number of peaks and the number of notches match between the left ear of the person to be measured with ID_A and the right ear of the person to be measured with ID_B. The data extraction unit 302 extracts the feature amount hpL_A of the left ear of the person to be measured with ID_A and the feature amount hpR_B of the right ear of the person to be measured with ID_B. The data extracted by the data extraction unit 302 is used as the extracted data.

The comparison unit 304 compares the extracted data (for example, feature amount hpL_A, feature amount hpL_A) with the user data (for example, user feature amount hpL_U) to obtain the degree of similarity. The comparison unit 304 obtains the second preset data that is most similar to the user data among the extracted data. The comparison unit 304 outputs the comparison result to the selection unit 305. The selection unit 305 selects the first preset data corresponding to the second preset data most similar to the user data. As described above, the first preset data includes the spatial acoustic transmission characteristic 1 and the spatial acoustic transmission characteristic 2.

For example, it is assumed that the feature amount most similar to the user feature amount hpL_U is the feature amount hpL_A of the left ear of the subject 1 with ID_A. The selection unit 305 selects the spatial acoustic transmission characteristics Hls_A and Hro_A corresponding to the feature amount hpL_A. The characteristic selected by the selection unit 305 is used as the selection characteristic, and the data of the selection characteristic is used as the selection data. The selected data is the first preset data included in one data set. The selection data contains a pair of spatial acoustic transfer characteristics.

The data extraction unit 302, the comparison unit 304, and the selection unit 305 perform the same processing on the left and right user feature quantities, respectively. For the user feature amount hpL_U of the user's left ear, a pair of spatial acoustic transmission characteristics (for example, Hls_A, Hro_A) is selected data. Similarly, for the user feature amount hpR_U of the user's right ear, the pair of spatial acoustic transmission characteristics becomes the selection data. The selection unit 305 selects selection data for each user's ear.

Then, the transmission unit 306 transmits the selection data (first preset data) to the out-of-head localization processing device 100. The transmission unit 306 performs processing (for example, modulation processing) according to the communication standard on the selected first preset data and transmits the selected first preset data. The transmission unit 306 transmits a pair of spatial acoustic transmission characteristics as selection data for each of the user's left and right ears.

In this way, the data extraction unit 302 extracts a part of the data set from the data set stored in the data storage unit 303 based on the peak and the notch. The comparison unit 304 compares the second preset data of the extracted data set with the user data. Then, the selection unit 305 selects the first preset data suitable for the user based on the comparison result between the second preset data and the user data.

The receiving unit 122 receives the selection data (first preset data) transmitted from the transmitting unit 306. The receiving unit 122 performs processing (for example, demodulation processing) according to the communication standard on the received first preset data. The receiving unit 122 receives a pair of spatial acoustic transmission characteristics as the first preset data regarding the left ear. The receiving unit 122 receives a pair of spatial acoustic transmission characteristics as the first preset data regarding the right ear.

Then, the filter setting unit 116 sets the spatial acoustic filter based on the first preset data. For example, the spatial acoustic transmission characteristic Hls_A included in the pair of spatial acoustic transmission characteristics becomes the spatial acoustic transmission characteristic Hls of the user U, and the spatial acoustic transmission characteristic Hro_A becomes the spatial acoustic transmission characteristic Hro of the user U. Similarly, the pair of spatial acoustic transmission characteristics included in the first preset data, which is the selected data, becomes the spatial acoustic transmission characteristics Hlo, Hrs of the user U. The spatial acoustic transmission characteristic Hlo and Hrs of the user U are selected from the data of the spatial acoustic transmission characteristic 1 of FIG. The spatial acoustic transmission characteristics Hlo and Hro of the user U are selected from the data of the spatial acoustic transmission characteristic 2 of FIG.

When the first preset data is the data after being cut out by the filter length, the filter setting unit 116 sets the first preset data as it is as a spatial acoustic filter. For example, the spatial acoustic transmission characteristic Hls_A becomes the spatial acoustic transmission characteristic Hls of the user U. When the first preset data is the data before being cut out by the filter length, the filter setting unit 116 performs a process of cutting out the spatial acoustic transmission characteristic to the filter length.

The out-of-head localization processing device 100 stores the out-of-head localization filter in a memory or the like. The filter setting unit 116 sets the spatial acoustic filter in the convolution calculation units 11, 12, 21, and 22 of FIG. Further, the filter setting unit 116 sets the inverse filter calculated by the inverse filter calculation unit 115 in the filter unit 41 and the filter unit 42. As shown in FIG. 1, the out-of-head localization processing device 100 performs arithmetic processing using a spatial acoustic filter corresponding to four spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs, and an inverse filter. The out-of-head localization processing device 100 uses four spatial acoustic filters and two inverse filters to perform the above-mentioned convolution calculation processing on the stereo input signal.

In this way, the data storage unit 303 stores the first preset data and the second preset data in association with each other for each person to be measured 1. The first preset data is data relating to the spatial acoustic transmission characteristics of the subject 1. The second preset data is data relating to the external auditory canal transmission characteristic of the subject 1. Specifically, the second preset data includes the feature amount of the external auditory canal transmission characteristic of the subject 1.

The comparison unit 304 compares the user data with the second preset data. The user data includes a user feature amount related to the external auditory canal transmission characteristic obtained by the user measurement. The comparison unit 304 obtains the similarity of the features. The comparison unit 304 obtains the similarity of the features. Then, the comparison unit 304 determines the person to be measured 1 and the left and right ears that are similar to the user's ear canal transmission characteristics.

The selection unit 305 reads out the first preset data corresponding to the determined subject and the left and right ears. Then, the transmission unit 306 transmits the selected first preset data to the out-of-head localization processing device 100. The out-of-head localization processing device 100, which is a user terminal, performs out-of-head localization processing using a spatial acoustic filter based on the first preset data and an inverse filter based on the measurement data.

By doing so, it is possible to determine an appropriate filter without the user U measuring the spatial acoustic transmission characteristics. Therefore, it is not necessary for the user to go to the listening room or the like, or to install a speaker or the like in the user's house. User measurement is performed with headphones on. That is, if the user U wears the headphones and the microphone, the external auditory canal transmission characteristics of the individual user can be measured. Therefore, an out-of-head localization with a high localization effect can be realized by a simple method. It is preferable that the headphones 43 used for the user measurement and the out-of-head stereotactic listening are of the same type.

Further, the user data and the second preset data are features based on the peaks and notches of the frequency amplitude characteristic. As a result, the amount of data to be processed can be reduced. In particular, the peak frequency, the peak value, the notch frequency, and the notch value are extracted as feature quantities. As a result, even when the data storage unit 303 stores preset data of a large number of subjects, matching can be appropriately performed with a small amount of data.

Further, in the method according to the present embodiment, it is not necessary to perform an auditory test for listening to a large number of preset characteristics. Therefore, the burden on the user can be reduced and the convenience can be improved. Then, by comparing the feature quantities of the person to be measured and the user, it is possible to select a person to be measured having similar characteristics. Then, since the acoustic transmission filter is generated from the first preset data of the selected subject's ear, a high out-of-head localization effect can be expected.

Next, the process of obtaining the similarity will be described with reference to FIG. FIG. 9 is a flowchart showing an example of the process of obtaining the similarity. In the present embodiment, the comparison unit 304 calculates the similarity using the inter-vector distance or the correlation coefficient. The comparison unit 304 may calculate the similarity using both the inter-vector distance and the correlation coefficient.

The data extraction unit 302 calculates the evaluation range (S21). The evaluation range is a frequency band for extracting peaks and notches. Here, the evaluation range is a frequency band of 4 kHz or higher. This is because individual characteristics are likely to appear in the frequency band of 4 kHz or higher. The lower limit of the evaluation range is not limited to 4 kHz, and may be, for example, 2 kHz. The upper limit of the evaluation range can be the maximum frequency of the frequency characteristic obtained by frequency conversion. If only the peaks and notches within the evaluation range have been extracted in advance, S21 can be omitted.

The data extraction unit 302 extracts data from the preset data stored in the data storage unit 303 (S22). That is, the data extraction unit 302 extracts the second preset data in which the number of peaks and the number of notches match in the evaluation range.

Next, the comparison unit 304 normalizes and scales the user data and the second preset data. For example, when calculating the Euclidean distance, the value (unit) of the evaluation range differs depending on the frequency and amplitude value. Therefore, the comparison unit 304 normalizes the peak frequency and the notch frequency so that the minimum frequency of the frequency band to be evaluated is 0 and the maximum frequency is 1. In the amplitude value, the comparison unit 304 normalizes the peak value and the notch value so that the maximum value of the amplitude value (peak value) to be evaluated becomes 1 and the minimum value of the amplitude value (notch value) becomes 0. .. For example, in the example shown in FIG. 6, the range of fp1 to fp3 is 0 to 1 on the horizontal axis (frequency axis). Similarly, on the vertical axis (amplitude axis), the range of gn1 to gp3 is 0 to 1.

Scale conversion changes the scale of envelope data so that discrete spectral data (frequency amplitude characteristics) are evenly spaced on the logarithmic axis. The frequency amplitude characteristics obtained by the frequency converter are evenly spaced in terms of frequency. That is, since the frequency amplitude characteristics are evenly spaced on the frequency linear axis, they are not evenly spaced on the frequency logarithmic axis. Therefore, interpolation processing is performed so that the frequency amplitude characteristic data are evenly spaced on the frequency logarithmic axis.

In the frequency amplitude characteristic, on the logarithmic axis, the lower the frequency range, the coarser the adjacent data spacing, and the higher the frequency range, the denser the adjacent data spacing. Therefore, the comparison unit 304 interpolates the data in the low frequency band in which the data interval is coarse. Specifically, the comparison unit 304 obtains discrete envelope data arranged at equal intervals on the logarithmic axis by performing interpolation processing such as three-dimensional spline interpolation. Envelope data that has undergone scale conversion is defined as scale conversion data. The scale conversion data is a spectrum in which the frequency is associated with the amplitude value or the power value.

The reason for converting to a logarithmic scale will be explained. It is generally said that human senses are converted to logarithms. Therefore, it is important to consider the frequency of the audible sound on the logarithmic axis. By performing the scale conversion, the data are evenly spaced in the above-mentioned sensory quantity, so that the data can be treated equivalently in all frequency bands. As a result, mathematical calculations, frequency band division and weighting become easy, and stable results can be obtained. The comparison unit 304 may convert the envelope data into a scale close to human hearing (referred to as an auditory scale), not limited to the logarithmic scale. As the auditory scale, a logarithmic scale (Log scale), a mel scale, a Bark scale, an ERB (Equivalent Rectangular Bandwidth) scale, or the like may be used for scale conversion. The comparison unit 304 scales the envelope data on an auditory scale by data interpolation. For example, the comparison unit 304 interpolates the data in the low frequency band in which the data interval is coarse in the auditory scale to make the data in the low frequency band dense. Data that are evenly spaced on the auditory scale are dense in the low frequency band and coarse in the high frequency band on the linear scale. By doing so, the comparison unit 304 can generate scale conversion data at equal intervals on the auditory scale. Of course, the scale conversion data does not have to be completely evenly spaced data in the auditory scale.

In S23 and S25, normalization and scale conversion do not have to be the same process. For example, the scale for finding the distance and the scale for finding the correlation coefficient may be different. Furthermore, only one of normalization and scale conversion may be performed. Alternatively, normalization and scale conversion may be omitted. The processing order of normalization and scale conversion may be arbitrary, whichever comes first.

Next, the comparison unit 304 calculates the distance between the feature vectors (S24). The process for obtaining the distance between the feature vectors will be described with reference to FIG. FIG. 10 is a flowchart showing a process of obtaining a distance. Here, the comparison unit 304 obtains the Euclidean distance of the feature vector for the user data and the second preset data. First, for initialization, l = 0, m = 0, and q = 0 (S31). q indicates the Euclidean distance.

The comparison unit 304 determines whether or not l = l_max (S32). l_max is an integer indicating the number of peaks. When it is determined that l = l_max is not (NO in S32), q + = || AP [l] -BP [l] || is set (S33). AP [l] is a two-dimensional vector indicating the l-th peak in the user data. BP [l] is a two-dimensional vector indicating the l-th peak in the second preset data. The comparison unit 304 obtains the Euclidean distance between the peaks of the l-th peaks.

The comparison unit 304 increments l (S34). Then, the processes of S32 to S34 are repeated until l = l_max. This gives the distance between all the peaks. That is, q is a value obtained by adding the distance between peaks only l_max times. When l_max = 3, the peak-to-peak distance between the first peaks (first peak-to-peak distance), the peak-to-peak distance between the second peaks (second peak-to-peak distance), and the third peak-to-peak distance. The inter-peak distance (third inter-peak distance) is obtained. In S33, q is the sum of the distances between the first to third peaks.

When it is determined that l = l_max (YES in S32), the comparison unit 304 determines whether or not m = m_max (S35). m_max is an integer indicating the number of notches. When m = m_max is not (NO in S35), q + = || AN [m] -BN [m] || is set (S36). AP [m] is a two-dimensional vector indicating the m-th notch in the user data. BP [m] is a two-dimensional vector indicating the m-th notch in the second preset data. The comparison unit 304 obtains the Euclidean distance between the m-th notches. In the above processing, AP [l], AN [l], BP [l], and BN [l] are data that have been normalized and scale-converted in S23.

The comparison unit 304 increments m (S37). Then, the processes of S35 to S37 are repeated until m = m_max. This gives the distance between all the notches. That is, q is the sum of the distance between l peaks and the distance between m notches. When l_max = 3 and m_max = 2, q is the sum of the first to third peak distances and the first to second peak distances.

Then, the comparison unit 304 sets q = q / (l_max + m_max). As a result, q becomes the average value of (l_max + m_max) distances. In this way, the distance between the feature vectors can be obtained. As described above, since the second preset data in which the number of peaks and the number of notches match are extracted, the distance between the feature vectors can be appropriately obtained.

Returning to the description of FIG. The comparison unit 304 calculates the correlation coefficient (S26). The process for obtaining the correlation coefficient will be described with reference to FIG. In the following processing, AP [l], AN [m], BP [l], and BN [m] are normalized and scale-converted data in S25.

The comparison unit 304 interpolates the feature amount data (S41). FIG. 12 shows an example of linearly interpolating feature data. Here, the user data has three peaks AP [1] to AP [3] and two notches AN [1] to AN [2]. Further, the second preset data has three peaks BP [1] to BP [3] and two notches BN [1] to BN [2]. The comparison unit 304 calculates the amplitude value between the peak and the notch by linear interpolation. Of course, not limited to linear interpolation, spline interpolation or the like may be used. The data obtained by interpolating the user data is referred to as the interpolated data linAF (f), and the data obtained by interpolating the second preset data is referred to as the interpolated data linBF (f).

The reason for generating the interpolated data will be described. The number of peaks and notches in the two data looking for correlation may be small. In that case, it becomes difficult to sufficiently evaluate the similarity between the two waveforms. Therefore, the data (amplitude value) between the peak and the notch is interpolated. As a result, the correlation of the external auditory canal transmission characteristics can be obtained with high accuracy.

Next, the comparison unit 304 calculates the evaluation range of the interpolated data (S42). For example, after interpolation, the maximum frequencies of the two data to be evaluated may differ. In this case, the maximum value of the evaluation range of the frequency axis is set up to the frequency with which the two maximum frequencies are smaller. In the example shown in FIG. 11, the peak frequency of BP [3] is larger than the peak frequency of AP [3]. Therefore, the interpolated data linAF (f) and linBF (f) are generated with the peak frequency of AP [3] as the maximum frequency. That is, the maximum value of the evaluation range is set based on the frequencies of AP [l_max], AN [m_max], BP [l_max], and BN [m_max].

When the peak frequency of AP [l_max] is larger than the notch frequency of AN [m_max] and the peak frequency of BP [l_max] is larger than the notch frequency of BN [m_max], the peak frequency of AP [l_max] and BP The smaller peak frequency among the peak frequencies of [l_max] is set as the maximum frequency of the interpolated data. When the peak frequency of AP [l_max] is smaller than the notch frequency of AN [m_max] and the peak frequency of BP [l_max] is smaller than the notch frequency of BN [m_max], the notch frequency of AN [m_max] and BN Of the notch frequencies of [m_max], the smaller notch frequency is set as the maximum frequency of the interpolation data.

The minimum value of the evaluation range of the frequency axis may be 4 kHz at which personal characteristics are likely to appear. Since the frequency range in which personal characteristics are likely to appear may differ depending on the method for measuring the auricle characteristics, the minimum value of the evaluation range of the frequency axis does not need to be 4 kHz. As described in S21, the minimum value can be in the range of 2 kHz to 4 kHz. The evaluation range calculated in S42 and the evaluation range calculated in S21 may be different or the same.

Then, the comparison unit 304 calculates the correlation coefficient (S43). Specifically, the comparison unit 304 calculates the correlation coefficient r between the interpolated data linAF (f) and the interpolated data linBF (f) in the evaluation range.

Returning to the description of FIG. The comparison unit 304 calculates the similarity based on the Euclidean distance q and the correlation coefficient r (S27). The smaller the value of the Euclidean distance q, the closer the distance, that is, the similar characteristics. The correlation coefficient r takes a value between -1 and +1 and the closer it is to +1 the more similar it is. Therefore, the smaller the value of (1-r), the more similar the characteristics. The comparison unit 304 sets the similarity to q + (1-r). Alternatively, the comparison unit 304 may set the similarity to q * (1-r). Further, the comparison unit 304 may weight q and (1-r). For example, the comparison unit 304 may multiply at least one of q and (1-r) by a coefficient for weighting.

The comparison unit 304 calculates the degree of similarity with the user characteristics for all the data extracted in S22. That is, the comparison unit 304 calculates the similarity with the user data for each of the second preset data having the same number of peaks and the same number of notches. Then, the selection unit 305 selects the first preset data corresponding to the second preset data having the highest similarity (S28). In the present embodiment, since the similarity is obtained based on the correlation coefficient and the distance, it is possible to select the subject 1 whose characteristics are similar to those of the user. Then, since the selection unit 305 selects the first preset data of the selected ear of the subject to be measured, a high out-of-head localization effect can be expected.

In the above description, the similarity is determined based on the correlation coefficient and the distance, but the similarity may be determined based on only one of the correlation coefficient and the distance. Further, in the present embodiment, since only the second preset data in which the number of peaks and the number of notches match is extracted, the number of dimensions of the feature vector can be made uniform. As a result, the distance between the vectors can be calculated appropriately, and more appropriate matching can be performed.

Further, in the above description, the correlation coefficient r between the interpolated data linAF (f) and the interpolated data linBF (f) was obtained, but the comparison unit 304 has the frequency amplitude characteristic F (f) and the interpolated data linBF (f). The correlation coefficient may be obtained. Alternatively, the comparison unit 304 may obtain the correlation coefficient between the smoothing characteristic SF (f) and the interpolated data linBF (f). In this case, the out-of-head localization processing device 100 may transmit the external auditory canal transmission characteristic f (t), the frequency amplitude characteristic F (f), or the smoothing characteristic SF (f) to the server device 300. Further, a part or all of the processing for obtaining the frequency amplitude characteristic F (f) or the smoothing characteristic SF (f) may be performed by the server device 300.

Modification example In the modification example, the second preset data is clustered. FIG. 13 is a table for explaining an example in which clustering is performed based on the number of peaks and the number of notches. In the first embodiment, in order to obtain the distance between the feature vectors, the second preset data in which the number of peaks and the number of notches match the user feature amount is extracted. Therefore, by clustering the second preset data in advance according to the number of peaks and the number of notches, the data extraction process can be performed quickly. For example, when the number of peaks is 3 and the number of notches is 2, the data extraction unit 302 extracts the second preset data included in the cluster 330.

Of course, the second preset data may be divided into clusters (groups) by using other than the number of peaks and the number of notches. For example, clustering may be performed according to the feature vector. Specifically, clustering can be performed using the k-means method or the like. Further, the clustering may be hierarchical clustering or non-hierarchical clustering. Further, it is preferable to perform clustering so that each cluster contains a sufficient number of second preset data.

The data extraction unit 302 may extract the second preset data of the cluster to which the user feature amount belongs. In the data storage unit 303, it is sufficient that the plurality of the second preset data are classified into two or more clusters. The data extraction unit 302 extracts the second preset data included in the cluster to which the user feature amount belongs.

The out-of-head localization processing device 100 and the measurement processing device 201 perform arithmetic processing for appropriately generating a filter according to the measurement result. The out-of-head localization processing device 100 and the measurement processing device 201 are a personal computer (PC), a tablet terminal, a smart phone, and the like, and include a memory and a processor. The memory stores processing programs, various parameters, measurement data, and the like. The processor executes a processing program stored in memory. Each process is executed when the processor executes the process program. The processor may be, for example, a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit), or the like. ..

It is possible to omit some blocks from the block diagrams of FIGS. 1 and 2. For example, in FIG. 2, the transmitting unit 121, the transmitting unit 306, the receiving unit 122, and the receiving unit 301 may be provided in other devices. Further, a plurality of devices may be dispersed to perform the processing. Of the flowcharts of FIGS. 5 and 9, some or all of the processing can be omitted. For example, S23 to S27 can be replaced with other comparison processing. That is, the comparison unit 304 may compare the user feature amount with the second preset data by a process other than S23 to S27.

Part or all of the above processing may be executed by a computer program. The programs described above can be stored and supplied to a computer using various types of non-transitory computer readable media. Non-transient computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)) are included. The program may also be supplied to the computer by various types of temporary computer readable media. Examples of temporary computer-readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

Although the invention made by the present inventor has been specifically described above based on the embodiment, the present invention is not limited to the above embodiment and can be variously modified without departing from the gist thereof. Needless to say.

U User 1 Subject 2L Left microphone 2R Right microphone 5L Left speaker 5R Right speaker 9L Left ear 9R Right ear 10 Out-of-head localization processing unit 11 Convolution calculation unit 12 Convolution calculation unit 21 Convolution calculation unit 22 Convolution calculation unit 24 Adder 25 Adder 41 Filter unit 42 Filter unit 43 Headphones 100 Out-of-head localization processing device 111 Characteristic acquisition unit 112 Frequency conversion unit 113 Smoothing unit 114 Feature quantity extraction unit 115 Inverse filter calculation unit 116 Filter setting unit 121 Transmission unit 122 Receiver unit 200 Measurement Equipment 201 Measurement processing equipment 300 Server equipment 301 Reception unit 302 Data extraction unit 303 Data storage unit 304 Comparison unit 305 Selection unit 306 Transmission unit

Claims (11)

An output unit that is attached to the user and outputs sound toward the user's ear,
A microphone unit that is attached to the user's ear and collects the sound output from the output unit.
A measurement processing device that outputs a measurement signal to the output unit and measures a sound collection signal output from the microphone unit.
An out-of-head localization filter determination system including a server device capable of communicating with the measurement processing device.
The server device
Data storage in which the first preset data relating to the spatial acoustic transmission characteristics from the sound source to the ear of the person to be measured and the second preset data relating to the characteristics of the external auditory canal transmission characteristics of the ear of the person to be measured are associated and stored. The unit includes a data storage unit that stores a plurality of the first and second preset data acquired for a plurality of subjects.
The out-of-head localization filter determination system is
A frequency conversion unit that obtains frequency characteristics by frequency-converting the sound-collecting signal picked up by the microphone unit.
A smoothing unit that smoothes the frequency characteristics,
A feature amount extraction unit that obtains a smoothed peak and notch of the frequency characteristic and extracts the feature amount of the frequency characteristic as a user feature amount based on the peak and notch.
A data extraction unit that extracts a part of the second preset data among the plurality of second preset data stored in the data storage unit based on the user feature amount, and a data extraction unit.
A comparison unit that compares the user feature amount with the extracted second preset data,
An out-of-head localization filter determination system including a selection unit for selecting a first preset data from a plurality of the first preset data based on a comparison result. The out-of-head localization filter determination system according to claim 1, wherein the data extraction unit extracts the second preset data by using the number of the peaks and the notches. The out-of-head localization filter determination system according to claim 2, wherein the data extraction unit extracts the second preset data in which the number of peaks and notches in a predetermined frequency band matches the user data. The feature amount is
A claim that includes the peak frequency of the peak in a predetermined frequency band and the peak value at the peak frequency, and also includes the notch frequency of the notch in the predetermined frequency band and the notch value at the notch frequency. The out-of-head localization filter determination system according to any one of 1 to 3. Any one of claims 1 to 4, wherein the comparison unit obtains the similarity based on the distance between the feature vector including the user feature amount and the feature vector including the feature amount of the second preset data. The out-of-head localization filter determination system described in. The second preset data according to any one of claims 1 to 5, wherein the similarity is obtained based on the correlation between the interpolation characteristic obtained by interpolating between the peak and the notch and the user characteristic. Out-of-head localization filter determination system. In the data storage unit, a plurality of the second preset data are classified into two or more clusters.
The data extraction unit
The out-of-head localization filter determination system according to any one of claims 1 to 6, wherein the second preset data included in the cluster to which the user feature amount belongs is extracted. A frequency conversion unit that obtains frequency characteristics by frequency-converting the sound collection signal obtained by collecting the measurement signal output from the output unit with the microphone unit.
A smoothing unit that smoothes the frequency characteristics,
A feature amount extraction unit that obtains a smoothed peak and notch of the frequency characteristic and extracts the feature amount of the frequency characteristic as a user feature amount based on the peak and notch.
A spatial acoustic filter setting unit that sets a spatial acoustic filter based on a spatial acoustic transmission characteristic associated with a feature quantity similar to the user feature quantity.
An out-of-head localization processing device including an inverse filter calculation unit that calculates an inverse filter that cancels the characteristics of the output unit based on the sound pick-up signal. Data storage in which the first preset data relating to the spatial acoustic transmission characteristics from the sound source to the ear of the person to be measured and the second preset data relating to the characteristics of the external auditory canal transmission characteristics of the ear of the person to be measured are associated and stored. A data storage unit that stores a plurality of the first and second preset data acquired for a plurality of subjects, and a data storage unit.
A data extraction unit that extracts a part of the second preset data among the plurality of second preset data stored in the data storage unit based on the user feature amount, and a data extraction unit.
A comparison unit that compares the user feature amount with the feature amount of the second preset data,
A selection unit for selecting the first preset data from the plurality of first preset data based on the comparison result is provided.
An out-of-head localization filter determining device in which the frequency characteristic of the external auditory canal transmission characteristic is smoothed and the feature amount is extracted based on the smoothed peak and notch of the frequency characteristic. An output unit that is attached to the user and outputs sound toward the user's ear,
It is an out-of-head localization filter determination method for determining an out-of-head localization filter for the user by using a microphone unit which is attached to the user's ear and has a microphone for collecting the sound output from the output unit. ,
The step of frequency-converting the sound pick-up signal picked up by the microphone unit to obtain the frequency characteristics, and
The step of smoothing the frequency characteristics and
A step of obtaining a smoothed peak and notch of the frequency characteristic and extracting the feature amount of the frequency characteristic as a user feature amount based on the peak and notch.
A step of extracting a part of the second preset data from the plurality of second preset data stored in the data storage unit based on the user feature amount, and a step of extracting the second preset data.
A step of comparing the user feature amount with the extracted second preset data,
An out-of-head localization filter determination method including a step of selecting a first preset data from a plurality of first preset data based on a comparison result. An output unit that is attached to the user and outputs sound toward the user's ear,
A computer is provided with an out-of-head localization filter determination method for determining an out-of-head localization filter for the user by using a microphone unit that is attached to the user's ear and has a microphone that collects the sound output from the output unit. It ’s a program to run
The method for determining the out-of-head localization filter is
The step of frequency-converting the sound pick-up signal picked up by the microphone unit to obtain the frequency characteristics, and
The step of smoothing the frequency characteristics and
A step of obtaining a smoothed peak and notch of the frequency characteristic and extracting the feature amount of the frequency characteristic as a user feature amount based on the peak and notch.
A step of extracting a part of the second preset data from the plurality of second preset data stored in the data storage unit based on the user feature amount, and a step of extracting the second preset data.
A step of comparing the user feature amount with the extracted second preset data,
A program including a step of selecting a first preset data from a plurality of first preset data based on a comparison result.

Images